Hirotaka Aoki, Nozomi Kobayashi, Mizuki Fuke, Tomonori Yasui (2023). Development of Scenario-Based Approach Aided by Key PSFs to Design of New Interface of Autonomous Vehicles Monitoring System: A Case Study. In: Human Factors in Transportation. AHFE Open Access, vol. 95, pp. 154–160. https://doi.org/10.54941/ahfe1003801
In the present paper, we develop an improved scenario-based design approach to design a new interface of the monitoring system for autonomous vehicles that adapts to the autonomous driving levels 4 to 5 condition. Although the future society with high-level autonomous vehicles such as those having levels 4 to 5 is clearly anticipated, there still remains much uncertainty relating to how autonomous vehicles are implemented in terms of platform technology (e.g., AI, image processing, etc.), social norms (e.g., laws, safety responsibility, etc.), context of use (e.g., human role, allocated tasks, etc.) and so forth. For example, it is unclear the followings: What can be done automatically by the platform technologies of autonomous vehicles (i.e., uncertainty in technological advancement), what kind of socio-technical support can be given when human operators have to intervene in a critical incident that cannot be managed automatically (i.e., uncertainty in tasks required to human), etc. Caused by these uncertainties, it is of great difficulty to design user-centered interface of autonomous vehicle control/monitoring systems since the users/operators’ tasks and needs are not well defined.The paper proposes a design approach adapting to the above-mentioned background issues. The most part of the approach is based on scenario-based design (Rosson and Carroll 2002). This is referred to as the technique to elicit effective specifications of interface based on scenarios in which how a human uses and interacts with a specific system are vividly described to capture the essence of interaction and physical design. In the approach, requirements/rough draft of basic specifications are elicited by interpreting desirable interaction/behavior described in the scenarios Although the scenario-based design is applicable for a new product design, creating successful scenario having appropriate level of details and in which key behaviors/tasks and their contexts are covered requires rich knowledge about interaction design as well as expertise/experiences in practice. In order to support the difficulty in creating scenarios, we propose a series of processes where scenarios are created and refined iteratively supported by a list of PSFs (performance shaping factors) directly connected with the system’s context of use. The list of PSFs are, in our processes, used as guidelines/heuristics not only to obtain insight for creating narrative description of a human, but also to keep consistency of envisioned usage episodes in scenarios (refined iteratively) by maintaining an orientation to identical key context factors. A case study was performed with a big manufacturing company in Japan where our design approach was applied to design a new interface of autonomous vehicles monitoring system. The potential target users of the system are operators working at a general operation center of public transportation in high-level autonomous vehicles condition. In the beginning of the design phase, the first version of scenario where sequences of possible and desirable operators’ behaviors were stressed was created by referring PSFs in driving contexts. By interpreting potential needs and decomposing them into requirements logically, the first version of design solution was developed. Along with both of iterative revisions of scenarios and subsequent design improvements, interface design was refined. Based on the results of this application and feedbacks from our collaborating company, we discuss feasibility of our approach as well as implications for development of systems for autonomous vehicle monitoring.
In the present paper, we develop an improved scenario-based design approach to design a new interface of the monitoring system for autonomous vehicles that adapts to the autonomous driving levels 4 to 5 condition. Although the future society with high-level autonomous vehicles such as those having levels 4 to 5 is clearly anticipated, there still remains much uncertainty relating to how autonomous vehicles are implemented in terms of platform technology (e.g., AI, image processing, etc.), social norms (e.g., laws, safety responsibility, etc.), context of use (e.g., human role, allocated tasks, etc.) and so forth. For example, it is unclear the followings: What can be done automatically by the platform technologies of autonomous vehicles (i.e., uncertainty in technolo- gical advancement), what kind of socio-technical support can be given when human operators have to intervene in a critical incident that cannot be mana- ged automatically (i.e., uncertainty in tasks required to human), etc. Caused by these uncertainties, it is of great difficulty to design user-centered interface
Development of Scenario-Based Approach Aided by Key PSFs 155
of autonomous vehicle control/monitoring systems since the users/operators’ tasks and needs are not well defined.
The paper proposes a design approach adapting to the above-mentioned background issues. The most part of the approach is based on scenario-based design (Rosson and Carroll 2002). This is referred to as the technique to elicit effective specifications of interface based on scenarios in which how a human uses and interacts with a specific system are vividly described to capture the essence of interaction and physical design. In the approach, require- ments/rough draft of basic specifications are elicited by interpreting desirable interaction/behavior described in the scenarios Although the scenario-based design is applicable for a new product design, creating successful scenario having appropriate level of details and in which key behaviors/tasks and their contexts are covered requires rich knowledge about interaction design as well as expertise/experiences in practice. In order to support the difficulty in creating scenarios, we propose a series of processes where scenarios are cre- ated and refined iteratively supported by a list of PSFs (performance shaping factors) directly connected with the system’s context of use. The list of PSFs are, in our processes, used as guidelines/heuristics not only to obtain insight for creating narrative description of a human, but also to keep consistency of envisioned usage episodes in scenarios (refined iteratively) by maintaining an orientation to identical key context factors.
DEVELOPMENT PROCEDURE PROPOSED Scenario-Based Design Approach
As mentioned before, our systems design approach is developed for desi- gning of HMI of autonomous vehicles monitoring systems. Since this is a new system development project, competing products analysis cannot be condu- cted. In this context, it can be expected that Scenario-based design (Rosson et al. 2001, Rosson et al. 2002) gives a great insight about determination of systems specifications. The overall image of the design process is depicted in Figure 1. Scenario in this scenario-based design is a narrative description where the user of a future system/product is concretely and vividly described. Various versions of scenarios are used at an early point in the early stage of the development processes in order to determine specifications. Since the scena- rios involves narrative descriptions of envisioned usage episodes, they enable design teams to focus not on technical aspects but on user experiences. By taking a top-down thinking process in which the user’s behaviors in the sce- narios are deployed to the system’s/product’s functionality to be implemented, preliminary users’ requirements and specifications can be elicited.
Though the scenario-based design seems promising, the success of the design is heavily depending on the quality of scenarios. A scenario which is really effective for design should have stories telling how users will use the system/product to accomplish tasks and relating other activities vividly including critical contexts which are the essences of interactions. The cre- ation of such effective scenarios requires both of rich expertise in theories (e.g., psychological theories relating to interaction designs) and good senses in writing. Considering the above-mentioned requirements, some supporting
Figure 1: Scenario-based design processes.
tool to create scenario is strongly required, especially from practical point of view.
Scenario Creation and Refinement Aided by PSFs List To support creation of effective scenario, we think that a predetermined list of items that should be described in scenarios is one possible tool. In the field of HRA (Human Reliability Analysis), performance shaping factors (PSFs) are considered to examine human error rates in various HRA methodologies (e.g., Lee et al. 2011). PSFs are referred to as the factors affecting human behaviour. For example, THERP (Technique for Human Error Rate Predi- ction), which is one of the most famous and traditional HRA methodologies, provides the PSFs in a nuclear power plant domain (Kirwan 1996, Kirwan et al. 1997). The PSFs in THERP system can apply for other domains which include human monitoring tasks. The PSFs in THERP system is consisted of the following three categories: External PSFs, stressor PSFs and internal PSFs (in detail, see Kirwan 1994). These PSFs are well-organized, and can be used as predetermined items that should be considered when creating a scenario.
Figure 2 shows a summary of scenario creation procedure. For example, the PSFs in THERP included in external PSFs are classified into three sub cate- gories. The first category named “situational characteristics” has nine items in total. Following the procedure shown in Figure 2, we examine whether the item is closely connected with the user’s behavior in the specific product. If the item has to be considered, we create a description (e.g., brief episode of usage), and keep the item in the revised list. In the end of the procedure, we can obtain descriptions as well as the items that are likely to be critical for the context of use. By synthesizing descriptions, a story representing how a user behaves is created.
Development of Scenario-Based Approach Aided by Key PSFs 157
Figure 2: Scenario creation processes.
CASE STUDY: DESIGN OF NEW INTERFACE OF AUTONOMOUS VEHICLES MONITORING SYSTEM Creation of Scenario Aided by PSFs
The present study is a collaborative work with a Japanese manufacturing company. The company started a project where it develops a HMI of autono- mous vehicles monitoring system. This is the first experience for the company to develop such system, since the level of autonomous vehicles in Japan is not so high meaning that this kind of system is recognized as future pro- ducts. To check the feasibility of our development procedure, we apply it to the early stage of the project. Ethics approval was granted by the Tokyo Insti- tute of Technology Review Board (2022150). Due to the strict non-disclosure agreement, the explanation about the interface is suppressed in the present paper.
Following the procedure, we created a first version scenario, which rou- ghly describe how a user (operator) behaves in a hypothetical work situation. Table 1 shows a part of the scenario and design implications obtained by interpreting the scenario. Italic words in parentheses in the scenario shows items of PSFs used to create the piece of episode. For example, “work hours” included in PSFs can be treated as a critical factor relating to the context of use of the monitoring system. From this item (work hours), we could recognize that the working hours and related human factors had to be included in the usage episodes. Very similar idea could be also obtained from the PSFs items of shit rotation, duration of stress and long, uneventful vigilance periods. By synthesizing the ideas, we could make the first sentence of the scenario (see Table 1).
Specifications Elicitation Process By examining the stories which vividly describe how an operator behaves, we could obtain insights that were useful for determination of specifications.
Table 1. Example scenario and elicited implications.
Table available in the original PDF paper.
Scenario created -first version- Implications for design Less workload should be avoided to keep vigilance level. Warning level should be defined appro- priately. Knowledge of typical adverse events should be taught in advance. Appropriate wording of adverse events, which is consistent with users’ mental model, is necessary… (omit) I frequently experience on intervention to autonomous vehicles every day (work hours, shift rotation, duration of stress, long, uneventful vigilance periods). The types of the interventions are very different (anticipatory requirements, interpretation, complexity). I feel a kind of pressure since I need to understand the reason why the autonomous system calls me immediately based on very limited information (long and short-term memory, task speed). Sometimes, the reason is very severe (task criticality). Caused by this, I sometimes feel impatient and irritated during working (stress). Honestly speaking, I want a little more information which help me to understand the situation/reason by which the system calls me (procedure required)… (omit)
Present scene, scene when the adverse event occurs, and relating contexts are necessary to understand the situation. Scenes transitions for at least a few seconds seems necessary. Clear mark that highlights the pede- strian is needed. In addition to visual information, audi- tory information should be perceived by the operator… (omit)
One day in the afternoon, the system calls me with a warning sound (perceptual requirements, cautions and warnings, fatigue). I look at the driving information display immediately, and try to recognize what adverse event occurs (procedure required). I can guess that the event is something relating to a pedestrian who can be seen in front of the car (interpretation, state of current practice or skill). The only thing I can understand quickly is that the event occurs at an intersection since the monitoring system shows a map around the car (perceptual requirements, decision making). I really feel nervous because something happens where a pedestrian is involved (interpretation, stress, task criticality). But still what really happens is not clear… (omit)
One of critical implications is underlined in Table 1. Reading the correspon- ding part of the scenario, we can easily generate the following question: How does the operator identify the current situation? It is obvious that the seeing the present scene is not sufficient because the objects involved in the
Development of Scenario-Based Approach Aided by Key PSFs 159
adverse event (e.g., pedestrian) may have already left from the view sight. This lets us generate a desirable design concept of interface enabling operators to understand the situation immediately by showing both of present status and additional status information for appropriate time interval in which the speci- fic adverse event is valid. By breaking down the design concept, the following basic specifications could be elicited:
Information display showing present status constantly.
Additional information display showing warning message of adverse event, and status before, during and after the adverse event whenever necessary.
Generation of First Version Prototype Reflecting the specifications elicited from the scenario, the first version of the prototype was developed. Figure 3 shows a schematic description of the pro- totype. Caused by the NDA, only a part of specifications which are directly connected with the output of above-mentioned design processes are roughly described in the figure. One of the notable features equipped in the prototype is an automatic scene playing in the sub display, which allows an operator to take over from the autonomous vehicles controlling systems smoothly.
Figure 3: Schematic description of the prototype.
CONCLUSION
As well known in human factors and ergonomics societies, a critical princi- ple of human-centred design, in general, is an iterative design process. We are now performing iterative design in which both of scenarios and prototypes are refined. The keys of our approach are as follows: (i) operators desirable behaviors are always focused and they are vividly described in scenarios, and (ii) the consistency among scenarios are always kept since the identical PSFs are used as references at any phases, which contributes to efficient and effe- ctive design processes. Though the paper does not show details of the design
processes, this can show the essence of the design methodology, which is use- ful for new products/systems design as was recognized by the collaborating practitioners.
ACKNOWLEDGMENT
This research was supported by Mitsubishi Electronic Corporation. We would like to acknowledge Kohei Tanaka, Kitamura Yoshiaki, Kazuyo Yoshimura for their great effort to this research.
REFERENCES
Kirwan, B., 1994.: Guide to Practical Human Reliability Assessment, Taylor & Francis, London. Kirwan, B., 1996. The validation of three Human Reliability Quantification
techniques—THERP, HEART and JHEDI: Part 1—technique descriptions and validation issues. Appl. Ergon. 27 (6), 359–373. Kirwan, B. et al., 1997. The validation of three Human Reliability Quantification techniques—THERP, HEART and JHEDI: Part II—Results of validation exercise. Appl. Ergon. 28 (1), 17–25. Lee, S. W., Kim, A. R., Ha, J. S., & Seong, P. H. 2011, “Development of a qualitative
evaluation framework for performance shaping factors (PSFs) in advanced MCR HRA”, Annals of Nuclear Energy, vol. 38, no. 8, pp. 1751–1759. Rosson, M. B., Carroll, J. M.: Scenario-Based Design, In Jacko, J. and Sears, A. (eds.), The Human-Computer Interaction Handbook: Fundamentals, Evolving Techno- logies and Emerging Applications, Lawrence Erlbaum Associates, pp. 1032–1050, 2002. Rosson, M. B., Carroll, J. M.: Usability Engineering: Scenario-Based Develo-
pment of Human Computer Interaction, Morgan Kaufmann Publishers, NY, USA, 2001Booher, Harold, ed. (2003). Handbook of human systems integration. New Jersey: Wiley.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-71-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional