Paul Weiser, Michael Preutenborbeck, Marcel Usai, Frank Flemisch (2025). From Human Computer Interaction to Human Systems Migration: Internal Interaction in Automated Road Vehicles. In: Intelligent Human Systems Integration (IHSI 2025): Integrating People and Intelligent Systems. AHFE Open Access, vol. 160, pp. 520–527. https://doi.org/10.54941/ahfe1005857
Vehicle automation has evolved from early systems, like cruise control, to more advanced technologies requiring deeper exploration and understanding of human-automation interaction and Human Systems Integration. Initially, research focused on human-computer interaction, but it later shifted towards a dynamic cooperation between humans and machines. As vehicle automation levels will vary in the future from partially to highly and fully automated systems, new safety concerns arise. These are particularly relevant for transitions of systems between automation levels and migrations of humans and technology between different configurations of the socio-technical systems. This paper describes a work-in-progress in the German DFG-project MiRoVA (Migration of Road Vehicle Automation), especially subproject 4, which focuses on internal interaction in the vehicles, e.g. between the automation and the driver. In this subproject we aim to address gaps in understanding the impact of automation migration on human-machine interaction. The focus is to explore how changes in automation levels affect human-machine cooperation and HMI design. This paper presents the fundamental aspect of human systems migration of vehicle automation, followed by resulting goals and the research concept created to investigate the impact of automation migration on human-machine interaction in human-in-the-loop simulations of traffic systems.
The history of vehicle automation is nearly as old as modern vehicles themselves, with the first forms of cruise control in cars, e.g., by Wilson- Pilcher, in the early 1900s, and the first airplane autopilot system, e.g. by Sperry (1915). These first systems were still primitive in design and the possible interactions. With more complex vehicle systems, and especially the rise of computer technology, automation also became more sophisticated and broader in scope. With this came the need to investigate the interaction between human and automation. This was initially done from a more
From Human Computer Interaction to Human Systems Migration 521
technology-oriented perspective, which was then opposed with a human- centered perspective (Billings, 1991), and combined with a balanced perspective (Flemisch, 2003). The main paradigm of the time was that of (human-computer) interaction, where the machine did nothing more than realize the orders given by the human over a relatively static human-machine interface.
Inspired by automation in aviation and automotive, Hoc (2000), based on the ideas of Rasmussen (1983), proposed a shift from this paradigm of interaction to a paradigm of cooperation to allow systems to be designed with dynamic function allocation and transitions of control between human and machine in mind.
The concept of cooperation within human-machine systems (HMS), specifically in the context of cooperative vehicle guidance (Flemisch et al., 2016), which draws inspiration from the H(orse)-Metaphor (Flemisch et al., 2003), has been a focus of research and development in the automotive domain for many years. While our work will focus on the local HMS between driver and vehicle, it is important to note that this cooperation also takes place between the local HMS and other traffic actors. A more holistic model of how such a system of systems (SoS) works together is shown in Figure 1.
Figure 1: Holistic system model of human-vehicle cooperation, with the human- machine system (HMS) (center), within a micro traffic system, which itself is embedded in a macro traffic system (Flemisch et al., 2024, based on Flemisch et al., 2023).
LEVELS OF AUTOMATION AND HUMAN SYSTEMS MIGRATION At the current time, no road vehicle is capable of fully automated driving and not all vehicles offer the same automation features. Multiple classification systems were developed that culminated in the formulation of an international standard for Driving Automation Systems for On-Road
Motor Vehicles (SAE, 2021). A basic overview of these levels is shown in Figure 2.
Furthermore, the automation that is offered by the vehicle may not be active at all times. It may also only be partially active, which means that there can be multiple levels of automation in the vehicle at the same time. Therefore, it is necessary to define transitions for the automation to take over or give back control of the vehicle to the user. A typical example where such a transition is safety critical, would be a situation for which the automation is not designed. In that case, the automation has to give back control to the driver. This is often abbreviated as the takeover request (TOR), and subject of ongoing research (e.g. Melcher et al., 2015; Wan et al., 2018).
Figure 2: Linear presentation of the migration of automation levels, e.g. VDA, BASt, SAE (Flemisch et al., 2024).
Flemisch et al. (2011) transfer the idea of human-technology migration to the domain of road traffic, expand it to a socio-technological perspective, and apply it to a migration and evolution capable human machine interaction and automation. This human systems migration will also lead to increasing automation of road vehicle systems. As a result, people will also become accustomed to newer, more automated support systems.
In a perfect world, this migration of automation levels would happen simultaneously with all road users, and all manufacturers would implement e.g. automation state transitions the same way. However, this is not reasonable to assume. In the future, as it is currently the case, there will be a mix of different automation levels present. This means that in the future, multiple migration paths have to be considered. The most straightforward migration is from one automation level to the next (upwards migration). Drivers may also change vehicles without changing the level of automation (sideways migration). This is still a relevant process, however, as the same automation features may be realized differently in the HMI. Lastly, there may be cases where drivers will have to use a vehicle with a lower level of automation (downwards migration). An example for a possible migration path is shown in Figure 3.
From Human Computer Interaction to Human Systems Migration 523
Figure 3: Example of a sideways and upward migration with focus on HMI. Notice especially, how the two human-machine-interfaces on the left are structured with a different transition scheme in mind. Based on Eom & Lee (2022) and ideas of the Vorreiter project (Flemisch et al., 2020), (Flemisch et al., 2024).
SAFETY CONCERNS IN REGARD TO HUMAN SYSTEMS MIGRATION There are multiple safety risks associated with human systems migration. In this work, we will focus on the human-machine system between the driver and vehicle, though a broadening to a micro- and macro-traffic system (see Figure 1) will most likely reveal even more risks.
For upwards migration, a significant factor is an over-reliance on and over- confidence in the automation. This was described by Flemisch et al. (2017) as the uncanny and unsafe valley of automation. The most critical region was identified to lie between SAE level two and level three/four systems, where a less automated system could be assumed to be more capable than it actually is. In the past, there were already multiple accidents with the involvement of automated vehicles, where an over-confidence of the human actors in the automation was identified as one of the main causes (National Transportation Safety Board NTSB).
In case of sideways migration, safety concerns can arise when an interaction of drivers and automation creates confusion or conflict. For example, imagine a vehicle transitioning from a radar-based adaptive cruise control system to a vision-based system for maintaining speed and distance. In both cases, the vehicle operates at SAE 2 (partial automation), but the new system relies on cameras to detect road conditions and other vehicles. If the system fails to detect an obstacle due to poor lighting or weather conditions
(e.g., heavy rain), the driver may not be prepared to take control immediately because they were used to the radar system, which performed better in those conditions.
In regard to downward migration, an over-familiarization with higher automation levels can lead to safety risks. This is especially risky, if the driver assumes an automation, e.g., a braking assistant, to be there, but it isn’t. In this case, they may not react fast enough in case of an emergency situation, because they assume that the vehicle would handle the event.
Considering these accidents and potential risks, we believe that changes in critical aspects of the human-machine system, the micro or macro traffic system are crucial. If the driver fails to account for such changes, it could lead to disadvantages, incidents, or even accidents. These changes could include shifts in environmental conditions, the behavior of automation levels, or transitions between different automation levels. Managing these changes relies heavily on the HMI, though this issue extends beyond just the HMI itself. However, apart from Morris (2020), who addresses migration problems in automation system functions related to risks and liability, little research has been conducted on migration as a change management process in road traffic.
OUR GOALS A review of the research history reveals two contrasting trends. On one hand, there has been limited exploration and insufficient attention paid to the migration process and its relationship with human-machine interaction, cooperation, and HMI. On the other hand, there are valuable models and insights that detail how humans, machines – such as vehicle automation – and the environment engage through HMI, influencing factors like mental models, situation awareness, workload, and trust. Building on the progress made in projects like DFG-CoInCar (e.g. Flemisch et al., 2024; Stoll et al., 2019), we are motivated to expand this existing knowledge to address the emerging challenges of human systems migration, especially in the context of human-machine interaction and cooperation, with a focus on the crucial role of the HMI. Understanding the various potential future states and the transitions between automation levels is essential to guide the future direction of research, development, and policy concerning traffic systems and automated vehicles.
Our core objectives are 1) to deepen scientific understanding of both the positive and negative impacts of migration on human-machine interaction, cooperation with automation, and related HMI, and 2) to present this knowledge to the research and development community in a way that facilitates its practical application.
In the context of our second goal, we aim to integrate our findings into a broader traffic system simulation. While our focus will stay on the HMI between driver and automated vehicle, we recognise that it is crucial to also consider how limitations in human interaction resources can impact interactions not only within the vehicle but also with external systems. Specifically, we will cooperate with other researchers to examine interactions
From Human Computer Interaction to Human Systems Migration 525
with other road vehicles – automated and non-automated – and vulnerable road users, such as pedestrians. These external interactions are critical to understanding the broader context of driver decision-making and the safe integration of automated systems into complex road traffic scenarios.
RESEARCH CONCEPT The research structure will follow the turbine model for human systems exploration (HSE) (Flemisch et al., 2022). The challenge is to condense the many interaction, cooperation, and HMI options into a format that can be efficiently used in design explorations. Those will then be implemented inside a static driving simulator, where the design space is explored iteratively with representative drivers and other stakeholders. To increase agility, aspects of automation or the interface are initially emulated using the ‘Theatre Method’, where a design team member acts out the role of the system, simulating interactions with the driver. The identified qualitative interaction patterns will then be formalized inside state machines, which are further developed in the direction of probabilistic networks.
Based on this, the patterns will be properly integrated into the driving simulator, so we can later validate them experimentally. The simulator setup already includes a simulation environment to create use cases, a driving automation system (Altendorf, 2018) that performs basic manoeuvres such as lane keeping, following, lane changing, and emergency braking, an interaction mediator (Baltzer et al., 2014; Baltzer, 2021) for HMI control and structured conflict resolution between humans and automation, and a pattern handler (Usai, 2024) to manage the interaction patterns developed.
This process will be done over multiple iterations. In each iteration, the qualitative system model will be sharpened and more interaction patterns will be identified, formalized, and validated.
CONCLUSION AND OUTLOOK In this work, we have presented the challenges of human systems migration with the example of vehicle automation, and the shortcomings of current research into this topic. We plan to address this research gap to identify positive and negative impacts of the migration between automation levels, specifically in the domain of road traffic. While the research in subproject 4 of MiRoVA is focused on the local human-machine systems, the overall goal is to integrate our findings of the subproject into the traffic system simulation of the MiRoVA research group, which also includes the micro- and macro traffic system.
ACKNOWLEDGMENT
The authors would like to thank the partners of MiRoVA for their invaluable support, especially IAD with Dr.-Ing. Bettina Abendroth, the FZD with Prof. Dr.-Ing. Stephen Peters, LfE with Prof. Dr. Klaus Bengler, and the IfV with Prof. Dr.-Ing. Peter Vortisch, as well as Marvin Baumann, Felix Friedrich, Kai Storms, Niklas Grabbe, Jonas Schulte and Tianyu Tang, in this challenging
endeavour to analyse the effects of automation migration in traffic systems. The research group on Migration of Road Vehicle Automation (MiRoVA) is funded by the German Research Foundation (DFG) under grant no. 523663459.
REFERENCES
Altendorf, E. (2018). Kooperative Fahrer-Fahrzeug-Systeme in der automatisierten Fahrzeugführung unter Verwendung von Spieltheorie und Deep Learning. (Dissertation No. RWTH-2019-08193). Lehr- und Forschungsgebiet Systemergonomie, RWTH Aachen University. Baltzer, M. C. A. (2021). Interaktionsmuster der kooperativen Bewegungsführung
von Fahrzeugen (Dissertation No. RWTH-2021-04755). Lehr-und Forschungsgebiet Systemergonomie, RWTH Aachen University. Baltzer, M., Altendorf, E., Meier, S., Flemisch, F., Stanton, N., Landry, S.,... & Vallicelli, A. (2014). Mediating the interaction between human and automation during the arbitration processes in cooperative guidance and control of highly automated vehicles: Basic concept and first study. Advances in Human Aspects of Transportation Part I, 439–450. Billings, C. E. (1991). Human-centered aircraft automation: A concept and guidelines
(Vol. 103885). National Aeronautics and Space Administration, Ames Research Center. Eom, H., & Lee, S. H. (2022). Mode confusion of human⣓machine interfaces for automated vehicles. Journal of Computational Design and Engineering, 9(5), 1995–2009. Flemisch, F. O., Adams, C. A., Conway, S. R., Goodrich, K. H., Palmer, M. T., &
Schutte, P. C. (2003). The H-Metaphor as a guideline for vehicle automation and interaction (No. L-18448). Flemisch, F. O., Preutenborbeck, M., Baltzer, M., Wasser, J., Kehl, C., Grünwald, R.,...
& Dahlmann, A. (2022). Human systems exploration for ideation and innovation in potentially disruptive defense and security systems. In Disruption, Ideation and Innovation for Defence and Security (pp. 79–117). Cham: Springer International Publishing. Flemisch, F., Abendroth, B., Peters, S., Bengler, K., Vortisch, P. (2024). MiRoVA Migration of Road Vehicle Automation, proposal to the DFG under grant no. 523663459. Flemisch, F., Altendorf, E., Canpolat, Y., Weßel, G., Baltzer, M., Lopez, D.,... &
Schutte, P. (2017). Uncanny and unsafe valley of assistance and automation: First sketch and application to vehicle automation. In Advances in Ergonomic Design of Systems, Products and Processes: Proceedings of the Annual Meeting of GfA 2016 (pp. 319–334). Springer Berlin Heidelberg. Flemisch, F., Herzberger, N., Usai, M., Baltzer, M., Schwalm, M., Voß, G.,... & Stiller, C. (2024). Cooperative Hub for Cooperative Research on Cooperatively Interacting Vehicles: Use Cases, Design and Interaction Patterns. In Cooperatively Interacting Vehicles: Methods and Effects of Automated Cooperation in Traffic (pp. 495–508). Cham: Springer International Publishing. Flemisch, F., Preutenborbeck, M.; Wasser, J.; Herzberger, N.: Vom Teufelsquadrat
zum nachhaltigen Vieleck und holistischen Bow-Tie Diagramm; Jahrestagung der Gesellschaft für Arbeitswissenschaft GfA “Nachhaltiges Lernen und Arbeiten”; Hannover 2023 (Best of Session).
From Human Computer Interaction to Human Systems Migration 527
Flemisch, F., Schieben, A., Heesen, M., Hesse, T., Beller, J. (2011). Aus der Vergangenheit (und Natur) lernen, Zukunft gestalten: Migrations-und evolutionsfähige Gestaltung der Mensch-Maschine Interaktion. In: 9. Berliner Werkstatt Mensch-Maschine: Aus der Vergangenheit lernen, Zukunft gestalten. Berlin. Flemisch, F., Winner, H., Bruder, R., & Bengler, K. (2016). Cooperative guidance, control and automation. Handbook of driver assistance systems, 1471–1481. Hoc, J. M. (2000). From human–machine interaction to human–machine cooperation. Ergonomics, 43(7), 833–843. Melcher, V., Rauh, S., Diederichs, F., Widlroither, H., Bauer, W. (2015), Take-
Over Requests for Automated Driving, Procedia Manufacturing, Volume 3, Pages 2867–2873, ISSN 2351–9789. https://doi.org/10.1016/j.promfg.2015.07.788 Morris, T. (2020). On the Migration of Risks and Liabilities for Increased Automation. In AIAA SciTech 2020 Forum (p. 0454). Rasmussen, J. (1983). Skills, rules, and knowledge; signals, signs, and symbols, and
other distinctions in human performance models. IEEE transactions on systems, man, and cybernetics, (3), 257–266. SAE International Recommended Practice, Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, SAE Standard J3016_202104, Revised April 2021, Issued January 2014. https://doi. org/10.4271/J3016_202104 Sperry, Lawrence B. (1915). The Sperry automatic stabiliser. Flight, p. 74, 29th
January 1915. Stoll, T., Imbsweiler, J., Deml, B., Baumann, M. (2019), Three Years CoInCar: What Cooperatively Interacting Cars Might Learn from Human Drivers, IFAC-PapersOnLine, Volume 52, Issue 8, Pages 105–110, ISSN 2405-8963. https://doi.org/10.1016/j.ifacol.2019.08.056 Usai, Marcel & Mandischer, Nils & Flemisch, Frank. (2024). Pattern Handler: Integrating Real World and Virtual Models of Human Systems Patterns to Regulate the Control Distribution in a Cooperative Automated Driving Task. 10.13140/RG.2.2.24723.69922. Wan, J. and Wu, C. (Dec 2018), The Effects of Lead Time of Take-Over Request
and Nondriving Tasks on Taking-Over Control of Automated Vehicles, in IEEE Transactions on Human-Machine Systems, vol. 48, no. 6, pp. 582–591, doi: 10.1109/THMS.2018.2844251.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2025): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-36-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional