Christian Gold, Klaus Bengler (2021). Taking Over Control from Highly Automated Vehicles. In: Advances in Human Aspects of Transportation: Part II. AHFE Open Access, vol. 16. https://doi.org/10.54941/ahfe100669
Future cars will be able to execute the longitudinal and lateral control and other subtasks of driving. Automation effects, known in other domains like aviation, rail traffic or manufacturing, will emerge in road transportation with consequences hard to predict from the present point of view. This paper discusses the current state of automation research in road traffic, concerning the take-over at system limits. Measurements like the take-over time and the maximum accelerations are suggested and substantiated with data from different experiments and literature. Furthermore, the procedure of such take-over situations is defined in a generic way. Based on studies and experience, advice is given concerning methods and lessons learned in designing and conducting take-over studies in driving simulation. This includes the test and scenario design and which dependent variables to use as metrics. Detailed information is given on how to generate proper control conditions by driving manually without automation. Core themes like how to keep situation presentation constant even for manual drivers and which measures to use to compare a take-over to manual driving are addressed. Finally, a prospect is given on further needs for research and limitations of current known studies.
The transferability of results from aviation for instance is very limited, because of many factors like variety of users, different object density, task complexity, time criticality and many more. That is why basic research in road transportation has to be done, caused by the fast progress of the development of automation in this specific area.
Automation can be divided into different stages with increasing levels of support by automation. With traffic-jam- assistance we have nowadays reached the level of partially automated driving. The task of the driver is changing from driving to monitoring the system and staying prepared to intervene at any point in time. A high driver benefit could be gained with the next level, the high automation. Following the definition of the BASt, the driver doesn’t have to monitor the system any more, but has to be prepared to take over control of the vehicle within a certain time buffer (Gasser, 2012). Accordingly, the NHTSA (2013) defines the same stages of automation, using a different wording. The BASt “Partially automated driving” corresponds to NHTSA “Level 2 - Combined Function Automation” and “highly automated driving” and is also called “Level 3 - Limited Self-Driving Automation”.
As soon as the driver doesn’t have to monitor the system, he can deal with tasks other than driving and is able to completely withdraw himself from the driving task. While the automation deals with the driving within its system limits, the human and his performance is crucial in take-over situations that occur on system limits or at the end of the automation scenario. Concerning the aspects of driver vehicle interaction, ways have to be found to keep the driver as involved in the driving task as possible and to be aware of the current availability of the driver at any point in time while driving highly automated to ensure the safety of the overall system.
OVERVIEW ON CURRENT RESEARCH Based on the conclusion of Reason (1990) that the cognitive structure of humans is not well suited for supervisory control and the “ironies of automation”, introduced by Bainbridge (1983) warning that the higher the level of automation becomes the more important the role of the human within the overall system gets, the impact of automated driving has to be critically examined. As car manufacturers become more confident about the feasibility of such automated systems, this field of research grew in recent years and led to plenty of publications. Beside automation effects like overreliance, skill degradation or reduced situation awareness, the authority transition (Saffarian, Winter & Happee, 2012) is one key controllability aspect of automated driving. Based on a review of literature regarding adaptive cruise control and active lane assist systems, Stanton and Young (1998) stated that there are going to be “some problems with reclaiming control of the vehicle in failure scenarios”. Except for those failure scenarios, transitions could likewise be caused by “road blockage, severe weather conditions, sudden maneuvers by another vehicle and operator preference” (Saffarian et al., 2012), wherefore detailed studies focusing transitions in automated vehicles are urgently required.
Several studies in recent years investigated impacts of different parameters on take-over situations, almost exclusively in driving simulator experiments. Automation effects of different manifestations have been found in nearly every known study. With higher automation levels, subjects tend to get more involved in non-driving related tasks (Carsten et al., 2012). Although one would expect, that distraction tends to worsen drivers’ reaction in take- over situations, Neubauer, Matthews and Saxby (2012a) measured a shortening in brake reaction times of subjects making a phone call while driving highly automated compared to subjects driving automated without any non- driving related task. The author assumed a higher activation level to be the reason for this finding. The direction of the influence of non-driving related tasks seems to depend on the modality and/or type of the task. For example, Merat et al. (2012) measured an influence of the involvement in a questionnaire on reaction times in critical take- over scenarios. Also Radlmayr (2013) found differences in take-over quality between a visual and a cognitive task. Beller, Heesen and Vollrath (2013) figured out that the communication of systems uncertainty leads to greater time to collisions (TTC) at system limits. Even if the subjects themselves chose whether to activate the automation or not, they show increased reaction times compared to manual driving subjects (Neubauer et al., 2012b). Concerning the minimum time budget a driver needs to take over control, Damböck et al. (2012) and Gold et al. (2013a) suggested budgets greater than 7 seconds for visually distracted drivers in different take-over situations. Significant automation effects have been observed with 4, 5 and 6 seconds time budget. Petermann-Stock et al. (2013) measured take-over times up to 8.8 seconds independently of the age group of the subjects. Also other transitions than from highly automated to manual driving were addressed in a study by Petermann and Schlag (2010) in a wizard-of-oz vehicle. Similar to findings in Gold et al. (2013b) they found increased problems through mode confusion compared to transitions between highly automated and manual driving.
TAKING OVER VEHICLE CONTROL As soon as the driver is out of the loop, he is likely to lose situation awareness by a shift of tasks, inattention, daydreaming, fatigue et cetera. This influences the shift back to the driving task, which is demanded within a limited time budget at system limits. A state where the automation and the driver both reach their limits has to be avoided as far as possible and therefore, as highly automated cars have system limits by definition, the take-over procedure has to be studied and the driver has to be granted as much time and support as possible. Taking over control from highly automated vehicles can be abstracted as shown in figure 1.
Figure 1. Generic take-over situation
The driver approaches a system limit in highly automated mode. With a certain time budget, the driver gets informed about approaching this system limit by a take-over request (TOR). Depending on the previous task of the driver, he has to interrupt his task and glance up to the scenery. With his eyes on road, the information perception and processing starts and a transition area is entered, where the level of automation is somewhere in between highly automated driving and manual driving. Depending on the system design, the vehicle continues the driving task until reaching the system limit, or, for example in case of a system failure, the driver has to perform the driving task immediately. In case of continued guidance by the automation, the driver has to indicate take-over readiness by inputs on a control element like the steering wheel. As soon as the driver starts his maneuver to respond to the system limit, the level of automation reaches “manual driving” and the take-over time ends.
There are more possibilities for transitions at system limits such as only delegating longitudinal or lateral guidance back to the driver. Those transitions had been considered in studies by Petermann and Schlag (2010) and Gold et al. (2013b). Whether those transitions help to improve overall performance of the human machine system could not be conclusively clarified yet, but problems with mode awareness seem to be more likely to occur.
HUMAN PERFORMANCE IN TAKE-OVER SITUATIONS The measurements of drivers’ performance in take-over situations can be divided into two aspects: Take-over times and take-over quality. Although the driver might steer, brake or accelerate, it is not ensured that he assessed the situation correctly and came to an appropriate decision. It is therefore not only very important to provide a sufficient time budget and to not force the driver into premature decisions but also to assess drivers’ inputs and measure the take-over quality. With shorter time budgets, the take-over times get shorter but the take-over quality worsens (cf. Gold et al. 2013a). Both aspects therefore always have to be considered when assessing humans’ performance in take-over situations.
Take-over time and reaction times Considering the take-over procedure, different time metrics can be measured. First of all, the take-over time - when the manual driving starts - is important. It includes the movement time to the driving position, situation perception and assessment and the decision process. Besides, a variety of times can be measured by methods like Eye-Tracking or based on drivers input on control elements. Depending on the situation and the required driver input, the end of the take-over time is not easy to define. Gold et al (2013a) suggests for a lane change a steering wheel angle of more than 2 degrees and a braking pedal position of more than 10 percent. Below those boundary values any driver input is supposed to be not related to a conscious maneuver but for the reason of vehicle stabilization which is not a reaction to the system limit. If the vehicle stabilization itself is the reaction to the system limit, the definition of the take-over time would have to be defined differently correspondingly.
Figure 2: Reaction times in Gold et al. 2013a
Figure 2 shows results regarding different reaction times in two take-over scenarios with 5 and 7 seconds time budget. In this experiment, next to the take-over time (intervention) the gaze reaction (the time until the first reaction after the TOR is measured), the road fixation (time until first gaze on scenery), the hands on time (time when hands touched the steering wheel), the first gaze in the side mirror and the activation of the turn signal (indicators) were recorded. Depending on the situation and experimental question, the measurement of several different times like brake-reaction time can be reasonable.
Take-over quality Measuring the drivers’ performance during and after taking over control is essential to assess the overall system composed of driver, vehicle and automation. There are several ways to assess drivers’ performance, depending on situation and necessary maneuver. If a stabilization of the vehicle is sufficient, common measures like standard deviation of lateral position or standard deviation of velocity can be measured. If the required maneuver is more complex, the measuring of take-over becomes elaborate. For measuring the quality of a lane change for example, lateral accelerations or the minimal observed time to collision can be recorded. Furthermore, the securing of the lane change by gazes in the mirrors or shoulder check can be examined.
Finally, it can be assessed whether the subjects chose the right maneuver as a response to the system limit. For instance, inappropriate braking or choosing the wrong lane can indicate insufficient take-over quality. As higher accelerations or smaller TTC clearly indicate a worse take-over quality, the determination as to whether a maneuver is appropriate is more complex as for example ethical aspects can influence this decision.
DESIGNING TAKE-OVER EXPERIMENTS The following suggestions are based on the experience of the author in several take-over experiments. As in other fields, take-over experiments examine the influence of an independent variable on a dependent variable. An imaginable scenario is that in a within subjects design the situation is kept constant and with repeated measures the independent variable (e.g. different non-driving related tasks) varies. For this purpose every subject has to experience the same situation at least two times. Results show, that even with different visual representation of one situation, take-over times and quality improve on the second situation. Figure 3 shows additional data of the experiment of Gold et al. (2013b). Situation “Road Construction” and “Situation Person” were presented twice to each subject with a slightly different appearance. In both situations, the subjects had to react to an object which was entering the current lane from the right. The influence of the learning effect is obvious. Several dependent variables are showing significance tested with a t-test and a significance level of 0.05.
Figure 3: Example for Learning Effects
Those learning effects complicate conclusions regarding the influence of the independent variable. A between subjects design, where subjects experience only one take-over, would avoid influences of learning effects, unfavorable on costs of bigger samples and therefore a higher experimental effort. Nevertheless, when planning the experimental design, the question has to be raised as to whether repeated measures are usable or if big samples with only one take-over situation have to be preferred.
To judge not only whether certain aspects have an influence on the take-over time and quality but also whether a certain take-over is sufficient from a controllability point of view, the take-over performance has to be compared to manual driving. It is, and has to be further discussed, not only how much the performance of the overall system has to improve, but also if such take-overs have to be compared to manual driving without any assistance, or to the state of the art with driver support by advanced driver assistance systems. Furthermore, if a baseline consisting of manual driving subjects shall be recorded, it has to be ensured that the representation of the situation matches the automated condition. Here, the most critical value is the point in time when the system limit becomes visible. As automated driving subjects only have a certain time budget to react on the system limit, this limit should become visible to the manual driving subjects at the exact same point in time. This is possible by hiding the system limit behind any object, like a leading vehicle. The authors suggest to preferably letting the limit emerge suddenly in a simulator by displaying the limit in the very moment of the tor. Accepting small losses of sense of reality of subjects detecting this sudden appearance, automated and manual driving subjects have the same preconditions for solving the situation and artefacts are prevented as much as possible.
CONCLUSION AND OUTLOOK The paper summarized questions and studies regarding automated driving and the take-over from automated vehicles. Directions were given and dependent measures suggested for designing such studies. By performing the experiments in real vehicles, future studies have to examine whether the transferability of driving simulator studies in this special research field is permissible. The impact of many kinds of influences on take-over time and quality also have to be further investigated and additional studies are needed regarding all kinds of automation effects probably emerging in automated vehicles, such as behavioral adaptation or skill degradation.
REFERENCES
Bainbridge, L. (1983). Ironies of Automation. Automatica, 19(6), 775–779. Beller, J., Heesen, M., & Vollrath, M. (2013). Improving the Driver-Automation Interaction: An Approach Using Automation Uncertainty. Human Factors: The Journal of the Human Factors and Ergonomics Society, 55(6), 1130–1141. doi:10.1177/0018720813482327 Carsten, O., Lai, F. C. H., Barnard, Y., Jamson, A. H., & Merat, N. (2012). Control Task Substitution in Semiautomated Driving:
Does It Matter What Aspects Are Automated? Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(5), 747–761. doi:10.1177/0018720812460246 Damböck, D., Farid, M., Tönert, L., & Bengler, K. (2012). Übernahmezeiten beim hochautomatisierten Fahren. 5. Tagung Fahrerassistenz, 5. Gasser. (2012). Rechtsfolgen zunehmender Fahrzeugautomatisierung: Gemeinsamer Schlussbericht der Projektgruppe.
Bremerhaven. Gold, C., Dambock, D., Lorenz, L., & Bengler, K. (2013a). "Take over!" How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 1938–1942. doi:10.1177/1541931213571433 Gold, C., Lorenz, L., Damböck, D., & Bengler, K. (2013b). Partially Automated Driving as a Fallback Level of High
Automation. 6. Tagung Fahrerassistenzsysteme, 28.-29. November, München, 6. Merat, N., Jamson, A. H., Lai, F. C. H., & Carsten, O. (2012). Highly Automated Driving, Secondary Task Performance, and Driver State. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(5), 762–771. doi:10.1177/0018720812442087 Neubauer, C., Matthews, G., & Saxby, D. (2012a). The Effects of Cell Phone Use and Automation on Driver Performance and
Subjective State in Simulated Driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), 1987–1991. doi:10.1177/1071181312561415 Neubauer, C., Matthews, G., Langheim, L., & Saxby, D. (2012b). Fatigue and Voluntary Utilization of Automation in Simulated Driving. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(5), 734–746. doi:10.1177/0018720811423261 NHTSA. (2013). Preliminary Statement of Policy Concerning Automated Vehicles. Retrieved from www.nhtsa.gov/ staticfiles/
rulemaking/ pdf/ Automated_Vehicles_Policy.pdf Petermann, I., & Schlag, B. (2010). Auswirkungen der Synthese von Assistenz und Automation auf das Fahrer-Fahrzeug System.
AAET 2010 – Automatisierungssysteme, Assistenzsysteme und eingebettete Systeme für Transportmittel, 383–403. Petermann-Stock, I., Hackenberg, L., Muhr, T., & Mergl, C. Wie lange braucht der Fahrer?: Eine Analyse zu Übernahmezeiten aus verschiedenen Nebentätigkeiten während einer hochautomatisierten Staufahrt. In 6. Tagung Fahrerassistenzsysteme, München, 2013 . Radlmayr, J. (2013). How varying traffic situations and secondary tasks affect the quality of the take-over process during highly
automated driving. Technische Universität München, München. Reason, J. T. (1990). Human error. Cambridge: Cambridge University Press. Saffarian, M., Winter, J. C. F. de, & Happee, R. (2012). Automated Driving: Human-Factors Issues and Design Solutions.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), 2296–2300. doi:10.1177/1071181312561483 Stanton, N. A., & Young, M. (1998). Vehicle automation and driving performance. Ergonomics, 41(7), 1014–1028.
doi:10.1080/001401398186568
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Aspects of Transportation: Part II
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2098-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional