Frederik Naujoks, Christoph Mai, Alexandra Neukum (2021). The Effect of Urgency of Take-Over Requests During Highly Automated Driving Under Distraction Conditions. In: Advances in Human Aspects of Transportation: Part I. AHFE Open Access, vol. 15. https://doi.org/10.54941/ahfe100646
Highly automated driving may improve driving comfort and safety in the near future. Due to possible system limits of highly automated driver support, the driver is expected to take over the vehicle control if a so-called take-over request is issued. One example of these system limits are missing or ending lines on motorways. This study focuses on the design of take-over requests in such situations. Using a motion-based driving simulator, N = 16 participants encountered different take-over situations in congested traffic that varied in their difficulty: ending lines on straight road (easy), temporary lines due to a work zone (moderate) and loss of lines in a situation with high curvature (difficult). The driver support consisted of a hands-off system that was taking over longitudinal and lateral control. Participants were asked to perform a secondary task while driving. Take-over requests were presented either visually or visual-auditory. Drivers’ hands-on times (i.e., time until driver puts hands back on the steering wheel) are lower if visual-auditory take-over requests are used in comparison to purely visual ones. Measures of lateral vehicle control also show an advantage of visual-auditory take-over requests. Differences between the take-over concepts are especially pronounced in difficult take-over situations.
Vehicle automation is thought to improve driving safety, comfort and traffic efficiency in the near future1. The introduction of such new vehicle applications on motorways is likely to cause a fundamental change to the nature of the driving task (Flemisch et al., 2012). Today’s assistive technologies aim at supporting the driver at different levels of the driving task (e.g. route planning and navigation, collision avoidance or lane keeping). Examples of market-ready driver support systems are adaptive cruise control (ACC) or lane departure assistance that assist the driver at keeping either longitudinal or lateral control (‘function specific assistance’, NHTSA, 2013). By now, assisting the driver at both longitudinal and lateral control is feasible (‘combined function’, NHTSA, 2013, e.g.,
Schaller, Schielen & Gradenegger, 2008). Such driver assistance systems are called semi-autonomous, due to the necessity to monitor the system’s activities permanently in order to control the vehicle at all times (). Recently, even higher levels of automation have been demonstrated in test vehicles and in various research projects (e.g. HAVEit, SARTRE, CityMobil). The aim of further development is to provide the possibility of driving in a highly automated mode without the need to monitor the system permanently. In this way the driver can benefit from an increase in driving comfort as one advantage of highly automated driving. However, there will still be situations that cannot be safely managed by the automation alone. Thus, the driver will be expected to take back the vehicle control within a 1 E.g., Spalder and Abel (2013) expect highly automated driving to be marked-ready by 2020.
Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8 suitable preparatory time frame (e.g., ).
Research question Automation of driving tasks can have an influence on driver performance and driver state as the driver’s role is shifted from actively controlling the vehicle to monitoring the system behaviour, as has been shown in the research literature regarding automation and human performance (e.g., ). Numerous studies have demonstrated effects of automation such as loss of skill (Stanton & Marsden, 1996), loss of situation awareness (Endsley, 1999) or
overreliance on the automation (Parasuraman & Riley, 1997). Previous studies have also shown that drivers tend to increase their engagement in secondary activities while driving with assistance systems (). Similar automation effects on driving behaviour are also expected during highly automated driving (Saffarian, De Winter & Happee, 2012). From a human factors perspective, a safe and efficient transition (so-called ‘take-over’) from automated back to manual driving (if system limits are reached) is therefore of high importance. One example of system limits during highly automated driving are missing or ending lines on motorways, which are normally needed for estimating the vehicles future path reliably (see for examples of such system limits in empirical studies). In this case, a safe transition from highly automated to manual driving has to be ensured. Specifically, suitable ways to prepare the driver to take back the vehicle control, for example by displaying that he/she needs to take back his/her hands on the steering wheel, have to be identified. One goal in the design of such take-over requests is consequently to alert the driver urgently in order to provoke an immediate reaction if a transition to manual driving is imminent. On the other hand, false alarms (i.e., take-over requests although the situation is safely managed by the automation) are to be expected, which in turn could lower the effectiveness of take-over requests (see Bliss & Acton, 2003; Cotté, Meyer & Coughlin, 2001; Lees & Lee, 2007; Yamada & Kuchar, 2006 for examples regarding false-alarm effects of driver warnings) and the willingness to use highly automated driving functions (see ). Especially, frequent and annoying false alarms may result in non-acceptance of highly automated vehicle applications (Dingus, Jahns,
Horowitz & Knipling, 1998; Bliss & Acton, 2003).
With this in mind, one important research question is the modality of effective take-over requests. However, there is little research on this topic to date. On the basis of research pertaining to driver warnings during manual driving, it can be expected that visual-auditory take-over requests may be suitable in case of imminent transitions to manual driving (). However, these urgent warning signals may be perceived as annoying in case of false alarm. In this paper, the behavioural effects of visual vs. visual-auditory take-over requests will therefore be examined in different take- over scenarios. The aim of the study is to determine, if there is a benefit (e.g., shorter reaction times) from take-over requests that are presented in an urgent manner (i.e., visual-auditory), or, if a visual notification may be sufficient. It may additionally be expected that drivers’ reactions to take-over requests will depend on the specific scenario (). Consequently, different take-over scenarios that varied in their difficulty to take back the vehicle control (e.g., straight vs. curved road geometry) were used.
METHOD
Experimental design The sample consisted of N = 16 participants. All participants were selected from an existing WIVW (Wuerzburg Institute for Traffic Sciences) test driver panel and had previously participated in extensive simulator training (see Hoffmann & Buld, 2006). During the simulator ride, the participants were assisted by a highly automated driver assistance system taking over both longitudinal and lateral control permanently when activated through pressing an activation button. The participants drove through a simulator course that included different situations in which control over the highly automated vehicle was given back to the driver. The study was conducted in the WIVW motion-based driving simulator using a mixed between-within subject design, with the between-subjects factor ‘modality of take-over requests’ (visual vs. visual auditory) and ‘driving scenario’ as a within-subject factor. The participants were randomly assigned to the between-subjects factor. The test scenarios consisted of different possible system limits regarding the determination of the safe vehicle path (e.g., missing lane markings). These situations varied with respect to their difficulty, that is, to what degree the participants had to take corrective actions in order to keep the vehicle within the lane (see Figure 1).
Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8
In an easy scenario, the control was given back to the driver because of missing lane markings on a straight road segment (missing lines). Thus, there was no need for the drivers to take immediate corrective actions.
In another scenario, the take-over request was issued at the beginning of a work zone where the drivers had to perform a lane change in order to follow temporary lines. This scenario was considered to be of moderate difficulty, as drivers had to react to the upcoming lane change.
Finally, the take-over request was issued during a curvy road segment without an obvious reason, which was considered a difficult take-over situation.
Figure 1. Scenarios used in the simulator run
Take-over requests were presented at the moment when the respective system limit was reached (e.g., beginning of missing or temporary lines). A three-lane motorway with a lane width of 3.75m was used. The whole drive was completed in dense traffic conditions with a speed of travel of 50kph. The participants were asked to perform a secondary task while driving in highly automated mode. The chosen task was designed to reflect distraction that is likely to occur during highly automated driving and consisted of reading various text passages taken from a weekly news magazine. During the simulator ride, the participants were free to choose between the different articles that had been preselected.
Procedure After a brief verbal description of the assistant system (activation and deactivation of the highly automated driving mode), the participants completed a short drive in order to become familiar with the assistance system. The scenarios were then completed in randomized order to control sequence effects. Furthermore, the simulator course included several non-critical scenarios in which no driver intervention was needed. The participants were asked to drive on the right lane in highly automated driving mode and to engage in the secondary task during driving. The whole simulator run took about 45 minutes.
Human-machine-interface The human-machine-interface was adopted from published research on highly automated driving. Specifically, the design of the take-over requests was concordant with the one used in the HAVEit project (Flemisch, Kaussner, Petermann, Schieben & Schömig, 2010). The display was positioned on a screen in the centre console where the menu system for the operation of sound, radio, etc. is normally positioned. Participants could switch between different driving modes (manual and highly automated) by pushing a button on the steering wheel and the driving mode was visually indicated on the screen (the visual indication of the driving mode was also adopted from the HAVEit-Project, see Flemisch et al., 2010, for a detailed description). The visual component of the take-over request consisted of a symbol indicating that the driver should take his hands back on the steering wheel (see Figure 2). The visual indication was deactivated by the driver in putting his/her hands on the steering wheel. In order to increase the salience of the visual indication, the picture was flashing with a rate of 5 Hz. In case of visual-auditory take-over
Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8 requests, a warning tone (1000 Hz, sinus) was additionally presented for one second.
Figure 2. Take-over request
Dependent variables Relevant measures regarding the drivers’ responses to the take-over requests, as well as the quality of lane keeping were recorded and analysed. Of particular interest were (1) the elapsed time until the drivers take their hands back on the steering wheel and (2) maximum and standard deviation of the lateral position (i.e., deviation from the centre of the lane) within a time interval of 30s after the take-over requests were issued.
Table 1: Dependent variables
Statistical procedures In a first step, detailed descriptive statistics of the above-mentioned parameters are provided, followed by a hierarchical test procedure: Regarding the impact of the independent variables (modality of take-over request and driving scenario), the above mentioned parameters are analysed using a linear-mixed model (LLM) with ‘modality’ and ‘scenario’ and the interaction between the two as fixed effects and the participants as random effects. Subsequently, differences between the modality conditions within the driving scenarios are analysed using t-tests for paired samples. A significance level of 5% is applied to all inferential statistical tests.
RESULTS
The drivers continued performing the secondary tasks (i.e., reading) until take-over requests were issued in most cases. However, in five cases (visual: 3 cases, visual-auditory: 2 cases), drivers paused performing the secondary task and put their hands on the steering wheel before a take-over request was issued. These cases were excluded from the following analysis.
Table 2 shows the descriptive statistics of the dependent measures for the valid cases. As is evident in Table 2, drivers generally benefit from visual-auditory compared with purely visual take-over requests: Longer mean reaction times, as well as poorer lane keeping performances (i.e., higher maximum lateral positions and higher SDLP-values) are found after the presentation of visual compared to visual-auditory take-over requests. This effect is also Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8 supported by the inferential statistics (significant main effect ‘modality’, see Table 3).
Table 2: Descriptive statistics (all situations) Percentile Parameter Condition N Mea nTable 3: Inferential statistics (LMM)
This positive effect (i.e., shorter reaction times and better lane keeping) of the visual-auditory presentation of the take-over requests is independent from the investigated driving scenario (non-significant interactions between modality and driving scenario in all investigated parameters, see Table 3). However, a scenario specific analysis of the dependent measures is depicted in Figure 3. As is evident in Figure 3, the differences between visual and visual- auditory take-over requests are mainly due to significant or marginally significant differences in the scenarios with high curvature and with temporary lines (work zone)2. Two drivers actually changed to the neighbouring lane in the high curvature scenario because they failed to take over the vehicle control timely enough. In contrast, no significant differences are found between the take-over requests in the easy driving situation (‘ending lines’).
2 Follow-up t-tests were used for further testing of the significant main effect ‘modality’ within the scenarios. It must be noted that the interaction between modality and scenario (Table 3) was non-significant. However, as we specifically expected, that differences between modality conditions will be more pronounced in difficult than easy scenarios, and given that the sample used in the study was rather small, this approach was seen as suitable.
Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8
Figure 3. Reaction times (hands-on, left), maximum lateral position (middle) and standard deviation of lane position (right) after the presentation of take-over requests
Figure available in the original PDF paper.
CONCLUSIONS
The study at hand investigated the effectiveness of visual and visual-auditory take-over requests during highly automated driving under distraction conditions using a motion-based driving simulator. The results of the experiment show that drivers’ reactions depend on the modality of the take-over requests. Specifically, the drivers’ hands-on times (i.e., time until driver takes hands back on the steering wheel) are lower if visual-auditory take-over requests are used in comparison to purely visual ones. Measures of later vehicle control also show an advantage of visual-auditory take-over requests. Taken together, the experiment demonstrates that purely visual take-over requests may not be sufficient to ensure safe transitions from highly automated to manual driving in some driving situations, particularly when drivers are heavily distracted as in the present study. It has to be noted that the modality effect found in this study may well depend on the specific design of the visual take-over requests. On the basis of previous research, a flashing display in the middle console was used. The results in this study may be limited to this type of visual displays with regard to the placement and size (placement on a screen in the middle console). Other approaches, e.g. using LED-technology or Head-Up Displays (HUD) may be more effective in getting the drivers’ attention and in preparing him/her to take back the vehicle control. However, in view of possible negative effects of false alarms on effectiveness and acceptance of take-over requests, future research efforts have to take into account the interplay between take-over situations, take-over request design and the current driver state in order to prevent false alarms and to ensure acceptance of highly automated driver support.
Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8
REFERENCES
Bliss, J. P. & Acton, S. A. (2003). Alarm mistrust in automobiles: how collision alarm reliability affects driving. Applied Ergonomics, 34(6), 499-509. Campbell, J. L., Richard, C. M., Brown, J. L. & McCallum, M. (2007). Crash warning system interfaces: Human factors insights
and lessons learned (No. HS 810 697). Washington, D.C: National Highway Traffic Safety Administration. Cotté, N., Meyer, J. & Coughlin, J. F. (2001). Older and younger driver’s reliance on collision warning systems Proceedings of the 45th Annual Meeting of the Human Factor Society (pp. 277-280). Santa Monica, CA: Human Factors and Ergonomics Society. Damböck, D., Farid, M., Tönert, L. & Bengler, K. (2012). Übernahmezeiten beim hochautomatisierten Fahren. Paper presented
at the 5. Tagung Fahrerassistenz, München. Dingus, T. A., Jahns, S. K., Horowitz, A. D. & Knipling, R. (1998). Human factors design issues for crash avoidance systems. In N. V. Barfield & A. Dingus (Eds.), Human factors in intelligent transportation systems (pp. 55-94). Mahwah, NJ: Erlbaum. Endsley, M. R. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task.
Ergonomics, 42(3), 462-492. Fancher, P., Ervin, R., Sayer, J., Hagan, M., Bogard, S. & Bareket, Z. e. a. (1998). Intelligent Cruise Control Field Operational Test (final report) (No. No. DOT HS 808 849). Springfield, Virginia: National Technical Information Service. Flemisch, F., Heesen, M., Hesse, T., Kelsch, J., Schieben, A. & Beller, J. (2012). Towards a dynamic balance between humans
and automation: authority, ability, responsibility and control in shared and cooperative control situations. Cognition, Technology & Work, 14(1), 3-18. Flemisch, F., Kaussner, A., Petermann, I., Schieben, A. & Schömig, N. (2010). HAVEit Deliverable D. 33.6: Validation of optimum concept for task repartition: HAVEit consortium. Gasser, T. M., Arzt, C., Ayoubi, M., Bartels, A., Buerkle, L., Eier, J., Flemisch, F., Haecker, D., Hesse, T., Huber, W., Lotz, C.,
Maurer, M., Ruth-Schuhmahcher, S., Schwarz, J. & Vogt, W. (2012). Rechtsfolgen zunehmender Fahrzeugautomatisierung (Berichte der Bundesanstalt für Straßenwesen, Reihe F84). Bremerhaven: Wirtschaftsverlag NW. Gold, C., Damböck, D., Lorenz, L. & Bengler, K. (2013). “Take over!” How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 57, pp. 1938-1942). San Diego: SAGE Publications. Hoffmann, S. & Buld, S. (2006). Darstellung und Evaluation eines Trainings zum Fahren in der Fahrsimulation. In VDI (Ed.),
Integrierte Sicherheit und Fahrerassistenzsysteme (pp. 113-132). Düsseldorf: VDI Verlag. Kämpchen, N., Aeberhard, M., Ardelt, M. & Rauch, S. (2012). Techniken fuer das hochautomatisierte Fahren auf der Autobahn.
Automobiltechnische Zeitschrift, 114(6). Kramer, A. F., Cassavaugh, N., Horrey, W. J., Becic, E. & Mayhugh, J. L. (2007). Influence of age and proximity warning devices on collision avoidance in simulated driving. Human Factors, 49(5), 935-949. Lees, M. N. & Lee, J. D. (2007). The influence of distraction and driving context on driver response to imperfect collision
warning systems. Ergonomics, 50(8), 1264-1286. Merat, N. & Jamson, A. H. (2009). How do drivers behave in a highly automated car Proceedings of the 5th International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (pp. 514-521). Iowa: The University of Iowa. Naujoks, F. & Neukum, A. (2014). Timing of in-vehicle advisory warnings based on cooperative perception. In D. d. Waard, K.
Brookhuis, R. Wiczorek, F. d. Nocera, R. Brouwer, P. Barham, C. Weikert, A. Kluge, W. Gerbino & A. Toffetti (Eds.), Proceedings of the Human Factors and Ergonomics Society Europe Chapter Annual Meeting (pp. 1-13). NHTSA. (2013). Preliminary Statement of Policy Concerning Automated Vehicles. Washington, DC: National Highway Traffic Safety Administration. Parasuraman, R., Hancock, P. A. & Olofinboba, O. (1997). Alarm effectiveness in driver-centred collision-warning systems.
Ergonomics, 40(3), 390-399. Parasuraman, R. & Miller, C. A. (2004). Trust and etiquette in high-criticality automated systems. Communications of the ACM, 47(4), 51-55. Parasuraman, R., Mouloua, M., Molloy, R. & Hilburn, B. (1993). Adaptive function allocation reduces performance cost of static
automation. In R. S. Jensen & D. Neumeister (Eds.), 7th International Symposium on Aviation Psychology (pp. 37-42). Parasuraman, R. & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors: The Journal of the Human Factors and Ergonomics Society, 39(2), 230-253. Rudin-Brown, C. M. & Parker, H. A. (2004). Behavioural adaptation to adaptive cruise control (ACC): implications for
preventive strategies. Transportation Research Part F: Traffic Psychology and Behaviour, 7(2), 59-76. Saffarian, M., De Winter, J. & Happee, R. (2012). Automated driving: human-factors issues and design solutions. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Schaller, T., Schielen, J. & Gradenegger, B. (2008). Stauassistenz-Unterstützung des Fahrers in der Quer-und Längsführung:
Systementwicklung und Kundenakzeptanz. Paper presented at the 3. Tagung Aktive Sicherheit durch Fahrerassistenz, Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8
Garching. Schmidt, G. & Krüger, H.-P. (2010). Violating the driver’s threat anticipation and the effect of different modalities for Forward Collision Warnings (FCW). Paper presented at the Paper presented at the HFES Europe Chapter Conference, Berlin. Scott, J. J. & Gray, R. (2008). A Comparison of Tactile, Visual, and Auditory Warnings for Rear-End Collision Prevention in
Simulated Driving. Human Factors, 50(2), 264-275. Seppelt, B. & Wickens, C. D. (2003). In-Vehicle Tasks: Effects of Modality, Driving Relevance, and Redundancy. Illinois: Aviation Human Factors Division, Institute of Aviation, University of Illinois. Spalder, E. & Abel, H. (2013). Die Evolution der Mensch-Maschine-Schnittstelle in Richtung automatisiertes Fahren - Das HMI
als Vertrauen bildende Maßnahme für den Fahrer. In VDI-Gesellschaft Fahrzeug und Verkehrstechnik (Ed.), Der Fahrer im 21.Jahrhundert (pp. 325-341). Düsseldorf: VDI Verlag. Stanton, N. A. & Marsden, P. (1996). From fly-by-wire to drive-by-wire: safety implications of automation in vehicles. Safety Science, 24(1), 35-49. Yamada, K. & Kuchar, J. K. (2006). Preliminary study of behavioral and safety effects of driver dependence on a warning system
in a driving simulator. IEEE Transactions on Systems, Man and Cybernetics, Part A, 36(3), 602-610.
Human Aspects of Transportation I (2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-4951-2097-8
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Aspects of Transportation: Part I
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2097-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional