Miriam Schäffer, Felix Unselt, Wolfram Remlinger (2024). Analysis of Physical Readiness for Take-Over in Automated Driving – Approach to Classify Non-Driving Related Activities According to Their Level of Complexity. In: Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems. AHFE Open Access, vol. 119, pp. 9–18. https://doi.org/10.54941/ahfe1004465
Publisher statistics will load when you open Metrics.
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER
Analysis of Physical Readiness for Take-Over in Automated Driving – Approach to Classify Non-Driving Related Activities According to Their Level of Complexity
Take-overs are becoming part of the use of automated driving functions, since these functions will be limited to certain Operating Driving Domains at first. As the OEM must guarantee safe driving at all times, safe take-overs of the driver are also part of these driving functions. During automated driving in SAE Level 4 more postures and interior adjustments will be possible. This will make the evaluation and prediction of the take-over procedure more complex. In addition to the cognitive transition, the physical adaptation of the driver and the interior back to a drivable state must also be taken into account. In this work, three factors were identified that characterize non-driving related activities in physical terms: whether an item is used, whether the interior is adjusted and whether seat, steering wheel or pedals are adjusted. On this basis, four complexity levels were derived. By assigning an NDRA to a complexity level, the driver's effort for physical adaptation to the driving task and capability to take over can be derived.
The development of automated driving functions is currently forced by many OEMs. Since 2017, it has been legally permitted in Germany for the driver to turn away from the driving task in automated mode SAE Level 3 (SAE J3016, 2021). In 2022, the world’s first series vehicle with approval for an SAE Level 3 automated driving function was approved as a Congestion Assis- tant. During automated driving in SAE Level 3, the driver of the vehicle can temporarily devote himself or herself to Non-Driving Related Activities (NDRA). However, the driver must enable a continuous readiness to take- over the vehicle control within a defined time window if the system reaches its limits.
Since July 2021, the use of SAE Level 4 functions has also been legally regulated, but within defined Operating Driving Domains (ODD) in German
public road traffic. Within these domains, the automated vehicle can handle all situations independently, including bringing the vehicle into a minimal risk condition in case of a critical system error. Therefore, the SAE Level 4 system will also demand for take-overs (Take-Over Request, TOR) when the automated vehicle leaves the ODD or the driver prefers to drive manually instead of reaching a vehicle condition of minimal risk. Take-overs will be more complex and extensive, since the SAE Level 4 system enables the driver a complete and continuous devotion. Nevertheless, longer durations for take- overs are expected, due to the larger functional scope of SAE Level 4 systems.
By providing the automated driving function, the OEM is responsible and liable for traffic and functional safety during the automated journey. From a legal point of view, this also implies the guarantee of a safe take-over and the subsequent manual driving through the human driver.
The take-over from different NDRAs has already been investigated many times (Yoon et al. 2021; Jarosch 2020; Radlmayr et al. 2019). Furthermore, Naujoks et al. (2018) already defined various dimensions of NDRAs which may influence the driver’s availability and take-over performance. Focus of the research is predominantly the cognitive or mental absence of the driver due to the NDRA. As mentioned above, during automated driving in SAE Level 4, the occupant may be in a variety of new postures (Fleischer & Chen 2020), some of which are not considered in today’s vehicles or are unknown today. Furthermore, the customer will demand a completely new interior design adapted to the NDRA, which in turn will be accompanied by new NDRA postures (Fleischer & Wendel 2021; Fleischer & Li 2021; Yang et al. 2020). Therefore, the restoration of the physical readiness of both the driver and the vehicle interior becomes a crucial aspect for OEMs in the prediction and calculation of the take-over procedure and take-over time and has been insufficiently studied so far. The Effort of interruption or termination of an NDRA is already defined as a NDRA dimension influencing the take-over performance (Naujoks et al. 2018). The basis for the assessment and predic- tion of take-over procedures must therefore be the assessment of the driver and the NDRA-adapted vehicle interior at the time of the TOR regarding their take-over capability. This article aims to introduce an approach to assign an NDRA to a certain level of complexity (LoC) to derive on the basis of this the condition of the driver regarding interruptibility, take-over capability and respective effort for taking over.
FACTORS CHARACTERIZING NDRAS IN PHYSICAL TERMS Pfleging & Schmidt (2015) define NDRAs as activities or tasks, which do not refer to the driving task. On the basis of a multitude of survey studies (Hecht et al. 2020; Naujoks et al. 2018; Jorlöv et al. 2017), the NDRAs desired by potential users during the automated journey in dependence of, e.g., the duration of the ride have been identified. For the investigation in this article, these NDRAs are reduced to the following activities (see Table 1, selection based on Fleischer & Chen 2020).
Schäffer et al. (2023) and Schäffer et al. (2021) already introduced the approach HoMoTo to structure, describe and calculate the process of taking
Analysis of Physical Readiness for Take-Over in Automated Driving 11
over an automated operating vehicle. The authors divide the whole take-over procedure into the sub-phases Hand-over, Move-over and Take-over, which can run sequentially or in parallel, and assign the related necessary tasks to the Driver, the Vehicle Interior and the Driving System. Since this article mainly examines the physical restoration of the ability to drive, only these parts within HoMoTo are addressed below.
Table 1. NDRAs considered in this work.
Table available in the original PDF paper.
NDRAs Making a phone call Relaxing Paperwork Observing landscape Sleeping Working with laptop Using smartphone Reading a book Using VR glasses Eating/drinking Reading newspaper Talking to the co-driver Using a tablet
The first sub-phase Hand-over includes all tasks for handing over the items used during the passive ride (see Figure 1) to the vehicle (stowing) and picking up objects the driver needs for driving (Schäffer et al. 2023). To investigate the physical readiness at the time of the TOR depending on the NDRA, based on the HoMoTo approach NDRAs can therefore be classified according to whether an item is required that needs to be stowed (see Table 2, items selected according to Fleischer & Chen 2020). E.g., for observing the land- scape or for sleeping basically no object is necessarily needed. For reading, in general a book, newspaper, smartphone or tablet is needed. Some NDRAs are possible both with and without an item, e.g., making a phone call.
Figure 1: Examples for NDRAs that require an item, Making a phone call (left), Using VR glasses (right).
The second sub-phase Move-over includes the tasks for the adaption of the driver and the vehicle back to a drivable condition (Schäffer et al. 2023). This step implies the adjustment of the NDRA-adapted components, like seat, steering wheel, pedals and other settings relevant for the NDRA.
In today’s vehicles, some of the NDRAs demanded by study participants are only realizable through an appropriate interior configuration. Although some NDRA-supporting configurations, like cup holders and trays, are
already installed in today’s vehicle cockpits or can be added afterwards, oth- ers require a complete new design of the center console or the instrument panel. E.g., working with a laptop may require a folding table (see Figure 2, left) or a retracted steering wheel additionally for higher freedom of move- ment (see Figure 2, right). Furthermore, NDRAs such as the use of a tablet contribute to a significant increase in comfort through an appropriate con- figuration, like additional arm rests in the door panel or additional trays (see Table 3). The re-adjustment of these components increases the complexity and take-over capability massively. An assessment of NDRAs depending on if an adjustment of the interior is required or leads to a significant comfort increase seems useful in this context.
Table 2. Classification of NDRAs according to whether an item is required (items selected according to Fleischer & Chen (2020)).
Table available in the original PDF paper.
NDRAs that do not require use of item NDRAs that require use of item Making a phone call (“hands free”) Making a phone call – smartphone Observing landscape Using smartphone – smartphone Talking to the co-driver Eating/drinking – food, drinks Relaxing Reading a book – book Sleeping Reading newspaper – newspaper Using a tablet – tablet Paperwork – paper, pencil Working with laptop – laptop Using VR glasses – VR glasses
Figure 2: Example for NDRAs that require interior adjustments, Working with laptop on table (left), Working with laptop on table with retracted steering wheel (right).
Studies have shown that participants demand for so called Non-Driving Postures (NDP) (Yang 2021; Fleischer & Chen 2020; Yang et al. 2019) and the respective seat adjustment during NDRAs. These new adjustments include, i.e., a reclined seat (see Figure 3, right) or rotatable seats facing the front-seat passenger or the opposite driving direction (Yang et al. 2019). Fleischer & Li (2021) and Yang et al. (2019) also identified the demand for higher spaces between steering wheel and torso during NDRAs com- pared to the neutral driving posture (see Figure 3, left) by moving the seat in x-direction. Furthermore, Fleischer & Wendel (2021) investigated desir- able backrest angles during different NDRAs. To return from the NDRA and the respective NDP back to the driving task the seat, the steering wheel, which
Analysis of Physical Readiness for Take-Over in Automated Driving 13
is already extended again, and the pedals have to be adjusted within Move- over. These adjustments are similar to the settings made before driving a new vehicle and concern the actual driver’s workplace. On the basis of this, the take-over procedure increases in effort depending on the NDP and if adjust- ments of the seat, steering wheel or pedals are necessary or may be demanded by the driver as they lead to an increase in comfort (see Table 4).
Table 3. Classification of NDRAs according to whether interior adjustments may be required or demanded by the driver.
Table available in the original PDF paper.
NDRAs that do not require adjustment NDRAs that may require adjustment Making a phone call Observing landscape Using smartphone Eating/drinking Eating/drinking Talking to the co-driver Relaxing Sleeping Reading a book Reading a book Reading newspaper Reading newspaper Using a tablet Using a tablet Paperwork Paperwork Working with laptop Working with laptop Using VR glasses
Figure 3: Neutral driving posture (left), example for an NDRA that leads to an increase in comfort due to seat adjustment, Relaxing or Sleeping (right).Table 4. Classification of NDRAs according to whether adjustments of seat, steering wheel or pedals may be required or demanded by the driver (based on Yang et al. (2019)).
Table available in the original PDF paper.
NDRAs that do not require adjustment NDRAs that may require adjustment Making a phone call Observing landscape Using smartphone Eating/drinking
Continued
Table 4. Continued NDRAs that do not require adjustment NDRAs that may require adjustment
Table available in the original PDF paper.
Talking to the co-driver Talking to the co-driver Relaxing Relaxing Sleeping Sleeping Reading a book Reading a book Reading newspaper Reading newspaper Using a tablet Paperwork Working with laptop Using VR glasses
LEVELS OF COMPLEXITY (LOC) Based on the factors introduced above, the condition of the driver and the vehicle interior at the time of the TOR can be assessed by if an item is used, if the interior is adjusted and if the seat, steering wheel or pedals are adjusted for the NDRA. The applicability of a factor is initially described by binary numerical codes: 0 (zero) means Not applicable, 1 (one) means Applicable. This allows to describe an NDRA by the assignment of 0 or 1 to three digits. E.g., an NDRA described by ‘000’ does not require the use of an item, no adjustment of the interior or of the seat, steering wheel or of the pedals is made. This may apply for making a phone call (“hands free”) in the neu- tral driving posture and in an interior that is set up for manual driving. An NDRA described by ‘111’ requires an item, and the interior as well as the seat, steering wheel or the pedals are adjusted. E.g., while working, in this case, a laptop is placed on a table and the seat is adjusted longitudinally and reclined backwards. In total, this binary coding results in eight categories into which NDRAs can be divided, also multiple times: 000, 100, 010, 001, 110, 101, 011, 111. According to Feess (2018), complexity can be determined by the num- ber and type of elements and their relationships to each other. In this case, the amount of simultaneously occurring factors introduced above leads to a higher complexity regarding the physical effort for taking over. This phys- ical complexity refers to how extensive the transition is at the time of the TOR back to the manual drive constitution. Based on the amount of applicable factors (see Tables 2, 3, 4), the eight categories are assigned to four complexity levels (see Table 5). The complexity of the physical tran- sition back to driving constitution increases with higher LoCs. It should be noted, that the categorization of the NDRAs to the different LoCs do not claim to be complete, but are merely intended to provide an under- standing of the approach. In principle, e.g., adjusting the interior and the seat can always lead to an increase in comfort and can be requested by the driver.
Analysis of Physical Readiness for Take-Over in Automated Driving 15
Table 5. Classification of NDRAs according to their LoC.
Table available in the original PDF paper.
Level of complexity NDRAs
Talking to the co-driver
LoC I – no factors (000)
Making a phone call (“hands free”)
Relaxing
Observing landscape
Making a phone call
Reading a book
Sleeping LoC II – 1 factor (100 | 010 | 001)
Using smartphone
Reading newspaper
Eating/drinking
Using a tablet
Paperwork
Talking to the co-driver
Working with laptop
Relaxing
Using VR glasses
Sleeping
Eating/drinking
Using a tablet
LoC III – 2 factors (110 | 101 | 011)
Reading a book
Paperwork
Reading newspaper
Reading a book
Reading newspaper
Working with laptop LoC IV – 3 factors (111)
LIMITATIONS AND DISCUSSION The approach introduced enables to categorize NDRAs on the basis of simple criteria to different LoC. Based on this, the physical effort to return to the driving task and the respective take-over readiness of the driver in physical terms can be derived.
As already been proven in many studies (Yoon et al. 2021; Jarosch 2020; Radlmayr 2020), the actual cognitive or mental state of the driver also influ- ences the take-over procedure immensely as a whole as well as individual physical tasks in particular. This interdependency and intercorrelation of the cognitive and physical state during take-over is not considered in the approach. E.g., Sleeping is seen as one of the more complex NDRAs regard- ing take-over capability and necessary take-over time because of the driver’s complete mental absence (Wörle et al. 2021), whereas in this approach, Sleep- ing can be assigned to LoC I. In addition, the cognitive condition of the driver is more difficult to predict than the actual physical state. However, this must also be taken into account when predicting the driver’s current readiness to take over through e.g., the integration of existing mental models. E.g., a linking with the novel software function Situation Awareness Management (Remlinger & Pomiersky 2021), which provides a method to determine the required situational awareness of the driver, must be further investigated, since the concept assists to verify, regain, and increase situation awareness as well (Pfeifer et al. 2022).
Furthermore, the three evaluation factors are considered to be of equal value and are not given any further weighting. However, a distinction is essen- tial to examine not only between the factors but also within, since different factors will affect the driver’s readiness in different degrees. E.g., the folding of the table is more complex than the stowing of a book, and the handling and stowage of different objects, such as a smartphone or a laptop, is associated
with different levels of effort. Also the handling of multiple items or more than one interior adjustment impact the take-over capability, which is why the investigation of the interdependence of the factors is required. In this case, a categorization of different items (or respective adjustments of interior, seat, steering wheel or pedals) through the assignment of, e.g., weighting factors and the integration in the LoC seems necessary.
For the transfer of the approach into the development process of auto- mated vehicles, development of the approach concerning the inclusion of further relevant individual factors of the driver during the preparation, like taking on shoes, or further posture characteristics is necessary. Also, future seat configurations like swivel seats must be considered. In this case, the adjustment back to the driving task is currently still partly unclear due to the limited space available in the vehicle interior. To this end, the approach is designed as an open system, which allows adding further factors and posture characteristics of the occupants.
To validate the approach, it is necessary to investigate the take-over time and take-over quality of NDRAs of different LoCs, e.g., in a driving simulator study. An integration of the approach presented into the HoMoTo-concept also seems promising. However, a uniform description format should be aimed for.
CONCLUSION
In the future, OEMs will have to deal with the safety-critical process of taking-over in the design of every vehicle interior variant and in the fur- ther development of every driving function. This paper aimed to bring a structure into the current investigation of the physical take-over readiness of the driver. In practical application, the approach aims to identify poten- tials in the vehicle interior to support the driver during the take-over. As a first approach, three factors influencing the take-over capability in depen- dence of the NDRA where identified: the item used, the adjustment of the interior and the adjustment of the seat, the steering wheel or pedals. The binary description of NDRAs by these factors resulted in eight cate- gories. These categories were than divided into four LoCs. By assigning an NDRA to one of these LoCs, a statement can be made about the actual driver’s physical readiness to take-over. Limitations exist, like the influence of the cognitive state, the weighting of the factors and the consideration of the intercorrelation of factors. Nevertheless, the approach serves a basis for the addition of further evaluation criteria and the integration into the existing HoMoTo-concept.
REFERENCES
Feess, E. (2018). Komplexität, https://wirtschaftslexikon.gabler.de/definition/komp lexitaet -39259/version-262672. Fleischer, M., Chen, S. (2020). How Do We Sit When Our Car Drives for Us?,
in: Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Posture, Motion and Health, Duffy, V. G. (Ed.). Cham: Springer International Publishing, pp. 33–49.
Analysis of Physical Readiness for Take-Over in Automated Driving 17
Fleischer, M., Li, R. (2021). Spatial Needs for Non-driving Related Activities, in: Advances in Simulation and Digital Human Modeling. Proceedings of the AHFE 2021 Virtual Conferences on Human Factors and Simulation, and Digital Human Modeling and Applied Optimization, Wright, J. L., Barber, D., Scataglini, S., Rajulu, S. (Eds.). Cham: Springer International Publishing, pp. 282–287. Fleischer, M., Wendel, N. (2021). Desirable Backrest Angles in Automated Cars, in:
HCI International 2021 – Late Breaking Posters: 23rd HCI International Confer- ence, Stephanidis, C., Antona, M., Ntoa, S. (Eds.). Cham: Springer International Publishing, pp. 521–526. Hecht, T., Darlagiannis, E., Bengler, K. (2020). Non-Driving Related Activities in Automated Driving – An Online Survey Investigating User Needs, in: Human Sys- tems Engineering and Design II. IHSED 2019. Advances in Intelligent Systems and Computing, Ahram, T., Karwowski, W., Pickl, S., Taiar, R. (Eds.). Cham: Springer International Publishing, pp. 182–188. Jarosch, O. (2020). Non-Driving-Related Tasks in Conditional Driving Automa-
tion, Ph. D. dissertation, Chair of Ergonomics, Technical University of Munich, Munich. Jorlöv, S., Bohman, K., Larsson, A. (2017). Seating Positions and Activities in Highly Automated Cars – A Qualitative Study of Future Automated Driving Scenarios, International Research Conference on the Biomechanics of Impact. Cham: Springer International Publishing, pp. 13–22. Naujoks, F., Befelein, D., Wiedemann, K., Neukum, A. (2018). A Review of
Non-driving-related Tasks Used in Studies on Automated Driving, in: Advances in Human Aspects of Transportation, Stanton, N. A. (Ed.). Cham: Springer International Publishing, pp. 525–537. Pfeifer, C., Pomiersky, P., Remlinger, W. (2022). Situation Awareness Management for Driver Take Over from Level 4, in: 22. Internationales Stuttgarter Symposium. Proceedings, Bargende, M., Reuss, H. C., Wagner A. (Eds.). Wiesbaden: Springer Vieweg, pp. 101–111. Pfleging, B., Schmidt, A. (2015). (Non-) Driving-Related Activities in the Car:
Defining Driver Activities for Manual and Automated Driving. Workshop on Experiencing Autonomous Vehicles: Crossing the Boundaries between a Drive and a Ride at CHI’15. Radlmayr, J., Fischer, F. M., Bengler, K. (2019). The Influence of Non-driving Related Tasks on Driver Availability in the Context of Conditionally Automated Driving, in: Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018), Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (Eds.). Cham: Springer International Publishing, pp. 295–304. Remlinger, W., Pomiersky, P. (2021). Fahrassistenzsystem für ein automatisiert
fahrbares Fahrzeug und computerimplementiertes Verfahren zum Assistieren eines Fahrers eines automatisiert fahrbaren Fahrzeugs, Germany DE 10 2021 209 251, still unpublished patent. SAE J3016 (2021). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Warrendale, PA, United States. Schäffer, M., Pomiersky, P., Remlinger, W. (2021). Hand Over, Move Over, Take
Over – What Automotive Developers Have to Consider Furthermore for Driver’s Take-Over, in: Automatisiertes Fahren 2021, Bertram, T. (Ed.). Wiesbaden: Springer Vieweg, pp. 127–142. Schäffer, M., Pomiersky, P., Remlinger, W. (2023). “Hand-Over, Move-Over, Take-Over” – HoMoTo as a Holistic Method to Analyze Take-Over Scenarios, in: Human Factors in Transportation. AHFE (2023) International Conference, Praetorius, G., Sellberg, C., Patriarca, R. (Eds.). USA: AHFE International.
Wörle, J., Metz, B., Baumann, M. (2021). Sleep inertia in automated driving: Postsleep take-over and driving performance, Accident Analysis & Prevention. Yang, Y. (2021). Driver’s Non-Driving Postures in Automated Driving: Model-
ing, Assessment, and Countermeasure, Ph. D. dissertation, Chair of Ergonomics, Technical University of Munich, Munich. Yang, Y., Fleischer, M., Bengler, K. (2020). Chicken or Egg Problem? New challenges and Proposals of Digital Human Modeling and Interior Development of Automated Vehicles, in: Advances in Additive Manufacturing, Modeling Systems and 3D Prototyping. Proceedings of the AHFE 2019 International Conference on Additive Manufacturing, Modeling Systems and 3D Prototyping, Di Nicolanto- nio, M., Rossi, E., Alexander, T. (Eds.). Cham: Springer International Publishing, pp. 453–463. Yang, Y., Klinkner, J. N., Bengler, K. (2019). How Will the Driver Sit in an Auto-
mated Vehicle? – The Qualitative and Quantitative Descriptions of Non-Driving Postures (NDPs) when Non-Driving-Related-Tasks (NDRTs) are Conducted, in: Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018), Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (Eds.). Cham: Springer International Publishing, pp. 409–420. Yoon, S. H., Lee, S. C., Ji, Y. G. (2021). Modeling Takeover Time Based on Non-Driving-Related Task Attributes in Highly Automated Driving, Applied Ergonomics, vol. 92.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-95-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional