INTRODUCTION
Advancements in semi-autonomous driving technology are amongst the most popular topics in transportation today. More than thirty companies have introduced or announced plans to develop semi-autonomous vehicles (SAV’s) with projections of fully autonomous vehicles in the next decade or two. The purported benefits of SAVs are many and include reduced traffic congestion (Fagnant and Kockelman, 2015), improved safety, increased fuel economy, more efficient parking options, and more accessible transportation for indi- viduals who are unable to drive due to permanent or temporary disability (Fagnant and Kockelman, 2015).
Some drivers remain distrustful of semi-autonomous vehicles (Lee and Kolodge, 2018) including older adults (Fagnant and Kockelman, 2015) who would stand to benefit from using this technology. The public’s perception of SAV’s is influenced by many factors including a lack of knowledge about the technology and by news reports of SAV crashes. Crash fatalities have drawn scrutiny from federal agencies including NHTSA and NTSB.
© 2022. Published by AHFE Open Access. All rights reserved. 394
Semi-Autonomous Vehicle Crashes: An Exploration of Contributing Factors 395
Understanding of the causes of SAV crashes can inform vehicle design and regulatory policy. Vehicle crashes do not have a singular cause. Consequen- tly, it is important to identify the role of other contributory factors. Articles appearing in the popular press cite a variety of issues including limitations of the technology, driver behavior, the changing role of the driver, a driver’s understanding of the operation of the SAV’s, and their assumptions about SAV capabilities or limitations. We explore each of these issues and their possible contributory role to SAV crashes.
CRASHES AND SAV’S Overall SAVs experience collisions at a higher rate per miles traveled than conventional vehicles; however, of the collisions experienced, approximately 94% were caused by factors not attributable to the operation of the SAV itself (Favarò et al., 2017). Additionally, SAV crashes are typically of a lower injury severity than conventional vehicles (Schoettle and Sivak, 2015). The most common type of SAV’s crashes are those where a conventional vehi- cle strikes the SAV from the rear (Petrovi´c et al., 2020). This type of crash occurs at double the rate of conventional vehicles (Favarò et al., 2017). One explanation for the larger proportion of rear end strikes is that drivers of conventional vehicles are not accustomed to the breaking behavior of SAVs (Petrovi´c et al., 2020). Drivers of conventional vehicles drive with a style that is more apt to satisfying their driving goals while SAVs follow prescribed road rules and signage (Nyholm, 2018). When an SAV approaches a stop sign or stoplight the vehicle will come to a complete stop whereas a conventional vehicle may perform a rolling stop allowing the following vehicle to proceed forward. Anticipating the behavior of the lead vehicle is further complicated when the driver of the following vehicle is not aware the lead vehicle is a SAV and if they are, whether it is being operated manually or semi-autonomously. SAV’s lack a means of communicating their operational mode (e.g., manual versus semi-autonomous), the detection of other road users including pede- strians, or its intentions. Drivers and SAV’s use turn signals, but only drivers convey information through body language.
DRIVER KNOWLEDGE OF SAV CAPABILITIES Driver behavior and not the SAV system per se can increase the risk of a crash. The Operational Design Domain (ODD) limitations define the operational conditions under which the semi-autonomous system is intended to operate. The ODD for a Tesla Model 3 includes the following (NTSB, 2017):
- Designed for use on limited-access highways.
- Designed for areas with no cross traffic and clear lane markings.
- Not for use on city streets where traffic conditions are constantly changing.
- Not for use on winding roads with sharp curves.
- Not for use in inclement weather conditions with poor visibility.
396 Mersinger and Chaparro
Drivers using advanced driver assistance systems outside of ODD defined conditions are well documented (NTSB, 2017). NTSB (2020;2017) investiga- tions of several SAV fatalities and anecdotal evidence including videos posted on YouTube show drivers employing SAV features on roads with cross traffic and roads that lack lane markings.
Tesla’s driver assistance system can be engaged in situations that violate the ODD. The operation of GM’s Super Cruise driver assistance system is more restrictive. The system can only be activated on limited-access roadways. An infrared illumination camera is used to monitor the drivers head pose to assess whether they are attending to the roadway. A lightbar on the steering wheel flashes green if the system determines that the driver is not attending to the roadway. Tesla’s system monitors the torque applied to the steering wheel as a surrogate indicator of driver engagement. The torque generated by the weight of the hands on the steering wheel is registered as driver interaction. Although GM’s Super Cruise system relies on a more behaviorally relevant criterion of driver engagement, gaze direction and attention are not always strongly correlated (Underwood, 2005).
Why might drivers operate their SAV’s in violation of the ODD? One pos- sibility is that drivers may be unaware or unfamiliar with their “supervisory” role and what it entails. Drivers may not be aware of the operational limita- tions of the automation defined in the ODD. Additionally, this new role or responsibility may not be congruent with human attentional and behavioral capabilities.
NEW ROLE OF THE “DRIVER” Knowledge of operating one vehicle often generalizes to other vehicles. This is not the case for SAV’s. When driving in semi-autonomous mode the dri- ver’s role changes from active control of the vehicle to that of a passive supervisor-control mode of system operations (Sheridan, 2002). The driver may be unaware of what is expected of them, including how the automated systems operate (Endsley, 2017), what system information should be moni- tored, the meaning of system malfunctions, warnings, and what actions to take. In addition, knowledge of operating one manufacturers SAV may not generalize to another due to different ODD’s, algorithms, and sensor suites with different capabilities and limitations.
Absence of Training Requirements Like other consumer products no requirement exists (other than licensure) for the owner to demonstrate a minimal level of understanding of advanced driver assistance systems but unlike other consumer products the consequ- ences of misuse can be more severe. Also, unlike operators in aviation or manufacturing, the driver is not required to demonstrate a minimum level of understanding of the system or operational competency as assessed through testing or simulation. Consumers have come to expect a plug and play experience and manufacturers have responded by eliminating barriers that prevent the purchaser from activating and using the technology as soon as possible. Training and other educational requirements are a barrier to this
Semi-Autonomous Vehicle Crashes: An Exploration of Contributing Factors 397
seamless experience. It follows that the performance of SAV operators will be more variable given the large range in consumer user understanding of these systems.
Even in cases where an operator may have developed a mental model of the advanced driver assistance system this information can become obso- lete as new software updates are released. Release notes and accompanying documents may lack sufficient detail explaining the operation of the new or modified features (Endsley, 2017).
Drivers’ Understanding of the ODD Also, the onus is on the vehicle owner to familiarize themselves with the ODD and the operation of the vehicle by reading the owner’s manual. The owner’s manual alerts the driver to known limitations of advanced safety fea- tures including lane assist and automatic breaking and are essential for the safe operation of the vehicle. Nevertheless, drivers often do not consult auth- oritative sources (Eby, 2017; McDonald, 2016). Crash reports and surveys of owners of vehicles equipped with advanced in-vehicle technology indi- cate drivers’ understanding of the operation of these technologies is poor (Llaneras, 2007). Drivers appear instead to rely on informal sources inclu- ding blogs, opinions of family or friends, or the web for information about the operation of these systems (McDonald, 2016).
Sustained Attentional Monitoring The new supervisory role also emphasizes vigilance or sustained attentive monitoring of on-going system operations with the goal of detecting system malfunctions. It is known that human operators perform sustained attention tasks poorly (Davies and Parasuraman, 1982). In the 1940’s, Mackwork (1948) reported evidence of a vigilance decrement wherein visual detection of signals declined after only 30 minutes of monitoring a display. Reports of miss rates for real monitoring or inspection tasks including X-rays, inspection of glassware and metal fasteners are also typically high ~20–40% but in some cases relatively low rates (i.e., 1—9%) are reported (Craig, 1984). Even so, these comparatively low miss rates are probably unacceptable for socially important tasks like driving. In the absence of detailed system knowledge, what information serves as a basis for predicting the SAV’s response to novel road situations? Below we discuss the role of system names and marketing, mental models, and anthropomorphisms.
Marketing The driver’s expectations and behavior may be subtly (or not) biased by system names and marketing materials that may prime expectations that do not match system capabilities. Marketing materials highlight the capability of these systems to see other vehicles and pedestrians and to avoid them without also noting the system’s limitations. Critics have noted that the use of terms like Autopilot or Pilot Assist and marketing campaigns like that accompan- ying the introduction of Drive Pilot by Mercedes-Benz may seed unfounded
398 Mersinger and Chaparro
expectations among users that the systems are more capable than they really are (Parasuraman and Riley, 1997).
Extension of Mental Models Most SAV drivers are not familiar with the technical limitations of sensors such as LIDAR, radar, computer vision systems, or how data from these sensors are processed to detect, categorize, and respond to road situati- ons. Because the operation of the system is opaque, operators have little insight into the processes that determine a SAV’s responses to situational factors. Instead, operators might rely on their knowledge of analogous systems (e.g., knowledge extension by analogy) like lane assist or adaptive/a- dvanced cruise control to predict the operation of systems that superficially appear similar but might operate differently. Given the systems’ opacity, the operator may infer relationships between external events and system actions that are in fact not linked (Woods, 2010).
When interacting with a conventional vehicle, inferences can be drawn about the other driver’s intentions based on one’s own experience, social norms, mental models, body language, and head and eye direction. Many of these cues are absent when interacting with a SAV. The rule sets that SAV’s follow that determine their action, and the information that they collect are not easily recognized or understood by the human driver (Surden and Wil- liams, 2016). While the driver can observe the surrounding data that the vehicle might sample and the vehicle’s behavior, they cannot fully understand why the vehicle behaves the way that it does.
Consider the case of a SAV approaching a parked police car or fire truck partially blocking their path. Operators may expect their SAV to either stop or to maneuver around the stopped vehicle. However, what is the basis of this expectation?
Anthropomorphisms The driver’s expectations might be anthropomorphic. The term anthropo- morphism refers to the attribution of human traits or behaviors to nonhuman objects. However, when drivers’ expectations for the vehicle are anthropo- morphic, they are not expecting the vehicle to share all human traits. The driver does not expect their vehicle to have emotion or feel pain, but rather, they expect their vehicles to make executive decisions in driving scenarios similar to their own. Humans exercise a theory of mind when predicting the behaviors of other humans, but these projections of behavior cannot be applied to SAVs as they have different capabilities and limitations. In the afo- rementioned scenario, a driver would stop, or they would maneuver around the truck. The driver would not consider continuing to drive straight as an option. Tesla’s owner’s manual cautions operators that the “Traffic-Aware Cruise Control” may not detect some objects and may not brake for statio- nary vehicles when driving over 50 mph. Seeing and avoiding an object may appear so basic, simple, so rudimentary that an operator might assume the automation has similar capabilities. However, the apparent ease with which we see and negotiate the environment belies the complexity of the perceptual
Semi-Autonomous Vehicle Crashes: An Exploration of Contributing Factors 399
process (Smallman and St. John, 2005). Thus, drivers may assume that the autonomous system’s perceptual capabilities are analogous to their own and that the system will respond to driving situations in a like manner. Yet, mul- tiple crashes with SAV’s striking stationary fire trucks parked on the side of the road clearly illustrate that this is not the case.
The AI and neural network are not as robust as humans in identifying objects seen from multiple viewpoints or under different environmental or situational conditions. The ability of a neural network to identify an object across multiple conditions is critically dependent on the training set of images and inclusion of scenes that represent the diversity of conditions that might occur in the natural environment. Something as simple as identifying a fire- truck at night or during the day with or without flashing lights may prove challenging for a neural network but is simple for a human observer who has never confronted the scene before.
TRUST AND AUTOMATION An operator’s failure to monitor automation is related to their perception of the reliability of the system. Extended interaction with high reliability systems engender an expectation that system failures are extremely rare and hence that frequent monitoring is unnecessary (Wickens et al., 2015). The operator’s trust in the system is calibrated to the system and is dependent on obtaining a rich set of experiences and instances where the system operates well or fails.
Users with higher automation trust spend more time attending to non- driving related tasks than users in lower automation trust groups (Zhang et al., 2021) and respond more slowly in scenarios where manual takeover of the SAV is necessary. Barring gross errors, operators become increasingly comfortable with the advanced features, monitoring the system’s operation less frequently (Zhang et al., 2021) and increasingly defer to system decisions.
CONCLUSION
We have identified factors that might contribute to SAV crashes including the effects of incorrect or undeveloped user mental models and poor operator understanding of system capabilities, limitations, and functionality. The effe- cts of these factors are compounded by system opacity, the absence of user training requirements, a means to communicate vehicle states (i.e., manual or semi-autonomous mode) or intentions to other road users, anthropomorphic based user reasoning, and requiring human operators to perform a supervi- sory/monitoring function. These issues highlight a number of human factors research needs including defining minimal educational requirements for ope- rators, providing user feedback to improve their understanding of current system operation, means of communicating system intentions with other road users, and ways to improve or modify monitoring tasks that are better suited to the user.
400 Mersinger and Chaparro
REFERENCES
Craig, A., 1984. Human Engineering: The control of vigilance. In Y. Warm (Ed.), Sustained Attention in Human Performance (pp. 247–291). Chichester, UK: Wiley. Davies, D.R. and Parasuraman, R. 1982, The Psychology of Vigilance, Academic
Press, London;New York. Eby, D.W., Molnar, L.J., Zakrajsek, J.S., Ryan, L.H., Zanier, N., Louis, R.M.S., Stanciu, S.C., LeBlanc, D., Kostyniuk, L.P., Smith, J. and Yung, R., 2018. Prevale- nce, attitudes, and knowledge of in-vehicle technologies and vehicle adaptations among older drivers. Accident Analysis and Prevention, 113, pp. 54–62. Endsley, M.R., 2017. Autonomous driving systems: A preliminary naturalistic
study of the Tesla Model S. Journal of Cognitive Engineering and Decision Making, 11(3), pp. 225–238. Fagnant, D.J. and Kockelman, K., 2015. Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice, 77, pp. 167–181. Favarò, F.M., Nader, N., Eurich, S.O., Tripp, M. and Varadaraju, N., 2017.
Examining accident reports involving autonomous vehicles in California. PLoS One, 12(9), p.e0184952. Lee, J.D. and Kolodge, K., 2018, September. Understanding attitudes towards selfdriving vehicles: Quantitative analysis of qualitative data. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 62, No. 1, pp. 1399–1403). Sage CA: Los Angeles, CA: SAGE Publications. Llaneras, R., 2007. Misconceptions and self-reported behavioral adaptations asso-
ciated with advanced in-vehicle systems: lessons learned from early technology adopters. Driving Assessment 2007 Fourth International Driving Symposium. Human Factors in Drivers Assessment, Training and Vehicle Design, pp. 299–305. Mackworth, N.H., 1948. The breakdown of vigilance during prolonged visual search. Quarterly Journal of Experimental Psychology, 1(1), pp. 6–21. McDonald, A.B., McGehee, D.V., Chrysler, S.T., Askelson, N.M., Angell, L.S. and
Seppelt, B.D., 2016. National survey identifying gaps in consumer knowledge of advanced vehicle safety systems. Transportation Research Record, 2559(1), pp. 1–6. National Transportation Safety Board. 2017. Collision Between a Car Operating with Automated Vehicle Control Systems and a Tractor-Semitrailer Truck Near Williston, Florida, May 7, 2016. Highway Accident Report NTSB/HAR-17/02. Washington, DC. National Transportation Safety Board. 2020. Collision Between a Sport Utility Vehi-
cle Operating with Partial Driving Automation and a Crash Attenuator, Mountain View, California, March 23, 2018. Highway Accident Report NTSB/HAR-20/01. Washington, DC. Nyholm, S., 2018. The ethics of crashes with self-driving cars: a roadmap, II. Philosophy Compass, 13(7), p.e12506. Parasuraman, R. and Riley, V., 1997. Humans and automation: Use, misuse, disuse,
abuse. Human Factors, 39(2), pp. 230–253. Petrovi´c, Ð., Mijailovi´c, R. and Peši´c, D., 2020. Traffic accidents with autonomous vehicles: type of collisions, manoeuvres and errors of conventional vehicles’ drivers. Transportation Research Procedia, 45, pp. 161–168. Schoettle, B. and Sivak, M., 2015. A preliminary analysis of real-world crashes
involving self-driving vehicles. University of Michigan Transportation Research Institute. UMTRI-2015-34
Semi-Autonomous Vehicle Crashes: An Exploration of Contributing Factors 401
Sheridan, T.B., 2002. Humans and Automation: System Design and Research Issues.
Human Factors and Ergonomics Society. Smallman, H.S. and John, M.S., 2005. Naïve realism: Misplaced faith in realistic displays. Ergonomics in Design, 13(3), pp. 6–13. Surden, H. and Williams, M.A., 2016. Technological opacity, predictability, and self-
driving cars. Cardozo Law Review., 38, p. 121. Underwood, G. ed., 2005. Cognitive Processes in Eye Guidance. Oxford University Press. Wickens, C.D., Hollands, J.G., Banbury, S. and Parasuraman, R., 2015. Engineering
Psychology and Human Performance. Psychology Press. Woods, D.D., 2010. Behind Human Error. Ashgate Publishing, Ltd. Zhang, Y., Ma, J., Pan, C. and Chang, R., 2021. Effects of automation trust in drivers’ visual distraction during automation. PLoS One, 16(9), p. e0257201.