Kristin Lennox, Iiona Scully, Julio Yanes, David Cades (2024). Assessing the Impact of Driver Assistance Technology: A Review of Non-Crash and Crash Studies. In: Advances in Human Factors of Transportation. AHFE Open Access, vol. 148, pp. 323–330. https://doi.org/10.54941/ahfe1005223
Level 2 advanced driver assistance systems (ADAS), sometimes referred to as partial automation systems, control both longitudinal and lateral motion of a vehicle under driver control and supervision. Level 2 systems are increasingly common in commercially available vehicles, and there has been extensive study of the potential impact of these systems on crash risk. Historically, studies of these systems have used proxies for crash risk, such as driver behavior and attentiveness, to predict the eventual influence of these systems on real-world crash rates. However, recently, real-world crash studies have been conducted for L2 systems from multiple manufacturers. This paper provides a review of both non-crash and crash-based evaluations of Level 2 systems, including a new analysis of crash data published by Tesla. Overall, while non-crash assessments of Level 2 systems have been mixed, all crash studies published to date point to a reduction in risk associated with such systems. This review also suggests improvements to non-crash studies that may increase their predictive value.
While there are currently no fully automated automobiles available on the consumer market, partial automation features are offered by multiple manu- facturers and are even standard in certain vehicle models. Level 2 (L2) driver assistance systems (SAE International, 2021; NHTSA, 2023) combine lon- gitudinal vehicle control through acceleration and deceleration with lateral control through vehicle steering. This can be achieved, for example, through the combination of an adaptive cruise control (ACC) feature and lane center- ing assist (LCA) feature. SAE International and the National Highway Traffic Safety Administration (NHTSA) define L2 systems as driver support/assis- tance features, where the driver is required to be in control, respond, and remain attentive to the driving environment. Examples of commercially avail- able L2 systems include Audi Traffic Jam Assist, Ford BlueCruise, GM Super Cruise, Nissan ProPILOT Assist, Volvo Pilot Assist, and Tesla Autopilot.
Although L2 systems may be marketed as convenience features, similar to cruise control, there is clearly the potential for these systems to influ- ence crash rates. Studying this effect, however, is non-trivial. Unlike simpler systems like automated emergency braking (AEB), there are no formal def- initions of target crash types expected to be impacted by L2 systems (e.g., Wang, 2019). Manufacturers may provide guidance regarding the intended use of L2 systems (e.g., road type or speed limitations), and they are acti- vated at the driver’s discretion. This makes it challenging to identify crashes and crash exposure for which L2 was active and that are appropriate for performance characterization.
As a consequence, the potential impact of L2 technology is often evaluated using proxy information, such as studies of driver gaze and attentiveness. Such studies may generate conflicting results and may not reflect the true impact of L2 systems on crashes. In this paper, we provide a review of such proxy studies attempting to gauge the effects of L2 systems, as well as a com- parison to two types of crash studies that have been successfully carried out for L2 systems: equip/non-equip and usage based. We include a new analysis for Tesla Autopilot crash data. We then compare the predicted effects from non-crash studies with the crash data available and discuss challenges and potential future opportunities for assessing crash risk of partial automation and driver assistance systems.
METHODOLOGY
We conducted a literature review with multiple search strategies including database searches and scanning the reference lists of relevant papers. The non-crash studies included in this review were selected for inclusion if they assessed the impact of L2 systems on driver behavior. Our written review of non-crash studies was non-exhaustive but includes a representative sample of non-crash assessments. For the crash studies, we included all research that we identified that included real-world crash data associated with L2 systems.
NON-CRASH ASSESSMENTS OF L2 SYSTEMS Some design intentions of L2 driver assistance are to reduce the effort of the driving task, maintain a safe distance from other vehicles, and maintain the vehicle in the lane, which in turn has the potential to result in safety benefits for the driver (e.g., Seppelt & Victor, 2016). Despite the potential benefits that these systems can provide, researchers also suggest that the reduced effort toward the driving task might have other unintended effects (e.g., Reagan et al., 2021).
A primary area of interest when estimating the potential benefits and lim- itations of L2 systems is driver engagement in the driving task. Research on ACC and L2 systems suggests that these systems have the potential to reduce driver workload (e.g., de Winter er al., 2014; Endsley, 2017; Biondi et al., 2018) and increase situational awareness (e.g., Beller et al., 2013; Endsley, 2017; Mueller et al., 2022), which in turn could free up attentional resources and provide potential benefits to the driver. For example, Mueller et al. (2021)
Assessing the Impact of Driver Assistance Technology 325
found that using an L2 system improved situational awareness for partici- pants who had experience using an L2 system and demonstrated that those with improved situational awareness spent more time looking to the forward, periphery, and side, as well as in the rear-view mirror, which the authors sug- gested indicates more active and dispersed scanning strategies. In addition to changes in situational awareness and workload, a longitudinal study of driver behavior while operating L2 systems demonstrated a reduction in speeding behavior when L2 systems were activated compared to manual driving by experienced drivers (Dunn et al., 2019).
On the other hand, researchers have also suggested that the use of L2 systems may provide drivers with the opportunity to engage in distract- ing non-driving related behaviors that involve manual or visual distraction (e.g., removing hands from the wheel or reduced scanning of the roadway and mirrors), which have been associated with increased crash risk (e.g., Cunningham et al., 2017; Dingus et al., 2016; Gershon et al., 2019). In an early case study using Tesla’s Autopilot, Endsley (2017) indicated she was able to look around and be more aware of traffic with the L2 system engaged, but that over time, she experienced mind-wandering and noted that her attention also deviated to competing tasks, like texting or adjusting the navigation or sound system. Similarly, Reagan et al. (2021) found an increase in visual-manual non-driving tasks, including cell phone activity and center stack activity, over a period of four weeks for inexperienced participants using Volvo’s Pilot Assist L2 system compared to manual driving.
Studies using glance behavior to examine visual attention to the roadway have shown mixed results. Morando et al. (2020) found that 64% of glances were toward the forward roadway when participants used L2 technology, compared to the 76% under manual driving control. In contrast, Shutko et al. (2018) found that participants kept their eyes on the roadway 89% of time with similar technology engaged, which they claim is consistent with manual driving (e.g., Tijerina et al., 2004). Further, Shutko et al. (2018) also diverge from Reagan et al. (2021) as they demonstrate that although partic- ipants engaged in a variety of non-driving tasks, engagement in these tasks was similar across periods of L2 system use and manual driving. This is con- sistent with findings from a subset of longitudinal L2 driving studies which found that drivers were not overly reliant on the system, operated L2 systems as intended, and did not engage in non-driving secondary tasks more when compared to manual driving (Dunn et al., 2019; Russel et al., 2018). Fur- thermore, interview research examining the potential for driver engagement in non-driving tasks suggests that while experienced L2 drivers may engage more in secondary tasks, they are likely adapting to the driving situation, and identify “safe,” less risky scenarios before engaging in these tasks (Lin et al., 2018). In summary, non-crash based research examining the potential impact of L2 systems on driver engagement in the driving task is mixed, with results indicating improved, degraded, or equivalent performance to manual driving depending on the assessment method.
CRASH ASSESSMENTS FOR L2 SYSTEMS As described previously, there are challenges involved in acquiring appro- priate data to directly assess the impact that L2 systems may have on crash rates. Crash rate calculations require information about relevant crashes and the potential exposure of vehicles to crashes. There are two primary strate- gies that have been used thus far for assessments of L2 impact on crash rates: equip/non-equip studies and use/non-use studies.
An equip/non-equip study compares the crash rate for vehicles with L2 technology equipped with “control” vehicles that do not have the technol- ogy equipped, but they do not include any information about whether the system was in use at the time of the crash. The Highway Loss Data Institute (HLDI) has carried out a series of equip/non-equip studies for L2 systems from multiple manufacturers, including Tesla (HLDI, 2017), Nissan (HLDI, 2021a), BMW (HLDI, 2021b), and Audi (HLDI, 2022). In all cases, rates of different types of insurance claims were assessed for vehicles with the rele- vant L2 system equipped and a peer group consisting of vehicles from the same manufacturer, and typically the same model, without any L2 technol- ogy. Exposure was measured as insured vehicle years. The specific claim types assessed included property damage liability, collision, and multiple types of medical claims.
For three of the four systems studied (Tesla Autopilot, BMW Driving Assis- tance Plus, and Audi Traffic Jam Assist), statistically significant reductions in one or more claim types were observed. All other results were indeterminate, possibly due to small sample sizes. Leslie et al. (2022) also carried out two studies comparing Cadillac models equipped with GM’s Super Cruise to sim- ilar Cadillac vehicles without Super Cruise. These included a comparison of the proportion of crashes involving equipped vs non-equipped vehicles rel- ative to the proportion of equipped vs non-equipped vehicles in the study population as well as a quasi-induced exposure analysis. Likely due to small sample sizes, neither study produced statistically significant results.
Equip/non-equip studies compare vehicles that could potentially use L2 technology to peers that cannot, but they do not include any information regarding actual usage of L2 systems. This may reduce both the estimated effect size for relevant technologies as well as the statistical power of hypoth- esis tests. In contrast, use/non-use studies directly compare crash rates for the same vehicles when they are or are not using L2 technology. However, publicly available crash databases do not generally contain sufficient infor- mation to determine either the state of an L2 system at the time of a crash or the amount of usage (exposure) for the L2 system (e.g., NHTSA, 2022).
Two studies have been published using manufacturer provided data regarding crashes with L2 systems in use versus manual driving. Leslie et al. (2022) carried out a comparison of crash involvement for vehicles using or not using Super Cruise on “Super Cruise Compatible Roads” to expected usage rates of Super Cruise based on telemetry data. However, the dataset was very small (8 relevant crashes), and the results were not statistically sig- nificant. Tesla publishes crash rates for its vehicles with Autopilot in use and not in use (Tesla, 2023), but the difference in the raw rates is not entirely
Assessing the Impact of Driver Assistance Technology 327
attributable to the effect of Autopilot. This is because Autopilot is generally used on highways, where the crash rate is known to be lower than for driving in general. Goodall (2023) developed a method for “normalizing” the crash rates that Tesla publishes in a manner that controls for these differences in road usage.
We have applied the method from Goodall (2023) to the most recent crash data published by Tesla, adjusting the non-Autopilot crash rate to reflect driv- ing exclusively on highways. This comparison may slightly overestimate the comparable Autopilot crash rate, as no adjustment is made to account for non-highway use included in the rate. The results are shown in Figure 1 and Table 1. Throughout the reporting period, the Autopilot crash rate is consis- tently lower than the highway only non-Autopilot rate, and the regression analysis indicates that this difference is statistically significant. In addition, the Autopilot crash rate exhibits a decreasing trend throughout the study period while the non-Autopilot rate appears to be constant.
Figure 1: A comparison of highway crash rates for Tesla vehicles with L2 technology in use (green) and not in use (blue) using methodology from Goodall (2023).
While studies of L2 systems frequently exhibit indeterminate results due to small sample sizes, in all cases where statistically significant differences are observed, they indicate a reduction in crash rates associated with L2 systems.
Table 1. Results of regression fit to normalized Tesla crash data.
This review of non-crash and crash assessments of L2 systems provides valu- able insights into both types of study. Crash assessments directly measure the real-world effects of L2 systems. However, they may suffer from lack of data availability. The lack of statistical significance for all results of the Super Cruise study (Leslie et al., 2022), as well as multiple results in the HLDI studies (HLDI, 2017; HLDI, 2021a; HLDI, 2021b; HLDI, 2022), reflect the challenges in acquiring sufficient data to evaluate the real-world effects of this relatively new type of technology.
It is also notable that both studies involving comparisons of crash rates with systems active vs inactive, required information from the manufacturers, namely, Tesla (Goodall, 2023) and GM (Leslie et al., 2022), and depended on vehicle telemetry. Equip/non-equip studies are typically carried out when technology is introduced, and they become less viable as technology becomes widespread. Therefore, the presence or absence of telemetry as part of vehicle design may be a critical limiting factor in future studies of L2 technology.
Non-crash assessments can provide valuable insights into design charac- teristics of systems and can be carried out early in a system’s development and deployment cycle when real-world crash information is not yet avail- able. However, these studies must be interpreted with caution, as the effects predicted based on proxies for crash risk may not be reflected in real-world assessments. It is particularly notable that the non-crash assessments of L2 systems are mixed, while all crash assessments to date point toward crash risk reduction associated with these systems.
One possible explanation for these mixed results stems from small sample sizes (e.g., Endsley, 2017 (1 participant); Reagan et al., 2021 (10 partici- pants)), or study populations that are not representative of the general popu- lation of L2 system users. Some studies that are ostensibly about L2 systems combine true L2 (ACC and lane centering) with L1 systems (ACC + reactive lane keeping), limiting the ability to generalize the conclusions when applied solely to either technology type (e.g., Dunn et al., 2009).
Driver familiarity with system performance may also be an important fac- tor. Trust level (high or low) in automation can lead to changes in driver behavior and engagement with the driving task, including mis- or nonuse of technology (e.g., Lee & See, 2004; Parasuraman & Riley, 1997). For exam- ple, Reagan et al. (2021) found that some of their participants reduced use of L2 technology over time, which they conclude may have been due to reduc- tion of trust, and therefore were not included in their analysis of in the second half of the study. The authors concluded that this may have contributed to greater differences between the first and second half of the study. Experienced users may also adapt their level of engagement to the particular driving situ- ation when the system is in use (Lin et al., 2018; Shutko et al., 2018). This is also consistent with research that suggests that Tesla Autopilot users report that they are comfortable looking away from the road “somewhat” longer even though they also report feeling responsible for the operation of their vehicle, believe that Autopilot requires supervision, and tend to have similar eyes-on-road behavior to manual driving (Shutko et al., 2018).
Assessing the Impact of Driver Assistance Technology 329
CONCLUSION
Overall, the literature indicates that driver behavior may be altered by the use of L2 systems, but there is no indication at this time that these changes have increased aggregate crash risk. In fact, all statistically significant results found in the literature point to a reduction in crash risk associated with these systems. There are also clear opportunities for improvements to non-crash research, such as ensuring adequate sample sizes and making clear distinc- tions between L2 and other types of driver assistance technology. Non-crash assessments will continue to play a key role in evaluating and improving partial automation systems, and continuous review and improvement to ensure that these results are generalizable to real-world driving are critically important.
REFERENCES
American Automobile Association (2022). Effectiveness of driver monitoring systems. Beller, J., Heesen, M. and Vollrath, M. (2013) ‘Improving the driver–automation interaction: An approach using automation uncertainty’, Human Factors, 55(6), pp. 1130–1141. Biondi, F. N. et al. (2018) ‘80 MPH and out-of-the-loop: Effects of real-world
semi-automated driving on driver workload and arousal’ in (ed.) Proceedings of the human factors and ergonomics society annual meeting. Los Angeles: SAGE Publications, 62(1), pp. 1878–1882. Cicchino, J. B. (2017) ‘Effectiveness of forward collision warning and autonomous emergency braking systems in reducing front-to-rear crash rates’, Accident Anal- ysis & Prevention, 99, pp. 142–152. Cunningham, M. L., Regan, M. A. and Imberger, K. (2017) ‘Understanding driver
distraction associated with specific behavioural interactions with in-vehicle and portable technologies’, Journal of the Australasian College of Road Safety, 28(1), pp. 27–40. De Winter, J. C., Happee, R., Martens, M. H. and Stanton, N. A. (2014) ‘Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence’, Transportation Research Part F: Traffic Psychology and Behaviour, 27, pp. 196–217. Dingus, T. A. et al. (2016) ‘Driver crash risk factors and prevalence evalua-
tion using naturalistic driving data’, Proceedings of the National Academy of Sciences, 113(10), pp. 2636–2641. Dunn, N., Dingus, T. and Soccolich, S. (2019) ‘Understanding the impact of technology: Do advanced driver assistance and semi-automated vehicle systems lead to improper driving behavior’, Virginia Tech Transportation Institute, American Automobile Association, Washington, DC. Endsley, M. R. (2017) ‘Autonomous driving systems: A preliminary naturalistic
study of the Tesla Model S’, Journal of Cognitive Engineering and Decision Making, 11(3), pp. 225–238. Highway Loss Data Institute. (2017). ‘Tesla Model S driver assistance technologies’, Loss Bulletin, 34(30), pp. 1–13. Highway Loss Data Institute. (2021). ‘2013–17 BMW collision avoidance features -
a 2021 update’, Loss Bulletin, 38(26), pp. 1–30.
Highway Loss Data Institute. (2021). ‘Advanced driver assistance systems on the 2017–19 Nissan Rogue’, Loss Bulletin, 38(1), pp. 1–16. Highway Loss Data Institute. (2022). ‘Audi’s traffic jam assist’, Loss Bulletin, 39(3),
pp. 1–9. Gershon, P. et al. (2019) ‘Distracted driving, visual inattention, and crash risk among teenage drivers’, American Journal of Preventive Medicine, 56(4), pp. 494–500. Goodall, N. (2023) ‘Normalizing crash risk of partially automated vehicles under
sparse data’, Journal of Transportation Safety & Security, pp. 1–17. Leslie, A. J. (2022) ‘Analysis of Field Effects Associated with the GM Super Cruise System’, UMTRI. Lin, R., Ma, L. and Zhang, W. (2018) ‘An interview study exploring Tesla drivers’
behavioural adaptation’, Applied Ergonomics, 72, pp. 37–47. Morando, A., Gershon, P., Mehler, B. and Reimer, B. (2020) ‘Driver-initiated Tesla Autopilot disengagements in naturalistic driving’ (ed.) in 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applica- tions, pp. 57–65. Mueller, A. S. et al. (2021) ‘Bears in our midst: familiarity with Level 2 driving
automation and attending to surprise on-road events’, Transportation Research Part F: Traffic Psychology and Behaviour, 90, pp. 500–511. National Highway Traffic Safety Administration (2022) Standing General Order on Crash Reporting for Automated Driving Systems. Available at: https://www.nhts a.gov/document/sgo-crash-reporting-adas-ads (Accessed: 11 November 2023). National Highway Traffic Safety Administration (2022) Automated Vehicle Safety.
Available at: https://www.nhtsa.gov/technology-innovation/automated-vehicles-s afety (Accessed: 27 November 2023). Reagan, I. J. et al. (2021) ‘Disengagement from driving when using automation during a 4-week field trial’, Transportation Research Part F: Traffic Psychology and Behaviour, 82, pp. 400–411. SAE On-Road Automated Driving Committee. (2021) Taxonomy and definitions
for terms related to driving automation systems for on-road motor vehicles (SAE J3016) SAE International. Seppelt, B. D. and Victor, T. W. (2016) ‘Potential solutions to human factors challenges in road vehicle automation’, Road Vehicle Aautomation, 3, pp. 131–148. Shutko, J., Osafo-Yeboah, B., Rockwell, C. and Palmer, M., 2018. Driver
Behavior While Operating Partially Automated Systems: Tesla Autopilot Case Study (No. 2018-01-0497). SAE Technical Paper. Tesla. Tesla Vehicle Safety Report. Available at: https://www.tesla.com/VehicleSafet yReport (Accessed: 19 May 2023). Tijerina, L. (1999) ‘Driver eye glance behavior during car following on the road’, SAE
Transactions, pp. 2369–2383. Wang, J. S. (2019) Target crash population for crash avoidance technologies in passenger vehicles (Report No. DOT HS 812 653). Washington, DC: National Highway Traffic Safety Administration. Victor, T. W. et al. (2018) ‘Automation expectation mismatch: Incorrect pre-
diction despite eyes on threat and hands on wheel’, Human Factors, 60(8), pp. 1095–1116.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Factors of Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-24-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional