Kelly Steelman, Holly Handley (2022). A Primer on the Human Readiness Level Scale (ANSI/HFES 400-2021). In: Human Factors in Transportation. AHFE Open Access, vol. 60, pp. 184–191. https://doi.org/10.54941/ahfe1002448
The Human Readiness Level (HRL) Scale is a simple 9-level scale for evaluating, tracking, and communicating the readiness of a technology for safe and effective human use. It is modeled after the well-established Technology Readiness Level (TRL) framework that is used throughout the government and industry to communicate the maturity of a technology and to support decision making about technology acquisition. The TRL framework, however, does not consider the technology’s readiness for human use. As human error is implicated in 60-90% of incidents and accidents across a range of domains, it is critical to consider a technology’s human readiness alongside its technological maturity. The HRL scale was developed to address this need and to complement and supplement the TRL. In 2019, Drs. See (Sandia National Laboratories) and Handley (HFES Science Policy Fellow; Old Dominion University) formed a working group of practitioners across DoD, industry, and academia to mature the HRL concept; evaluate its usability, reliability, and validity; and develop a standard. The resulting ANSI/HFES 400-2021 Standard defines the HRL scale and provides guidance on how to apply them within a system development process. The standard provides questions to guide evaluation activities, with exit criteria and examples of the supporting evidence required to progress from one level to the next. The HRLs map on to three phases of the development process: Basic Research and Development (HRL 1-3), Technology Demonstrations (HRL 4-6), and Full-Scale Testing, Production, and Deployment (HRL 7-9):HRL 1: Basic principles for human characteristics, performance, and behavior observed and reportedHRL 2: Human-centered concepts, applications, and guidelines definedHRL 3: Human-centered requirements to support human performance and human-technology interactions establishedHRL 4: Modeling, part-task testing, and trade studies of human systems design concepts and applications completedHRL 5: Human-centered evaluation of prototypes in mission relevant part-task simulations completed to inform designHRL 6: Human systems design fully matured and demonstrated in a relevant high-fidelity, simulated environment or actual environmentHRL 7: Human systems design fully tested and verified in operational environment with system hardware and software and representative usersHRL 8: Human systems design fully tested, verified, and approved in mission operations, using completed system hardware and software and representative usersHRL 9: System successfully used in operations across the operational envelope with systematic monitoring of human system performance The proposed presentation is part of an HFES initiative to socialize HRLs within the government, industry, and academia. The presentation will provide concrete examples drawn from the transportation sector to illustrate how HRLs can be applied throughout a human systems integration process.ReferencesHFES/ANSI (2021). Human Readiness Level Scale in the System Development Process (ANSI/HFES 400-2021). Retrieved from https://www.hfes.org/publications/technical-standardsSalazar, G., See, J. E., Handley, H. A., & Craft, R. (2020, December). Understanding human readiness levels. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 64, No. 1, pp. 1765-1769). Sage CA: Los Angeles, CA: SAGE Publications.See, J. E. (2021). Human Readiness Levels Explained. Ergonomics in Design, 10648046211017410.
The Human Readiness Level (HRL) scale is a nine-level scale designed to sup- plement the Technology Readiness Level (TRL) scale widely used throughout U.S. Department of Defense (DoD) (Government Accountability Office, 2020). While the TRLs assess the maturity of a technology before and after it is integrated into a developing system, the HRLs were developed to focus on the readiness of the system for use by human operators and maintainers (Handley & Savage-Knepshield, 2020).
The goal of the HRL scale is to indicate the state of integration within the system with respect to humans and technology. The evaluation of the HRL focuses on the system’s readiness for human usability and incor- porates the human element as part of the systems lifecycle development. HRLs have the potential to minimize the cost of design changes, through early identification of human issues, and reduce human error in fielded systems by tracking the mitigation of identified issues through subsequ- ent HRL assessments (Salazar et al., 2021). The HRL scale shifts attention from lagging indicators of human readiness, such as human error in fiel- ded systems, to leading indicators – namely, the evidence-based measures of system usability readiness derived through the application of the HRL scale (See, 2021).
A Primer on the Human Readiness Level Scale (ANSI/HFES 400-2021) 185
The HRLs were first proposed as a measure to communicate on the rea- diness of a system for use by human operators and maintainers in 2010 (Acosta, 2010). Additional progress was made by a DoD Human Systems Integration (HSI) working group that refined the scale with more detailed definitions and descriptions (Phillips, 2015). In 2019, a new HRL working group consisting of a cross section of DoD, industry and academic experts served as the writing committee for an HRL standard; the Human Factors & Ergonomics Society (HFES)/American National Standards Institute (ANSI) 400-2021 “Human Readiness Level Scale in the System Development Pro- cess” was approved in 2021 (Handley, 2021). The standard defines the nine levels of a Human Readiness Level (HRL) scale and provide guidance for their application (HFES/ANSI, 2021). The nine HRLs are defined as follows:
HRL1: Basic principles for human characteristics, performance, and behavior observed and reported
HRL2: Human-centered concepts, applications, and guidelines defined
HRL3: Human-centered requirements to support human performance and human-technology interactions established
HRL 4: Modeling, part-task testing, and trade studies of human systems design concepts and applications completed
HRL 5: Human-centered evaluation of prototypes in mission-relevant part-task simulations completed to inform design
HRL 6: Human systems design fully matured and demonstrated in a relevant high-fidelity, simulated environment or actual environment
HRL 7: Human systems design fully tested and verified in operational environment with system hardware and software and representative users
HRL 8: Human systems design fully tested, verified, and approved in mission operations, using completed system hardware and software and representative users
HRL 9: System successfully used in operations across the operational envelope with systematic monitoring of human-system performance Human readiness levels (HRLs) were designed to mirror the TRL’s and their familiar nine-level scale. However, rather than focusing solely on tech- nology, HRLs consider the readiness of the technology or system to support the human (See et al., 2019). They provide a set of supporting questions at each level that describe the fundamental activities to ensure human conside- rations are appropriately addressed and an objective, structured approach for assessing a system’s readiness for human use. The HRL scale addresses a need in both the DoD and industry for a metric to convey Human System Integra- tion (HSI) maturity and related program risk (Schwartz & Dodson, 2021). HSI practitioners can utilize the HRL standard to demonstrate the importa- nce of HSI investment, quantify the success of their efforts accurately and reliably, and gain leadership buy-in.
Handley and Savage-Knepshield (2020) examined the suitability of HRL use for an army acquisition program. They reviewed previous HSI asses- sments that tracked the program’s progress across acquisition milestones and using the HRLs identified which issues persisted over time. Their analysis
186 Steelman and Handley
mapped documented HSI issues to the HRL underlying questions. Results demonstrated that when issues were resolved during system development, the HRL increased, and when they remained unresolved, the HRL remained flat thereby providing evidence that HRLs could be used to convey the rea- diness of a system to interface with a human operator (Savage-Knepshield, Hernandez & Sines, 2021). The use of the assigned HRL rating readily com- municates the readiness of a technology or system for human use to program and acquisition managers to support decisions regarding future program direction and allocation of funding (See, 2021).
APPLYING ANSI/HFES 400-2021 TO ADVANCED DRIVING SYSTEM (ADS) TECHNOLOGY The ANSI/HFES 400-2021 standard (HFES/ANSI, 2021) includes an appen- dix with evaluation guidance that includes examples of representative eva- luation activities, criteria for exiting one level and transitioning to the next, and descriptions of the type of supporting evidence that should be provided to indicate that these criteria have been met.
Below we outline representative evaluation activities at each HRL, using Honda’s (2021) Traffic Jam Pilot (TJP) as an example. TJP is an SAE Level 3 feature that allows the system to fully take over the driving task in congested traffic. At SAE Level 3, also referred to as Conditional Driving Automa- tion, an ADS is capable of performing all routine aspects of the driving task within a specific operational design domain (ODD), provided the fallback- ready driver is receptive to requests to intervene and able to respond in the event of an automation failure (SAE, 2021). For TJP, the ODD includes slow-speed driving in a single lane. The system monitors the environment and provides both lateral and longitudinal control under this narrow set of roadway conditions and speeds. When the system is active, the driver is not required to monitor the system and may use the infotainment system to engage in other tasks like watching movies or navigation. The system uses vibration in the seatbelt to alert the driver to regain control of the vehicle and colored lights illuminate the steering wheel to indicate which advanced driving feature is activated. Cameras monitor the driver at all times, and the system is able to make an emergency stop if the driver is non-responsive.
In Table 1, we present the exit criteria and a sample evaluation question, drawn from Appendix C of ANSI/HFES 400-2021. We’ve listed only one of the many recommended evaluation questions included in the standard appendix. At all HRLs, human system experts are responsible for specifying which of the recommended evaluation questions are relevant and whether those questions have been answered positively. The examples below are based on publicly available information about the TJP system. TJP technology pre-dates the ANSI/HFES 400-2021 standard, and the authors do not have specific knowledge about Honda’s approach to assessing human readiness. However, the example does provide a representative use case for how HRLs could have been applied within the development process.
A Primer on the Human Readiness Level Scale (ANSI/HFES 400-2021) 187
Table 1. HRL evaluation questions and examples for TJP system.
Table available in the original PDF paper.
HRL 1: Basic Human Research Exit Criterion: In order to exit HRL 1 and advance to the next level, human involvement relevant to the developing concept or proposed applications should be identified and characterized at a basic level. Sample Evaluation Question C.1.1.1: Have key human behaviors, capabilities, and limitations been identified? Example: HSL 1 requires initial consideration of both the activities that the driver will be allowed to engage in while TJP is engaged and the human performance characteristics that are relevant to capturing the driver’s attention to re-engage in the driving task. TJP uses haptic feedback in the seatbelt to alert the driver of the need to resume the driving task. Documentation might include studies on the efficacy of various haptic warning types.
HRL 2: Human Centered Design Guidelines Exit Criterion: In order to exit HRL 2 and advance to the next level, key human-centered design and training principles, standards, and guidance for human interaction with the technology or for human performance augmentation should be established. Sample Evaluation Question C.2.1.2: Have key human-centered design principles, standards, and guidance been established? Example: HSL 2 requires the identification of relevant design standards. As TJP uses haptic feedback in the seatbelt to capture driver attention and to initiate a driver takeover, this would include the identification of haptics-related ISO standards (e.g., ISO 9241-910, Ergonomics of human-system interaction — Part 910: Framework for tactile and haptic interaction and ISO 9241-920, Ergonomics of human-system interaction — Part 920: Guidance on tactile and haptic interactions.)
HRL 3: Human Centered Requirements Exit Criterion: In order to exit HRL 3 and advance to the next level, required human-centered analyses are completed and human-centered requirements and KPPs are identified and flowed into high-level system requirements. Sample Evaluation Question C.3.1.5: Have situation awareness information flow and sharing requirements across teams of human or automated system components been identified? Example: The TJP feature is available only under a specific ODD when other Advanced Driver Assistance Systems (ADAS; e.g., Adaptive In-Lane Driving and active lane change) are engaged. When ADAS are engaged, the driver is responsible for monitoring. When Level 3 features are engaged, the system is responsible for all aspects of the driving task. Transitioning between ADAS and the more highly automated Level 3 driving features requires communication between the automated system components and the human user. During HRL3, the information sharing requirements regarding the driver’s role, the automation mode, and the actions that the vehicle is taking (changing lanes) should be identified.
Continued.
188 Steelman and Handley
Table 1. Continued.
Table available in the original PDF paper.
HRL 4: Part-Task Testing Exit Criterion: In order to exit HRL 4 and advance to the next level, human interactions and performance are evaluated and characterized, using analytical tools, modeling, and part-task testing with rapid prototypes and mockups. Task analyses and human-machine function allocations are updated. Human systems designs, HSI domain strategies, and human performance results must be deemed satisfactory by human systems experts. Sample Evaluation Question C.4.1.12: Have relevant human performance data been collected and evaluated to determine whether human performance metrics are successfully met, based on modeling and part-task testing? Example: HRL 4 evaluation activities may include laboratory-based simulator studies to assess the effectiveness of visual, auditory, and haptic alerts on takeover behavior on representative tasks and fallback condition.
HRL 5: Prototypes in Mission-Relevant Simulations Exit Criterion: In order to exit HRL 5 and advance to the next level, human interactions and performance with developing system prototypes are evaluated and characterized in the context of more realistic mission-relevant part-task simulations with higher fidelity and users independent from the design team. Human performance results must be deemed satisfactory by human systems experts. Sample Evaluation Question C.5.1.12: Have relevant human performance data been collected and evaluated to determine whether human performance metrics are successfully met, based on prototype testing in mission-relevant part-task simulations? Example: HRL 5 evaluation activities may include the use of a fixed based simulator to assess takeover performance, fallback condition, situation awareness, and system usability during the transitions between ADAS and the more highly automated Level 3 driving features.
HRL 6: Human Systems Design Fully Matured Exit Criterion: In order to exit HRL 6 and advance to the next level, human interactions and performance are evaluated and characterized in the context of high-fidelity simulated or actual environments with a functional and realistic prototype, representative users, and the full range of usage scenarios and tasks. Human performance results must be deemed satisfactory by human systems experts. Sample Evaluation Question C.6.1.2: Has the full range of user scenarios and tasks been tested in high-fidelity simulated or actual environments? Example: HRL 6 evaluation activities may include the use of a high-fidelity simulator or a test vehicle on a closed track to assess the effects of extended use of the infotainment system and other personal technologies on situation awareness and takeover performance.
HRL 7: Human Systems Design Fully Tested Exit Criterion: In order to exit HRL 7 and advance to the next level, human interactions and performance are evaluated and characterized in an operational environment with the final development system, representative users, and the full
Continued.
A Primer on the Human Readiness Level Scale (ANSI/HFES 400-2021) 189
Table 1. Continued.
Table available in the original PDF paper.
range of usage scenarios and tasks. Human performance results must be deemed satisfactory by human systems experts. Sample Evaluation Question C.7.1.11: Have relevant human performance data been collected and evaluated to determine whether human performance metrics are successfully met, based on testing with the final development system in an operational environment? Example: HRL 7 evaluation activities may include data collection using the final developed system on a closed test track, with representative users and a range of scenarios.
HRL 8: Human Systems Design Verification and Approval Exit Criterion: In order to exit HRL 8 and advance to the next level, human interactions and performance are evaluated and characterized with the production system in mission operations during the full range of usage scenarios and tasks completed by representative users. Human performance results must be deemed satisfactory by human systems experts. Sample Question C.8.1.1: Has the range of user scenarios and tasks been tested with the production system in mission operations? Example: Prior to commercial release, Honda’s TJP-equipped Legend sedan was tested on over 1.3 million km of road (Honda, 2021). HRL 8 evaluation activities would require that this testing and evaluation include representative users and a full range of usage scenarios, including emergency and other non-normal events. During these tests, human performance data should be collected and documented, and any human performance issues should be resolved.
HRL 9: Operational Use and Monitoring Exit Criterion: Like TRL 9, HRL 9 does not have a distinct or well-defined conclusion. A fielded system continues to be monitored, tested, and evaluated to ensure it supports the mission as intended. Sample Evaluation Question C.9.1.5: Is user training for operation of the fielded system being evaluated for required modifications? Example: In 2021, Honda’s TJP-equipped Legend sedan was released in limited quantities in Japan. HRL 9 evaluation activities would include ongoing human systems monitoring to ensure that the intended levels of performance are achieved. Human systems experts should be involved in the design of driver training offered through the manufacturer or through leasing agencies.
CONCLUSION
HRLs are an important tool for evaluating and communicating about a tech- nology’s readiness for human use. Rapid advances in automated driving systems (ADS) have motivated the development of shared language for clas- sifying these systems and guidance to ensure clear and consistent safety stan- dards and communication among stakeholders; HRLs are another important contribution to this effort. Further, ANSI/HFES 400-2021 provides a cri- tical tool for supporting the National Highway Traffic Safety Association (NHTSA) Automated Vehicle (AV) 2.0’s recommended self-assessment. This (currently) voluntary self-assessment is designed to support those involved in
190 Steelman and Handley
ADS testing and deployment to demonstrate to the public that they are: “(1) considering the safety aspects of ADSs; (2) communicating and collaborating with DOT; (3) encouraging the self-establishment of industry safety norms for ADSs; and (4) building public trust, acceptance, and confidence through transparent testing and deployment of ADSs” (NHTSA, 2017, p. 16). Achie- ving these goals, however, requires a common language for describing human readiness and clear guidance on the types of milestones that must be met before an automated driving feature is deemed ready for operational use. In support of this, the Human Factors and Ergonomics Society recently released Policy Statement on Autonomous and Semi-Autonomous Vehicles (2.0) that advocates for NHTSA to require human factors assessment through all steps of the design, development, and deployment and that HRLs are adopted as a mechanism for assessing and communicating AV readiness for human use (HFES, 2022).
ACKNOWLEDGMENT
The authors thank Dr. Michael Clamann for his feedback on this manuscript.
REFERENCES
Acosta, H. (2010, May 9–13). Human readiness levels: Implementing HSI – Connecting some dots [Panel discussion]. 81st Annual Scientific Meeting of the Aerospace Medical Association, Phoenix, AZ, United States. Government Accountability Office. (2020). Technology readiness assessment guide:
Best practices for evaluating the readiness of a technology for use in acquisition programs and projects (Report No. GAO-20-48G). https://www.gao.gov/assets /710/704769.pdf Handley, H. (2021). Editorial - Introduction to the Special Issue on Human Readiness Levels, Ergonomics in Design: The Quarterly of Human Factors Applications, 29(4) p. 4. Handley, H., & Savage-Knepshield, P. (2020, October 5–9). Evaluating the utility
of human readiness levels (HRLs) with human systems integration assessments (HSIAs) [Paper presentation]. Human Factors and Ergonomics Society 64th International Annual Meeting, Chicago, Illinois, United States. HFES/ANSI (2021). Human Readiness Level Scale in the System Development Process. (HFES/ANSI 400-2021). Retrieved from https://www.hfes.org/publications/ technical-standards HFES (2022). Policy Statement: Autonomous and Semi-Autonomous Vehicles (2.0).
Retrieved from https://www.hfes.org/Public-Policy/Policy-Statements Honda (2021, March 4). Honda launches next generation Honda SENSING Elite safety system with Level 3 automated driving features in Japan. https://bit.ly/3G NuEoH International Organization for Standardization. (2009). Ergonomics of human-
system interaction — Part 920: Guidance on tactile and haptic interactions (ISO Standard No. 9241-920). Retrieved from https://www.iso.org/standard /42904.html International Organization for Standardization. (2011). Ergonomics of humansystem interaction — Part 910: Framework for tactile and haptic interaction (ISO Standard No. 9241-910). Retrieved from https://www.iso.org/standard /51097.html
A Primer on the Human Readiness Level Scale (ANSI/HFES 400-2021) 191
National Highway Traffic Safety Administration. (2017). Automated driving systems 2.0: A vision for safety. Washington, DC: US Department of Transportation, DOT HS, 812, 442. Phillips, H. (2015, May 4–8). DOD HFE TAG: Human readiness level (HRL) wor-
king group. [Paper presentation]. DOD HFE TAG 69th Meeting, Orlando, FL, United States. SAE. (2021). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (Standard No. J3016). Retrieved from https: //www.sae.org/standards/content/j3016_202104/ Salazar, G., See, J. E., Handley, H. A. H., & Craft, R. (2021). Understanding human
readiness levels. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 64(1), 1765–1769. https://doi.org/10.1177/1071181320641427 Savage-Knepshield, P., Hernandez, C, & Sines, S. (2021). Exploring the Synergy Between Human Systems Integration and Human Readiness Levels: A Retro- spective Analysis, Ergonomics in Design: The Quarterly of Human Factors Applications, 29(4) pp. 16–24. Schwartz, B. & Dodson, J. (2021). Human Readiness Levels Promote Effective System Integration, Ergonomics in Design: The Quarterly of Human Factors Applications, 29(4) pp. 11–15 See, J. (2021). Human Readiness Levels Explained, Ergonomics in Design: The
Quarterly of Human Factors Applications, 29(4) pp. 5–10. See, J. E., Handley, H., & O’Neil, M. (2019, October 15–17). History and current status of human readiness levels [Working Group In-Person Meeting 1]. Joint HRL Working Group Meeting, Sandia National Laboratories, Albuquerque, New Mexico. https://www.osti.gov/servlets/purl/1766758
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-36-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional