Kelly Dickerson, Michael Hildebrandt, Niav Hughes, Heather Watkins, John Grasso, Isaac Wang, Peter Gerhardstein (2024). Technology Affinity in the Nuclear Domain: An Initial Assessment of Operator Attitudes Towards Technology. In: Usability and User Experience. AHFE Open Access, vol. 156, pp. 12–20. https://doi.org/10.54941/ahfe1005410
Many nuclear power plants in the U.S. and worldwide are currently monitored and controlled via control rooms outfitted primarily with analog instrumentation and controls (I&C). Due to obsolescence issues with parts for analog I&C along with a desire to leverage efficiency gains associated with digital systems, there is an increasing focus on modernization activities for both safety and non-safety systems. Digital modernization can include the addition of digital instrumentation, sensors, and control systems; some that are visible to operators and some that are not. Operator-facing digital controls include the human system interface (HSI). The U.S. Nuclear Regulatory Commission (NRC) is responsible for reviewing the safety of control room designs at nuclear power plants. Part of the NRC’s review process includes an assessment of how changes to control room HSIs impact usability and support safe operations. Safety and usability are often assessed from a workload, performance, and situation awareness perspective. Recent research into the factors that influence system evaluation found that users with high affinity for technology (i.e., a user with generally positive affect towards technology) perform differently on usability tests than users with less affinity towards technology. The present study had two aims: (1) to evaluate the potential usefulness of a recently developed affinity for technology interaction (ATI) survey for usability and human factors research in the nuclear domain and (2) to determine if affinity, as measured by the ATI, is related to perceived system usability and operator performance during simulated digital control room tasks. The first phase of the research was the collection of ATI data from students and a group of control room operators to establish potential baseline differences in technology affinity between populations. The second phase of the research was the collection of ATI and system usability survey (SUS) data from a small crew of operators to determine if there was an association between technology affinity and perceived system usability (assessed via the system usability survey [SUS]) during performance of simulated operational scenarios. The results are discussed in the context of how future regulatory guidance related to operator-facing digital systems may need to consider additional usability metrics as well as the broader safety implications for individual differences in technology affinity resulting in potentially biased usability ratings on scales like the SUS.
Technology Affinity in the Nuclear Domain: An Initial Assessment 13
INTRODUCTION
The worldwide production and use of nuclear power as an energy source is regulated by a regulatory body in each country. The U.S. Nuclear Regula- tory Commission (NRC) regulates the commercial use of nuclear power in the United States. Part of that regulatory role is to review and oversee the design, construction, operation and decommissioning of commercial nuclear power plants. The review and oversight of commercial nuclear power plants includes examining the human factors engineering (HFE) processes used by licensees during the design and testing of their control rooms. The way these reviews are conducted is described by The Standard Review Plan (NUREG- 0800), which provides an overview of all the guidance staff may use during the review process. Chapter 18 of NUREG-0800 specifically addresses HFE. There are two other NUREGs that are used by NRC staff and licensees that primarily address HFE. NUREG-0700, which contains control room design guidance and NUREG-0711, which contains the HFE criteria that are commonly used by licensees as they develop their HFE plans but was pri- marily developed to be used by staff when they review those plans for safety assurance.
NUREG-0711 recommends that licensees have a plan for the assessment of HFE of their control room designs and that these assessment techniques include consideration of operator workload and performance. NUREG- 0700 provides guidance for the review of workstations containing primarily computer-based HSIs which might include reviewing an assessment usability of hand-held devices, for example.
Currently, the nuclear industry is pursuing modernization activities that involve transitioning control room I&C from fully analog to hybrid analog- digital, and with the advent of advanced reactors, future control rooms are expected to be partially or even fully digital. There are several drivers for this kind of modernization, including availability and cost of analog com- ponents, the desire for improved efficiencies in operator performance, and the perception that digital displays will lead to better operator performance. However, digital control rooms can induce just as much cognitive demand as analog control rooms. Hughes, Reinerman-Jones, Lin, Matthews, Barber, and Dickerson (2023) found that when fully analog and fully digital con- trol room simulators were directly compared, global measures of workload indicated equivalent load between the two types of control rooms, but when reviewing workload subscales, it appeared that the sources of demand shifted depending on the type of control room. While observing shifting sources of demand, useful instruments like the NASA-TLX (Hart, 1988, 2006) are not able to provide an explanatory foundation for why this effect occurred. In other words, it could be something induced by the changes to the display of system information, or it could be related to operator attitudes about technology having a differential influence on perceived workload. The impli- cations of some of these findings should be considered in the development of guidance for modern control room design and review.
Measuring Attitudes Towards Technology: The Affinity for Technology Interactions (ATI) Scale The ATI scale is a 9-item survey that was designed to measure a persons’ ten- dency to engage in “intensive interactions with technology” or avoid these kinds of interactions (Heilala et al., 2023; Franke, Attig & Wessel, 2019). ATI represents a stable personality trait that reflects one’s intrinsic motivation to actively engage in intensive technology interaction, which may manifest as a tendency to explore and understand systems. This trait is rooted in the construct of “need for cognition,” indicating an individual’s inherent drive to engage in cognitive activities that require thinking. High need for cogni- tion suggests a propensity to seek, evaluate, and integrate multiple sources of information to make sense of and solve problems in one’s surroundings. Simi- lar to the assessment for need for cognition, the ATI scale evaluates individual differences in the inclination to approach situations and activities. While the need for cognition assessment examines differences in engaging in cognitively demanding tasks, the ATI scale captures variations in engaging in intensive technology interaction.
Research suggests that the ATI personality dimension is associated with more effective coping strategies for technology, including problem-solving and process learning (Franke et al., 2019). Understanding this user attribute is important as it can skew study results, particularly those that have smaller participant samples. For example, users who score high in technology affinity may also tend to be people who enjoy actively engaging to understand tech- nical systems and therefore are better able to cope with usability issues (see Attig, Wessel & Franke, 2017). This means that a high affinity user could perceive a system to be more usable than a user with low technology affinity assessing the same system. There is also a suggestion that high affinity individ- uals are better able to adapt to changes in highly technical systems, meaning they require less training to reach proficiency and make fewer errors.
Attig, Wessel, and Franke (2018) validated these perspectives empirically by looking at the potential link between the ATI and the system usability survey (SUS). Their study measured performance, ATI, and SUS data from participants using four different types of interactive websites (web-based newspaper, a route planner, a shopping site, and a discussion forum). Mean- ingful correlations were found between the ATI and SUS for the online route planning and the shopping website, but not for the less interactive sites (news- paper and discussion forum), suggesting that for interactive systems with high navigational demands that the ATI may uncover users who are likely to provide high usability ratings. Furthermore, attitudes towards technology have also been found to influence the perceived usability of several other types of digital devices and tools, such as activity trackers (Hofbauer & Rodriguez, 2022), traceable AI systems (Schrills & Franke, 2022), educa- tional apps (Winkler, Akyildiz & Herczeg, 2019) and smart car interfaces (Avramidis et al., 2021). Despite this awareness of the influence of attitudes towards technology in the consumer technology domain, there are no exam- ples in the literature of the role that affinity for technology has on perceived
Technology Affinity in the Nuclear Domain: An Initial Assessment 15
system usability and operator performance for safety-related systems or sys- tems with a safety-related component like nuclear power plants. However, there is broad research support for the notion that other individual differ- ences in attitudes towards technology can predict performance outcomes. For example, researchers have found that dispositional trust, which refers to expectations of trustworthiness and is often measured as automation com- placency potential, can significantly impact outcomes in human-automation interactions (Merritt, Unnerstall, Lee & Huber, 2015). Findings demonstrate that detection of automation failures is significantly lower when individuals score higher on measures of complacency (Muir, 1987; Parasuraman, Molloy & Singh, 1993; Parasuraman and Riley, 1997). Other research demon- strates that positive beliefs about automation’s performance, as indicated by the Perfect Automation Schema, are associated with increased sensitivity to automation errors (Merritt et al., 2015). Individuals with higher expecta- tions of technology experience greater trust breakdowns when system errors violate their mental model of the technology. These situations can lead to unanticipated system usage behaviors (i.e., over-trust leading to misuse or under-trust leading to disuse), which become prominent in instances where uncertainty is high. Furthering our understanding of attitudes towards tech- nology is necessary, as they can predict performance outcomes, which are crucial for safe operations.
THE PRESENT STUDY The purpose of this study was to assess the usefulness of the ATI for applica- tion in the nuclear domain in the context of hybrid or fully digital control room environments. To that end, this study compares samples of univer- sity students to nuclear power plant control room operators. One general concern about the ATI is that, for individuals who work in highly techni- cal fields, affinity can be strongly (and potentially differently) influenced by personal and professional interactions with technology. This study examines that potential by using the original, “unmodified”English language ATI scale and a “modified” version of the scale which focuses only on questions that seem more relevant to professional interactions with technology.
METHODS
The ATI is quick to administer and well-suited to integrating into larger studies aimed at answering other types of technology related questions (see Schrills & Franke, 2022; Babamiri et al., 2022 for example). The ATI data for the present study were collected during two larger studies.
The first ATI data collection was conducted with university students who took part in a larger, survey-based, assessment aimed at understanding the factors that contribute to digital media overuse. The second ATI data col- lection was conducted during two separate simulation-based studies aimed at understanding performance, workload, and situation awareness of nuclear power plant operators. The focus of this paper is on the use of the ATI for the nuclear domain and the students are used as a baseline comparison group.
As such, the methods will focus only on ATI-related methodology and will not address the larger study contexts.
Participants Participants were recruited from two populations: university students and nuclear power plant operators. Five students and one operator were removed from the final sample due to failure to complete one or more sections of one or more survey. The final sample size was N = 60 (36 students and 24 operators).
Survey Administration and Modifications ATI data was collected twice for each participant within each participant group (students and control room operators). One administration was the unmodified 9-item English language ATI survey. The other administration was a modified version of the ATI which asked only questions that appeared highly relevant to control room operations. Table 1 provides the ATI ques- tion. The unmodified ATI was presented to participants with an instruction that set a personal use context. The modified ATI was presented to partici- pants with an instruction which established a professional/occupational use context (university, power plant control room).
A subset of 12 operator participants completed the SUS. The SUS was administered in its original form and was given to participants at the end of their testing session.
Table 1. Survey questions. Note that the bolded ATI questions are the subset of items that were repeated for the professional use context (modified) ATI.
Table available in the original PDF paper.
ATI Questions SUS Questions
I found the system unnecessarily complex.
I enjoy tinkering with technical systems. I think that I would like to use this system frequently. I enjoy testing the functions of new technical systems.
I thought the system was easy to use.
I primarily deal with technical systems because I have to.
When I have a new technical system in front of me, I try it out extensively.
I think that I would need the support of a technical person to be able to use this system. I enjoy spending time becoming acquainted with new technical systems.
I found the various functions in this system were well integrated. It is enough for me that a technical system works; I don’t care how or why.
I thought there was too much inconsistency in this system. I try to understand how a technical system works exactly.
I found the system very cumbersome to use.
I would imagine that most people would learn to use this system very quickly. It is enough for me to know the basic functions of a technical system.
I felt confident using the system.
I try to make full use of the capabilities of a technical system.
I needed to learn a lot of things before I could get going with this system.
Technology Affinity in the Nuclear Domain: An Initial Assessment 17
Design and Analysis There were three hypotheses this study aimed to examine. 1. Students will have greater affinity for technology than operators. 2. The context of the affinity survey (personal vs. occupational use) will influence the affinity ratings of both participant groups. 3. For the subset of operators who com- pleted the system usability survey (SUS), there will be a positive correlation between affinity and perceived system usability, such that higher ATI scores are associated with higher SUS scores. Participants responded to each ATI question using a 6-point Likert scale with the following anchors: 1 = com- pletely disagree, 2 = largely disagree, 3 = slightly disagree, 4 = slightly agree, 5 = largely agree, and 6 = completely agree. Responses to the three negatively worded items (3, 6, and 8) were reverse coded.
Participant SUS scores were calculated using the approach reported in the literature (Brooke, 1996; 2013). For odd numbered items, 1 was subtracted from the raw score. For event numbered items, 5 was added to the raw score. The sum of these scores was then multiplied by 2.5 to produce the final SUS score. Generally, a score of greater than 68 is considered average usability across all types of digital systems.
RESULTS AND DISCUSSION
Since this was the first (to our knowledge) use of the ATI in the nuclear domain, the primary interest was to examine if the sample of control room operators differed from a sample of college students since college students were one of the groups used by the ATI creators to validate the survey.
Table 2. Average (SE) scores for the standard (“personal use context”) and modified (“professional use context”) ATI surveys.
Table available in the original PDF paper.
Participant Type ATI Personal Use Context ATI Professional Use Context Operators 4.43 (.17) 4.60 (.16) Students 3.61 (.12) 3.24 (.12)
ATI Results There was a significant difference in technology affinity between participant types (students, operators) for both the personal (F(1,58) = 16.39, p < .001) and professional (F(1,58) = 48.32, p < .001) use framing context. However, the difference between personal and professional technology affinity exhib- ited different patterns. Students rated their affinity for technology lower when given the “classroom” as the framing context when compared to personal use of technology (see Table 2). Conversely, operators rated their affinity for technology as lower when the framing context was their personal use, com- pared to their affinity for technology in the context of a nuclear power plant control room (Table 2).
SUS Results Twelve nuclear power plant operators provided SUS ratings following their participation in a series of scenarios related to control room operations. The
average SUS score was 83.96 (SE = 2.80). The range of SUS scores (79.50– 97.50). These scores are well above the reported literature average SUS score of 68 (Bangor, Kortum & Miller, 2009; Brooke, 2013).
Pearson’s correlation tests were used to determine if there was a correla- tion between operators SUS ratings and their affinity for technology. Neither correlation (Personal Use [r2 = −.065], Professional Use [r2 = −.156]) was significant p > .05, suggesting that for this sample of operators there was no association between technology affinity and perceived system usability as rated by the SUS.
Figure 1: There was a weak but not statistically significant association between the ATI and the SUS.
CONCLUSION
This study is the first to our knowledge to use the ATI in the context of nuclear control room operations. The technologies used in a nuclear control room are distinct from those used in daily life in that they are complex and represent a range of digital and analog displays and tools. Given the distinct char- acteristic of the technology landscape, there was reason to believe that the standard ATI questions would be perceived as less applicable. To address this, all participants completed two versions of the ATI, an unmodified version, where the instructions provided personal day-to-day use of technology as the context, and a modified version, where the professional environment (either school-related, or control room-related) was the framing context. Interest- ingly, students rated their professional affinity for technology lower than their personal affinity, and control room operators rated their professional affinity higher than their personal affinity for technology. These results sug- gest that additional research into other individual factors (e.g., age, gender, educational background) as well as technology interaction-related factors is needed.
Additionally, the range of ATI scores for the 12 operators who also completed a SUS survey was compressed and limited to the high affinity portion of the scale for both the unmodified and modified ATIs (4.22–5.44
Technology Affinity in the Nuclear Domain: An Initial Assessment 19
and 4.00–5.40, respectively). It is possible that since the sample contained only high affinity users, there was not an opportunity to observe an asso- ciation between usability and the ATI. Similarly, the average SUS rating was high (M = 83.9), which could have limited any influence that affinity had on perceived system usability. Specifically, the system tested during the simulator study was highly usable and therefore individual factors, like tech- nology affinity, may have had limited influence. The samples of students and operators were very small relative to other studies that have used the ATI. Follow-up studies with larger samples and a greater range of ATI scores will be needed to determine if affinity for technology is a factor driving perceived usability.
Ongoing modernization activities in the nuclear industry are changing the control room environment substantially. Increasing the amount and com- plexity of technology in the control room will necessarily increase the need for easy-to-use tools for human factors researchers, regulators, and systems designers for assessing how new systems impact the ability of operators to perform tasks safely.
ACKNOWLEDGMENT
This document is a work of authorship of multiple institutions. The views expressed herein are those of the authors and do not necessarily represent an official position of the U.S. Nuclear Regulatory Commission, Bingham- ton University, or the Halden Human Technology Organization project. This material is declared as a work of the U.S. Government and is not subject to copyright protection in the United States. The authors would like to thank the Halden Human Technology Organization Project for their support on this research project.
REFERENCES
Attig, C., Wessel, D., & Franke, T. 2017. Assessing personality differences in humantechnology interaction: An overview of key self-report scales to predict successful interaction. In HCI International 2017–Posters’ Extended Abstracts: 19th Inter- national Conference, HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Proceedings, Part I 19 (pp. 19–29). Springer International Publishing. Avramidis, M., Kellner, C., Staudt, J., Zimmermann, V., & Gerber, N. 2021.
Evaluating a smart car interface in terms of usability, user experience and user acceptance. Mensch und Computer 2021-Workshopband. Babamiri, M., Heidarimoghadam, R., Ghasemi, F., Tapak, L., & Mortezapour, A.
2022. Insights into the relationship between usability and willingness to use a robot in the future workplaces: Studying the mediating role of trust and the moderating roles of age and STARA. PloS one, 17(6), e0268942. Bangor, A., Kortum, P., & Miller, J. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3), 114–123. Brook, J. 1996. SUS: A “quick and dirty” usability scale. Usability evaluation in
industry. Brooke, J. 2013. SUS: A retrospective. Journal of usability studies, 8(2), 29–40.
Franke, T., Attig, C., & Wessel, D. 2019. A personal resource for technology interaction: development and validation of the affinity for technology interaction (ATI) scale. International Journal of Human–Computer Interaction, 35(6), 456–467. Hart, S. G., & Staveland, L. E. 1988. Development of NASA-TLX (Task Load Index):
Results of empirical and theoretical research. In Advances in psychology (Vol. 52, pp. 139–183). North-Holland. Hart, S. G. 2006, October. NASA-task load index (NASA-TLX); 20 years later.
In Proceedings of the human factors and ergonomics society annual meet- ing (Vol. 50, No. 9, pp. 904–908). Sage CA: Los Angeles, CA: Sage publications. Heilala, V., Kelly, R., Saarela, M., Jääskelä, P., & Kärkkäinen, T. 2023. The Finnish version of the Affinity for Technology Interaction (ATI) Scale: Psycho- metric properties and an examination of gender differences. International Journal of Human–Computer Interaction, 39(4), 874–892. Hofbauer, L. M., & Rodriguez, F. S. 2022. How is the usability of commercial
activity monitors perceived by older adults and by researchers? A cross-sectional evaluation of community-living individuals. BMJ open, 12(11), e063135. Hughes, N., Reinerman-Jones, L., Lin, J., Matthews, G., Barber, D., and Dickerson, K. 2023. Human performance test facility (HPTF) volume 2 – com- paring operator workload and performance between digitized and anlog simulated environments (RIL 2022-11). U. S. Nuclear Regulatory Commission. Merritt, S. M., Unnerstall, J. L., Lee, D., & Huber, K. 2015. Measuring individual
differences in the perfect automation schema. Human factors, 57(5), 740–753. Muir, B. M. 1987. Trust between humans and machines, and the design of decision aids. International journal of man-machine studies, 27(5–6), 527–539. Parasuraman, R., Molloy, R., & Singh, I. L. 1993. Performance consequences
of automation-induced ‘complacency’. The International Journal of Aviation Psychology, 3(1), 1–23. Parasuraman, R., & Riley, V. 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors, 39(2), 230–253. Schrills, T., & Franke, T. 2022. How do Users Experience Traceability of AI Systems?
Examining Subjective Information Processing Awareness in Automated Insulin Delivery (AID) Systems. ACM Transactions on Interactive Intelligent Systems. U. S. Nuclear Regulatory Commission. 2020. Human-System Interface Design Review Guidelines (NUREG-0700, Rev.3). Washington, D. C.: U. S. Nuclear Regulatory Commission. U. S. Nuclear Regulatory Commission. 2016. Standard Review Plan for the Review of
Safety Analysis Reports for Nuclear Power Plants: LWR Edition—Human Factors Engineering, Chapter 18, NUREG-0800, Rev. 3. Washington, DC: US Nuclear Regulatory Commission. U. S. Nuclear Regulatory Commission. 2012. Human Factors Engineering Program Review Model, NUREG-0711, Rev. 3. Washington, DC: US Nuclear Regulatory Commission. Winkler, T., Akyildiz, D., & Herczeg, M. 2019, June. SpelLit: A Collabora-
tive Body-and Space-related Interactive Learning Game for School Children to acquire Reading and Writing Skills. In EdMedia+ Innovate Learning (pp. 63–71). Association for the Advancement of Computing in Education (AACE).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Usability and User Experience
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-32-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional