Si̇nan Apak (2026). Effects of Artificial Intelligence Decision Support Systems on Operator Trust and Workload. In: Industrial and Systems Engineering. AHFE Open Access, vol. 219, pp. 32–40. https://doi.org/10.54941/ahfe1007773
AI-based decision support systems (AI-DSS) used in production lines are being increasingly adopted due to their potential to improve operator performance in complex and uncertainty-laden operations. These systems support operators throughout the process in terms of algorithmic accuracy and computational capacity. Another reason for their preference is their direct relationship with operator trust and perceived workload over time. This study aims to examine the effects of AI-based decision support systems on operator trust and workload from the perspective of human factors and ergonomics. By addressing the operator’s interaction with artificial intelligence within the system, the study focuses on the relationship between operators’ trust in system output and cognitive workload. The system outputs were tested by operators within scenarios offering different levels of AI support and were evaluated using multidimensional measurement tools. Subjective and objective indicators reflecting operator workload, together with time-based measurements, were considered jointly. Trust in the system was examined through dimensions such as transparency, predictability, and behavior in the face of errors. The findings suggest that when the output provided to the operator by AI-based support systems are not designed appropriately, they may lead to either over-trust or excessive caution among operators. As a result, both the accuracy of the operator’s decisions and situational awareness may be weakened. It was observed that as the level of automation increases, cognitive workload decreases; however, the ability to respond quickly and accurately to unusual situations within the system also declines. From the operators’ perspective, it was determined that adaptive and highly explainable AI solutions support trust in the system and contribute to maintaining workload balance. Overall, this study demonstrates that ergonomic principles play a decisive role in influencing operator performance in the design of AI-supported decision systems. As a fundamental design criterion, maintaining the balance between sustained performance, operator trust, and cognitive workload is essential. In this regard, the research contributes to the development of evaluation and design approaches for AI-supported decision systems in industrial, healthcare, and safety-critical application areas within lean manufacturing environments. The findings are expected to guide design processes toward establishing a more robust and effective foundation for human–AI interaction.
Lean smart manufacturing (LSM), which integrates lean production principles with digital technologies, is employed to enhance efficiency in line with lean principles, reduce downtime, and enable production systems to adapt more rapidly to changing conditions. Artificial intelligence–supported decision- making systems (AI-DSS) are widely used to assist operators in functions such as predictive maintenance, anomaly detection, and the continuous monitoring of processes. While these systems significantly improve technical performance, they may also create a comprehension gap in cases where operators are unable to fully comprehend the logical processes underlying the recommendations presented to them. In such situations, the effective utilization of the benefits provided by the system by the operator becomes more difficult, and the reliability of human–machine interaction is adversely affected.
In line with lean manufacturing principles, system transparency, visual control, and the operator’s command over the process make the explainability problem arising in AI-based systems even more pronounced. Many artificial intelligence applications possess a “black-box” nature, in that they do not present their decision-making mechanisms to the user in an explicit and understandable manner. This situation makes it particularly difficult to interpret system outputs in production processes characterized by intense time pressure. The explainable artificial intelligence (XAI) approach offers a framework aimed at alleviating this problem by seeking to communicate the operating logic of algorithms to users in a comprehensible way. Studies in the literature have shown that explainability can contribute to a more balanced formation of trust in human–AI interaction and can reduce unnecessary cognitive workload (Gunning et al., 2019; Adadi and Berrada, 2018).
This study examines the effects of different levels of explainability in AI-supported decision-making systems on operator trust and cognitive workload. It evaluates whether systems that provide explanatory information outputs contribute to the appropriate calibration of operator trust, reduce monitoring requirements, and improve operator interaction in maintenance- oriented decision-making tasks on a press line. This study examines whether explainable AI-DSS interfaces improve trust calibration and reduce cognitive workload without undermining situational awareness in lean press-line maintenance decisions.
LITERATURE REVIEW
With the rapid advancement of technology, the widespread adoption of artificial intelligence in industrial environments is regarded as a comprehensive socio-technical transformation affecting the entirety of production processes. While data-intensive systems and high levels of automation are reshaping the functioning of manufacturing systems, the job description of operators is also changing in parallel. Today’s operators are expected not only to possess physical competence, but also to interpret the inferences, recommendations,
34 Apak and warnings generated by the system and integrate them into decision- making processes. In this way, the cognitive roles of operators are becoming increasingly critical. The success of AI-based solutions in manufacturing environments depends not only on the technical accuracy of algorithms or their processing power. The process depends on what operators understand about AI support systems, how much they trust these systems, and how effectively they can use the information provided. Thus, human–AI interaction constitutes a fundamental component of modern manufacturing environments in terms of sustainability and reliability.
Trust calibration and cognitive ergonomics emerge in the literature as two closely related themes. While automation systems can reduce the physical or routine workload of operators, the output structure of black- box AI systems requires operators to interpret warnings and suggestions, thus creating cognitive workload for operators. Such black-box notifications can cause operators to hesitate, develop an inappropriate level of trust in the system, or resist system recommendations. Therefore, regardless of the system’s technical capabilities, human-centered AI research evaluates this contradiction. As Shneiderman (2020) states, AI systems should be designed to be understandable to operators, foster trust, and support real-time actions.
Table 1: Literature comparison on AI-DSS, operator trust, and workload.
Table available in the original PDF paper.
Author / Year
Focus of the Study Key Findings (Trust and Workload Relationship)
Lean/Smart Manufacturing Context
Hoffman et al. (2018)
Confidence Measurement and Calibration
Lack of transparency creates “distrust” in the operator, increasing the mental workload.
The need for explainability in decision support systems.
Kiangala and Wang (2024)
Industrial AI and Human-Machine Interface
“Visual Management” integration in smart factories.
Visual evidence provided with AI recommendations reduces the operator’s cognitive effort and stabilizes confidence.
Predictive maintenance (PdM) applications in press lines and heavy industry.
Moraes et al. (2023)
Lean Smart Manufacturing: The clash of lean discipline and AI flexibility.
Lean and Industry 4.0 Alignment AI-DSS interfering with standard workflows (Standard Work) can create “role confusion” and increased workload for the operator.
The studies classified in Table 1 describe three main points. First, operators’ attitudes are not always the same. These attitudes vary with the explanatory power of the outputs of the AI decision support system. Second, if the outputs are not immediately accepted by the operator, cognitive work is generated in the operator to perceive the problem. The third main point is the situation where lean manufacturing principles do not align with the
Effects of Artificial Intelligence Decision Support Systems on Operator Trust 35 outputs. All these findings reveal that the technical performance of the AI decision support system cannot improve the process; rather, how these outputs are perceived by the operator is what affects performance.
Integration of artificial intelligence is a cognitive adaptation process and the synthesis of the studies of Table 1 shows that it is not a technical achievement. The key findings of this comparative study may be summarized in three major categories. Transparency and Trust, Linear Relationship; As noted by Hoffman (2018) and Kiangala and Wang (2024), operator trust is not a fixed acceptance but a dynamic process which is triggered by the transparency of the system. When the press lines are the high- risk environment, and the operator cannot provide the answer to the question why I got this warning, the possibility of disusing the system is increased. The Explainable Artificial Intelligence (XAI) makes sure that trust is adjusted at this important level, which reduces the dependency of the operator on the system to a reasonable level. Converting Cognitive Workload and Mental Muda are described by the works of Sharma (2025) and Moraes (2023) reveal that the lean manufacturing principle of waste prevention should include not only the physical processes but also cognitive ones.
METHODOLOGY
An exploratory quasi-experimental within-subject study was conducted with three experienced press-line operators in an automotive manufacturing setting. Each participant completed maintenance-related decision tasks under two interface conditions: (1) a black-box AI-DSS interface that displayed warning messages and standard error codes only, and (2) an explainable AI-DSS interface that additionally displayed causal information, confidence-related cues, and recommended corrective actions. Four anomaly scenarios derived from sensor-based press-machine data were presented in each condition. Dependent variables included response time, decision accuracy, NASA-TLX workload scores, and Jian Trust Scale scores (Hart and Staveland, 1988; Jian et al., 2000; Gunning, 2017). Because of the small sample size, analysis was descriptive and comparative rather than inferential.
The participant group consisted of three press line operators with technical responsibilities who had prior experience working in a lean manufacturing environment. Although the sample size was limited, the study was designed as an exploratory investigation focusing on detailed operator responses in a realistic industrial context rather than statistical generalization. Participants interacted with a simulated control panel fed with production-related data on a controlled workstation designed for ergonomic work without exposure to external environmental effects.
The artificial intelligence-based decision support system (AI-DSS) used in the study was trained over a month using real-time sensor data collected from a high-tonnage hydraulic press. This dataset consisted of 14 telemetric variables representing the machine’s operating condition, such as piston
36 Apak pressure, die closing speed, stroke count, oil temperature, vibration frequency, motor temperature, current draw, and voltage fluctuation. Based on this data, four anomaly scenarios were created for representative failure situations that operators might encounter. This allowed researchers to compare operator responses under standardized conditions when performing maintenance- related decision-making processes.
The data collection process focused on operator performance and psychometric indicators. Response time and decision accuracy were used as operational performance metrics for the operator. Operators’ cognitive workload was assessed using the NASA Task Load Index (NASA-TLX), which captures mental demand, physical demand, temporal demand, performance, effort, and frustration. The Jian Trust Scale, a widely used 12-item tool, was used to assess operator trust in automation. Operators’ responses were considered to measure whether the AI interface provided explanatory answers. Both analytical and comparative approaches are necessary to understand operator attitudes (Creswell & Creswell, 2017). Operators’ responses under two different interface implementation conditions were compared, and workload, confidence, and performance values were compared between the two scenarios.
Throughout the experiment, operators completed the relevant maintenance tasks in both interface conditions, based on the same anomalies. In each scenario, operators made a decision by examining the system output. The cognitive workload assessment was completed by measuring the time it took for these decisions to translate into a result. Response time and decision accuracy were recorded for each condition. Due to the exploratory approach and small sample size, the results were obtained by comparing the patterns between the conditions.
FINDINGS In lean manufacturing, when determining the effectiveness of AI-based decision support systems, the accuracy of prediction and the way the output is delivered to the operator create different cognitive workloads. The findings show that both elements affect system performance. In the black-box environment, operators exerted more effort to make sense of the warnings, increasing the monitoring load. Operators’ trust responses were more variable in this environment; they sometimes trusted the system and sometimes were reluctant to follow the recommendations. The explainable interface provided a more balanced level of trust. Operators’ situational awareness was also higher in this interface. These patterns demonstrate that trust in AI-assisted operations is a matter of calibration. Calibration is shaped by transparency, predictability, and actionability.
Table 2: Conceptual comparison of manual, AI-DSS, and XAI-supported operation.
Table available in the original PDF paper.
XAI-Powered Ideal Scenario (Recommendation)
Parameters Traditional Lean Operation (Manual Tracking)
AI-DSS Supported Press Line (Current Status)
Cognitive Workload (NASA-TLX)
High: Mental fatigue increases due to the effort to understand “black box” decisions.
Low: Thanks to explainable data, decision-making speed increases and uncertainty decreases.
Medium: Physical exertion is high, but the decision process is under the control of the operator.
Trust Calibration
Low/Unstable: Due to false positive alerts, the operator tends to disable the system.
Experience Oriented: Operators rely on their own senses (sound, vibration).
Optimized: Knowing the margin of error of the system, the operator develops data-driven confidence.
Error Detection Speed
Low: Intervention is made after the malfunction occurs (Reactive).
Very High: A warning is given hours before the fault occurs (Proactive).
Very High + Significant: The fault warning is presented with the “cause” information.
Lean Harmony (Muda - Waste)
Minimum: Both physical and mental processes reach their simplest state.
High: Process waste is high due to unplanned downtime and scrap products.
High: The operator has complete control of the process.
High: A symbiotic flow of information is achieved between the operator and the AI.
Low: The operator simply falls into the “screen viewer” position (Out-of-the- loop).
It has been observed that when the results provided by the AI decision support system do not align with the natural functioning of the production line, the operator views the result with skepticism. It has also been observed that the operator spends time evaluating the output during this process. Operators working in a lean manufacturing environment focus on predefined processes, established routines, and production targets. Moreover, since operators work under task time pressure, even seemingly correct warnings can be perceived negatively if they disrupt the existing flow or do not adequately respond to the application in the field.
These types of situations require operators to verify the accuracy of AI outputs as well as to check how well they fit into actual shop-floor decision- making processes. The applicability of the system output and whether it’s a warning the operator has encountered before are as important as the correct recommendation the system provides. Even if support systems produce flawless outputs, the operator must perceive these outputs as reasonable and useful within the production process. Outputs that the operator cannot easily understand are not accepted within the process. Ultimately, what matters is whether the recommendation works within the human-machine interaction that defines the production environment.
DISCUSSION AND IMPLICATIONS The findings provide comprehensive implications regarding the fundamental design principles of using artificial intelligence as a decision support system in a press line where lean manufacturing is implemented. It was observed that operators used the AI interface design and evaluated the results. The comprehensibility of the AI-powered decision support system indicates that it is preferable as a supportive technological design component in the operators’ decision-making process.
Theoretically, it shows that operators’ trust in artificial intelligence is not a one-dimensional attitude. Their attitudes are shaped in the production environment by factors such as transparency, predictability, and compatibility with workflows. The system outputs have made the operator part of a dynamic calibration process. Therefore, the operators’ trust in the AI system’s output depends on the production goals determined in the planning process. The operator’s trust in the system is related to their ability to make sense of the incoming notification. From a practical point of view, the widespread use of the tested system is related to the extent to which operators perceive the incoming warnings. System outputs that are not sufficiently explanatory create a kind of ‘cognitive waste’ by shifting the operator’s attention from productive intervention to interpretive monitoring activities. One of the main goals of lean manufacturing systems is to eliminate unnecessary burdens. Such cognitive inefficiencies should not be ignored. Therefore, the quality of explanations provided by AI-powered decision- making tools in lean manufacturing environments is extremely important. The clarity, applicability, and compatibility of the recommendations with actual workflows should be checked throughout the process. An effective AI application in lean manufacturing goes beyond technical accuracy. It acts as a supporting element in the operator’s decision-making process. This requires a design approach that can provide explanations consistent with operational realities on the production floor.
LIMITATIONS This study has limitations in its design. These include the limited number of operators used in the sample, the fact that the study was conducted in a simulated workstation environment based on real production data, and the fact that the analysis relies primarily on descriptive comparisons. The limited sample size indicates that the study is not generalizable. Instead, the application should be interpreted as an exploratory study. Although it simulated the real production environment to some extent, it did not fully reflect the variability found in real production environments. The observed differences between interface conditions are not statistically validated effects. Differences in operator behavior vary according to the individual’s level of perception. It is recommended that future studies include larger participant groups, test the application in different production environments, and use inferential statistical methods.
CONCLUSION
This study investigated the effects of artificial intelligence decision support systems on operator trust and cognitive workload. The results show that operators’ trust in the decision support system is related to the level of perception of the system’s output. In a lean manufacturing environment, cognitive workload waste occurs with the time it takes for the operator to understand the stimulus given by the system. Therefore, it is extremely important that the system outputs are understandable, interpretable, and applicable by operators working under real production constraints. In this case, determining the success of AI-based decision support systems solely by the system’s technical performance is insufficient.
When AI outputs are not clearly understood by the operator, operators are forced to expend additional effort to interpret them, increasing their cognitive workload. In such cases, the operator’s level of trust in the support system becomes less stable, and situational awareness may decrease. Explainable outputs support more appropriate trust calibration and make the decision- making process more manageable. From this perspective, the explanatory nature of the system should be considered a fundamental component of human-centered AI design in industrial environments.
In conclusion, this study contributes to the literature by demonstrating that the usability of AI-DSS by operators depends on how system outputs are communicated to them. In the context of lean manufacturing, explainability plays a dual role. While these systems, which provide quick warnings about problems encountered by operators, deliver outputs aimed at enabling rapid problem-solving, messages that the operator cannot perceive cause the system to take longer and subject the operator to additional cognitive load. Explainable AI applications highlight the need for a practical and human- centered design for sustainable and secure industrial decision support systems.
REFERENCES
Adadi, A., & Berrada, M. (2018). “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI).” IEEE Access, 6, 52138–52160. Creswell. J.W. and Creswell, J.D. (2017) Research Design: Qualitative, Quantitative,
and Mixed Methods Approaches. 4th Edition, Sage, Newbury Park. Gunning, D. (2017). Explainable Artificial Intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), https://www.darpa.mil/program/explainable- artificial-intelligence. Gunning, D., Stefik, M., Choi, J., Miller, T., Magar, S., & Yang, G. D. (2019). “XAI-
Explainable Artificial Intelligence.” Science Robotics, 4(37). Hart, S.G. and Staveland, L.E. (1988) Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology, 52, 139–183. Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018). Metrics for explainable
AI: Challenges and prospects. XAI Metrics, arXiv preprint arXiv:1812.04608. Jian, J. Y., Bisantz, A. M., & Drury, C. G. (2000). Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics, 4(1), 53–71.
and generative AI chatbot human machine interface to improve a factory troubleshooting downtime in the context of Industry 5.0. The International Journal of Advanced Manufacturing Technology, 132(5), 2715–2733. Moraes, A., Carvalho, A. M., & Sampaio, P. (2023). Lean and Industry 4.0: A review of the relationship, its limitations, and the path ahead with Industry 5.0. Machines, 11(4), 443. Sharma, R. (2025). AI-Powered Robotics and Automation: Revolutionizing
Industries. In Artificial Intelligence and Machine Learning Algorithms for Engineering Applications 246–267. CRC Press. Shneiderman, B. (2020). Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. International Journal of Human–Computer Interaction, 36(6), 495–504.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Industrial and Systems Engineering
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-95-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional