Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Requirements for Haptic Virtual Training Systems in the Automotive Industry

1 Institute of Industrial Engineering and Ergonomics (IAW), Aachen, 52062 Germany ↗
2 Ford Motor Company, Cologne, 50735, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1004467

Abstract

Due to challenges associated with the speed of change and the variety of products, assembly training is becoming more and more important. Virtual reality (VR) can improve training processes but is largely limited to visual and auditory feedback. Manual assembly tasks are difficult to train only with visual and auditory feedback. Haptic-enhanced assembly systems can bridge this gap by enabling haptic interaction in the virtual environment. Integrating haptics into virtual training can improve virtual training processes in terms of a better understanding of assembly operations. This paper aims to analyze human and technological requirements for training systems in the context of the automotive industry. For this purpose, a qualitative expert study was conducted with nine experts from the automotive industry. Based on an interview guideline, requirements were derived for the assembly training system and the integration of haptic interaction modalities. A focus group discussion was conducted to evaluate and validate the requirements in terms of relevance and practicality.

INTRODUCTION

Manufacturing processes have become increasingly complex, involving complicated assembly procedures and the integration of various com- ponents. Training workers to perform these tasks efficiently and effec- tively is crucial for maintaining high productivity and quality standards. The importance of worker training is also highlighted by the automo- tive industry’s quality management standard (IATF 16949 2016), published in 2016.

Traditional training methods, such as hands-on or on-the-job training, can be costly and time-consuming (Gavish et al., 2015). They require phys- ical setups, specialized equipment or experienced instructors (Brough et al., 2007). Virtual assembly training systems offer a cost-effective and efficient alternative by simulating assembly environments without the need for physi- cal resources (Hirt et al., 2019). For this reason, many companies are already using virtual trainings (Chris Morris, 2018).

© 2024. Published by AHFE Open Access. All rights reserved. 27

Virtual training allows workers to understand assembly processes in a controlled and safe virtual environment. Trainees can practice assembling components, understand the sequence of operations, and learn to identify errors that may occur during production. By providing immediate feedback, virtual training can help workers refine their skills, improve their efficiency, and reduce the risk of errors in real-world production (Xie et al., 2021).

In the industry, virtual training is often conducted using desktop computers as training equipment. The availability of advanced immersive technologies, such as virtual reality (VR), augmented reality (AR) and mixed-reality (MR) can significantly enhance the realism and immersion of virtual assembly set- tings. These technologies enable trainees to visualize and interact directly with virtual components, tools, and environments, enhancing their learn- ing experience and skill acquisition for complex manufacturing processes. Thus, the use of immersive technologies promises to improve virtual training (Liberatore and Wagner, 2021).

While immersive training applications are becoming increasingly popular, there is still a research gap in the area of haptic enhanced virtual training. However, especially haptics provide physical feedback to users and offers another dimension of immersion (Preutenborbeck et al., 2022). In the context of assembly training, users can not only see and hear virtual environments, but also physically interact and feel them, providing more direct access to practical skills (Xia et al., 2012). In addition, haptic feedback can help users better understand complex motoric techniques by providing a better sensory experience (Sigrist et al., 2013).

Before implementing and introducing new training technologies in the industry, it is necessary to know the requirements for immersive virtual training methods. Requirements analysis is an important phase in systems engineering (Haberfellner et al., 2019) and Human Systems Integration (Flemisch et al., 2020) because it helps define the scope, goals, and expec- tations of stakeholders. Here, both the functional and the non-functional requirements, i.e. those related to the perceived system qualities, have to be captured.

This paper presents an in-depth requirement analysis involving different stakeholders from the automotive industry, e.g. production workers, indus- trial engineers and production ramp-up engineers. The objective is to analyse all requirements needed to develop a haptic enhanced virtual assembly system that enables a cost-effective, safe and immersive approach to virtual training.

RELATED WORK

In a systematic literature review, related work in the field of haptic enhanced assembly systems in manufacturing were identified in advance (Preutenbor- beck et al., 2023). The approaches of Al-Ahmari et al., Kind et al. and Buchholz et al. are the most relevant, as the focus of this publication is on the requirements for haptic enhanced assembly training in the automotive industry.

Al Ahmari et al. present the design of a system for virtual assembly sim- ulation. The main functions include planning of assembly operations and

Requirements for Haptic Virtual Training Systems in the Automotive Industry 29

training of assembly workers. The system uses a virtual environment for inter- active assessment and training. It provides visual, auditory and haptic, i.e. tactile, and kinesthetic feedback. The authors describe technical requirements such as data translation, integration of different hardware and software sys- tems, and real-time collision detection. In this way, it is possible to develop the system that help train operators for assembly operations and bridge the gap between planning and execution of assembly (Al-Ahmari et al., 2016).

Kind et al. describe an architecture that enables workers to perform virtual assembly simulations with force feedback in VR environment. The architec- ture consists of three different cores: The visualization core is used to load CAD data from various sources into the virtual scene. The simulation core emulates the physical behavior of the virtual model and computes forces applied to the force feedback device. The interaction core serves as the user interface and provides force feedback to the user. Within this architecture, the authors identify technical requirements, such as the ability to load CAX mod- els, simulate the physical behavior of the virtual models, calculate the forces applied to the haptic device, and provide a user interface with force feedback. Requirements related to the application include the ability to verify criti- cal assembly tasks, implement modular end effectors for flexible use cases, and analyze time measurements to optimize assembly tasks and sequences (Kind et al., 2020).

Buchholz et al. present an approach for VR-based interaction with virtual factory models. The system allows visualization of production processes in 3D and adaptation of interaction mechanisms to individual users. Through the integration of haptic interaction modules the developed system enables early validation of virtual prototypes in different use cases, e.g. assembly training. The authors identify requirements for the virtual assembly system such as intuitive visualization of complex assembly processes and the need to integrate different VR technologies. However, the authors focus more on technical perspective, e.g. and less on human or ergonomic perspective (Buchholz et al., 2017).

In summary, none of the presented approaches provides an in-depth anal- ysis of user requirements for a haptic enhanced assembly training in VR. A user-oriented and systematic investigation of the requirements that are important for the design and implementation of a haptic enhanced assembly training in VR is missing.

METHODOLOGY

The objective of the interviews was to identify requirements for haptic enhanced virtual training. To achieve this, nine experts from the automo- tive industry were interviewed. They were selected based on their experience in production ramp-up and training processes. The interviews were con- ducted either remotely by Zoom or in person, depending on the location of the participants, and lasted approximately 60 minutes. The guideline did not contain any predefined answer options but ensured the comparability of statements by different experts on the topics. The interview guideline was structured according to elements of a work system based on a standard of

the REFA association, see Figure 1. In this way, all potential requirements for the training could be covered, e.g. work task, tool, worker related aspects and environmental influences (REFA, 2023).

While the discussion with the participants followed a flexible approach, the interview guideline provided a general direction for the conversation. This allowed an individual focus on each participant’s specific areas of expertise. The questions were organized into thematic categories to ensure a logical flow of the interview and minimize jumps in context for the participants (Kallio et al., 2016).

Figure 1: Assembly station as a work system according to REFA (REFA, 2023).
Original PDF Figure 1:

After the interviews were completed, the recordings were transcribed and coded by using the software MAXQDA 2022. This allowed a more detailed analysis of the interview data. The analysis was carried out using a combined deductive and inductive approach for qualitative content analysis according to Mayring and Fenzl (Mayring and Fenzl, 2014). The transcribing phase was followed by coding of the material, in which a code system was used to iden- tify relevant text passages for the requirements analysis. In the third phase, the coded data was analyzed by establishing relationships among categories. This allowed to improve the code system and to extract additional aspects from the interviews and include them in the requirements analysis.

RESULTS AND DISCUSSION

Based on the interviews and the qualitative content analysis, 28 requirements were identified. These requirements were reviewed in a focus group discus- sion within the project team to ensure that they were complete, consistent, and feasible. The following section summarizes the results of the evaluation. On the one hand, this refers to the presentation of the objectives and success factors of virtual training. On the other hand, the requirement specification is presented, which was based on the expert interviews and the qualitative content analysis. Table I and Table II show the functional (f) and quality

Requirements for Haptic Virtual Training Systems in the Automotive Industry 31

attributes, sometimes referred to as non-functional (nf.) requirements, of the haptic virtual assembly training system.

Objectives of Virtual Assembly Training

Figure 2 illustrates the objectives of virtual assembly training. The number of mentions results from the fact that multiple mentions were allowed. The most important objective is to enable employees to understand the assembly process in order to ensure a successful ramp-up of new products. In this way, errors and, if necessary, rework on vehicles and the associated costs can be avoided. In addition, virtual training processes can reduce the time and cost of the training itself. Software-based training requires fewer trainers than physical training and is therefore suitable for accelerating learning processes. However, the correct manual execution of assembly processes cannot be guar- anteed. Therefore, further development in the field of haptic enhanced virtual training processes is very promising. Another advantage is the training relia- bility. Standardized procedures ensure that important information is passed on to the trainee. With the completion of the training, it can be documented that the assembly process has been learned.

Figure 2: Objectives of virtual assembly training.
Original PDF Figure 2:

Requirements for Training Content

Table 1 shows the requirements of the training system related to the work system, i.e. the work task, the tool handling, worker-specific aspects and environmental influences: Work Task Requirements For training of the process steps, the training system must provide the assem- bly instructions (No.1). To be able to present the correct instructions at any time, it must also be possible to identify all assembly steps, which are per- formed by the worker (No.2). For training of ergonomic body postures, the system requires body-tracking (No.3). The system should also be able to evaluate workers’ postures during the execution of assembly processes and suggest ergonomic improvements (No.4). Haptic technologies, e.g. haptic

gloves, should be integrated into the system for training the correct man- ual execution of assembly operations. This refers to certain hand operations, e.g. clipping (No.5) or gesture-based concepts for larger movements, e.g. pressing in plastic cladding (No.6). For training of the spatial layout, all process-relevant locations, e.g., of material boxes, must be integrated into the virtual environment (No.7). In this context, the level of detail is an impor- tant non-functional requirement, since there is a trade-off between realism of the environment and task orientation (No.8). The required range of motion and tracking area of the training system depends on the size of the consid- ered workstation. The training system should also take into account relevant safety measures (No.9). This includes wearing safety shoes and other safety equipment. For this purpose, the worker can be represented in the virtual environment in the form of an avatar.

Table 1. Work system requirements.

Table available in the original PDF paper.

Work System Requirements

No. Description f. nf.

x

Work task Process steps 1 Assembly instructions for process steps x x 2 Assembly step identification x Posture 3 Posture identification and body tracking x 4 Posture evaluation mechanism, e.g. ergonomics x Manual execution 5 Assembly gestures, e.g. for manual operations, e.g. clipping x 6 Provide haptic feedback for manual activities x Spatial layout 7 Implementing spatial workstation layout, including material boxes etc.

x

8 Appropriate detail of workstation layout x Safety measures 9 Implementing safety instructions, e.g. gloves and safety shoes x Work tool Handling 10 Implementing tool manipulation and functions, e.g. buttons

11 Implementing tool sounds x 12 Realistic tool handling, e.g. vibration x 13 Stability/ usability in tool operation x

x

Worker User Interface 14 Inclusive interface design x 15 Different languages x 16 Integration of a chat option between trainer and trainee x Environment Factory 17 Simulation of factory environment, e.g. production noise

18 Appropriate level of detail x

Work Tool Requirements The training system should allow a realistic interaction with the tools, includ- ing their handling and execution in the virtual environment (No.10). Haptic devices, e.g. by simulating mechanical resistances of buttons, can support the execution of tool functions. For the realistic interaction, sounds and haptic feeling of tools need to be implemented (No.11, No.12). During the assembly process, the sounds and haptic impressions vary depending on

Requirements for Haptic Virtual Training Systems in the Automotive Industry 33

the use-situation, e.g. when a correct torque of a screw is achieved. There- fore, a realistic implementation of the tool behaviour not only serves to increase immersion, but also is an essential part of quality assurance by the worker. Increasing the usability of the tool handling is a very impor- tant non-functional requirement (No.13). The system must run stable to ensure smooth and user-friendly operation. This is particularly important for acceptance, as one of the key success factors, see Figure 3.

Figure 3: Success factors of virtual assembly training.
Original PDF Figure 3:

Worker-Specific Requirements The training system must have an inclusive interface design that enables nav- igation and operation to ensure accessibility and usability for all, including people with limited vision or other impairments (No.14). This also includes making the training system available in different languages. This allows peo- ple with different cultural backgrounds and different language skills to use the system effectively. A multilingual interface and translation of training content into multiple languages are essential to reach a broad user base (No.15). The system should provide an integrated chat function that allows direct commu- nication between trainer and trainee (No.16). This feature should operate in real time and can support both text-based and voice-based communication to enable effective guidance and feedback, independent from the location of the trainer.

Environmental Requirements The training system should be able to simulate a realistic factory environment, including production-typical noise levels (No.17). In addition, the system should provide an appropriate level of detail of the surrounding to allow an accurate and effective employee training (No.18). The key aspects of the assembly processes and environment should be adequately considered.

Didactic Requirements Table 2 shows the didactic requirements for the training system. The didactic requirements include features that improve the workers motivation, as well as the analysis and documentation of the training results. To increase moti- vation, the training system should provide an introduction level to the virtual environment to familiarize participants with relevant aspects and work tools

(No.19). The system should also include different levels of training that are consistent with current training methods. This ensures that participants acquire their skills and knowledge gradually and systematically (No.20). The system should integrate gamification elements such as game sounds to make learning more fun and interactive (No.21). The system should also have a user-friendly interface and an appealing interaction design to facili- tate the learning process and increase usability (No.22). The system should be able to measure the times required to perform the various assembly tasks (No.23). In addition, the training system should have an assembly error anal- ysis to facilitate the identification and correction of failures (No.24). The evaluation of the variables collected by the system should provide a high information value, e.g. through a high granularity (No.25). In this way, complicated assembly steps can be identified during the training sessions. The requirements for documentation relate primarily to automation and the presentation of results. The system should enable automated documenta- tion of training results, preferably through a “one-click”-feature, to make the process efficient and timesaving for the trainer (No.26). Training results should be presented in a motivating way, such as high scores or other perfor- mance indicators, to encourage trainees and increase their motivation to learn (No.27). Since the training results sensitive performance data, it is necessary to consider whether a detailed evaluation is provided only to the trainee.

Table 2. Didactic requirements.

Table available in the original PDF paper.

Didactics Requirements No. Description f. nf.

x Motivation 19 Familiarization with the virtual environment at the beginning of each training 20 Include different training levels, consistent to current training methods x 21 Include gamification elements, e.g. game sounds x 22 Learning friendly interface and interaction design x Analytics 23 Implementation time measurements x 24 Implementation of assembly error analysis x 25 High informative value of the assessment, e.g. granularity x Documentation 26 Automated documentation of the training results (“one click”) x 27 Motivating presentation of the training results, e.g. high scores x

Success Factors of Virtual Assembly Training At the end of the interview, all experts were asked to name success factors that determine successful assembly training. The most important success fac- tor mentioned by six experts is the training time. This refers primarily to the time required by production workers to train new assembly processes or workstations. However, the time required to implement the training system is also relevant. This time was subsumed under the second success factor, effort.

The acceptance of the virtual training is also important, as this can influ- ence the motivation and success of the training. The survey of success factors was intended as an initial assessment of potential requirements, but was

Requirements for Haptic Virtual Training Systems in the Automotive Industry 35

expanded to include a more specific analysis of requirements at level of the single design elements of the assembly training system.

CONCLUSION AND OUTLOOK Virtual training systems offer a cost-effective and efficient alternative by simulating assembly environments without the need for physical resources. By integrating haptic components, virtual training can be made more real- istic. This allows participants to visualize and interact directly with virtual components, tools and environments to understand complex manufacturing processes. To ensure successful implementation, a comprehensive analysis of the requirements for such immersive virtual training methods is necessary.

This paper describes a requirement analysis in which different stakeholders from the automotive industry were involved in the design and development of a haptic enhanced assembly training system. Through guided interviews with experts, 28 requirements were identified, which were validated in a focus group discussion afterwards. The analysis of the requirements and the validation form the basis for the explorative design and development of an effective and user-friendly virtual training system for assembly pro- cesses. The integration of haptic feedback promises that participants develop a better understanding of assembly processes and thus avoid errors before they occur in real production. Future work can focus on the exploration and implementation of design concepts that meet the identified requirements. Fur- ther development and use of immersive technologies in training could also be explored in other sectors and areas of the manufacturing industry to further improve the efficiency and quality of training.

ACKNOWLEDGMENT

This research was funded by the Ford Motor Company under the Ford Alliance Program (FA0394).

REFERENCES

Al-Ahmari, A. M., Abidi, M. H., Ahmad, A. and Darmoul, S. (2016) ‘Development of a virtual manufacturing assembly simulation system’, Advances in Mechanical Engineering, vol. 8, no. 3, 168781401663982. Brough, J. E., Schwartz, M., Gupta, S. K., Anand, D. K., Kavetsky, R. and Pettersen, R.

(2007) ‘Towards the development of a virtual environment-based training system for mechanical assembly operations’, Virtual Reality, vol. 11, no. 4, pp. 189–206. Buchholz, C., Kind, S. and Stark, R. (2017) ‘Design of a Test Environment for Planning and Interaction with Virtual Production Processes’, Procedia CIRP, vol. 62, pp. 547–552. Chris Morris (2018) Why Walmart and other F500 companies are using vir-

tual reality to train the next generation of American workers [Online], CNBC Work. Available at https://www.cnbc.com/2018/10/29/why- f500-companies-use-virtual-reality-to-train-workers-of-the- future.html?utm_source=headtopics&utm_medium=news&utm_campaign=2018- 10-30 (Accessed 3 August 2023).

Flemisch, F., Bielecki, K., Hernández, D. L., Meyer, R., Baier, R., Herzberger, N. D.

and Wasser, J. (2020) ‘Let’s Get in Touch Again: Tangible AI and Tangible XR for a More Tangible, Balanced Human Systems Integration’, in Ahram, T. (ed) Intelligent Human Systems Integration 2020: Proceedings of the 3rd International Conference on Intelligent Human Systems Integration (IHSI 2020): Integrating People and Intelligent Systems, February 19-21, 2020, Modena, Italy, Cham, Springer International Publishing AG, pp. 1007–1013. Gavish, N., Gutiérrez, T., Webel, S., Rodríguez, J., Peveri, M., Bockholt, U. and Tecchia, F. (2015) ‘Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks’, Interactive Learning Environments, vol. 23, no. 6, pp. 778–798. Haberfellner, R., Weck, O. de, Fricke, E. and Vössner, S. (2019) Systems Engineering,

Cham, Springer International Publishing. Hirt, C., Holzwarth, V., Gisler, J., Schneider, J. and Kunz, A. (2019) ‘Virtual Learning Environment for an Industrial Assembly Task’, Proceedings 2019 IEEE 9th International Conference on Consumer Electronics - Berlin (ICCE-Berlin): 2019 ICCE-Berlin took place September 8-11, 2019 in Berlin, Germany. Berlin, Germany, 9/8/2019 - 9/11/2019. Piscataway, NJ, IEEE, pp. 337–342. Kallio, H., Pietilä, A.-M., Johnson, M. and Kangasniemi, M. (2016) ‘Systematic

methodological review: developing a framework for a qualitative semi-structured interview guide’, Journal of advanced nursing, vol. 72, no. 12, pp. 2954–2965. Kind, S., Geiger, A., Kießling, N., Schmitz, M. and Stark, R. (2020) ‘Haptic Interaction in Virtual Reality Environments for Manual Assembly Validation’, Procedia CIRP, vol. 91, no. 3, pp. 802–807. Liberatore, M. J. and Wagner, W. P. (2021) ‘Virtual, mixed, and augmented reality: a

systematic review for immersive systems research’, Virtual Reality, vol. 25, no. 3, pp. 773–799. Mayring, P. and Fenzl, T. (2014) ‘Qualitative Inhaltsanalyse’, in Baur, N. and Blasius, J. (eds) Handbuch Methoden der empirischen Sozialforschung, Wiesbaden, Springer VS, pp. 543–556. Preutenborbeck, M., Herzberger, N. and Flemisch, F. (2023) Virtual Assembly

Systems in Manufacturing Industry: Systematic Literature Review. unpublished. Preutenborbeck, M., Usai, M., Herzberger, N. and Flemisch, F. (2022) ‘Multimodal Human Systems Exploration with Tangible XR for the Internet of Production: An expert survey’, Production Management and Process Control, July 24-28, 2022, AHFE International. REFA (2023) Arbeitssystem [Online]. Available at https://refa-weiterbildung.de/arbe

itssystem/ (Accessed 4 August 2023). Sigrist, R., Rauter, G., Riener, R. and Wolf, P. (2013) ‘Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review’, Psychonomic bulletin & review, vol. 20, no. 1, pp. 21–53. Xia, P., Lopes, A. M., Restivo, M. T. and Yao, Y. (2012) ‘A new type haptics-

based virtual environment system for assembly training of complex products’, The International Journal of Advanced Manufacturing Technology, vol. 58, 1–4, pp. 379–396. Xie, B., Liu, H., Alghofaili, R., Zhang, Y., Jiang, Y., Lobo, F. D., Li, C., Li, W., Huang, H., Akdere, M., Mousas, C. and Yu, L.-F. (2021) ‘A Review on Virtual Reality Skill Training Applications’, Frontiers in Virtual Reality, vol. 2.

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-95-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →