Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Technical and Socio-Technical Success Factors of AI-Based Knowledge Management Projects

1 ifaa – Institute of Applied Industrial Engineering and Ergonomics, 40474 Düsseldorf, Germany ↗
2 Chair of Digitalization in Mechanical Engineering, Hochschule Niederrhein University of Applied Science, 47805 Krefeld, Germany 3consipio Software Engineering GmbH, 74074 Heilbronn, Germany 4sensrec service UG, 47057 Duisburg, Germany ↗
3 consipio Software Engineering GmbH, 74074 Heilbronn, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1005354

Abstract

The demographic change has a large impact on the labour market and poses a challenge to companies. With many employees going into retirement within the next 10 years, it is not just the workforce itself leaving the firms, but also their experiential knowledge that the workers gained over the years. Much of it is tacit and thus unobtainable through common documentaries of work processes. Keeping it inside of the company is crucial to ensure productivity and educate the upcoming generation of workers in their company. The project “KI_eeper – Know how to keep” has the goal to capture experiential knowledge and provide it to the workers during the production process automatically through an AI-based assistance system. The system is currently under development and requires careful consideration of the users’ needs at the production line. By choosing a participative approach, the employees are directly in touch with the developers and can influence the development of the system significantly. Managing both the available technical capabilities as well as the demands of the employees towards the system at the same time is key to have a successful outcome of the project. This paper shares the essential success factors both on the technical and socio-technical level to secure a seamless integration of an AI-based assistance system into production processes, based on a case study in a German manufacturing company.

INTRODUCTION

Due to the demographic change and the imminent retirement of the Baby Boomer-generation from the 1960s, companies are facing the challenge to identify their experiential knowledge and make it accessible for the upcoming workforce generations. Capturing experiential knowledge which is mostly subconscious and exclusive to long-year workers is difficult. Cur- rent approaches for knowledge capturing and assistance are both extensive

© 2024. Published by AHFE Open Access. All rights reserved. 68

Technical and Socio-Technical Success Factors 69

and costly. Because of the increasing complexity of industrial production processes, it would be necessary to capture and process various specific infor- mation in texts, images, and video (Mohajan, 2016). More so, employees need an intuitive and comfortable way to access the information directly during their work practice. In the research project “KI_eeper – Know how to keep”, an AI-based assistance system with self-learning capability for experience-reliant tasks in production areas is developed and tested. Cap- turing the experiential knowledge as well as the buildup of the assistance system will be automated in a resource-saving way.

The KI_eeper assistance system is developed and currently piloted at the company apra-norm in the surface technology (ST) department. Their tasks are especially complex because of the variety of products that all need differ- ent treatments which requires the workers to have extensive knowledge and experience. The impact of the first station, the hanging station, is especially significant. Their performance influences the cycle time. Mostly the wrong treatment of the products and the wrong combination at this station cause that the ST do not fulfil the cycle time and need more resources than planned.

The KI_eeper AI algorithm should learn from the experienced team mem- bers and captures their knowledge directly in the work process. The goal of the first phase (implementation) is to ensure a highly qualitative data pool for the AI to learn with. In the second phase (assistance) the data base will be used to generate assistance functions. The implementation of the software to build up the database at the apra-norm ST is being initiated.

The human-centred approach of the entire system development is focused on a design philosophy close to the worker’s routine and the user acceptance of the technology. Empowering and even enthuse the employees to work with the new AI-based technology is one of the objectives in the project. Therefore, the project-team created and piloted an participative approach, which gives the employees the possibility to bring in their ideas and needs in the development process of their assistance technology. This approach and especially the implementation process of the software will be presented in this contribution.

THE IMPORTANCE OF CO-CREATION AND EMPLOYEE ACCEPTANCE IN DIGITALIZATION PROJECTS The focus of this paper is to emphasize how important it is for an organi- zation’s adoption of digital technologies, especially Artificial Intelligence, to involve employees into the design of the system before brought into their work routine (Stowasser & Suchy, 2021). There are especially two main effects: Building up employee acceptance towards the newly implemented systems and ensuring a system design that is aligned with the needs of the employees in their work processes (Ottersböck et al., 2024a). The first steps are taken by establishing transparency through communication and information flow. Using tools like workshops, actively supporting spreading information through meetings and internal media are common practices to achieve that objective (Ben Rehouma et al., 2020). The employee acceptance is essentially driven by giving the employees full agency over the outcome of the project and have significant impact on the technical design. By this

approach, you establish an emotional connection and make it their “own” project (Ottersböck et al., 2024b). By participating in the development pro- cess, trust is built towards both the developers of the digital system or artificial intelligence as well as the system itself. Reflexive approaches are recommended to put the user ultimately into the centre of the design process (Osterheider et al., 2023). Technological factors that are chosen in the right degree have a positive impact on employee acceptance and by co-creating the system, the knowledge of the team regarding the work process is integrated into the system to meet the requirements of the workers, deal with possible ethical concerns and avoid undesired development (Ottersböck et al., 2024b; Na et al., 2022).

CASE STUDY KI_EEPER ASSISTANCE SYSTEM IN SURFACE TECHNOLOGY The KI_eeper assistance system is being developed and piloted at apra-norm in surface technology (ST). In this area of work, currently over 2900 differ- ent products are treated in their own distinct way. The products are hung on a hanging conveyor belt system at the first station and then pass through the other two stations of the ST: the painting station and the quality con- trol. Especially the employees of the first station, the hanging station, must perform well. Due to the high variety of products, they need comprehensive knowledge, such as the correct alignment of the products on the conveyor trolleys or the knowledge of whether any pre-treatments such as rinsing, acid baths, or masking activities are necessary. It is also essential to combine products, as some products need more time in the paint station than others. A fitting combination ensures that the 2.2–minute production cycle can be maintained, and the resources of the other stations are optimally utilized. An assistance system is wanted even by experienced employees, as the complex- ity of processing increases with each new product variant (Ottersböck et al., 2023). The designed and developed AI system should learn independently in the work process from all experienced team members and then support all workers.

Figure 1: Technical development KI_eeper in three phases: data capture, AI training, assistance (Ottersböck et al., 2024b).
Original PDF Figure 1:

Technical and Socio-Technical Success Factors 71

This first phase of data capture (Phase 1) forms the data base for training the AI system (Phase 2) and is essential for the identification of experiential knowledge and the development of the assistance functions (Phase 3) (see Figure 1). The employees have a special status here, as they contribute signif- icantly to building a high-quality data pool without receiving any apparent added value in the form of assistance functions at this stage. Already in this phase, the work process will change, and there will be additional effort due to manual data entry into the system. The challenge, therefore, lies particularly in enabling and inspiring employees to work with the new data capturing system. To ensure this, an employee-centred approach was developed and tested.

Approach, Tools and Success Factors to Ensure Success on the Sociotechnical Level From the outset, employees in the pilot area of surface technology were participatively involved in the project. In an initial information meeting, employees were informed about the background, goals, and procedures of the project, as well as the potential of AI in production. The meeting also served to address and alleviate existing fears as well as concerns regarding the use of technology and AI. The contents of the meeting were discussed in advance with internal project managers and the works council to check if the content and the special technical words fit to the target group or need to be more explained. Visual representations were used extensively to manage language barriers. The workshop formed the basis for subsequent analytical and participative measures (Ottersböck et al., 2024a).

As part of a process and requirements analysis, employees of ST were sur- veyed about their activities, challenges in the work area, and their affinity for technical systems. Silent and participatory observations during task execution provided insights into the process flow, experience-driven tasks, and where digital assistance can offer help. In a consolidation workshop, the results of the analysis were discussed and completed with the employees of ST. At a so-called “ELSI+UX workshop”, employees evaluated the technical con- cept based on deployment scenarios in their work processes. The focus of the evaluation was on the ethical, legal, and social implications of technol- ogy (ELSI) (Greenbaum, 2015). Additionally, employees had the opportunity to assess the user-friendliness and functions of the system (Ottersböck et al., 2024a). User-friendliness is a key factor of acceptance for newly implemented digitalized system, especially AI (Leitenbauer et al., 2023).

The measures that were presented in the first section of this chapter are usually driven by the project lead that acts like a third party connecting the perspectives of the company lead and the team that will use the assistance system in their daily work. Nonetheless, also the production department itself can be active and drive the development of the AI-based assistance system without the help of the AI developers and only small support from project leaders.

In the daily shop floor meeting, employees were informed by the depart- ment head, the internal project leader, the works council, external project

managers, and developers about the introduction of the software. The intro- duction to the software was done “learning-by-doing" in the work process. Employees were supported during their activities by project participants and the department head. Through participatory observations during the first deployment, initial optimization opportunities were identified. Additionally, a flip chart was placed in the work area. This is used to continuously collect improvement suggestions from the employees.

To summarize, the essential success factor at the sociotechnical level is the direct inclusion of employees in the development process. It is crucial to rely on both the individual and the collective feedback of the team. The base for a successful implementation is thus achieved by co creation, to foster user acceptance and thorough process intelligence. It is achieved through infor- mation events, workshops, and analyses, but also through direct feedback in the implementation phase during the piloting of the system in ST (Ottersböck et al., 2024b).

Methodology and Success Factors in the Technological Dimension The technic team developed a software for data acquisition in ST based on the results of the requirements and process analysis with ST staff. The software can be used for the entire ST process. For this purpose, three active stations were implemented, where the data entry by the employees takes place. These are a main station where the hanging process takes place and two confirma- tion stations at the coating and quality control stations. At the latter, the quality of the process is assessed, which determines whether the data set is added to the data pool. In addition, a passive photo sensor station was installed, which generates an image of the passing trolley without any active input from the operator. A schematic representation of the ST including all input stations is shown in Figure 2.

Figure 2: Schematic representation of the complete ST process including the stations which actively and passively collect data (Ottersböck et al., 2023 (modified)).
Original PDF Figure 2:

Technical and Socio-Technical Success Factors 73

A web-based system was chosen for the implementation of the software. This decision was based on the fact that a much higher level of mutual pro- cessing between web applications can be achieved than with isolated desktop systems. User access can be carried out from different end devices (e.g. tablets or desktop) and, in addition to simpler maintenance, greater security can also be achieved (Sotnik et al., 2023). For this purpose, a web server was set up which represents the backend and the database, that can be accessed by n-many end devices (see Figure 3).

Figure 3: Structure of the software: stations (Ottersböck et al., 2024b).
Original PDF Figure 3:

In the specific project of apra-norm, three active stations were set up as described, on which end devices were installed. In the case of the hanging station, a desktop system was installed to allow several applications to run simultaneously in the future. For the other two active stations, only a tablet was used. The passive photo station does not require a terminal device to collect data for the AI-system. It will be triggered from an external RFID reader (see Figure 4).

Figure 4: Structure and organization of the software: processes (Ottersböck et al., 2024b).
Original PDF Figure 4:

The application consists of two main parts: data capture and the assistance function. Both areas are realized through sequential processes of successive actions. Here, flexibility and reusability were paramount, which is why a modular and generic approach was chosen. Each sub-action of a process can be configured in any order and has specific settings that can be predefined. This allows for quick adjustments to be made in changing conditions with- out the need for programming. The number of processes and the sub-actions contained in each process are arbitrary. Directly repeating carriage contents can be copied into the following entry using a repeat button, so that dou- ble manual entry can be avoided. However, inputs do not always have to be made active by persons. Data can also be recorded by externally connected devices/sensors and transmitted to the system (see Figure 5).

Figure 5: Structure of the software: process example (Ottersböck et al., 2024b).
Original PDF Figure 5:

The collected data is used to train the AI network. For the assistance func- tion, this network is then queried in real-time based on input parameters for a solution approach and then provides a suggestion that is graphically or textual displayed to the employees at the station terminal. An optimal imple- mentation of the graphical interface was a prerequisite to enthuse employees for the system and not to deter them from using it. In the graphic design, particular attention was paid to a clear and easy-to-use structure of the user interface.

In a production environment, an employee usually does not have much time to deal with complicated user interfaces. The UI (User Interface) should therefore be intuitive and not overloaded, so that it quickly becomes clear how to operate the application without instructions. Another reason for a simple and visual UI design is to create a system that is as accessible as possible so that inexperienced users and employees can also use the system without restrictions.

All possible problems while using the system and needs of the employees must be taken in account which is done by directly accompanying the staff in their first steps of using the system. Individual employees at the workstation where the system is piloted, the entire department team and the organization

Technical and Socio-Technical Success Factors 75

as a whole face distinct challenges during this phase that need to be addressed accordingly.

Collaboration and regular exchange with the employees led to a successful interim result. Even though the technical conception was already very mature, challenges in the implementation in practice still arose. Through exchange with the employees, emerging problems or suggestions for improvement were quickly identified and resolved (Ottersböck et al., 2024b). As a result, specific system functions were created, for example, the ability to retrieve previously entered wagon configurations again, instead of having to enter them each time, as it is not uncommon in larger orders to have wagons that are exactly identically loaded.

CONCLUSION

The implementation of the software at apra-norm in ST went off on a sat- isfactory start. Employees could quickly work with the system thanks to its intuitive usability. As a key success factor on the technical level, the maturity of the system was constantly evaluated. The software was optimized gradu- ally in several testing phases and is still undergoing improvements. The data pool to train the AI algorithm is now slowly getting built up with a prototype implementation of the data capture system, both for automatic and manual inputs. The aim was to avoid dissatisfaction of the employees within missing functionalities of the system or a technic which is not user-friendly. The par- ticipative way fostered user-acceptance and gave the employees in the ST the possibility to take part on the development process of the system. This also had a positive effect on the user-friendliness as the feedback of the employees facilitated a custom-tailored software solution, fitted to the demands of the users. Through the iterative nature of the approach, the project has not run out of ways to improve the processes and align the technical system with it. An important objective of the project is to present approaches both on the technical and socio-technical level that are generally applicable and are not limited to the special cases of the firms in this project.

ACKNOWLEDGMENT

This research project is funded by the German Federal Ministry of Educa- tion and Research (BMBF) within the framework of the program “Zukunft der Arbeit” (funding codes 02L20C500–02L20C505). Furthermore, we thank the Project Management Agency Karlsruhe (PTKA) for supporting the project. The authors are responsible for the content of this publication.

REFERENCES

Ben Rehouma, M., Geyer, T. and Kahl, T. (2020). Investigating Change Management Based on Participation and Acceptance of IT in the Public Sector: A Mixed Research Study. International Journal of Public Administration in the Digital Age. Volume 7, pp. 51–70. doi: 10.4018/IJPADA.20201001.oa4.

da Silveira, G., Borenstein, D. and Fogliatto, F. (2001). Mass Customization: Literature Review and Research Directions. International Journal of Production Economics. Volume 72, pp. 1–13. doi: 10.1016/S0925-5273(00)00079-7. Geis-Thöne W (2021). Mögliche Entwicklungen des Fachkräfteangebots bis zum

Jahr 2040: Eine Betrachtung der zentralen Determinanten und Vorausberechnung. IW-Report. Volume 11, pp. 1–40. Greenbaum, D. (2015). Expanding ELSI to all areas of innovative science and technology. Nature Biotechnology. Volume 33, No. 4, pp. 425–426. Leitenbauer, L., Sorko, S. and Lutz, M. (2024). Reducing Change Resis- tance: Stakeholder-based Approach for Extended Reality (XR) Implementa- tions. Tehniˇcki glasnik. Volume 18, No. 1, pp. 115–121. doi: 10.31803/tg- 20230508224609. Mohajan, H. (2016). Sharing of Tacit Knowledge in Organizations: A Review.

American Journal of Computer Science and Engineering. Volume 3, No. 2, pp. 6–19. Na, S., Heo, S., Han, S., Shin, Y. and Roh Y. (2022). Acceptance Model of Artificial Intelligence (AI)-Based Technologies in Construction Firms: Applying the Technology Acceptance Model (TAM) in Combination with the Technology– Organisation–Environment (TOE) Framework. Buildings. Volume 12, No. 2, p. 90. doi: 10.3390/buildings12020090. Osterheider, A., Klapperich, H., Stein, E., Weiler, T., Endter, C., Huldtgren, A.

and Müller, C. (2023). Conceptualization of the Understanding of Participa- tion and Co-Creation in Interdisciplinary Research Groups developing Digital Health Technology: An Exploratory Study: Conceptualization of the Understand- ing of Participation and Co-Creation. Proceedings of Mensch und Computer 2023 (MuC’23). Association for Computing Machinery, New York, NY, USA, pp. 534–538. doi: 10.1145/3603555.3608572. Ottersböck, N., Discher, A., Cost Reyes, C., Ternes, J., and Dander, H. (2024b) Implementierung eines selbstlernenden KI-Assistenzsystems in der Produktion: Mitarbeiterzentrierte Vorgehensweise bei der Softwareeinführung zum Aufbau des Datenpools. In: 70. GfA-Frühjahrskongress – “Arbeitswissenschaft in-the-loop” (in print), Stuttgart, Germany. Ottersböck, N., Prange, C., Rusch, T. and Dander, H. (2023). Entlastung von

Beschäftigten in komplexen Produktionsumgebungen durch informatorische, KI- basierte Assistenztechnologien - Erfahrungswissen ermitteln, erhalten, trans- ferieren und Prozesse optimieren. AI: MAG - The World of Tomorrow Volume 1, pp. 27–28. Ottersböck, N., Urban, I., Cost Reyes, C., Peters, S. and Boiteux, C. (2024a).

Employee Acceptance for AI Based Knowledge Transfer. Conception, Realization and Results of an ELSI+UX Workshop. 5th International Conference on Industry 4.0 and Smart Manufacturing (in print), Lisbon, Portugal. Sotnik, S., Shakurova, T. and Lyashenko, V. (2023). Development Features Web- Applications, International Journal of Academic and Applied Research (IJAAR) Volume 7, No. 1. Stowasser, S., and Suchy, O. (2022). Einführung von KI-Systemen in Unternehmen.

Gestaltungsansätze für das Change Management. Whitepaper. Plattform Lernende Systeme, Munich, Germany.

Cover of the proceedings volume Human Factors and Systems Interaction

Explore the complete digital volume

Human Factors and Systems Interaction

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-30-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →