Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Feedback-Driven Adaptive AR Assistance for Intralogistics: Design and Initial Evaluation

Universität der Bundeswehr München, 85579 Neubiberg, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1007510

Abstract

Manual order picking remains a central intralogistics activity, but performance is constrained by non-value-added travel and by search and verification effort at the shelf. Augmented-Reality pick-by-vision systems promise context-sensitive guidance directly in the field of view of the workers, yet practical deployment must cope with deviations such as empty compartments without breaking task flow. This paper presents the design and prototype implementation of a feedback-driven adaptive assistance concept for picking. Following a Design Science Research process, requirements were derived from a scenario analysis and an expert interview, then realized in a Unity-based simulation prototype that combines egocentric route guidance with bin-level highlighting and a deterministic correction mode. When a stock shortage is reported at the target shelf, the system switches to a predefined reserve location and recalculates guidance accordingly. The prototype was assessed in an exploratory qualitative think-aloud study (N = 5). Participants reported high confidence in task completion (mean 8.8/10) and highlighted the route line and shelf framing as helpful cues. They also noted usability issues with the visibility and affordance of the stockout reporting control. This highlights an AR trade-off, as prominent overlays can increase guidance visibility but may obstruct the environment.

INTRODUCTION

Order picking in picker-to-parts warehouses is still dominated by manual work and therefore by human performance. In such settings, non-value- added movement and search activities dominate the time budget. Travel accounts for roughly half of working time and shelf-level search for around one fifth (De Koster et al., 2007). Since travel does not create direct added value, it offers high leverage for optimization (De Koster et al., 2007).

At the same time, intralogistics organizations face pressure from both demand and labor dynamics. The paper situates this work against declining labor availability in logistics (Hempfing and Schwemmer, 2019) and increasing efficiency demands in intralogistics (Jäckl and Voigt, 2024). In parallel, e-commerce continues to drive order volumes and therefore picking throughput requirements (bevh - Bundesverband E-Commerce und Versandhandel Deutschland e.V., 2024).

Feedback-Driven Adaptive AR Assistance for Intralogistics: Design 107 Augmented-Reality (AR) pick-by-vision systems aim to externalize guidance by presenting context-sensitive information and visual cues directly at the point of work (Reif and Günthner, 2009; Rejeb et al., 2021). This work argues that such systems should not treat the worker as a passive receiver of instructions, but as a human-in-the-loop partner who can feed back deviations that are difficult to detect automatically in changing environments (Schirner et al., 2013; Quandt et al., 2023). Building on this framing, this paper designs and prototypes an adaptive assistance concept that reacts to a reported stock shortage by switching to a reserve target and rerouting guidance without leaving the task context.

This paper is organized as follows. Background and design considerations summarize related work and key implications for AR picking assistance. The method section describes the requirements process and study procedure. Design rationale and implementation detail how requirements were translated into the Unity prototype, and the evaluation and discussion report findings from the think-aloud study.

BACKGROUND AND DESIGN CONSIDERATIONS From a user perspective, shelf-level errors often stem from the cognitive effort of locating a storage position and identifying the correct item. Also, search inefficiencies lead to time losses and increase the risk of picking errors (“mis-picks”) (Rejeb et al., 2021).

Here, pick-by-vision is positioned as combining software-based flexibility with digital guidance and hands-free work compared to static paper-based methods and hardware-bound alternatives (Reif and Günthner, 2009). The scenario analysis further emphasizes acceptance factors for heterogeneous workforces. Older employees may prefer paper-based processes, perceived ease of use (PEU) is treated as critical for acceptance, and heterogeneous language proficiency motivates a text-light, symbol-based representation of work instructions (Rohacz and Strassburger, 2021; Reif and Günthner, 2009). The requirements framing also reflects cognitive ergonomics. Information density should be limited to essentials to avoid overloading working memory (Sweller, 1988) and to reduce the risk that prominent overlays capture attention at the expense of events in the environment (“attentional tunneling”) (Wickens and Alexander, 2009).

For navigation cues, the prototype follows an egocentric dynamic-line approach. The dynamic trajectory lines in virtual environments can reduce navigation task time compared to map-based tools by providing egocentric cues (Yi and Lee, 2022).

METHOD

The work follows Design Science Research (DSR), producing knowledge for practice through the construction and evaluation of an artifact (Hevner, 2007). In the relevance cycle, the requirements derive from a scenario analysis in a logistics company and a guided expert interview with management, focusing

on process deficits, acceptance factors, and infrastructural constraints. It also applies communicative validation (member checking) to confirm interpreted statements and derived requirements (Quandt et al., 2023).

Requirements are formalized as functional (FA) and non-functional (NFA) requirements, following the distinction between system behavior and quality attributes (Glinz, 2007). The catalog includes hands-free operation (FA-01), visual attention guidance (FA-02), information reduction (FA-03), process validation with explicit confirmation (FA-04), and data security (FA-05) (Reif and Günthner, 2009; Billinghurst et al., 2015; Syberfeldt et al., 2015; Quandt et al., 2023). Non-functional requirements capture usability and cognitive ergonomics, including intuitive usability (NFA-01) grounded in PEU as an acceptance factor (Rohacz and Strassburger, 2021), perceptual certainty with attention management (NFA-02) (Wickens and Alexander, 2009), and latency and performance considerations for comfort (NFA-03).

In the design cycle, the requirements are instantiated in a Unity-based desktop simulation prototype to validate interaction logic before hardware- specific deployment.

DESIGN RATIONALE AND TRACEABILITY This section makes the requirements-to-design relationship explicit. It links each identified deficit to the corresponding requirements, design decisions, implementation choices, and evaluation signals. The prototype is intentionally narrow in scope. It operationalizes a single high-impact deviation case, a reported stock shortage. It relies on deterministic control to keep system behavior predictable for users operating in a dynamic environment (Millington, 2019).

High search and orientation effort in picker-to-parts systems motivated FA-02 on visual attention guidance and NFA-01 on intuitive usability. The design therefore combines egocentric route cues with a bin-level confirmation frame. This supports both navigation and shelf-level verification. The prototype implements the route cue by projecting a dynamic guidance line in ARNavigation. The line is derived from a baked NavMesh route (Unity Technologies, 2016). HighlightTrigger activates the shelf frame only when the user enters a defined interaction radius.

The risk of visual overload from constant overlays informed FA-03 on information reduction and NFA-02 on perceptual certainty. The design consequently shows interaction elements only when they are task relevant. It removes highlights outside the shelf context. The prototype realizes this through context-sensitive activation and deactivation on trigger entry and exit. It also applies occlusion handling to preserve depth ordering between virtual guidance and static geometry (Macedo and Apolinario, 2023).

Deviations such as empty compartments can interrupt task flow. This motivated FA-04 on process validation and guidance. The design provides an explicit feedback control at the shelf and a predictable correction mode. In the prototype, ARManager embodies a finite-state decision model. It switches from the primary target to a predefined reserve target when feedback is provided. It updates status cues through color changes (Quandt et al., 2023; Millington, 2019).

Feedback-Driven Adaptive AR Assistance for Intralogistics: Design 109 Finally, NFA-03 on latency minimization guided decisions for robust and comfortable interaction in longer use. The design avoids per-frame expensive computations and stabilizes camera motion. The prototype therefore uses event-driven path recalculation. It includes a startup delay to ensure NavMesh readiness. Camera control constraints and jitter reduction were added as well. These choices were motivated by virtual motion sickness considerations (Yi and Lee, 2022).

PROTOTYPE DESIGN AND IMPLEMENTATION The prototype was implemented in Unity as a desktop simulation environment that represents a warehouse aisle layout. A component-oriented structure is used where scripts are attached to GameObjects, and a central manager coordinates process state and UI feedback. Common object-oriented design patterns (e.g., singleton and observer) are used to structure responsibilities and event propagation (Gamma et al., 1994).

Navigation is implemented in the ARNavigation component. Route computation relies on NavMesh Agent infrastructure of Unity, which uses an A* search over a polygonal navigation mesh (Unity Technologies, 2016). The A* evaluation function and the role of admissible heuristics for optimal paths (Hart et al., 1968; Russell and Norvig, 2010). In the prototype, the computed route is rendered via a continuously updated line projected onto the floor to provide egocentric guidance (see Figure 1).

Figure 1: User view in the unity-based simulation prototype at the start of an order. The head-up display presents task information, while a floor-projected route line provides egocentric navigation guidance.
Original PDF Figure 1:

At the shelf, HighlightTrigger provides visual attention guidance by activating a frame when the user enters the interaction zone. The frame both confirms that the correct bin has been reached and focuses attention on the target compartment. To minimize clutter, the frame is deactivated when leaving the zone and interaction elements follow the same context-sensitive principle to reduce user effort (Sweller, 1988).

Adaptive deviation handling is realized through state-based control logic of ARManager. When the worker reports a stock shortage using the on-site feedback button, the system enters a correction mode, switches the active target to a predefined reserve location, and recalculates navigation. The state variable that distinguishes standard and correction modes ensure deterministic behavior and clear state transitions (Millington, 2019).

Perception stability is considered a quality objective of the system. Correct occlusion of virtual guidance by static geometry is addressed to preserve depth perception and avoid misleading superimposition (Macedo and Apolinario, 2023). It also motivates technical measures for stable motion and reduced jitter to mitigate virtual motion sickness risk (Yi and Lee, 2022).

Unity 2022.3 LTS and Visual Studio Code were used for development. NavMesh baking was configured with an Agent Radius of 0.8, a Max Slope of 45°, and a Voxel Size of 0.266. Navigation is activated after a 0.5-second startup delay to ensure that the NavMesh has initialized.

EVALUATION AND DISCUSSION The prototype was evaluated in a qualitative user study conducted in the desktop simulation. Data collection followed the think-aloud method to capture participant reasoning during task execution and to surface mismatches between user mental models and the implemented logic (Van Someren et al., 1994).

The sample comprised five male participants aged 19 to 23 (N = 5). The authors justify this size using probabilistic arguments for discovering usability issues in iterative design, that a large portion of relevant problems can be identified with five participants (Turner et al., 2006) and framing this as an efficient strategy in early-stage evaluations (Nielsen, 2000).

Participants were briefed to act as new employees in a large warehouse with limited knowledge of the local language. They performed an order- picking task in which they had to locate and remove “Torx screws (M8)” with a specified quantity, and they were instructed to use the problem-solving mechanism of the system if an “empty compartment” occurred. The procedure thereby explicitly tested the correction mode under a known disturbance.

Measures combined subjective and qualitative data. Confidence in task completion was rated on a 1–10 scale, yielding a mean of 8.8/10. Qualitatively, participants repeatedly highlighted the navigation line (see Figure 1) as the most helpful element for quick orientation, and they described the bin-level frame as supportive for confirming the correct compartment. After reporting a stock shortage, all participants understood the rerouting behavior and rated it as useful.

Feedback-Driven Adaptive AR Assistance for Intralogistics: Design 111 The evaluation also revealed concrete usability issues. Several participants initially overlooked the stockout reporting control at the shelf, suggesting that its placement and affordance should be refined. One participant perceived the camera/movement lock (introduced to reduce motion sickness risk) as a potential crash, and acoustic feedback was described as too quiet in at least one case.

The results indicate that externalized visual guidance can support confident task completion, but they also expose interface trade-offs. Increasing visual prominence can improve recognizability, yet prominent overlays may obstruct the environment and contribute to attentional tunneling (Wickens and Alexander, 2009).

CONCLUSION

The paper presented a feedback-driven adaptive AR assistance concept for manual order picking and instantiated it as a Unity simulation prototype. The system combines egocentric route guidance and shelf-level highlighting with a deterministic correction mode that reroutes to a predefined reserve location when a stock shortage is reported.

Methodologically, the study was conducted as a desktop simulation without quantitative process time measurement, and interaction differed from real walking and AR headsets use. The small, homogeneous sample limits generalizability, and the implemented adaptation covers a defined stockout case.

Future work should validate the interaction concepts on AR headsets, broaden the evaluation to operational user groups, and integrate feedback events with a backend to support process improvement.

ACKNOWLEDGMENT

This paper is based on the bachelor’s thesis of LHH at Universität der Bundeswehr München. The authors MSN and SP would like to thank dtec. bw – Digitalization and Technology Research Center of the Bundeswehr for funding the project “Competencies for the Digital Working World (KoDiA) - Empowerment for Digitalization”. dtec.bw is funded by the European Union – NextGenerationEU. They are also grateful to all interview participants.

AI DISCLOSURE The authors used generative artificial intelligence tools to assist with language translation, editing, rephrasing, and improving readability of the manuscript. The AI tools were not used to generate scientific content, data, results, or conclusions. All responsibility for the content remains with the authors.

REFERENCES

bevh - Bundesverband E-Commerce und Versandhandel Deutschland e.V. (2024).

Interactive Commerce in Germany 2023. Excerpt from the 2023 results. URL: https://www.bevh.org.

Billinghurst, Mark, Adrian Clark, and Gun Lee (2015). “A survey of Augmented Reality”. In: Foundations and Trends in Human-Computer Interaction 8.2-3, pp. 73–272. De Koster, Rene, Tho Le-Duc, and Kees Jan Roodbergen (2007). “Design and control of

warehouse order picking: A literature review”. In: European Journal of Operational Research 182.2, pp. 481–501. Gamma, Erich et al. (1994). Design patterns: elements of reusable object-oriented software. Pearson Education. Glinz, Martin (2007). “On Non-Functional Requirements”. In: 15th IEEE International

Requirements Engineering Conference (RE’07). IEEE, pp. 21–26. DOI: 10.1109/RE. 2007.45. Hart, Peter E., Nils J. Nilsson, and Bertram Raphael (1968). “A formal basis for the heuristic determination of minimum cost paths”. In: IEEE transactions on Systems Science and Cybernetics 4.2, pp. 100–107. Hempfing, Alexander, and Martin Schwemmer (2019). Skilled labor shortage in

logistics: Measurement, structure, and areas for action. White paper. Nuremberg: Fraunhofer Institute for Industrial Engineering IBK and Working Group for Supply Chain Services SCS. Hevner, Alan R. (2007). “A Three Cycle View of Design Science Research”. In: Scandinavian Journal of Information Systems 19.2, pp. 87–92. Jäckl, Felix and Felix Voigt (2024). TMG Study: Automation and Digitalization in

Intralogistics. Technical Report. Stuttgart: TMG Consultants GmbH. Macedo, Marcio C. F. and Antonio L. Apolinario (2023). “Occlusion Handling in Augmented Reality: Past, Present and Future”. In: IEEE Transactions on Visualization, and Computer Graphics 29.2, pp. 1590–1611. Millington, Ian (2019). AI for Games. 3rd ed. Boca Raton: CRC Press. ISBN: 978-

1138483972. Nielsen, Jakob (2000). Why You Only Need to Test with 5 Users. Nielsen Norman Group. URL: https://www.nngroup.com/articles/why-you-only-need-to-test-with- 5-users/ (accessed on December 17, 2023). Quandt, Moritz et al. (2023). “Challenges in Designing and Implementing Augmented

Reality-Based Decision Support Systems for Intralogistics: A Multiple Case Study”. In: IFIP International Conference on Advances in Production Management Systems. Springer, pp. 343–356. Reif, Rupert, and Willibald A. Günthner (2009). “Pick-by-vision: Augmented Reality supported order picking”. In: The Visual Computer 25.5, pp. 461–467. Rejeb, Abderahman et al. (2021). “The potentials of augmented reality in supply chain

management: a state-of-the-art review”. In: Management Review Quarterly 71.4, pp. 819–856. Rohacz, Anke, and Steffen Strassburger (2021). “The acceptance of augmented reality as a determining factor in intralogistics planning”. In: Procedia CIRP. Vol. 104. Elsevier, pp. 1209–1214. Russell, Stuart J. and Peter Norvig (2010). Artificial Intelligence: A Modern Approach.

3rd. Upper Saddle River, NJ: Prentice Hall. Schirner, Gunar et al. (2013). “The future of human-in-the-loop cyber-physical systems”. In: Computer 46.1, pp. 36–45. Sweller, John (1988). “Cognitive Load During Problem Solving: Effects on Learning”.

In: Cognitive Science 12.2, pp. 257–285. Syberfeldt, Anna et al. (2015). “Visual assembling guidance using augmented reality”.

In: Procedia CIRP 37, pp. 291–296. Turner, Carl W., James R. Lewis, and Jakob Nielsen (2006). “Determining Usability Test Sample Size”. In: International Encyclopedia of Ergonomics and Human Factors. Edited by Waldemar Karwowski. CRC Press, pp. 3084–3088.

Feedback-Driven Adaptive AR Assistance for Intralogistics: Design 113 Unity Technologies (2016). Unity User Manual (Version 5.4). Online documentation

for Unity NavMesh. Unity Technologies. URL: https://docs.unity3d.com/540/ Documentation/Manual/index.html. (accessed on January 29, 2026) Van Someren, Maarten W., Yvonne F. Barnard, and Jacobijn A. Sandberg (1994). The think aloud method: A practical guide to modeling cognitive processes. London: Academic Press. Wickens, Christopher D. and Amy L. Alexander (2009). “Attentional tunneling

and task-management”. In: International Journal of Aviation Psychology 19.2, pp. 182–199. Yi, Chen and Yih-Chuan Lee (2022). “Exploring the Effect of Navigation Tool Design on Virtual Environment’s Navigation and Revisiting Experience”. In: Proceedings of the 18th ACM SIGGRAPH International Conference on Virtual- Reality Continuum and its Applications in Industry. ACM, pp. 1–9.

Cover of the proceedings volume Human-Computer Interaction & Emerging Technologies

Explore the complete digital volume

Human-Computer Interaction & Emerging Technologies

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 979-8-950676-09-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →