Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Design Process for Augmented Reality (AR) Experiences from the Perspectives of UX and Game Designers

Loyola University New Orleans, New Orleans, LA 70118, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1005487

Abstract

This paper presents a cognitive study comparing the approaches of User Experience (UX) designers and game designers in Augmented Reality (AR) experience design. The AR Haunted College is a scenario-based sequential AR experience that transforms the Monroe Hall building at Loyola University New Orleans into a chilling Halloween spectacle. The design of AR Haunted College involved contributions from both UX designers and game designers. We characterize the distinct approaches of UX and game designers by analysing the design process of the AR Haunted College developed in an agile project management framework. Our results show common and distinct HCI and game design principles between UX and game designers’ perspectives. Building on the findings of the analysis, we propose a set of design principles as a basis for an AR design framework and associated heuristics.

INTRODUCTION

Augmented Reality (AR) technology has experienced rapid growth and integration across various industries, providing innovative and immersive experiences for users. AR applications extend from gaming and entertainment to real-world problem-solving, offering new dimensions for engagement and interaction. Within the realm of AR development, professionals with diverse backgrounds collaborate to create impactful experiences, including game designers and User Experience (UX) designers. Game designers bring their expertise in crafting captivating narratives, interactive mechanics, and engaging gameplay elements to the table. Their focus lies in creating experiences that captivate and entertain users, often with a strong emphasis on storytelling and user engagement. On the other hand, interaction designers are dedicated to ensuring that AR experiences are user-friendly, intuitive, and deliver on the needs and expectations of the end-users. They are concerned with aspects such as accessibility, usability, and overall user satisfaction. This research seeks to uncover and analyze the distinct approaches of game designers and UX designers in developing AR experiences across various domains, fostering innovation and improvement in the field of Augmented Reality.

© 2024. Published by AHFE Open Access. All rights reserved. 279

280 Kim and Lee

We present an AR design project, AR Haunted College, that provides a set of sequential AR experiences in a scenario-based Halloween theme. The design of AR Haunted College involved contributions from both UX designers and game designers. We analyze the design process of the AR Haunted College in each perspective of UX designers and game designers to identify their unique design approaches.

This research addresses the following research question:

  • What are the distinct approaches between UX design and game design that influence the design and user interactions in designing augmented reality (AR) experiences?

This research addresses the lack of studies on design principles for the next generation of AR experiences by proposing a new approach that investigates two different disciplines related to AR experience design.

DESIGN PRINCIPLES FOR AUGMENTED REALITY While there is some research on augmented reality design guidelines, most of it is narrow, specific to particular problems, and does not provide generalized principles for AR experiences (Dunleavy, 2014; Dünser et al., 2007; Krüger and Bodemer, 2022; Liang, 2018, 2016).

Dünser et al. (2007) investigated how to apply HCI design principles in an AR design. They introduced eight design principles and heuristics with some examples: affordance, reducing cognitive overhead, low physical effort, learnability, user satisfaction, flexibility in use, responsiveness and feedback, and error tolerance. Dunleavy (2014) introduced design principles for augmented reality learning: Enable and then challenge, drive by gamified story, see the unseen. These design principles emphasize the use of affordance and reducing cognitive overhead in AR learning settings. Kourouthanassis et al. (2015) proposed five design principles for the development of mobile augmented reality applications.: Use the context for providing content, deliver relevant-to-the-task content, Inform about content privacy, Provide feedback about the infrastructure’s behavior, Support procedural and semantic memory. Liang (2016; 2018) proposed six AR design principles for older adults (Diminished Augmentation, Modality- rich Augmentation, Instantaneous Augmentation, Accurate Augmentation, Augmented Augmentation and Transparent Augmentation) based on five key features of AR design (i.e., Changeability, Synchronicity, Partial one to one, Hidden Reality and Registration). Krüger and Bodemer (2022) applied two basic multimedia design principles in AR learning environments: the spatial contiguity principle with visual learning material, and the coherence principle with audiovisual learning material, leveraging AR-specific contextuality potentials.

The existing research on AR design principles mentioned above relies heavily on human-computer interaction (HCI) design principles and usability principles (Dunleavy, 2014; Dünser et al., 2007; Liang, 2016; 2018), or focuses on the technical characteristics of AR technology (Krüger and Bodemer, 2022; Schmalstieg and Hollerer, 2016), rather than on the holistic

Design Process for Augmented Reality (AR) Experiences 281

user experience of AR design. As recent AR experiences have become much more complex, with multiple purposes and multimodal interaction, existing HCI design principles are limited in their ability to lead to successful AR experiences.

Recent research on AR design (Alqahtani and Kavakli-Thorne, 2020; Li et al., 2017; Yu et al., 2022; Zuo et al., 2022) has incorporated game design aspects to support educational purposes, demonstrating the potential of game-based learning with AR to facilitate learning in various contexts. The CybAR Game (Alqahtani and Kavakli-Thorne, 2020) is an AR mobile application that teaches cybersecurity concepts and demonstrates the consequences of actual cybersecurity attacks through feedback. To develop CybAR Game, the researchers conducted a focus group to identify key components that needed to be addressed in the design. The key components are roles, goals and objectives, outcome and feedback, rewards, and interaction. These key components include some parts of both HCI design principles and game design principles. Li et al. (2017) reviewed previous studies of AR games for learning. They proposed five considerations for designing AR learning games from their study: learner groups, learning objectives, AR features, game mechanics, and social interactions. Zuo et al. (2022) studied situating learning in AR fantasy and presented design considerations for AR game-based learning. They emphasized the use of narratives and simple interactions to increase learning effects as design considerations.

The existing research on AR game design for learning experiences mentioned above has primarily focused on applying specific game elements to achieve learning objectives, rather than implementing comprehensive game design principles. In this study, we investigate the approach of game designers towards AR design, exploring how game design principles can be integrated with HCI design principles for augmented reality experiences.

AR HAUNTED COLLEGE To investigate different perspectives on interaction design and game design for AR experiences, we developed a series of AR experiences involving contributions from both UX designers and game designers. The AR Haunted College is a scenario-based sequential AR experience that transforms the Monroe Hall building at Loyola University New Orleans into a chilling Halloween spectacle. Comprising 18 AR experiences that begin at the entrance of the building and culminate on the 5th floor, the AR Haunted College offers users a cohesive narrative to engage with. Some experiences involve jump scares from animated objects, while others require user actions like avoiding zombies or entering specific rooms to complete missions. The AR Haunted College was created using Adobe Aero, and its AR designs include 2D and 3D objects, videos, layered 2D elements, 3D-scanned objects, a blend of video and animated 2D elements, and various sound effects. Interactions within the AR Haunted College involve a dynamic mix of tap interactions and proximity interactions.

282 Kim and Lee

Figure 1 shows some of the AR experiences in AR Haunted College. Figure 1a is the first user story. Users face the AR entrance to the haunted house at the entrance of the Monroe Hall building. Ghosts come and go at this entrance, and when the user opens the door, they receive a welcome message and are guided to the next location. Figure 1b shows a user story on the 3rd floor. As the user runs away from a group of small skeletons, they happen to discover a ticket office shrouded in darkness. The skeleton standing in front of the ticket booth looks at the user and asks if they have been invited to Halloween Town or if they have come to conduct official business with Dr. Kim. If the user replies that they have been invited to Halloween Town, the skeleton gives them a ticket. On the other hand, if the user declares that they want to meet Dr. Kim, the skeleton immediately provides Dr. Kim’s room number. Figure 1c shows a user story on the 4th floor. Upon arriving at the exhibition hall, the user finds four students, 3d scanning models, transformed into plaster statues. Approaching Lila cautiously, she explains they are cursed and pleads for help. Then, the user approaches Shay’s statue, who reveals her sketchbook is cursed and requests its destruction. Finally, approaching Jada’s statue, who informs them about a sketchbook on the 5th floor that can lift the curse. The user is instructed to seek clues in the posters on the wall to break the curse. Figure 1d shows a user story on the 5th floor. The user opens locker number 117. Inside, among the thick dust, lies an old sketchbook. As the user picks it up, it comes to life, blinking and writhing as if trying to escape the user’s grasp. The user quickly flicks a lighter and the sketchbook bursts into flames, disappearing in a puff of ash.

Figure 1: AR Experiences within the AR haunted college.
Original PDF Figure 1:

Design Process for Augmented Reality (AR) Experiences 283

RESEARCH METHOD The design of AR Haunted College involved contributions from both UX designers and game designers. Five interactive design major students enrolled in the Agile Development Process course developed an AR design project using an agile project management framework, an iterative approach to managing software development projects that focuses on continuous releases and incorporates customer feedback with every iteration. Students completed a sprint, a fixed period of two weeks, during which they designed and completed a certain portion of the project. The design project included 4 sprints, and each sprint included a daily meeting called the daily scrum, design work, development work, design testing, a sprint review meeting, and a sprint retrospective meeting.

Additionally, six students enrolled in the Game Design course actively engaged in AR design projects, participating in 4 focus group discussions, one focus group after each sprint, to identify issues and potential improvements from a game design perspective. The evaluation process comprised two sessions: testing and focus groups. The protocol, including the informed consent document, has been reviewed and approved by our IRB, and we obtained informed consent from all students to conduct the study.

To examine the UX designers’ perspective on designing the AR experience, we recorded all daily scrum meetings (16 meetings), sprint review meetings (4 meetings), and sprint retrospective meetings (4 meetings). The verbal data reflects how UX designers approach design, highlighting the design aspects they focus on during the design process and self-evaluation, outlining encountered design challenges, and detailing interactions and user experiences they aim to improve and strengthen. For investigating the viewpoint of game designers in shaping the AR experience, we recorded all focus group sessions (4 sessions). The verbal data illustrates how game designers approach the design of AR experiences, the design aspects they focus on in AR design, the game design principles they apply to AR design, and how they use game elements in non-game AR design. All verbal data collected from both the Agile Development Process course and the Game Design course was transcribed for analysis.

THEMATIC ANALYSIS OF COLLECTED DATA We characterize the distinct approaches of UX and game designers by identifying emerging themes from the verbal data. Through our analysis, we identified various themes related to design principles, the design process, project management, technical consideration, team collaboration, and documentation. After completing the initial round of coding and grouping with the identified codes, we extracted only the coded data relevant to both HCI design principles (Masmuzidin and Aziz, 2019; Nielsen, 2005, 1994) and game design principles (Brathwaite and Schreiber, 2009), and then proceeded to interpret the data. We categorized the emerging themes into perspectives from UX designers and game designers, and extracted relevant design principles based on these themes.

284 Kim and Lee

UX Designers’ Perspective Five themes emerged from the verbal data of the Agile Development Process course, from the perspective of UX designers:

  • Graphics and Visual Experience: Students frequently discussed the quality of graphics, noting specific improvements in visual effects and overall immersion. Additionally, suggestions were made to further enhance visual elements by incorporating dynamic effects such as smoke, blood, and moving objects. However, concerns were raised regarding text readability, prompting discussions on the necessity for better contrast and background colors to improve clarity. Students primarily focused on the quality of graphics, dynamic graphic effects, and visual information readability, which are related to HCI design principles of Aesthetic and Minimalist Design.
  • Interactivity and Engagement: Discussions regarding interactive elements and user experience underscored the importance of incorporating engaging elements to prolong user engagement. Suggestions included adding game-like features such as puzzles, challenges, and interactive characters to enhance the overall experience. Furthermore, students emphasized the need for intentional and thematic connections between different elements to maintain coherence throughout the user journey. This relates to HCI design principles of Engagement, Consistency, and Informative Feedback, emphasizing the relationship between interaction and the game design principle of Theme (i.e., a background story that provides context for what the players need to accomplish).
  • Audio and Sound Effects: The analysis revealed a recurring theme surrounding the role of audio and sound effects in shaping the user experience. Students highlighted the significance of sound effects and background music in setting the mood and enhancing immersion. Recommendations focused on improving audio cues, including timing adjustments and ensuring alignment with narrative and visual elements to foster a cohesive experience. The relevant HCI design principles are Affordance, Signifier, and Visibility of System Status. This is also related to the game design principles of Theme.
  • Spatial Design and Exploration: Spatial design has emerged as a crucial aspect of the user experience, particularly in the context of augmented reality (AR) experiences. Students acknowledged the importance of spatial design in creating immersive experiences, suggesting strategies to optimize physical space for exploration and improve the placement of AR elements. Discussions also centered on reducing clutter and optimizing cues to prevent overwhelming users during interaction. Since the AR interface type utilizes the Exploring type of interaction, maximizing physical and digital exploration is crucial in AR user experience. The relevant HCI design principles are Affordance, Signifier, and Exploration of Virtual World. This is also related to the game design principles of Game Space (i.e., the entire area of the game).
  • Narrative and Storytelling: Lastly, students emphasized the pivotal role of narrative and storytelling in driving user engagement and maintaining

Design Process for Augmented Reality (AR) Experiences 285

interest. Discussions included refining narrative structures, integrating riddles and puzzles, and incorporating physical artifacts to complement digital narratives and encourage deeper exploration. The relevant HCI design principle is Engagement. This is also related to the game design principles of Game Dynamics (i.e., patterns of play that arise from mechanics once they are set in motion by players) and Theme.

Game Designers’ Perspective Six themes emerged from the verbal data of the Game Design course, from the perspective of Game designers:

  • Interactivity and Engagement: Students expressed enthusiasm for the interactive elements of the AR experience, such as the storyline, quests, and treasure hunt-like clues. Suggestions were made to amplify interactivity by integrating more interactive elements and puzzles, which could deepen user engagement and immersion. This result closely aligns with the perspective of UX designers.
  • Graphics and Realism: Feedback emphasized the importance of high- quality graphics and realism in virtual elements. Students discussed the realistic appearance of virtual characters, such as zombies, and their seamless integration into the physical environment. Suggestions were made to enhance graphics quality and improve the seamless integration of virtual and physical elements, thereby further enhancing the immersive nature of the experience. This result suggests that the emphasis on graphics quality aligns with the perspective of UX designers. However, game designers prioritize the relationship between graphics style and the physical environment or context, placing additional emphasis on consistency. The relevant HCI design principles in this context are Aesthetic and Minimalist Design, and Consistency.
  • Clarity of Instructions and Balance: Mixed feedback was received regarding the clarity of instructions. While some students found the instructions clear and helpful, others felt confused or uncertain about what to do next in certain parts of the experience. Suggestions were made to strike a balance between providing clear instructions and allowing for exploration and discovery, ensuring users can navigate the experience smoothly while still feeling a sense of agency and autonomy. In HCI design principles, it is important to provide clear instructions, which aligns with the design principle of Help and Documentation. However, in game design, ambiguity of Objectives (i.e., goals or outcomes that players can work towards) and Success Criteria (i.e., ways of knowing when the objectives are met) can enhance the modeling of certain scenarios. Thus, achieving a balance in the clarity of instructions is considered crucial in this theme.
  • Difficulty, Mechanics, and Feedback: Students provided insights on the difficulty level of the experience and proposed adjustments, such as offering clues or varying difficulty levels for tasks. Suggestions included refining mechanics like object scanning and providing more intuitive feedback for player actions. Emphasis was placed on the importance

286 Kim and Lee

of delivering meaningful feedback and rewards for player actions to enhance the overall experience and foster continued engagement. The relevant HCI design principle is Engagement, and the relevant game design principles are Level Design (i.e., crafting the layout, challenges, and player experience of individual stages), Mechanics (i.e., rules of the game), and Rewards (i.e., incentives that reward success).

  • Story, Characters, and Atmosphere: Students discussed the presence of a captivating story and characters within the AR experience. Suggestions were made to further develop characters and incorporate additional storytelling elements to enhance immersion. Feedback stressed the importance of crafting a spooky atmosphere within the physical environment to complement the virtual elements, including recommendations for incorporating details like cobwebs and bats, which could heighten the overall sense of immersion and suspense. This is related to the game design principles of Theme and Game Space, including environment design, level design, and character design.
  • Game State and Progression: Students recommended providing visual cues or indicators to help users track their progress and understand their current objectives. Feedback included suggestions for implementing an inventory system and item collection mechanics to enrich gameplay and progression, granting users a sense of achievement and advancement as they traverse through the experience. The relevant HCI design principle is Affordance and Signifier, and the relevant game design principles are Game State (i.e., a collection of all relevant virtual information that may change during play), Mechanics (e.g., Progression of Play), and Rewards.

DISCUSSION

Through the analysis, we found common and distinct HCI and game design principles between UX and game designers’ perspectives, as shown in Table 1. While UX designers primarily focused on HCI design principles such as Shneiderman’s Eight Golden Rules (Masmuzidin and Aziz, 2019) and Neilsen’s Heuristics (Masmuzidin and Aziz, 2019; Nielsen, 2005, 1994), they also considered some game design principles (Brathwaite and Schreiber, 2009). Conversely, game designers primarily centered on game design principles (Brathwaite and Schreiber, 2009) but also took into account certain HCI design principles (Masmuzidin and Aziz, 2019; Nielsen, 2005, 1994). Based on the results of our analysis, we propose a set of design principles that encompass both HCI and game design principles. These principles serve as the foundation for an AR design framework and associated heuristics. The 9 HCI principles include Aesthetic and Minimalist Design, Engagement, Consistency, Informative Feedback, Affordance, Signifier, Visibility of System Status, Exploration of Virtual World, and Help and Documentation. Additionally, the 9 game principles consist of Theme, Game Space, Game Dynamics, Objectives, Success Criteria, Level Design, Mechanics, Rewards, and Game State.

Design Process for Augmented Reality (AR) Experiences 287

Table 1. Comparison of design principles in UX and game designers perspective.

Table available in the original PDF paper.

HCI Design Principles Game Design Principles

  • Theme

UX Designer

  • Aesthetic and Minimalist Design
  • Game Space
  • Engagement
  • Game Dynamics
  • Consistency
  • Informative Feedback
  • Affordance
  • Signifier
  • Visibility of System Status
  • Exploration of Virtual World
  • Theme

Game Designer

  • Aesthetic and Minimalist Design
  • Game Space
  • Engagement
  • Objectives
  • Consistency
  • Success Criteria
  • Affordance
  • Level Design
  • Signifier
  • Mechanics
  • Help and Documentation
  • Rewards
  • Game State

While our study was conducted based on a specific design project, the identified design principles still have limitations in terms of generalizability. Additionally, the game design principles need to be redefined and refined to apply to general AR design. For instance, the principle of Game State should be redefined as Experience State, encompassing both the Visibility of System Status and Experience State concepts. Another example is Game Space, which needs to be redefined as AR Experience Space, encompassing both the physical space and the spatial range of digital elements. However, they can serve as valuable design guidelines or evaluation heuristics that can be applied to various AR experience designs and evaluations. Moving forward, we plan to develop an AR design framework by adjusting and refining the design principles we identified to suit general AR design. Subsequently, we will apply this framework to AR design in various contexts, leveraging the insights gained from the results of this study.

CONCLUSION

This paper describes a cognitive study that investigates the design process for Augmented Reality (AR) experiences from the perspectives of User Experience (UX) designers and game designers. To identify distinct design approaches of UX designers and game designers in designing AR experiences, we analyzed the design process of a scenario-based sequential AR design project, AR Haunted College, which involved contributions from both UX designers and game designers. Our analysis reveals both common and distinct HCI and game design principles between the perspectives of UX and game designers. As a result, we propose a set of design principles, consisting of 9 HCI design principles and 9 game design principles, to serve as the foundation

288 Kim and Lee

for an AR design framework. The main contributions of this paper are providing a foundation for developing a theoretical framework for designing AR experiences and uncovering the distinct approaches of game designers and UX designers in AR experience design.

REFERENCES

Alqahtani, H., Kavakli-Thorne, M., 2020. Design and evaluation of an augmented reality game for cybersecurity awareness (CybAR). Information 11, 121. Brathwaite, B., Schreiber, I., 2009. Challenges for game designers. Course Technology/Cengage Learning Boston, Massachusetts. Dunleavy, M., 2014. Design principles for augmented reality learning. TechTrends

58, 28–34. Dünser, A., Grasset, R., Seichter, H., Billinghurst, M., 2007. Applying HCI principles to AR systems design. Kourouthanassis, P. E., Boletsis, C., Lekakos, G., 2015. Demystifying the design of

mobile augmented reality applications. Multimedia Tools and Applications 74, 1045–1066. Krüger, J. M., Bodemer, D., 2022. Application and Investigation of Multimedia Design Principles in Augmented Reality Learning Environments. Information 2022, 13, 74. Li, J., Van der Spek, E. D., Feijs, L., Wang, F., Hu, J., 2017. Augmented reality

games for learning: A literature review, in: Distributed, Ambient and Pervasive Interactions: 5th International Conference, DAPI 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Proceedings 5. Springer, pp. 612–626. Liang, S., 2018. Establishing design principles for augmented reality for older adults.

Sheffield Hallam University (United Kingdom). Liang, S., 2016. Design principles of augmented reality focusing on the ageing population, in: Proceedings of the 30th International BCS Human Computer Interaction Conference. BCS Learning & Development. Masmuzidin, M. Z., Aziz, N. A., 2019. The adaptation of Shneiderman’s golden

rules and nielsen’s heuristics on motivational augmented reality technology design for young children, in: 2019 IEEE 9th International Conference on System Engineering and Technology (ICSET). IEEE, pp. 62–67. Nielsen, J., 2005. Ten usability heuristics. Nielsen, J., 1994. Enhancing the explanatory power of usability heuristics, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Celebrating Interdependence - CHI ‘94. Presented at the the SIGCHI conference, ACM Press, Boston, Massachusetts, United States, pp. 152–158. https://doi.org/ 10.1145/191666.191729 Schmalstieg, D., Hollerer, T., 2016. Augmented reality: principles and practice.

Addison-Wesley Professional. Zuo, T., Jiang, J., Spek, E. V. der, Birk, M., Hu, J., 2022. Situating learning in AR fantasy, design considerations for AR game-based learning for children. Electronics 11, 2331.

Cover of the proceedings volume Human Interaction and Emerging Technologies (IHIET 2024)

Explore the complete digital volume

Human Interaction and Emerging Technologies (IHIET 2024)

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-33-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →