Natalia Jerzak ab, Francisco Rebelo bc (2022). Heuristic Evaluation for Serious Games (HESG): Using HESG for Designing and Evaluating Serious Games. In: Advances in Ergonomics In Design, Usability & Special Populations: Part I. AHFE Open Access, vol. 18. https://doi.org/10.54941/ahfe100779
Heuristic evaluation has become an extensively acknowledged method of usability evaluation in software development. Serious games became an useful method in teaching because they can provide meaningful learning activities and challenges. It makes us considerer serious games as important research area. Available evaluation methods for games are falling short in evaluation process of serious games due to lacking comprehension and not covering the seriousness therefore, we took the challenge of proposing a brand new heuristic method for the evaluation of serious games - Heuristic Evaluation for Serious Games (HESG). HESG consists of three modules, Game Play, Entertainment and Usability and Game Mechanics. Each module can be used remotely, autonomously and allows certain variations depending on the needs of designers and evaluators. The purpose of HESG is to create a complete and easily applicable instrument for evaluating different types of serious games. The main objective is to provide researchers and game developers with complete method to evaluate serious games and specialized training simulators in order to assure that used games are meeting the objectives, standards of quality and improve learning and training process. HESG has proven to be a comprehensive easily applicable method, a good tool to evaluate the special training purposes and may significantly help the professionals to develop good usability serious games.
Ergonomics In Design, Usability & Special Populations I (2022)
INTRODUCTION
Heuristic evaluation (HE) has become a key factor in designing new applications, tools and technologies. Regarding HE in area of video game development we can find still many uncovered zones. There are several heuristics evaluations for video games but there are no standardized methods that are taking into account serious games. When considering the serious games as dispositive, that can be used with educational and training purpose, we should assure that the used games are satisfying the educational and training needs. This need makes us think about the importance of conducting a formal evaluation of serious games. Our intention is propose holistic, complete, easily applicable, comprehensive method that will take into account the nature of serious games. Heuristic Evaluation for Serious Games (HESG) has some similarities to other heuristic. During the process of we applied the specific context of the serious games.
The used methodology was based in literature review and focus group meetings with the students of game design course. The literature review was developed in direct science database and gave important information to define the main variables to build the heuristics. The focus group meetings involved: designers, ergonomists and educational professionals and were conducted in order to discuss the main strategies to implement the heuristics.
HEURISTIC EVALUATION Usability inspections methods are playing important role in the designing a well-structured, effective, learning and training tool such as video game. Heuristics Evaluations (HE) are qualitative method used by the experts in order to evaluate given problems. HE is the method of finding the usability problems so that they can properly attended and resolved by implementing evaluators (Nielsen, 1990). That is why, HE can be perceived as more subjective method then the other usability inspections methods because it is strongly embedded in the skills of the evaluators and their experience (Nielsen and Mack, 1994). It is possible that those characteristics might have a negative impact but while implementing skilled evaluators into the process we can easily uncover the problems, potential areas of conflicts and inconsistency.
Serious Games The positive outcomes of serious games are the topics of many investigations. There is the consensus that serious games are the potential tools designed to have the educational impact. Firstly, serious games can significantly raise students engagement and motivation (Lim et al., 2006) (Kim et al., 2009). Secondly, implementing serious games is suitable for beginners and it allows to take more active role in the education process (Martin et al., 2011). Games are providing a feeling of accomplishment, sense of triumph that encourages the immersion in the learning process (Prensky, 2003). They are improving problem solving, critical thinking and allow collaboration, socialization with other players (Klopfer and Yoon, 2004). Serious games are perfect method of assessing the knowledge of the students and they allow receiving the immediate feedback on performance. There is very strong visible emotional connection between a student and learning material that allow students to do the meaningful choices (Johnson et al., 2012). Serious games allow the learners to experience the situations that are normally hard to be experienced in real-life situations because of many limitations such as time, space, safety etc. The idea behind the serious games is to intentionally create the learning bridge between the experience of daily life and learning styles (Proserpio and Gioia, 2007) (Vahey et al., 2006) and provide the situated learning that is long lasting compared to standard learning. What is more, according to many studies, students are preferring games and simulators over the standard class exercises (Chin et al., 2009). We should not perceive the games as the remedy for all but focus on the benefits of using them in the areas of higher-level thinking and social skills (Connolly et al., 2012).
Heuristic Evaluation for Games Nowadays serious games are considered as fast growing field with varied areas of implementations. Conducting the heuristic evaluation can be a valuable and is gaining more relevance in game design process. There are several
Ergonomics In Design, Usability & Special Populations I (2022) heuristics evaluations in the area of games but they are partially applicable or not applicable to serious games. The main reason for that is the huge difference between video games and serious games. First and main goal of serious game is the educational and training role: meeting specific educational purpose, specific learning aspects, training objectives that are offered in many new forms rather than entertainment that is main characteristics of games (Michael and Chen, 2005).
We will briefly present the main HE applicable for games. Malone's list of Heuristics for instructional games in 1982 began the development process of Game Heuristics. He presented the first research that was focusing on developing special challenges, crucial elements of the fantasies (Malone, 1982). Working in this environment can trigger learning, knowledge acquisition by placing the player in the situation where he believes his knowledge is inadequate, inconsistent or non parsimonious and therefore, he is in need of fulfilling this lack of it in a natural way. According to Malone, we can foster the curiosity development through the game and have the direct impact on learning outcomes. The idea of HE was followed by Melissa Federoff who in 2002 created the completed list of Heuristics (Federoff, 2002). She divided her heuristics into three main groups: game interface, game mechanics and game play. The limitation of this HE could be seen in applicability only at the preliminary stage of game design and not covering properly the immersion and the emotional features of the games. Heuristics HEP - Heuristics to Evaluate the Playability of games (Desurvire et al., 2004) were introduced in 2004, they were useful in the process of game design. The researchers identified the four categories of their heuristics: game play, game mechanics, game usability and introduced new class: game story. Game story was including the narrative plot of the game and the character development during the game play. The limitation of HEP can be connected with applicability only at the preliminary stage and some scale of generalization that could lead to problems with objectivity. Since there are many heuristics for computer games and due to changing environment, there was a need of proposing the heuristics for game available on the other platforms differ from personal computer. First study that was focusing on playability designed for mobile games, taking into account the mobile context was conducted by (Korhonen and Koivisto, 2006). This set of heuristics is measuring the playability for the mobile games including mobile specifications and limitations such as battery limits or size of the screen. One year later the same authors has proposed an extended version of previous heuristics for mobile games that additionally included the multi-player dimension (Korhonen and Koivisto, 2007). Another notable evaluation that can help in creation specially crafter heuristics for serious games was conducted by (Jegers, 2008). His usability and playability heuristics were designed for persuasive games: games that are using one of the three characteristics: i) mobile and place/time independence, ii) social interactions and iii) integration between physical and virtual worlds. Proposed heuristics are focusing on the aspects that have not been covered before by other heuristics in the area of game development such as involving interactions between the players and with the game environment. (Jegers, 2008). While conducting evaluation and creation of the game we should also focus on the social aspect of the game - giving the possibility to communicate and share. In 2009 the Usability heuristics for networked multiplayer games - Networked Game Heuristics (NGH) (Pinelle et al., 2009) we introduced. In NGH heuristics the influence is put on new usability issues that are connected with multi-playing features. The heuristics were build after detailed examination of the reviews of multiplayer games available online. It was the first set of heuristics that was derived from real problem of network games reported by the end users in the game portals: GameSpy and GameSpot. In 2009 Desurvire together with Wiberg proposed another heuristics evaluation: PLAY (Desurvire and Wiberg, 2009). Play was initially refining the proposed list of heuristics HEP by the following dimensions: multiple types of games and genders. Game usability heuristics (Play) covers the areas of game play, skill development, tutorials, strategy and challenges, game story, immersion, coolness and usability and game mechanics. Those features made PLAY applicable at the early stage of game development and also as the help for developers during the cycle of creating the game. (Omar and Jaafar, 2010) had proposed the first set heuristics that could be used in the area of educational the game. Playability Heuristics for Educational Games (PHEG) is divided into five issues that are interface, pedagogical/educational, multimedia, content and playability. PHEG is treating the educational aspect in rather simple way, mentioning the areas worth the attention and do not take the wide spectrum of seriousness of the games, such as special training simulators, games for military. Another limitation lies only on focusing on computer games, serious games can be available on every platform and are not limited only to personal computers.
HEURISTIC EVALUATION FOR SERIOUS GAMES (HESG) Serious Games can easily fall short or even bring harm when users are not engaged and the learning objective is not reached. The existing heuristics for games are not dealing with seriousness feature of serious games that is one of the main assets of Heuristic Evaluation for Serious Games. Secondly, the existing sets of heuristics were not
Ergonomics In Design, Usability & Special Populations I (2022) describe in details and that was leading to problems with adaptation them to the purpose of serious games. Thirdly, in many areas the existing heuristics evaluation were overlapping, many point were repeated which was making them unclear. All those aspects were the imperative reasons for introducing the brand new method that would cover the issues connected with learning and training.
Methodology
Evaluation of serious games should not only focus on numeric score but more on finding areas for improvement. HE for serious games not only ought to be used at the prototype stage but also on enhancing the successful interaction with learning content, the user's experience and improving the final product. When taking in mind the specialized training simulators we should center on attitude change, habit formation and training specialized behavioral responses. That training features of specialized simulators have not, to our knowledge, been presented in any other heuristic evaluation for game development before.
This Heuristic Evaluation for Serious Games was designed, used, and multiply revised in a very specific context of serious games. This HESG method is based both on relevant literature study, we have analyzed nine different heuristics for games, various studies and field experience, provides a framework to help assessing serious games.
HESG derive from previously created HE for games, up-to-date information applied in the specific learning context of the serious games. The new proposed methodology is based on our personal experience in video games, ergonomics, varied literature reviews and the experience as game players. The interaction with experts in the field, game developers, game players had proven the completeness of the heuristic in terms of evaluation of the educational and training purposes. The focus group meetings involved: designers, ergonomists and educational professionals and were conducted in order to discuss the main strategies to implement the heuristics.
Results
We have conducted the tryouts of the methodology during the focus groups with the master students of game design course of University of Lisbon in order to see if the tool is comprehensive and understandable. Additionally, we gathered the opinion from the game players in order to receive end-user feedback and by ergonomic specialist. All the three group provide the meaningful feedback, their suggestions were included in the final phase of the tool. The tryouts of HESG and constant improvement of the tool has continued till the target groups could use the method as intended without additional help.
HESG is designed to help at every stage of the serious game's development from the concept stage, through testing prototypes up to evaluating final products. HESG has modular structure, consist of three module: Game Play, Entertainment, Usability and Game Mechanics. Each module can be used remotely and autonomously depending on the needs of designers and evaluators.
HESG - Game Play Module The first module, Game Play, describes the ways the game is playable. It is very crucial for game development and for the player who does not get bored, on the contrary he experiences smooth interaction with the game environment and game mechanics. Game Play covers the biggest amount of different variables among which we can see the need of providing the appealing story if required, continuing and lasting acknowledgment, attention, recognition. Additionally, the game should challenge the player and provide coherent environment of the game. Successful serious games should make meaningful goals and objectives available, give the player the control and autonomy, allow him to concentrate and when desired permit different social interactions with other player that can foster the player's experience. The detailed list of issues composing the module of Game Play is presented in the table below (see Table 1.).
Table 1: Heuristic Evaluation for Serious Games (HESG) - Game Play
Table available in the original PDF paper.
1. GAME PLAY 1.1. Game story*: The narrative story/game plot. *This section is applicable only when the serious game has a story or includes story elements. 1.1.1. The game story relates to the player's experience or real life situations.
Ergonomics In Design, Usability & Special Populations I (2022)
1.1.2. The game story is discovered through out of the game, introduced at the beginning and/or during the game play. 1.1.3. The game story encourages to spend time thinking of possible outcomes, scenarios in the game. 1.2. Enduring the play: To hold, sustain the player interest and engage him during the play. 1.2.1. a. The game does not put burden on the player (such as tiredness, fatigue, stress, feeling of discomfort). b. *Unless that is the objective of the game (e.g. specialized training games or simulators for military, air force, firefighters, etc. purposes). 1.2.2. a. The players should not be burdened with tasks they do not find important. b. *Unless that is the objective of the game (e.g. specialized training games or simulators for military, air force, firefighters, etc. purposes). 1.2.3. a. The game detects the player's repetitive failures during the game play and provides means to recover from them (e.g. special hints) instead of repetitively penalizing the player. b. *Unless that is the objective of the game (e.g. specialized training games or simulators for military, air force, firefighters, etc. which purpose is to form certain habits and behaviors). 1.2.4. The game is interesting and engaging (providing continued or lasting acknowledgment, attention, concentration or recognition) for the player. The player is interested enough to continue playing rather than quitting the game. 1.2.5. The game provides fair outcomes (objective, in accordance with relative merit or significance) to conducted actions. 1.3. Challenge, Strategy and Pace: The game should be sufficiently challenging, paced by different levels of intensity and pressure and match the player's level skills. 1.3.1. a. The game is paced in order to apply pressure but without frustrating the player. b. *Unless that is the objective of the game (e.g. specialized training games or simulators for military, air force, firefighters, etc. which purposes is to apply pressure and frustrate the player) or the pace is not essential for a certain type of game. 1.3.2. The game offers different tempos during the game play (the standard exercises. e.g. battle against enemies tends to follow a bell-curved pattern, on the other hand the tempo during final exercise (e.g. boss-fight, final stage task) rises logarithmically. 1.3.3. The game is easy to learn at the beginning but is harder and challenging to master. 1.3.4. The first actions conducted in the game are obvious and result in immediate and positive feedback. 1.3.5. The learning challenges offered in the game should be adequate, yet challenging enough for the player to keep interest in the game. 1.3.6. The game offers varied difficulty levels and/or tasks so the player has greater challenge while developing the mastery. The level of challenge should increase as the player progresses through the game. 1.3.7. The game challenges are triggering a positive game experience rather than a negative one. 1.3.8. The game is suitable for different learning styles (e.g. pragmatic, active, reflective, theoretic style) and it allows and encourages the usage of different learning styles, strategies and tactics. *This feature is not applicable for games requiring only one, single strategy or learning style. 1.4. Consistency in Game Environment: The game environment is coherent, reasoned. 1.4.1. The game world reacts to the player's actions and if needed, remembers the player's passage in the game, 1.4.2. The changes made to the game world are persistent and noticeable when backtracking to them. 1.4.3. The Artificial Intelligence (AI) is reasonable, balanced with the player's actions and yet, if possible, offers elements of unpredictability. 1.5. Goals and objectives (specific targets within the general goal): Reason to continue and proceed towards the success in the game. Games should provide the player with clear long-term, intermediate (middle), short (moment-to-moment) goals; principal and secondary goals at appropriate time. The objectives within the goals should be clear for the player. 1.5.1. The goals of the game are clear for the player. The game is visibly presenting main (overriding, principal) goals and clear objectives within the goals. 1.5.2. The game gives meaningful rewards that are immersing the player more deeply into the game by moving the player into higher level or into unlocking special achievements. Players should be rewarded appropriately for their effort and skill development during learning process. 1.5.3. a. The needed skills are taught early in order to be used and practiced during the game play, or right before the skill is needed to proceed through game play.
Ergonomics In Design, Usability & Special Populations I (2022) b. *Unless the objective of the game is to encourage the player to learn the skills on his own through e.g. trial-and-error. 1.6. Players Perception of Control: Players should feel in control over their actions, characters, environment in the game. 1.6.1. The game provides control and influence onto the game world (actions influencing the game world matter and shape the game world). 1.6.2. The game provides control over game characters/units and movements within the game world*. Applicable only if the game has a character or a unit. 1.6.3. The game provides control over taken actions, reactions and used strategies. 1.6.4. The game has a content building feature. *Applicable only if the game allows content creation- serious games e.g. for architects, designers etc. 1.7. Player's Concentration: Games should require concentration, mental effort and the player should be able to focus without unnecessary distraction. 1.7.1. a. The game should provide stimuli from different sources. b. *Depending on the context, purpose, type of the game, the player can experience different amounts and/or types of stimulus. E.g. to practice his reactions to certain type and/or amount of stimulus. 1.7.2. a. The game must provide stimulus that are worth the player's attention. There should not be unnecessary distractions from the game task/activity. b. *Unless, depending on the context and/or game purpose, some of the stimulus might not be worth the player's attention but they play a certain role in the game e.g. to cause distraction and train players' behaviors and reactions. 1.7.3. The game should quickly grab the player's attention and concentration, and maintain the player's focus. 1.7.4. a. The game should have a high workload (the amount of work assigned to or expected to be done in) that is appropriate for the players’ perceptual, cognitive, and memory limits. b. *Unless the objective is to train certain behaviors, actions, reactions and habits and /or verify different capacity restrictions during high workload, stress level or inappropriate perceptual, cognitive and memory limits. 1.8. Social Interactions: Games should support and create opportunities for meaningful social interaction. *These characteristics apply only when the game allows social interactions between players. 1.8.1. The game supports competition and cooperation between players. 1.8.2. The game supports social interaction between players (chat, messages, forum etc.). 1.8.3. The game supports social communities inside and/or outside the game.
HESG - Entertainment Module Entertainment is feature that furthers the interest, focus and provide amusement, engagement, enjoyment, fun and emotional connection for the player while playing. We have divided this module into emotional impact, coolness and humor and immersion. Emotional Impact should be understood as a intensive form of attraction that allows the player to experience personal involvement and emotional engagement in the game. Serious games should also take advantage of coolness and humor as a triggers in learning and training process - players are engaged thanks to interesting activities and eager to replay the game. The player's involvement in the game is achieved in a natural way and players are immersed deeply into the game. The detailed list of Entertainment Module is presented in the table below (see Table 2.).
Table 2: Heuristic Evaluation for Serious Games (HESG) - Entertainment
Table available in the original PDF paper.
2. ENTERTAINMENT (feature that furthers the interest, focus and provide amusement, engagement, enjoyment, fun and emotional connection). 2.1. Emotional Impact: Deep, intensive form of attraction, fascination, association and identification with the game. 2.1.1. The game encourages the development of an emotional engagement with the game world and/or game characters. *This feature can be verified by observation, questionnaires, special tests, and through the interviews. 2.1.2. The game encourages the personal involvement emotionally (e.g. the game scares, threats, thrills, rewards, Ergonomics In Design, Usability & Special Populations I (2022) punishes) and intuitively (e.g. sounds of the environment). 2.2. Coolness & Humor: Absorb the attention through usage of fun, surprises, paradoxes, likeable activities, challenge, fun. 2.2.1. The game provides tasks that are fun, challenging with no boring and uninteresting activities. 2.2.2. Learning the game rules, game environment is not boring but is a part of fun. 2.2.3. The game is enjoyable enough to be replayed again or it allows players to replay specific learning activities when needed. 2.2.4. The game supports cognitive curiosity through usage of e.g. surprises, paradoxes, humor that make the learning process interesting and fun. 2.3. Immersion: The game encourages deep experience yet effortless natural involvement in the game.
2.3.1. The game provides the feeling as though the world in the game is going on whether the character is there or not. *Applicable for specific serious game, e.g. real time simulators. 2.3.2. If possible and needed, the game utilizes instinctive, audio and visual content to further the players’ immersion in a game. 2.3.3. While playing, the player becomes less aware of his surroundings. This feature can be verified by observation, questionnaires, special tests, and through the interviews. 2.3.4. While playing, the player becomes less self-aware and less worried about everyday life or self. This feature can be verified by observation, questionnaires, special tests, and through the interviews. 2.3.5. The game allows the player to get involved in the game naturally rather than be constantly aware of the learning process This feature can be verified by observation, questionnaires, special tests, and through the interviews. 2.3.6. While playing, the player experiences an altered sense of time. This feature can be verified by observation, questionnaires, special tests, and through the interviews.
HESG - Usability and Game Mechanics Module The last module, Usability and Game Mechanics, is addressing the issues of user interactions with the game and its mechanism. This module is stressing the importance of providing the player with specialized documentation and tutorials that are mimicking the game and presenting the game rules in a natural, not forced manner. Serious game should provide the detailed feedback on the player's progress, identify the player advancement easily without interfering with the game. Screen layout and error prevention are also very crucial part of the last module of HESG. The detailed items of Usability and Game Mechanics are presented below (see Table 3.).
Table 3: Heuristic Evaluation for Serious Games (HESG) - Usability and Game Mechanics 3. USABILITY AND GAME MECHANICS 3.1. Documentation and Tutorial 3.1.1. The game provides useful tutorial(s) (e.g. special audio, visual instructions, initial levels in the game, pre- game, demo etc.). 3.1.2. The tutorial mimics the game play, teaches the rules of the game or gives the feeling of playing the game. 3.1.3. The game does not force to access the tutorial in order to start the game allowing to skip the tutorial e.g. in case of re-playing the game or for already advanced players. 3.1.4. The game does not force to read the manual or documentation in order to play. 3.2. Feedback 3.2.1. Upon initially starting the game there is enough information to get started to play. 3.2.2. Mechanics/controller actions have consistent mapped and learnable responses. The game controls are basic enough to learn quickly, yet if necessary can be expandable for advanced options. 3.2.3. Game controls are consistent within the game and follow standard conventions. 3.2.4. The score, status level, learning outcomes and goals are easy to identify without interfering with the game play. 3.2.5. The game provides different types of feedback (music, sound effects, controller vibration) that are appropriate to the game type and game style. 3.2.6. a. The game reacts in a consistent, immediate, challenging and/or exciting way to the actions of the player.
Table available in the original PDF paper.
Ergonomics In Design, Usability & Special Populations I (2022) b. *Unless reacting immediately is not the purpose of the game. 3.2.7. The feedback on progress toward the goals and success/failure is provided to the player. 3.3. Screen Layout
3.3.1. The user interface is well-organized and consistent (in color, typographic, dialogue and user interface design, there is a stylistic treatment across the game). 3.3.2. Learning objects and tasks might be varied but all menu instructions, tips or error messages are appearing in the same place on screen. 3.3.3. The game gives the control over the game interface. 3.3.4. The game gives the control over input devices. 3.3.5. The menu layers in the game are well-organized and minimalist to the extent of being intuitive. 3.4. Error Prevention 3.4.1. The game allows to play and get involved quickly and easily with tips and/or progressive or adjustable difficulty levels. *Unless the game does not allow different difficulty levels. 3.4.2. a. The learning context is given sensitively during the game play to avoid the situation of being stuck or need to rely on a manual for help. b. *Unless that is the intention/purpose of the game. 3.4.3. The game gives sense of control over the game (game can be easily turned off and on, and be saved in different stages). 3.4.4. The players' error is avoided, the game does not allow to make errors that are detrimental to the game and should provide support in recovering from errors. 3.4.5. The game provides means for error prevention and recovery through the use of warning messages. 3.4.6. a. The game does not allow to make the irreversible errors that are preventing from further play. b. *Depending on the context and type of the game, the irreversible errors can be allowed if they are serving as certain learning outcome. 3.4.7. The game should provide clear error messages; that does not include programmer code and are precisely indicating the problems together with solutions to recover from the error without additional help.
HESG - application In order to make HESG suitable and wide enough to serve as an affective evaluation tool for serious games and specialized training simulators we have built it in a particular way. HESG is allowing to be used modularly and there are possible different variations - additional conditions. All those supplementary circumstances are included in the presented method as a special condition of applicability. The features that allow the evaluators to make a choice are carefully described.
Therefore, when there is no need to apply certain features; e.g. the given serious game does not have game story or does not allow the social interaction; the evaluators can omit them easily or adapt only certain features connected with evaluated serious game or training simulator.
HESG was elaborated in order to serve as a evaluation method for simulators and training games aimed at specific learning and training outcomes - attitude change, habit formation and training behavioral responses in simulators for military, air force, firefighters. In those specialized serious games the objective can be different then in the standard serious game. Taking into account that unique requirements and additional objectives HESG is giving the possibility of choosing between alternatives a (for standard serious game) and b (for specialized training simulators).
Among the particular objectives of specialized serious games and training simulators we can find the following issues:
1. Deliberately put the player in uncomfortable situations of stress, tiredness, fatigue, feeling of discomfort.
2. Burdening the player with the tasks, routine in order to create certain habits and train reactions and behaviors.
3. Pacing the game in order to apply lots of pressure to frustrate the player.
Ergonomics In Design, Usability & Special Populations I (2022)
4. Limit the information in order to encourage the player to learn the skills on his own through e.g. trial-and- error without or with limited help.
5. Depending on the context, purpose, type of the game allow the player to experience different amounts and/or types of stimulus. Some of the stimulus might not be worth the player's attention but they play a certain training role in the game e.g. to cause distraction and train behaviors and reactions.
6. Train certain behaviors, actions, reactions and habits by putting different capacity restrictions such as high workload, stress level or inappropriate perceptual, cognitive and memory limits.
7. Allow the irreversible errors when they are serving as certain learning outcome.
CONCLUSIONS AND FUTURE WORK It is necessary to ensure that used and developed serious games are contributing to its purposes. It led us to the need of conducting a thoroughly, accurate evaluation with comprehensive, objective, easily applicable and free of influence method. In this paper we present the new method to validate specialized educational and training games. During the process of development we begin with the methodical revision of the concept of serious games, that furthermore, was followed by analysis of the existing heuristics for games. It allowed us to identify the areas of tangency and recognize the areas, that were not covered before by any other heuristics. We have identified the new areas: the educational and training components that are the integral part of serious games and crucial aspect of new proposed heuristic evaluation - HESG.
HESG is proven to be useful tool for conducting the evaluation of serious games at every stage of game development. Additionally, thanks to HESG's modular structure is possible to use modules remotely or their specific variations depending on the need of game developers.
The interaction with experts in the field, future game developers, game players had proven the potencial of the heuristic in terms educational and training evaluation but the detailed further study on game play, immersion, usability and game mechanism is needed to officially validate the method.
Moreover, HESG allows conducting comparison between games when it is needed to take decisions on choosing the right serious games depending on the objectives, purposes, costs etc. The new proposed methodology is based on our personal experience in video games, ergonomics, varied literature reviews about serious games, existing heuristics evaluations and the experience as the video game players. HESG is presenting many new aspects that are related to the specific type of serious games that are specialized training simulators. It must be noted that there is a need for further studies that will formally validate the heuristics. We are expressing our deep interest to conduct the following validation in the future as the part of our further studies on heuristic evaluation for serious games.
Ergonomics In Design, Usability & Special Populations I (2022)
REFERENCES
Chin, J., Dukes, R., & Gamson, W. (2009). Assessment in Simulation and Gaming A Review of the Last 40 Years. Simulation \& Gaming, 40(4), 553-568. Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. (2012). A systematic literature review of empirical
evidence on computer games and serious games. Computers \& Education. Desurvire, H., & Wiberg, C. (2009). Game usability heuristics (play) for evaluating and designing better games: The next iteration. In Online Communities and Social Computing (pp. 557-566). Springer. Desurvire, H., Caplan, M., & Toth, J. A. (2004). Using heuristics to evaluate the playability of games. CHI'04 extended abstracts
on Human factors in computing systems, (pp. 1509-1512). Federoff, M. A. (2002). Heuristics and usability guidelines for the creation and evaluation of fun in video games. Master dissertation, Citeseer. Jegers, K. (2008). Investigating the applicability of usability and playability heuristics for evaluation of pervasive games. Internet
and Web Applications and Services, 2008. ICIW'08. Third International Conference on, (pp. 656-661). Johnson, L., Adams, S., & Cummins, M. (2012). The NMC horizon report: 2012 higher education edition. Austin, Texas: The New Media Consortium. Kim, B., Park, H., & Baek, Y. (2009). Not just fun, but serious strategies: Using meta-cognitive strategies in game-based
learning. Computers \& Education, 52(4), 800-810. Klopfer, E., & Yoon, S. (2004). Developing games and simulations for today and tomorrow’s tech savvy youth. TechTrends, 49(3), 33-41. Korhonen, H., & Koivisto, E. M. (2006). Playability heuristics for mobile games. Proceedings of the 8th conference on Human-
computer interaction with mobile devices and services, (pp. 9-16). Korhonen, H., & Koivisto, E. M. (2007). Playability heuristics for mobile multi-player games. Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts, (pp. 28-35). Lim, C. P., Nonis, D., & Hedberg, J. (2006). Gaming in a 3D multiuser virtual environment: Engaging students in science
lessons. British Journal of Educational Technology, 37(2), 211-231. Malone, T. W. (1982). Heuristics for designing enjoyable user interfaces: Lessons from computer games. Proceedings of the 1982 conference on Human factors in computing systems, (pp. 63-68). Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: Seven years of
forecasts and convergence. Computers \& Education, 57(3), 1893-1906. Michael, D. R., & Chen, S. L. (2005). Serious games: Games that educate, train, and inform. Muska \& Lipman/Premier-Trade. Nielsen, J. (1990). Paper versus computer implementations as mockup scenarios for heuristic evaluation. Proceedings of the IFIP Tc13 Third interational Conference on Human-Computer interaction, (pp. 315-320). Nielsen, J., & Mack, R.L. (1994). Usability Inspection Methods. John Wiley & Sons, New York, NY. Omar, H., & Jaafar, A. (2010). Heuristics evaluation in computer games. Information Retrieval \& Knowledge Management,
(CAMP), 2010 International Conference on, (pp. 188-193). Pinelle, D., Wong, N., Stach, T., & Gutwin, C. (2009). Usability heuristics for networked multiplayer games. Proceedings of the ACM 2009 international conference on Supporting group work, (pp. 169-178). Prensky, M. (2003). Digital game-based learning. Computers in Entertainment (CIE), 1(1), 21-21. Proserpio, L., & Gioia, D. A. (2007). Teaching the Virtual Generation. Academy of Management Learning \& Education, 6(1),
69-80. Vahey, P., Tatar, D., & Roschelle, J. (2006). Using handheld technology to move between the private and public in the classroom. Ubiquitous computing: Invisible technology, visible impact, 187-210.
Ergonomics In Design, Usability & Special Populations I (2022)
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Ergonomics In Design, Usability & Special Populations: Part I
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2106-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional