Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evaluation of Systematically Developed Gamification Strategies with Game-Balance Simulation Tools

Department for Product Safety and Quality Engineering, University of Wuppertal, Wuppertal, 42119, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1001517

Abstract

is a modern method of motivational design. It aims to motivate people to perform Desired Actions in a specific context. The development of successful gamification strategies requires a consistent, systematic approach. This consists of context and user analysis, technology ideation, and mechanics design. Ultimately, in the practical application, the so-called testing, it must be examined what influence the gamification strategies actually have in order to make eventual adjustments before the implementation takes place in the target context. A prior, objective evaluation possibility does not exist and is therefore often limited to the empirical experience of gamification designers. The balancing of video games can be tested with digital simulation tools (e.g. machinations.io), which show the described relationships in flow charts. This research work shows that gamification strategies for products can be mapped and evaluated with the help of game balancing simulation tools to make better statements about the probability of success. Conclusions and requirements for the general simulation of gamification strategies with game balancing tools are drawn from an exemplary application. A structured process with instructions for the implementation of gamification strategies in game-balance simulation tools helps gamification developers in practical applications.

INTRODUCTION AND PROBLEM STATEMENT Gamification is a modern approach to increase motivation by using “game design elements in non-game contexts”(Deterding, 2011). The goal of gamifi- cation is mostly to extrinsically and intrinsically increase people’s motivation to perform certain actions, behaviors, or decisions by introducing video game-related elements into real-life situations. Gamification projects in most cases follow a structured design process to support successful development. A challenging aspect of this is the evaluation of the developed strategy. This is usually done by a prototypical implementation with a small test group followed by an empirical analysis of the impact. The implementation in test environments and the empirical methods are mostly very time-consuming and offer only sufficient indications of the actual effectiveness of the gamification

© 2022. Published by AHFE Open Access. All rights reserved. 150

Evaluation of Systematically Developed Gamification Strategies 151

strategy since the results depend on the respective boundary conditions of the tests. Furthermore, it is difficult to map different user types in a rea- listic relation, the relation between user types and assigned or unassigned gamification elements and the resulting decision to take a Desired Action with the methods mentioned above. Other options have not yet been establi- shed (Morschheuser et al., 2017). An objective simulation approach would add significant value to the evaluation of gamification strategies by summa- rizing, exploiting, and illustrating various empirical findings in the field of gamification concerning individual contextual influences.

Game balancing describes the adaption of parameters, scenarios, and beh- aviors in video games to create a balance between frustration due to too high demands or boredom due to underchallenge (Koster, 2004). Mach- inations.io is a browser-based game balancing tool. Through flowcharts, video game contexts can be visualized and actively simulated. The goal is to optimize games in terms of user experience and to identify potential pro- blems and errors, such as game scenes that cannot be overcome by gamers (Machinations, 2021).

This research investigates the principal suitability of machinations.io as a simulation tool for gamification strategy projects. In an explorative study, the relationships between user types, game elements, motivation, and performa- nce of Desired Actions are examined. Implications for future simulations of gamification strategies are provided.

THEORETICAL BACKGROUND Gamification

The term gamification first appeared in 2002 in a paper by management consultant Nick Pelling and has gained more and more attention in many areas over time. In 2011, the first scientific conference on the topic was held (Fleisch, 2018). Gamification is a relatively young approach that started in the field of software development. In the meantime, gamification methods are already successfully used in many companies (Ellenberger et al., 2020; Kessing & Löwer, 2020; Reiners & Wood, 2015). The concept of gamifica- tion attempts to use the potential of video games in a meaningful and targeted way. The goal is to increase people’s motivation by offering new incentives to increase interest in activities and make overcoming challenges more attra- ctive. By systematically designing gamification strategies, positive motivation potentials are developed and Desired Actions can be triggered. As gamifica- tion aims to influence the behavior of the users, Desired Actions define the target of a gamification project (Chou, 2016).

User Types User Types provide a characterization of the users in a gamification con- text regarding their attitude or core motivation to use a respective context. Marczewski combined the concept of Bartle’s Four Player Types (Bartle, 1996) with the Self-Determination Theory (Deci & Ryan, 1980) to create the User Types HEXAD (Marczewski, 2015). Marczewski describes the six

152 Kessing and Löwer

Figure 1: Gamification user types HEXAD according to Marczewski (Marczewski 2021).
Original PDF Figure 1:

different User Types, which have different basic motivations. The intrinsically motivated types are Achiever, Socialiser, Philanthropist, and Free Spirit. They are motivated by relatedness, autonomy, mastery, and purpose. The other two types, whose motivations are extrinsic rewards and change, are Disru- ptor and Player. The HEXAD model enables the classification of people in gamified applications (see Fig. 1).

Through a survey on his website, Marczewski performed a distribution of people into User Types HEXAD (Marczewski, 2021): Philanthropists 27%, Free Sprits 22%, Achievers 17%, Socialiser 16%, Players 15%, and Disru- pter 3%. However, this list only represents the dominant user types of the test participants. For a detailed description, Marczewski also presents fur- ther sub-types, proposes a user type evolution, and defines mixed user types to form a more realistic framework.

Gamification Elements Gamification elements form the basic repertoire for gamification design. Gamification elements are elements derived from video games, which are used there for a specific purpose. Classic examples are points, rankings, and badges. These elements have already been categorized in various frameworks and described as solution-neutrally as possible for the application in gamifica- tion projects. Based on a previous analysis in a gamification project, suitable gamification elements can be selected and then designed for the individual use case. The “Periodic Table of Gamification Elements” by Marczewski consi- sts of 52 gamification elements, which are assigned to the six User Types HEXAD and two general categories (Marczewski, 2015). Santos et al. also showed that the assignment of specific gamification elements to certain user types is possible (Santos et al., 2021a).

Game Balancing and machinations.io Game balancing describes a continuous process in video game development to optimize the gaming experience. Therefore, even after the release of a video game, so-called balance patches are often implemented to address game bala- nce issues. This is especially necessary for games that regularly release new content for the players since the influence of the new elements can only be tested under the appropriate test conditions. However, necessary changes

Evaluation of Systematically Developed Gamification Strategies 153

often arise only when the dynamics of the game evolve through a large number of players (Becker & Görlich, 2021).

Machinations.io is a browser-based platform for designing, balancing, and simulating game systems. It allows displaying arbitrary game systems in an interactive diagram, to set parameters, define elements and their relations to each other, and to visualize how the systems work. This allows to identify and fix potential balance problems before the release of video games and without extensive programming effort (Machinations, 2021). Various basic tools are available that can be combined to create complex logics. A source generates elements according to determinable conditions, a pool collects elements at intermediate points, and a gate can distribute elements. Resource connecti- ons route elements, while state connections represent conditions. Registers can mathematically compute relationships. The End-Condition terminates the run.

RESEARCH QUESTIONS AND METHODS The explanations above show that the evaluation of developed gamification strategies has so far taken place through exemplary implementation in a test environment with subsequent empirical evaluation. The implementation in test environments and the empirical methods are mostly very complex and offer only insufficient indications of the actual effectiveness of the gamifica- tion strategy since the results depend on the respective boundary conditions of the tests. Furthermore, it is difficult to map different user types in a realistic relation, the relation between user types and assigned or unassigned gamifi- cation elements and the resulting decision to take a Desired Action with the methods mentioned above. An objective, solution-neutral and effort-reducing way to simulate gamification strategies does not exist yet. Game balancing tools like machinations.io are not used as they are not designed for this use- case. This leads to the following research questions:

RQ1: Can gamification strategies be simulated in game balancing tools? RQ2: What are the core elements which have to be modeled to simulate gamification strategies in game balancing tools?

RQ3: How can the relationships between the identified elements be defined?

The approach consists of four main steps: 1. Defining the necessary elements to represent motivational relationships in game balancing. tools. 2. Linking the identified elements with relations based on scientific literature to a theoretical model. 3. Basic exemplary implementation of the theoretical model to machinations.io. 4. Fundamental evaluation of the implementation. The first step is performed using model matching from the literature. For this, explicit literature from gamification and gamification application is used to define recurring elements and schemes. The linkage of the elements is given by logical connections and by an adaption of scientific findings. The imple- mentation to machinations.io is performed by an explorative modeling study. The evaluation will take place by an exemplary execution of the model.

154 Kessing and Löwer

MODELING GAMIFICATION By analyzing scientific literature on gamification, the necessary elements of gamification design can be defined. According to Deterding (2017) (“The use of game design-elements in non-game contexts”), there are at least the ele- ments of game design (or gamification elements according to Marczewski) and the context to be gamified. Chou defines gamification as human-focused design in contrast to function-focused design, which is intended to increase the motivation to perform a specific Desired Action (Chou, 2016). As alre- ady explained before, the classification of characters via user types plays an important role. Accordingly, the following central aspects result, which have to be considered in a simulation: User, User Types, Gamification elements, Motivation, Context impact, Desired Actions

The relations between the elements are defined as follows. The people in the gamified context (users) are divided into different user types according to their characteristics and the user spectrum of the context to be gamified. The user types have an average basic motivation to perform the Desired Action, which is defined by the target of the gamification project. This motivation is influenced positively, negatively, or not by gamification elements. Gamifi- cation elements have an individual influence on the different user types. The adapted motivation influences the decision to take a Desired Action. The following questions have to be answered to complete the theoretical model: What is the percentage distribution of users across User Types HEXAD? What is the relation of a possible motivational change due to gamification elements and user types? How does a motivational change affect the decision to take a Desired Action?

The distribution of the users to the user types is dependent on the individual use case of the prospective gamification project. Hence, this distribution is assumed as given. In this particular research, the survey results of Marczewski (2021) are taken as they have a high participation rate and thus validity for a general assumption (see chapter “User Types”).

The correlation of whether a motivational change is triggered by a gamifi- cation element among the user types is derived numerically from the findings of Tondello et al, 2017. Tondello et al. describe the relations of gamifica- tion elements (gameful design elements) to the User Types HEXAD in an exploratory factor analysis. In an online survey, participants were analyzed regarding their user type and then asked for their preferences in given game design elements. The results can be calculated to a direct correlation factor. For example, the gameful design element Guilds/Teams is defined by a soci- alization value of .668 and an altruism value of −.430. The user type Free Spirit shows correlations of .003 to socialization and .149 to altruism. The socialization and the altruism values are each multiplied and then added up to the direct correlation factor. In this example, the Guilds/Teams element has an influence of −.0621 (or −6.21%) on the Free Spirit user type. For the implementation in the simulation model, this value is interpreted as a 6.21% probability for the Free Spirit users to be negatively influenced in its motivation to take the Desired Action.

Evaluation of Systematically Developed Gamification Strategies 155

Figure 2: Theoretical model and example user journey (pointed line) (UT = User Type, GE = Gamification Element, x = calculated influence of GE1 on UT1 based on Tondello et al. 2017, PI = positive influenced users, NI = negative influenced users, NOT = not influenced users, BV = motivational base value (here 50%)).
Original PDF Figure 2:

As a negative influence on the motivation does not automatically lead to a decision to not take the Desired Action, a logical relation between the motivational change and the decision for the Desired Action has to be imple- mented. As there are no concrete scientific findings on this relation yet, a statistic distribution is considered here. The basic distribution of the probabi- lity to take a Desired Action without the influence of gamification elements is modeled as a Gaussian distribution on a 0 to 100 percent scale around the pre-defined motivational base-value of the corresponding user type with sigma being 10%. Hence, if the motivational base value to take the Desired Action is 50%, the users will spread from 10% to 90% probability to take the Desired Action, with sigma1-interval to be between 40% and 60%, sigma2- interval from 30% to 70% (sigma=2), etc. If a user is influenced positively or negatively by a gamification element, he will only be distributed on the respective positive or negative half of the Gaussian distribution. For example, a Free Spirit user is influenced negatively by the element Guilds/Teams. The base value to take the Desired Action is 50%, so the user has a 68.2% chance to be distributed to the sigma1-interval. This follows a 50-40% probability to take the Desired Action.

The theoretical model and the example user are shown in Figure 2.

RESULTS

According to the model described in the chapter before, the impact of gami- fication elements on the decision of users to perform a Desired Action depending on user type characteristics is modeled exemplarily for one user type and two gamification elements with machinations.io. The simulation model includes three sub-steps:

1. Calculation of motivation direction influence through gamification elements 2. Calculation of motivation change distribution 3. Calculation of the decisions for a Desired Action

156 Kessing and Löwer

Figure 3: Excerpt from the model implementation in machinations.io.
Original PDF Figure 3:

Step 1: This step defines if the gamification elements have a positive, neu- tral, or negative influence on the motivation of the users with respect to their user type. In this particular example, the user type Free Spirit is considered. The two chosen gamification elements are Guilds or teams with a calculated value of −6.21% and Unlockable or rare content with a value of 16.18%. The user type is implemented as a source that represents the users. Each user interacts with both gamification elements, which are implemented as random gates. The chances of the paths to positive, neutral, or negative influence are related to the calculated values based on Tondello et al. (2017). The moti- vational changes are represented by a pool each. As one user generates two simulation elements, the pools have to be normalized later in the simulation.

Step 2: This step defines how much the motivation changes based on a Gaussian distribution. The users who are positively influenced by the gamifi- cation elements are distributed on the positive half of a Gaussian distribution while the negatively influenced users are arranged in a negative half. Neu- tral influenced users are distributed randomly in the Gaussian distribution positive and negative which is realized by a 50/50 chance to be distributed negative or positive. Every simulation element which represents a motivati- onal change in the pools triggers a source that distributes the elements via a random gate on a Gaussian distribution.

Step 3: This step defines how the motivational change influences the deci- sion of the users to perform the Desired Action. For this, a scale from 0 to 100% is used which represents the probability to perform the Desired Action. A base value for each user type, which is determined by the context can be preset and manually adjusted via an active slider during the simulation. The sigma value of the Gaussian distribution is set by 10%. If a user gets distribu- ted to the respective interval, he is assigned to the middle value. The formulas in the simulation diagram also realize that the Gaussian distribution does not generate probabilities over 100% and below 0%.

Finally, the calculated values are combined and analyzed. With different combinations of the pools of taken and not taken Desired Actions, the success rates of the gamification elements for the user types can be calculated (see Fig. 3).

In the given exemplary implementation for the Free Spirit user type with a base value of 50% and the Guilds or teams and Unlockable or rare content gamification elements, the simulated success quota results in 5 to 10% more performed Desired Actions in comparison to the base value.

Evaluation of Systematically Developed Gamification Strategies 157

CONCLUSION

This research shows that in general the simulation of the influence of gami- fication elements on the motivation and thus the decision for or against a Desired Action is possible. Hence, RQ 1 can be answered positively. Gamification strategies can be simulated in game balance tools.

The core elements, which have to be implemented to complete a simula- tion model have been identified, which are User, User Types, Gamification elements, Motivation, Context impact, and Desired Actions. Thus, RQ 2 can also be answered positively. Also, it was possible to connect the simulation elements with logical relations based on scientific findings from Marczewski, Tondello et al., and a Gaussian distribution. This results in a final simulation model with comprehensive and consistent simulation results. Thus, RQ 3 can also be answered positively.

IMPLICATIONS AND LIMITATIONS There exist some limitations to this study that have to be mentioned to support future research on this topic.

The target of simulating gamification strategies for evaluation purposes is not to precisely predict the impact of gamification elements on the per- formance of Desired Action but to give an objective orientation on how different gamification elements might work in a given context. In general, motivation and the impact of gamification is highly dependent on individual factors. These factors have to be analyzed more in future research to create the possibility for a better consideration in simulation contexts.

The number of identified simulation objects in RQ 2 gives a basis for simulation purposes but can definitely be increased. Especially the ele- ments which represent the individual context influence have to be examined more intensively and summarized to give simulation users a more intuitive interface.

There exist alternatives to the research studies chosen for connecting the simulation elements with logical connections. The User Types HEXAD from Marczewski are a common framework for categorization of human behavior in gamification research contexts but other frameworks exist that might also apply for the purpose of simulating gamification such as Bartle’s four Player Types (Bartle, 1996) or BrainHex (Nacke et al, 2014).

The influence of gamification elements on user types is currently a highly relevant topic in gamification research and several approaches exist such as (Santos et al, 2021a) or (Krath & von Korflesch, 2021). Tondello et al. (2017) give a direct possibility to generate numeric relations between user types and gamification elements, but this has to be handled as an approximation as humans on average do not only fit to one user type (Marczewski, 2021) and also change user types over time (Santos et al. 2021b).

The Gaussian distribution on the amount of motivational change is an approximation due to missing research findings in this area. When there are findings available they have to be implemented to the simulation model to describe this relation more realisticly.

158 Kessing and Löwer

REFERENCES Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD research, 1(1), 19. Becker, A., & Görlich, D. (2020). What is game balancing? - an examination of

concepts. ParadigmPlus, 1(1), 22–41. Chou, Y. (2016): Actionable Gamification. Octalysis Media, Milpitas, CA, USA Deci, E. L., & Ryan, R. M. (1980). Self-determination Theory: When Mind Mediates Behavior. The Journal of Mind and Behavior, 1(1), 33–43. http://www.jstor.org/stable/43852807 Deterding, S., et al. (2011). From Game Design Elements to Gameful- ness: Defining "Gamification". New York, NY: ACM. Beitrag im Internet: http://dl.acm.org/citation.cfm?id=2181037, abgerufen am 14.05.2020 Ellenberger, T.; Harder, D.; Brechbühler Pešková, M. (2020). Gamification in Unternehmen. In: Schellinger, Jochen; Tokarski, Kim Oliver; Kissling-Näf, Ingrid. Digitale Transformation und Unternehmensführung: Trends und Perspektiven für die Praxis, p. 55–81. Wiesbaden: Springer Fachmedien Fleisch, H., Mecking, C. & Steinsdörfer, E. (2018). Gamification4Good: Gemeinw-

ohl spielerisch stärken (1. Aufl.). Edition Stiftung&Sponsoring: v.1. Erich Schmidt Verlag. Berlin. ISBN: 9783503177967 Kessing D.; Löwer, M. (2020). Gamification and Design Thinking - A motivational analysis of an international, interdisciplinary, team-based university course. Interactive Collaborative and Blended Learning – ICBL2020, 14.-16.10.2020, Hamilton, Canada, DOI: 10.13140/RG.2.2.28591.36002 Koster, R. (2004) Theory of Fun for Game Design. Phoenix. Paraglyph Press. Krath, J., & von Korflesch, H. F. O. (2021). Player Types and Game Element Prefe-

rences: Investigating the Relationship with the Gamification User Types HEXAD Scale. In X. Fang (Ed.), HCI in Games: Experience Design and Game Mecha- nics. HCII 2021. Lecture Notes in Computer Science, vol 12789 (pp. 219–238). Springer Nature. https://doi.org/10.1007/978-3-030-77277-2_18 Machinations.io (2021). What is Machinations. https://machinations.io/docs/home/, last checked 02/14/2022 Marczewski, A. (2015): Even ninja monkeys like to play. Gamification, game

thinking & motivational design, Gamified UK, independently published, ISBN 1514745666. Marczewksi, A. (2021). Gamified UK User Type HEXAD Results, https://gamified .uk/UserTypeTest2016/user-type-test-results.php?lid=#.YaVGPboxk2w, last che- cked 02/14/2022 Morschheuser, B., Hamari, J., Werder, K. & Abe, J. (2017): How to Gamify? A

Method For Designing Gamification. In Proceedings of the 50th Hawaii Interna- tional Conference on System Sciences, Hawaii International Conference on System Sciences 2017, https://doi.org/10.24251/HICSS.2017.155 Nacke, Lennart E.; Bateman, Chris; Mandryk, Regan L. (2014). BrainHex: A neurobiological gamer typology survey. Entertainment Computing, 5(1), 55–62. doi:10.1016/j.entcom.2013.06.002 Reiners, T.; Wood, L. C. (2015). Gamification in Education and Business. Cham.

Springer International Publishing, ISBN 978-3-319-10207-8. Santos, A.C.G., Oliveira, W., Hamari, J. et al. (2021a) The relationship between user types and gamification designs. User Model User-Adap Inter 31, 907–940 (2021). https://doi.org/10.1007/s11257-021-09300-z Santos, A. C. G., Oliveira, W., Hamari, J. et al. (2021b). The relationship between

user types and gamification designs. User Model User-Adap Inter 31, 907–940 (2021). https://doi.org/10.1007/s11257-021-09300-z Tondello, Gustavo F., Alberto Mora, and Lennart E. Nacke. (2017). “Elements of gameful design emerging from user preferences.” Proceedings of the Annual Symposium on Computer-Human Interaction in Play. 2017.

Cover of the proceedings volume Creativity, Innovation and Entrepreneurship

Explore the complete digital volume

Creativity, Innovation and Entrepreneurship

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-07-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →