Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Emotion induction strategies in driving simulator for validated experiments

TU Dortmund University, Institute of Control Theory and Systems, Engineering, Otto-Hahn-Str. 8, 44227 Dortmund, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1001156

Abstract

Triggering emotions in a driving simulator is not easy as the virtual environment reduces the reality of the situations. This contribution deals with the induction of emotions in drivers during the simulation and addresses the possible hindrances in the design and implementation phases. For this purpose, an experiment is con-ducted on a driving simulator with 20 participants, 5 females and 15 males, aged 22 to 30 years old. First, important emotions that may recur in driving situations are presented. Then, the process of evoking emotions in drivers is clarified, three different strategies, namely monotonous, event-driven, and combination, are de-scribed, and the intensity of emotion evoked by each modality of the stimuli is ex-amined. In addition, a mapping from three-dimensional to discrete emotion space including seven states is presented. Finally, to evaluate the concepts discussed, the results regarding driver emotion and scenario validation are presented and general recommendations are provided.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

INTRODUCTION

Human error endangers road safety and results in a contemplative number of injuries every year. In this respect, the emotional state of the drivers has a major impact on their performance, especially in critical situations (Dargahi Nobari et al. 2022). Improving the emotional state can help improve the traffic situation during manual driving. In addition, driver state in automated vehicles is extensively studied in driving simulators in the context of developing driver-vehicle interaction strategies (Dargahi Nobari et al. 2020). However, triggering emotions in a simulator is complicated as the virtual environment reduces the reality of the situation for the driver. Moreover, the duration of the experiments is not long enough and the drivers do not have sufficient time to go into their role. While on the other hand, conducting experiments regarding the driver's emotional state in real vehicles on the road jeopardizes the traffic members since certain states degrade the driver's performance and increase the likelihood of an accident. Thus, most critical situations or devastating emotional states must be studied in the simulation environment and the first challenge in the investigation of drivers’ emotions is creating a realistic driving situation in virtual environments and evoking strong emotions in drivers during experiments in driving simulators.

DESIGN OF AN EXPERIMENT ON A DRIVING SIMULATOR In general, emotion models are dimensional or discrete. The dimensional models decompose emotion into several (two or more) continuous components. An example of this type is the PAD model (Mehrabian, 1980) with three dimensions, pleasure, arousal, and dominance. The discrete models, on the other hand, assign different categories to emotion. In most experiments in driving simulators, discrete emotions are primarily studied because the induction and measurement of the continuous dimensions are difficult. This contribution investigates emotion in a discrete manner as well. For this purpose, categories relevant to the driving context are first selected. Fear is a repeatedly studied emotion that responds to an experienced risk and can be a sign of perceived danger during driving activity (Fuller, 1984). According to the results achieved by Jeon et al. (2014), anger and happiness are two emotions that can deteriorate the performance of drivers. Sadness is another examined category that can increase driver’s risk-taking behavior (Hu et al. 2013) and reduce driving performance (Jeon, 2016). Moreover, drivers show ineffective responses to the surprising events (Horrey and Wickens, 2004), manifesting a footprint of surprise emotion in the driver’s performance. In addition, Ihm et al. (2018) discuss frustration as a disruptive emotion for the driving task. Therefore, in this contribution, distinguished driving scenarios are created to induce the emotions of happiness, sadness, surprise, anger, fear, and frustration. A further scenario that does not evoke

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 any emotion (neutral) is also provided as a baseline for observer assessment.

Setup and Sample A static driving simulator with SCANeR studio1 1.8 software as a platform is utilized in the experiment. The simulator is equipped with a RealSense2 camera and a Trust Gaming GXT 252 Emita streaming microphone3. These feeds provide the possibility to capture participants’ facial behavior, body movements, and speech.

The participants were chosen among students of the university varied in nationality, driving frequency, and education level. In total 20 participants, 5 females and 15 males, aged from 22 to 30 years old (mean = 26.1, standard deviation = 2.1), took part in the experiment. Each subject drove seven trials resulting in 140 trials. Thereof 5 trials were removed because of motion sickness and 3 because of technical problems.

Scenarios For the experiment, an introductory scenario alongside seven emotion inducing scenarios are designed. The introductory scenario lasts 10 min total. Other scenarios range from 6-10 min in length and are in three distinct forms: monotonous, event- driven, and combination. Monotonous means that the scenario is designed to evoke the intended emotion and maintains that emotion as the scenario progresses. Event- driven scenarios consist of individual events throughout the scenario and rely on those events to evoke the intended emotion. The final method, combination, uses a mixture of the two previously mentioned approaches. Inducing participants in these types of scenarios requires both specific programmed events and influencing the overall environment. Initial events and the environment trigger the intended emotion, while subsequent events and ongoing use of the environment, such as consistently high traffic, allow the emotion to persist or grow. All of the scenarios utilize itineraries to guide the subject through the designated part of a map with the exception of the introductory scenario, where the drivers are free to choose their route and another scenario, in which the road has no exit. Scenarios have various surroundings including highway, city, and country roads. A summary of the emotion inducing factors is included in Table 1.

A baseline scenario is established to record the driver’s neutral state. The scenario utilizes a two-lane country road. To add realism, traffic is present in a limited form. By limiting the amount of traffic, the scenario does not induce any negative feelings.

The happiness inducing scenario occurs on a highway adjacent to a lake. The lake area is specifically ideal due to providing a serene environment promoting positive feelings with the subject, in contrast to the bustle of a city setting. To further promote 1 https://www.avsimulation.com/solutions/ 2 https://www.intelrealsense.com/depth-camera-d435/ 3 https://www.trust.com/en/product/21753-gxt-252-emita-streaming-microphone

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 this scenario, traffic stays in a restricted state (less than 500 cars per hour on each side) to minimize the possibility of traffic-induced stress. Additionally, music plays throughout the scenario to further promote happiness (Krumhansl, 1997, Zentner et al. 2008).

(a) Scenario for surprise elicitation

(b) Scenario for anger elicitation

(c) Scenario for fear elicitation

(d) Scenario for frustration elicitation Figure 1. Scenes of different scenarios with defined events A small city section is chosen to elicit sadness in the other scenario. Very few cars or pedestrians exist thereby isolating the participants. The overall environment is dark, cloudy, and rainy, with the rain value being set at 40%. The participant experiences and hears rain in the background but not to the point of being overwhelmed. Music plays here as well (Adagio for Organ & Strings in G Minor) (Krumhansl, 1997, Peretz et al. 1998).

In the scenario with surprising events, the goal of each event is to evoke surprise with no tendency toward fear as the two emotions are studies separately in this experiment. There are seven total events in the scenario (see Table 1 and Figure 1a).

Mountainous terrain is selected to elicit fear in drivers. There is only one other car in the entire route as it is an aid in one of the events. There are five events in total. In

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 the first event, a deer runs across the road in front of the driver (Figure 1c). Unlike Lisetti and Nasoz’s child version of this scenario (2005), the participants have an opportunity to miss hitting the deer.

Table 1. Summary of emotion induction in scenarios Form Intended Surroun- ding Emotion induction methods emotion Traffic Time of Weather Events / Stimuli day nous

Table available in the original PDF paper.

City - Dusk Rainy Music - Albinoni’s Adagio

Monoto

Neutral Country - Daytime - - Happiness Highway Low Daytime Sunny Music - La Primavera Sadness Surprise City - Daytime -

  • Video clip (Gross et al. 1995)
  • Heavy driven Eventdisappears
  • Herd of cows
  • Fog activation
  • Fog deactivation
  • Logo flashing

snowfall

  • Bus
  • Deer crossing
  • Video clip (Deng Fear Country Low Nighttime Foggy Music - A night on the bare mountain et al. 2017)
  • Detached trailer
  • Water covers
  • Long red light
  • Aggressive

Combination

road

  • Brake failure Anger City High Daytime -
  • Slow tractor
  • Speed bumps
  • Long turn
  • Construction
  • Pedestrian

traffic

  • Long red light
  • Decelerating
  • Red light
  • Pedestrian vehicle ahead Frustration City Race with time penalty Daytime - Navy sound; Visual message on screen

Inspired from Yan et al. 2018, a scenario is designed to evoke anger, which takes place in the center of a city. To incite the emotion of anger, the participant encounters multiple events and annoyances, which leads to elapsed time. There are a total of nine events (see Table 1 and Figure 1b).

Drawing inspiration from Fuller, 1984, the frustration inducing scenario is in the format of a race, both against the clock and another car. The events in this scenario aim to agitate the driver during the race but are subtle such as pedestrians crossing the road at a red light (Figure 1d). The subtle approach reduces the tendency toward anger. In this scenario, the participants have 6 min to reach the goal and penalties apply if the participant does not obey traffic rules.

Procedure The experiment is structured in a 2 h format. Each participant gets instructions on the experiment’s procedure, signs a consent form, and fills a personal information questionnaire. The experiment starts with an introductory scenario to acquaint the

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 participants with the simulator. The scenario also serves to neutralize any apprehension of the participant prior to the experiment. The remaining scenarios are randomized to mitigate influence as a result of emotional ordering. Due to the number of participants, the randomization of the order has been limited. A total of four orders are selected such that all scenarios occur at least once in the first half and once in the second half of the experiment (Table 2). Before activating a scenario, the participants hear an introduction message that provides specific information applicable to the given scenario.

Table 2. Order of emotions induced during the experiment Order 1 Sadness Neutral Fear Happiness Surprise Frustration Anger Order 2 Neutral Sadness Happiness Surprise Fear Anger Frustration Order 3 Frustration Anger Surprise Fear Happiness Neutral Sadness Order 4 Anger Frustration Happiness Fear Surprise Sadness Neutral

Table available in the original PDF paper.

EVALUATION OF THE DESIGNED EXPERIMENT There are different questionnaires designed to capture a participant's emotion. Questionnaires can address either dimensional or discrete emotions, or in some cases both. In terms of dimensional emotion questionnaires, the self-assessment manikin (SAM) (Bradley and Lang, 1994) is utilized, which is a pictorial questionnaire that measures pleasure, arousal, and dominance. An oral survey is conducted as well to gather the subjective choice of discrete emotions. In the oral survey, participants are expected to indicate their emotional state whenever they are asked during the experiment. In addition to the data collected from the participants, a video of the drivers’ facial expressions and behavior is recorded and played to an observer after the experiment to assign emotion labels to the frames from a human perception perspective.

Oral and SAM questionnaires occur throughout each of the scenarios. For all emotion evoking scenarios, participants complete both questionnaires at the end of the scenario. For the scenarios with anger, happiness, sadness, and neutral emotions, the oral questionnaire is given twice during the scenario. For the fear inducing scenario, the system instructs the participant to fill out the SAM after each event. After each surprising event, the participant gives both verbal and SAM responses. During frustration induction, no questionnaire is given to maximize the effect of the scenario.

To decide on the emotion of drivers during each scenario or event, the results of SAM are, first analyzed and mapped to the discrete emotion space. Then, the mappings are compared with the oral and observer responses to obtain the final result. The primary motivation for the observer assessment step lies within the scenarios designed for fear and surprise elicitation, in which every participant’s video is reviewed for reactions unless the SAM and oral scores strongly correspond. The reason why observer assessment is deemed necessary rather than relying solely on the questionnaire in other scenarios is to factor out trials suspicious of motion sickness and to investigate subconscious reactions. These reactions include common signs of

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 emotions, such as eyes widening in fear and the position of the head moving back with wide eyes in surprise.

Mapping 3D to Discrete Emotions Mapping of the SAM to the discrete emotion space is accomplished based on previous researches (Ahn et al. 2010, Trnka et al. 2016, Bălan, 2020, Verma and Tiwary, 2017, Hussain et al. 2011). In case of a tie, the most recent work takes precedence. As shown in Table 3, each of the dimensions is divided into several classes. The number of classes for pleasure, arousal, and dominance are chosen such that a clearer distinction can be drawn between the discrete emotions. Then the assignment to the discrete emotion space is made which is summarized in Table 4 and visualized in Figure 2. As frustration does not appear in any of the four papers with a dominance score, it is excluded from the dominance set.

Figure 2. Mapping 3D to discrete space
Original PDF Figure 2.
Table 3. Class divisions of the individual dimensions Dimension Classes Pleasure Very low (VLP) Low (LP) Normal (NP) High (HP) Very high (VHP) Arousal Low (LA) Normal (NA) High (HA) Very high (VHA) Dominance Low (LD) Normal (ND) High (HD)

Table available in the original PDF paper.

Table 4. Mapping ranges for each discrete emotion Emotion Ahn et al. 2010 Trnka et al. 2016 Balan et al. 2020 Verma and Tiwary, 2017 Our conclusion Neutral - - - - NP, NA, ND Happiness - VHP, HA, HD VHP, HA, HD HP, NA, ND VHP, HA, ND Sadness - VLP, NA, ND VLP, LA, LD LP, LA, ND VLP, LA, ND Surprise - - HP, VHA, ND - HP, VHA, ND Anger VLP/LP, HA/VHA LP, HA, ND LP,VHA,HD - LP, VHA, HD Fear - VLP, HA, ND VLP, HA, LD - VLP, HA, LD Frustration LP, NA/HA - - - LP, HA Results and Discussion

Table available in the original PDF paper.

In this contribution, the scenarios are designed to trigger only seven emotional states. Thus, if the rated point in the dimensional emotion space is located in the undefined

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 areas of the space (free spaces in Figure 2), the Euclidean distances to all seven emotions are calculated and the nearest emotion is assigned to this point.

A scenario (an event) is counted as a successful induction if the intended emotion is achieved at some point during the scenario (event). The success rates of each scenario are shown in Table 5a. It can be seen that the success rates of the scenarios developed based on the monotonous method (neutral, happiness, and sadness induction) are all below 50%. The elements used in these scenarios are the time of day including dusk and dawn, weather conditions ranging from rainy to sunny, and the playing of an emotion inducing piece of music throughout the trip.

The surprise induction scenario, which is the only event-driven form, is the most effective scenario, with an average success rate of 94%. The seven events in this scenario are the playing of a surprise video clip, heavy snowfall, sudden disappearance of a bus in front of the ego-vehicle, a herd of cows crossing the road, sudden activation and deactivation of fog, and a flashing logo on the simulator's main screen. The success rate of each event in this scenario is shown separately in Table 5b. Table 5. Successful emotion evocation [%]

(a) For each emotion in the related scenario Neutral Happiness Sadness Surprise Fear Anger Frustration 30 45 25 94 79 61 44 (b) For surprising events

Video

Flashing

clip

Heavy snowfall

Bus disappears

Herd of cows

Fog activation

Fog deactivation logo 29 53 41 6 53 24 71 (c) For fear inducing events

Deer crossing Video clip Detached trailer Water on road Brake failure 58 26 47 68 37 The fear inducing scenario, based on the combination method, ranks second with a success rate of 79%. The included monotonic element is the occurrence of the scenario at night and fog with a scary music playing throughout the scenario (Table 1). The scenario is further broken down into its specific events. The success rates are shown in Table 5c.

The scenario for eliciting anger with a combination method has a success rate of 61%, which is lower than that of surprise and fear induction, but higher than that of all monotonous scenarios. A high traffic condition is designed as a monotonous emotion inducing element and nine events are added to create a combination form. The results show that 36% of the participants feel anger during the anger inducing scenario only in the first half of the scenario, due to the traffic-related events of slow tractor ahead, speed bumps, excessively long turn, and construction zone.

Finally, the frustration induction scenario is not as effective as the other combination scenarios with 44% success. A possible reason for this could be the lower number of events in this scenario (red light and a pedestrian crossing the road), which means that frustration is mainly induced by monotonous elements. The average

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 result for all emotions is 54%.

These results highlight the positive effect of events on emotion elicitation. According to the results, the event-driven form is recommended to elicit stronger emotions in driving simulator experiments than the monotonous form. When viewing events, it is recommended to avoid repeating an event as this may reduce its effectiveness (water on the road and brake failure). In addition, the presentation of a video clip is not recommended because a clip can hardly be touching in such a short period of time. However, only a few events were studied in this paper. Further studies are needed to select the appropriate events for each emotion.

REFERENCES

Ahn, J., Gobron, S., Silvestre, Q., Thalmann, D. (2010). Asymmetrical facial expressions based on an advanced interpretation of two-dimensional Russell’s emotional model. Proceedings of ENGAGE Bălan, O., Moise, G., Petrescu, L., Moldoveanu, A., Leordeanu, M., Moldoveanu, F.

(2020). Emotion Classification Based on Biophysical Signals and Machine Learning Techniques. Sym-metry, 12(1), 21 Bradley, M. M., Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the seman-tic differential. Journal of behavior therapy and experimental psychiatry, 25(1), 49-59 Dargahi Nobari, K., Albers, F., Bartsch, K., Braun, J., Bertram, T. (2022). Modeling

Driver-Vehicle Interaction in Automated Driving Dargahi Nobari, K., Albers, F., Bartsch, K., Bertram, T. (2020). Online Feedback Control for Driver-Vehicle Interaction in Automated Driving. Advances in Human Aspects of Transportation, 159–165. DOI:10.1007/978-3-030-50943- 9_21 Deng, Y., Yang, M., Zhou, R. (2017). A new standardized emotional film database

for Asian culture. Frontiers in psychology, 8, 1941 Fuller, R. (1984). A conceptualization of driving behaviour as threat avoidance.

Ergonomics, 27(11), 1139-1155 Gross, J. J., Levenson, R. W. (1995). Emotion elicitation using films. Cognition & emotion, 9(1), 87-108 Horrey, W. J., Wickens, C. D. (2004). Driving and side task performance: The effects

of display clutter, separation, and modality. Human factors, 46(4), 611-624 Hu, T. Y., Xie, X., Li, J. (2013). Negative or positive? The effect of emotion and mood on risky driving. Transportation research part F: traffic psychology and behaviour, 16, 29-40 Hussain, M. S., AlZoubi, O., Calvo, R. A., D’Mello, S. K. (2011). Affect detection

from multichannel physiology during learning sessions with AutoTutor. In International Conference on Artificial Intelligence in Education (pp. 131-138). Springer, Berlin, Heidelberg Ihme, K., Unni, A., Zhang, M., Rieger, J. W., Jipp, M. (2018). Recognizing frustration of drivers from face video recordings and brain activation measurements with functional near-infrared spectroscopy. Frontiers in human neuroscience, 12, 327 Jeon, M. (2016). Don’t cry while you’re driving: sad driving is as bad as angry

driving. International Journal of Human–Computer Interaction, 32(10), 777-790

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Jeon, M., Walker, B. N., Yim, J. B. (2014). Effects of specific emotions on subjective judgment, driving performance, and perceived workload. Transportation research part F: traffic psychol-ogy and behaviour, 24, 197-209 Krumhansl, C. L. (1997). An exploratory study of musical emotions and

psychophysiology. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 51(4), 336 Lisetti, C. L., Nasoz, F. (2005). Affective intelligent car interfaces with emotion recognition. In Pro-ceedings of 11th International Conference on Human Computer Interaction, Las Vegas, NV, USA Mehrabian, A. (1980). Basic dimensions for a general psychological theory:

Implications for person-ality, social, environmental, and developmental studies (Vol. 2). Cambridge, MA: Oelgeschlager, Gunn & Hain Peretz, I., Gagnon, L., Bouchard, B. (1998). Music and emotion: perceptual determinants, immediacy, and isolation after brain damage. Cognition, 68(2), 111-141 Trnka, R., Lačev, A., Balcar, K., Kuška, M., Tavel, P. (2016). Modeling semantic

emotion space using a 3D hypercube-projection: an innovative analytical approach for the psychology of emotions. Frontiers in psychology, 7, 522 Verma, G. K., Tiwary, U. S. (2017). Affect representation and recognition in 3D continuous valence–arousal–dominance space. Multimedia Tools and Applications, 76(2), 2159-2183 Yan, L., Wan, P., Qin, L., Zhu, D. (2018). The induction and detection method of

angry driving: Evidences from EEG and physiological signals. Discrete Dynamics in Nature and Society Zentner, M., Grandjean, D., Scherer, K. R. (2008). Emotions evoked by the sound of music: characterization, classification, and measurement. Emotion, 8(4), 494

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Cover of the proceedings volume Human Systems Engineering and Design (IHSED2021): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED2021): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-7923-8987-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →