Víctor De Nalda Tárrega, Andrés Soler, Nicolás Palomares, Javier Silva, Jose Laparra, Jose Solaz (2022). Human Autonomous Vehicle (HAV): From sickness prevention to emotional response. In: Human Factors in Transportation. AHFE Open Access, vol. 60, pp. 533–541. https://doi.org/10.54941/ahfe1002487
Dynamic driving simulators have been a key tool to reduce time and costs during the design and development of new automobile models or advanced driving assistance systems (ADAS) (Lyga et al., 2020). However, more efforts are needed to enhance the acceptance of new technologies by considering the human factors early in the process.Due to the high development costs or to the low fidelity of the final result, to develop and build a medium cost dynamic driving simulator with a good level of immersivity is not an easy task. The simulator should have a high level of immersivity, to achieve high correlation with field operational tests, and minimize the sickness effect of the simulators. A simulator with these capabilities can be used to validate different devices or systems from the automobile field with users, measuring physiological signals, behavior, movements and telemetry data.IBV has achieved this dynamic driving simulator by using an open-source software, CARLA (Team, n.d.), in which the different scenarios can be simulated. This simulator is based in a client server architecture, a motion platform with 6 degrees of freedom and 550 kg of payload (“Motion Platform PS-6TM-550 (6DoF, 550kg) - Motion Systems,” n.d.), three main screens with two rearview screens and an HMI (Human Machine Interface). This HMI can be used as an additional screen showing relevant travel information to the user, as a display of the user´s signals or as a panel with a driving assistance system among other things.The possibility of validating different products with a dynamic simulator of these characteristics, in which emotional responses associated to the different driving conditions can be generated and evaluated, with defined and customized experimentation conditions, and without having to develop their own simulator, can save a lot of time and money to the different companies.
(Chan et al., 2010), very useful for the training of novice drivers, and in the research fields (Boyle and Lee, 2010) (Brookhuis and de Waard, 2010).
Dynamic driving simulators are normally focused on technology develo- pment, and not as human factors centered as they should be (Wynne et al., 2019). As the new driving technologies (autonomous driving and different driver assistance systems) are being developed, there is a need in knowledge of how these new technologies can be easily accepted by the users, which level of adaptation they can manage to have and at which speed, and how it physically affects the human body. Running these experimentations can be expensive, dangerous or arise ethical issues if the experimentations would take place in real life driving situations.
A certain level of fidelity and immersivity must be accomplished by the simulator, as the intention of the experimentation is to extrapolate the results to real life driving scenes. The ability of a dynamic simulator to achieve this is called validity, and despite the frequent use of dynamic simulators in research, it is rarely achieved (Wynne et al., 2019).
The level of immersivity is achieved by involving and making the user beli- eve in the artificial environment created for the experimentation. Findings suggest how visual and motion configurations affect the experience in dri- ving simulator (Yeo et al., 2020). The immersivity of the simulator can only be achieved when visual, dynamic, sound and haptic stimulations are well integrated and synchronized. A field of view (FOV) high enough must be displayed in a big enough screen (or various different screen displayed close to each other) (Yeo et al., 2020); a dynamic platform that replicates the acce- leration being generated in the simulation, so that the user can feel them (Kami´nska et al., 2022); an evolving sound system playing different sound effects that could happen in different traffic situations (Elmsley, 2019) and the inclusion of elements that can vibrate or generate haptic alerts are exam- ples of the different qualities that a dynamic simulator can have in order to be considered fully immersive.
The validation of the results obtained defines the extent to which the simu- lator elicits the same driving behaviors that occur when driving in the real world (Mullen et al., 2011). Therefore, focusing in the development of the previously mentioned aspects, to guarantee the immersivity and validation of a new simulator is of the upmost importance.
The development of these simulators, in which fidelity in all of these aspe- cts is achieved, has always required a high budget. The Human Autonomous Vehicle focuses in these aspects during its development (Figure 1). The cost of high-end simulators usually runs in the one hundred-thousand-dollar range with less complex simulators still priced at tens of thousands of dollars (Rod- seth et al., 2017). However, in the Biomechanics Institute of Valencia (IBV), a simulator for all the automobile agents, and not only for the OEM, that fit with market needs has been developed.
DEVELOPMENT OF THE HUMAN AUTONOMOUS VEHICLE (HAV) During the preliminary stages of the simulator development, the tasks the simulator was going to be able to test, were defined. Besides assuring a high
HAV - Human Autonomous Vehicle - From Sickness Prevention to Emotional Response 535
Figure 1: User on the HAV.
level of immersivity and validity, allowing to achieve high correlation in the results comparing them with field operational tests; minimizing the simulator sickness was a priority task.
Simulator Sickness Simulator sickness is a side effect of driving simulators which may reduce the user’s performance and well-being, due to its various symptoms: from pallor to vomiting (Dahlman et al., 2009). Different theories try to explain why this kind of motion sickness occur. The theory of sensory conflict, where the motion information sensed by the vestibular and visual receptors and the non-vestibular proprioceptors do not match the expected information; the evolutionary theory, where the disturbances in motor control are evaluated by the human body as the ingestion of toxins and the nausea and vomiting occurs to eliminate the ingested toxins; and the posture instability theory, where motion sicknesses is attributed to behavioral issues and caused by prolonged postural instability (Du˙zma´nska et al., 2018).
Therefore, a simulator that has the capability to reduce these symptoms regardless the cause, can improve the results of the studies taken place and increase the user comfort. The following measures were considered when designing and mounting the HAV to reduce simulator sickness symptoms:
The platform replicates the three linear accelerations and three angular velocities by spawning a virtual Inertial Measurement Unit (IMU) in the ego vehicle. This accelerations and velocities are sent, in real time for each frame, to the platform manager script with a server-client architecture. The maximum delay that can occur between the visual signal in the main screens and the movements that the user feel is the frequency at which the simulation is happening.
The layout of the three main screens and the user seat is calculated so that the distance of the screens and the angle between them, simulate the FOV of the simulation scene.
A postprocess algorithm (Panini projection) is applied to the final image to avoid distortions. These kinds of post processing algorithms are very
536 de Nalda Tárrega
Figure 2: eValanz.
common when a really wide image (FOV > 110°) is represented in linear screens. Without it, images in the peripheral view zone are distorted and can enhance motion sickness when stared directly into them.
Different seats with vibration and haptic signals can be mounted on the simulator. Some specific vibration configurations may have a positive impact on the sickness (Dahlman et al., 2009).
The users balance can be measured, before and after the experimentations in the simulator, with a platform device designed and developed by the IBV (eValanz, Figure 2). One of the most significant signals of motion sickness is the loss of balance. We can then objectively measure if a user has suffered of motion sickness and to which degree. With this, we can decide how valid the results of the recent experimentation are and gather more information for future updates in the simulator to achieve even better results in reducing motion sickness. After every iteration of the design process, a reduction of the users affected by simulation sickness has been witnessed.
Immersivity and Validity Considering that one of the main goals in the development of the HAV, was to achieve a simulator that could validate solutions from the automobile (and other transport industries) for the use in the real world, achieving a sufficient level of immersivity and validity was critical. To ensure this, different stages of development were implemented:
1. The dynamic platform (“Motion Platform PS-6TM-550 (6DoF, 550kg) - Motion Systems,” n.d.) with the 6 degrees of freedom was mounted, ensuring the user feels the motion that the ego car is suffering. High acce- lerations and steep turns can be replicated by the platform, receiving in real time the accelerations vector provided by the software. 2. Inclusion of an HMI and rearview screens. These screens allow the user to access to all the information as if it was in a real car, making it easier to
HAV - Human Autonomous Vehicle - From Sickness Prevention to Emotional Response 537
Figure 3: Technician launching a scenario and measuring physiological signals.
change lane, see the velocity and autonomy of the car and display several driving alerts 3. In a third stage, an overhead LEDs system was installed, allowing the light intensity of the simulation room to be controlled. This facilitates the immersion of the user for low-light simulated scenes (sunset, night, tunnels). 4. Home cinema speaker’s were installed. The level of immersivity increases when, during the simulation, the sound effects of the environment and the own car, are being played throughout the sound system.
However, all of these features have to be evaluated to see how the user is really reacting against all of these stimuli. One of the key aspects for obtaining valid results in simulation experimentations with the HAV, is the capability to measure physiological signals, gaze tracking and facial analysis (Figure 3). A system of several sensors and cameras (contactless and non-invasive sen- sors in some cases or contact sensors with high precision in others) installed throughout the simulator allow to measure these different signals. The pos- sibility to use these kinds of devices with the output data of the simulator, is very helpful when evaluating human factors in new or developing driving systems (autonomous driving, haptic communication in the user’s seat) and see how the user reacts and accepts this new technology.
The main difference between the HAV and other dynamic simulators is achieving all of these features with a medium cost budget, due to a simulator engine based on the open source software Carla.
Software and Hardware Development The software of the simulator is based on the Carla open source simu- lator. Carla is an urban driving simulator with different traffic interse- ctions, pedestrians, street signs, street lights… (Zapridou et al., 2020) Carla has been used to validate autonomous driving architectures, as seen in (Gómez-Huélamo et al., 2021). The simulation uses a client server
538 de Nalda Tárrega
architecture, where the server is running in Unreal Engine, and then different scripts connect to the server as clients.
These client scripts are sending and retrieving information in real time, allowing the simulator to run in real time. The client agents allow to complete the following functions in the simulator:
Spawn and control different agents in the simulator.
Edit the scenario (weather and light conditions, props…)
Real time communication with the platform and storage of telemetry data.
Activate different external triggers (sound, haptic and visual signals).
Control all the different views of the simulator.
Measure physiological signals during the experimentations.
To assure a constant framerate flow of information, the communication between the server and different client script is done in synchronous mode. Therefore, the computing capability of the computer must be high enough to ensure a high enough framerate of the screens and simulation steps.
In the other hand, the hardware development of the HAV consists of the 6 degrees-of-freedom platform, designed by Motion Systems, the screens distri- buted among the platform, steering wheel, pedals and different add ons (such as the home cinema sound system, seat with haptic actuators…) The movement of the platform is controlled with a script that receives the accelerations from the simulator and converts them to directly control the actuators of the platform and replicate the movements in the simulation. Besides the three main screens, located in front of the user, there are actually three additional screens in the platform. The first two are located at each side of the user, replicating the rearview mirrors, by playing the images recorded on RGB cameras positioned at the location and orientation of conventional rearview cameras; the last one is located just in front of the user, and works as an HMI display.
Besides the physiological signals, the telemetry data of the different vehicles is also stored for every frame of the simulation and can be analyzed later (accelerations of the vehicle, reaction time of the user when certain events are triggered, evaluation of driving skills...) This data can be extremely important when analyzing the driving patterns of the user under different circumstances that can alter its emotional state among other experimentation activities.
A cockpit is mounted in the platform, where the seat, steering wheel and pedals are installed. With the steering wheel and the pedals, the ego veh- icle is controlled when manual driving mode is selected. Furthermore, a home cinema sound and light-intensity controlled systems are installed in the simulation lab.
EXPERIMENTATION RESULTS WITH HAV So far, and while the development of the HAV has been taking place, throu- ghout the iterative stages aforementioned and gradually improving different aspects of the simulator, different studies and experimentations have been completed with HAV (Figure 4). Some examples of the studies completed
HAV - Human Autonomous Vehicle - From Sickness Prevention to Emotional Response 539
Figure 4: HAV’s logo.
and how the functionalities available in HAV have allowed to execute them, are listed below:
Generation of different autonomous driving modes scenarios, to evaluate the emotional response to different critical situations in the Diamond pro- ject (“DIAMOND project- Addressing gender-specific needs in transport systems,” n.d.). Different scenarios where created where both, the ego veh- icle, as the other actors, were controlled autonomously, and their behavior would differ depending on the autopilot agent assigned to the vehicle. The vehicle dynamics would adapt to the driving mode.
Generation of different scenarios with a system of hazardous and dan- gerous events detection. These events would trigger different alerts in forms of haptic signals in the seat, or visual and auditive signals. The communication between the simulator and different external agents was implemented.
Experimentations where the telemetry stored in real time is later analyzed, to be able to reach conclusions in how the different arousals in the sce- narios affect the user’s driving behavior. An object to collect and measure telemetry was implemented.
Generation of scenarios with long, tiring and monotonous roads and night atmosphere, to enhance sleepiness and test different sleepiness detection methods.
Experimentations with autonomous driving conditions, where the user was completing other activities unrelated to driving, and had to regain control of the vehicle as the driving mode swifts to manual. A fast and smooth transition between manual and autonomous driving was needed.
New adaptive driving mode is currently in development, where the driving mode adapts to emotional model system developed by the IBV (Silva et al., 2021). The intention with this model is to also consider the user emotions when defining au autonomous driving mode or agent.
CONCLUSION
The development of this simulator intends to offer the possibility of having immersive driving simulation-based experimentations at an affordable price. The motivation and design process that took place to achieve it have been explained. Some key aspects as immersivity and reduction of simulator sickness have been improved in every phase of the design.
A brief explanation of how the system works is given and several examples of different experimentations that have taken place in the simulator, and how
540 de Nalda Tárrega
the different developments in the design have allowed to complete them, are mentioned.
As future work, new assets are planned to be incorporated as VR compa- tibility with the simulation software and platform, more complex autopilot agents and several features more that help HAV becoming a more immersive dynamic simulator.
ACKNOWLEDGMENT
This work was funded by the European Union’s Horizon 2020 Research and Innovation Program SUaaVE project: “SUpporting acceptance of automated Vehicles”; under Grant Agreement No. 814999.
REFERENCES
Boyle, L.N., Lee, J.D., (2010). Using driving simulators to assess driving safety.
Accident Analysis & Prevention, Assessing Safety with Driving Simulators 42, 785–787. https://doi.org/10.1016/j.aap.2010.03.006 Brookhuis, K.A., de Waard, D., (2010). Monitoring drivers’ mental workload in driving simulators using physiological measures. Accident Analysis & Prevention, Assessing Safety with Driving Simulators 42, 898–903. https://doi.org/10.1016/j. aap.2009.06.001 Chan, E., Pradhan, A.K., Pollatsek, A., Knodler, M.A., Fisher, D.L., (2010). Are dri-
ving simulators effective tools for evaluating novice drivers’ hazard anticipation, speed management, and attention maintenance skills? Transportation Research Part F: Traffic Psychology and Behaviour 13, 343–353. https://doi.org/10.1016/j. trf.2010.04.001 Dahlman, J., Falkmer, T., Ledin, T., Golding, J., Linköpings universitet, Hälsouniversitetet, Linköpings universitet, Institutionen för klinisk och experimentell medicin, (2009). Psychophysiological and Performance Aspects on Motion Sickness. DIAMOND project- Addressing gender-specific needs in transport systems [WWW
Document], n.d. . diamond-project.eu. URL https://diamond-project.eu/ (accessed 3.2.22). Du˙zma´nska, N., Strojny, P., Strojny, A., (2018). Can Simulator Sickness Be Avoided?
A Review on Temporal Aspects of Simulator Sickness. Frontiers in Psychology 9. Elmsley, A., (2019). How to increase user immersion with (mostly) audio. The Sound of AI. URL https://medium.com/the-sound-of-ai/how-to-increase-user-immersio n-with-mostly-audio-2c7ba6f8777b (accessed 3.9.22). Gómez-Huélamo, C., Del Egido, J., Bergasa, L.M., Barea, R., López-Guillén, E.,
Arango, F., Araluce, J., López, J., (2021). Train Here, Drive There: Simu- lating Real-World Use Cases with Fully-Autonomous Driving Architecture in CARLA Simulator, in: Bergasa, L.M., Ocaña, M., Barea, R., López-Guillén, E., Revenga, P. (Eds.), Advances in Physical Agents II, Advances in Intelligent Systems and Computing. Springer International Publishing, Cham, pp. 44–59. https://doi.org/10.1007/978-3-030-62579-5_4 Kami´nska, D., Zwoli´nski, G., Laska-Le´sniewicz, A., adamek, łukasz, (2022). Vibrating Tilt Platform Enhancing Immersive Experience in VR. Electronics 11, 462. https://doi.org/10.3390/electronics11030462 Lyga, Y., Lau, M., Brandenburg, E., Stark, R., (2020). Validation of Immersive Design
Parameters in Driving Simulation Environments. pp. 136–142. https://doi.org/10. 1007/978-3-030-51064-0_18
HAV - Human Autonomous Vehicle - From Sickness Prevention to Emotional Response 541
Motion Platform PS-6TM-550 (6DoF, 550kg) - Motion Systems [WWW Document], n.d. URL https://motionsystems.eu/product/motion-platforms/ps-6tm- 550/ (accessed 3.8.22). Mullen, N., Charlton, J., Devlin, A., Bedard, M., (2011). Simulator validity: Behavi-
ours observed on the simulator and on the road. Handbook of Driving Simulation for Engineering, Medicine and Psychology 1–18. Rodseth, J., Washabaugh, E.P., Al Haddad, A., Kartje, P., Tate, D.G., Krishnan, C., (2017). A novel low-cost solution for driving assessment in individuals with and without disabilities. Applied Ergonomics 65, 335–344. https://doi.org/10.1016/j. apergo.2017.07.002 Silva, J., Belda-Lois, J., Iranzo, S., Mateo, B., Nalda-Tárrega, V. de, Palomares,
N., Laparra-Hernández, J., Solaz, J., (2021). Emotion State Induction for the Optimization of Immersion and User Experience in Automated Vehicle Simulator. Proceedings of the 5th International Conference on Computer-Human Interaction Research and Applications. https://doi.org/10.5220/0010722700003060 Team, C., n.d. CARLA [WWW Document]. CARLA Simulator. URL http://carla.or g// (accessed 3.9.22). Wynne, R.A., Beanland, V., Salmon, P.M., (2019). Systematic review of driving simu-
lator validation studies. Safety Science 117, 138–151. https://doi.org/10.1016/j.ss ci.2019.04.004 Yeo, D., Kim, G., Kim, S., (2020). Toward Immersive Self-Driving Simulations: Reports from a User Study across Six Platforms, in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, pp. 1–12. Zapridou, E., Bartocci, E., Katsaros, P., (2020). Runtime Verification of Autonomous
Driving Systems in CARLA, in: Deshmukh, J., Niˇckovi´c, D. (Eds.), Runtime Veri- fication, Lecture Notes in Computer Science. Springer International Publishing, Cham, pp. 172–183. https://doi.org/10.1007/978-3-030-60508-7_9
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-36-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional