Ilhan Yunus, Fikri Farhan Witjaksono, Elif Naz Başokur, Jenny Jerrelind, Lars Drugge (2022). Analysis of human perception models for motion sickness in autonomous driving. In: Human Factors in Transportation. AHFE Open Access, vol. 60, pp. 414–422. https://doi.org/10.54941/ahfe1002473
1 Volvo Car Corporation, SE-405 31 Gothenburg, Sweden; The Centre for ECO Vehicle Design, Engineering Mechanics, KTH Royal Institute of Technology SE-100 44 Stockholm, Sweden ↗
2 Volvo Car Corporation, SE-405 31 Gothenburg, Sweden; Department of Mechanical and Maritime Sciences (M2), Chalmers University of Technology, SE-412 96 Gothenburg, Sweden ↗
3 Volvo Car Corporation, SE-405 31 Gothenburg, Sweden; Dept. of Electrical Engineering and Information Technology, ETH Zürich 8092 Zürich Switzerland ↗
4 The Centre for ECO Vehicle Design, Engineering Mechanics, KTH Royal Institute of Technology SE-100 44 Stockholm, Sweden ↗
Autonomous vehicle technologies are rapidly growing and are expected to change transportation habits radically. Autonomous cars increase the likelihood of motion sickness by allowing everyone in the vehicle to become passengers and perform non-driving tasks such as reading, working, and socializing. Comfort is one of the critical factors in the acceptance of autonomous vehicles. This makes accurate estimation of motion sickness a necessity in the development stages of autonomous vehicles. The sensory conflict theory is a widely accepted theory that explains the mechanism of motion sickness. Computational models based on the sensory conflict theory are used to predict motion sickness and contain two main parts: a human perception model and a nonlinear fitting function to the subjective feeling of motion sickness. Models of the human perception, including the dynamics of the vestibular system, are used to calculate the difference between sensory inputs and the predicted motions in the brain, i.e. the conflict signal, which is the primary cause of motion sickness. One of the main limitations of motion sickness prediction is how to mathematically model human perception because of the complexity of the psychophysiological systems. The aim of this work is to implement and analyse different human perception modelling techniques, such as observer framework in the control theory and optimal estimator approach using Kalman filters, to evaluate their abilities to integrate with motion sickness prediction. In this study, the different human perception models are implemented and analysed using MATLAB / Simulink and the advantages, as well as disadvantages of the models, are discussed.
motion sick (Sivak and Schoettle, 2015). The vestibular system plays a pri- mary role in motion sickness, as given by the fact that people who do not have functioning vestibular organs do not become motion sick. There is a strong belief that motion sickness during travelling is caused by conflicting sensory information about the human state of motion. The vestibular system is a sensory system that is responsible for providing the brain with infor- mation about motion, head position, and spatial orientation. Therefore, a clear understanding of the dynamic behaviour of the vestibular system and human perception is crucial for analysing discomfort. The aim of this study is to analyse human perception estimation algorithms for identifying motion sickness in Autonomous Driving (AD). Provided analyses include computer simulations (in Matlab/Simulink) that can also be used for a better under- standing of the model behaviour and human perception which further help to model motion sickness.
The block diagram in Figure 1 represents a general structure of the motion sickness models based on the sensory conflict theory, which consists of two main parts. The first part consists of mathematical models of the spatial orien- tation perception of humans, including the dynamics of the vestibular system. The vestibular organs consist of semicircular canals, which detect angular motions, and otoliths, which detect gravito-inertial force (GIF), and their dynamics are simply modelled as transfer functions. The second part of the sensory conflict theory-based model focus on modelling the subjective feeling of motion sickness using a conflict signal that comes from the human perce- ption. The difference between the calculated signal by the neural system and sensed signal causes the conflict signal. The conflict signal is used as an input to a nonlinear function that accounts for the subjective feeling of motion sickness.
HUMAN PERCEPTION MODELING For decades, researchers have configured models by conducting experi- ments and examining the responses of subjects to find an appropriate and reliable model of human perception. Merfeld developed a model in Luen- berger (e.g. linear) observer framework (Merfeld and Zupan, 2002) which is one of the common methods for human perception modelling and builds on his previous work on squirrel monkeys (Merfeld, 1995). Haslwanter
Table 1. Overview of human perception models in the literature.
modified the Merfeld model to fit with the eye movement response during Off-Vertical Axis Rotation (OVAR) experimental tests done on human test subjects (Haslwanter et al., 2000). The observer approach has the advantage of merging multiple internal models into one observer to monitor different states and enable interaction between them. However, the gains are free para- meters, so the modeler fine-tunes them to align with empirical observations. Furthermore, the estimates of the states are carried out without considering noise. However, the human sensory system is noisy and inaccurate; therefore, it is crucial to account for noise or uncertainty during state estimation.
The estimator approach to model human perception was pioneered by work by Borah et al. where they modelled the human perception using the Kalman filter estimator (Borah et al., 1979). The estimator is essentially representing the internal model that processes the error between the affe- rent response and the estimated sensory signal and replaces the static gain in the Luenberger observer models, with the Kalman gain which takes into account the fact that the measurement of the sensory dynamic and its process is noisy and non-deterministic. Further development was done by Pommel- let by building an Extended Kalman Filter (EKF) model (Pommellet, 1990). The main difference to the Borah Kalman Filter is the use of quaternion inte- gration to model the rotational velocity cue to relative gravity orientation, the use of visual saturation to take into account the vection phenomena, as well as the noise dynamics which are significantly different from the Borah model. Selva (Selva, 2009) proposed an Unscented Kalman Filter (UKF) based model which further developed the Pommellet EKF model. The improvement is based on that the accuracy of the EKF model is limited mathematically into first-order in terms of the posterior distribution states. Karmali and Merfeld implemented a Particle filter estimator since the statistical accuracy level observed in Kalman Filter, EKF and UKF are limited due to the Gaussian distribution assumption made in these methods (Karmali and Merfeld, 2012). The Particle Filter approach could lead to relatively high computational cost, but on the other hand it also uses a distributed and parallel processing of the processing dynamics which represent more realistic computational pro- cessing in the brain. An overview of human perception models in literature can be seen in Table 1.
Vestibular Organ Modeling There is a common consensus in the research community that the Otolith could be modelled as an overdamped mass-spring-damper system based on
Analysis of Human Perception Models for Motion Sickness in Autonomous Driving 417
the hair cell deflection of the organ which influences the afferent response to the CNS. The latest development has been done by Telban and Cardullo (Tel- ban and Cardullo, 2005) and this is the most complete representation of the transfer function according to a literature review of various otolith models done by Asadi (Asadi et al., 2016). In observer-based human perception models, commonly a unity matrix is used to model the otolith organ dyna- mics. In the optimal estimator-based human perception models (e.g. Selva), the transfer function by Telban and Cardullo (Telban and Cardullo, 2005) is used. This transfer function is mathematically represented as TOTO = AFR(s)
Equation reproduced from the original PDF.Equation reproduced from the original PDF.
where τL is the lead time constant, τl is the long time constant and τs is the short time constant, KOTO is the static Otolith gain, AFR is the Afferent Firing Rate and f is the gravito-inertial force (GIF).
The other part of the vestibular organ modelling is the Semicircular Canal. It is commonly modelled using a torsion pendulum with a high degree of damping and specific parameters. The angular velocity vector is represented by ωs. In observer-based human perception models, the semicircular canal organ is modelled with a 2nd order transfer function developed by Fernandez and Goldberg (Fernandez and Goldberg, 1971), (Telban and Cardullo, 2005) and (Asadi et al., 2017).
= τlτas2
Equation reproduced from the original PDF.
(80s + 1) (5.73s + 1)
ωs
(τ as + 1) (τ ls + 1) = 5.73·80 s2 where τa is the adaptation operator and τl is the long time constant. Has- lwanter used the same transfer function with selected τa = 7 and τl = 190 (Haslwanter et al., 2000).
Motion Sickness Modeling A widely recognised theory proposes that motion sickness is caused by a signal mismatch between the internal model in the brain and received signals from sensory organs. The sensory conflict theory-based models are focused on the mechanism of motion sickness and use a modelled vestibular system and Central Neural System (CNS). Current motion sickness models are based on the observer framework of control theory approaches of modelling human motion perception. The neural mismatch model was proposed by Reason (Reason, 1978) and after modelled mathematically by Oman (Oman, 1990). Bos and Bles proposed the Subjective Vertical Conflict (SVC) theory model called 1D-SVC, which interprets Oman’s heuristic model (Bos and Bles, 1998 and 2002). Bos and Bles proposed a nonlinear fitting function to experimen- tal results of subjective human motion sickness feeling due to vertical motion disturbances by McCauley (McCauley, 1976). Braccesi and Cianetti propo- sed an extended version of the 1D-SVC model to a 3D-SVC model called
vestibular 3D model (UNIPG) and visual-vestibular models (UNIPGSeMo) (Braccesi et al, 2011a+b). A 6-DOF motion sickness model was proposed by Wada et al. (Wada et al., 2015) and Kamiji et al. (Kamiji et al., 2007). The Wada model expands the previously 1-DOF Bos and Bles model into a 6-DOF model by considering the interaction between the semicircular canals and the otolith organ. In this model, the nonlinear fitting method, feedback conflict mechanism, sensory dynamics, and internal model directly adapt from the Bos and Bles model. Moreover, the model also includes the rotational velo- city cues contribution on the relative gravity representation from the Merfeld model. Recently, the 6-DOF mathematical model of motion sickness seve- rity prediction has been extended by considering visual inputs (Wada et al., 2020). The main difference between the model by Wada and the previous human perception model by Merfeld is that the efference copy of body dyna- mic is modelled as a feedforward static gain. The static gain is then developed later into a dynamic feedforward gain (Wada, 2021) based on the sequential learning of the body dynamics by utilizing the Recursive Gaussian Process Regression (RGPR) method.
Evaluation and Test Methods Two physiological parameters are considered in this study to determine the relationship between the human perception models and motion sickness models: The velocity storage time constant (τVS) and the subjective vertical time constant (τSV).
The velocity storage mechanism was firstly introduced by Raphan et al. (Raphan et al., 1979) as a mechanism that when the sensed perception affe- rents from cupula were passed to the CNS, they were processed with filters which have a certain time delay constant of a leaky integrator due to the neural storage, hence giving the prolonged rotational velocity sensation. The velocity storage time constant could be observed by calculating the time that it takes for the rotational velocity perception to decay to 36.8% of the peak value during trapezoidal input stimulus which is done in the Earth Vertical Axis Rotation (EVAR) tests.
The subjective vertical time constant could be observed by calculating the time it takes to reach the magnitude of 63.2% of the steady-state final condition during the Fixed Radius Centrifugation (FRC) test.
In this study, the human perception models and motion sickness models are assessed by their ability to predict these two parameters (τSV and τVS) in comparison with the experimental test results.
The EVAR validation for velocity storage time (τVS) was simulated by inputting the stimulus input of a trapezoidal yaw motion with the peak of 60 deg/s reached in a second (60 deg/s2 rotational acceleration) during the acceleration and then keeping a constant peak value for 45 seconds. After that, the human was decelerated at the same rate as the acceleration. The simulation results are then compared with experimentally measured velocity storage time constant in yaw direction for adults (e.g. the average 17.4 s for normal subjects using eye movements recordings), according to Bertolini et al. (Bertolini et al., 2012).
Analysis of Human Perception Models for Motion Sickness in Autonomous Driving 419
Figure 2: Simulation results for human perception and motion sickness models where θsens denotes sensed GIF tilt, θsubj denotes subjective GIF tilt perception, ωsubj denotes the subjective rotational velocity perception. (a): velocity storage time constant evalu- ation under EVAR test. (b): the subjective vertical time constant evaluation under FRC test.
The subjective vertical time constant (τSV) in the Fixed Radius Centrifuga- tion (FRC) test was experimentally performed by Merfeld (Merfeld, 2001). The FRC test comprises seated human subjects rotated in a simulator with a peak yaw velocity of 250 degree/s after 10 s of stimulation. Due to the centri- petal force experienced at a radius of 0.54 m from the center of the rotation, the human subject will experience lateral acceleration as well with ay = −ω2r. The consequence of this experimental stimulation is that the direction betw- een the perception of the GIF wrt. gravity compared against the direction of the sensed GIF wrt gravity, will be delayed with a certain time constant before the GIF perception goes past the steady-state sensed GIF and eventu- ally reaches a steady-state condition as well. This time constant is called the subjective vertical time constant (τSV) and is calculated experimentally as an average of 28.1 s according to a study by Merfeld for the facing the motion setup while the humans were rotated in the clockwise direction (Merfeld, 2001). Special laboratory equipment was used to record the perceptual GIF tilt from subjects’ perceived earth horizontal axis.
RESULTS AND DISCUSSION
From the simulation results presented in Figure 2 (a), it can be observed that the velocity storage time constant for the Merfeld (τVS = 17 s), Haslwanter (τVS = 13.4 s) and Selva (τVS = 16.7 s) models are in good agreement with the experimental velocity storage time constant of 17.4 s. The Pommellet model has given a velocity storage time constant (τVS = 20.8 s) which is higher with respect to the experimental results due to noise tuning (selected process noise Q = 0.3) in the model parameters. In addition, the Vingerhoets (τVS = 27.1 s) and the Newman model have overestimated the velocity-time constant (τVS = 26.1 s), due to the adaptation of the free gain parameters. However, Pommellet, Vingerhoets and Newman’s models are still in the range of experimentally determined adult human responses.
For the subjective vertical time constant evaluation, it can be seen in Figure 2 (b) that Merfeld and Newman’s models give the closest resemblance to the experimental result of average 28.1 s. The transient response of the
Vingerhoets model behaviour is similar to Newman’s model with a longer time delay to sense GIF tilt. The simulation results also show that the Wada motion sickness model could not predict the tilt angle perception (subjective vertical constant) as observed in the experimental tests but agreed with the physical stimulus which was also demonstrated by (Uefune et al., 2016). The results indicate that the human perception models which fit with the expe- rimental results of human perception tests could be integrated with motion sickness models to improve their capability of motion sickness prediction. Irmak et al. (Irmak et al., 2021) have experimentally investigated the relati- onship of motion perception parameters (e.g. velocity storage and subjective vertical time constants) to motion sickness.
The Selva model which represents the estimator model with the biologi- cal noise in the measurement and process of the model is possibly the most plausible modelling source for further improvement of the available motion sickness models (e.g. Wada or UniPG), assuming that the noise are Gaussian distributed. Alternatively, the Particle filter approach by Karmali and Mer- feld could also be used in the case that the sensory dynamics noise are of non-Gaussian distributed type.
CONCLUSION
The main contribution of this paper is that different human perception models were implemented and evaluated under test conditions. The simula- tion results show that the implemented human perception models are able to predict the human adult range of sensation measured by experimental tests. These human perception models can also be tuned to fit with different expe- rimental results of human perception. The study also indicates that accurate human perception modelling could improve the prediction of conflict signal estimation. Integrating accurate human perception models in motion sickness models can improve their capability to predict motion sickness feelings. For future studies, there is still a need to further investigate the integration of human perception models, which includes visual and somatosensory input, as well as improve the nonlinear function part that predicts the feeling of motion sickness.
ACKNOWLEDGMENT
The authors would like to thank the Centre for ECO2 Vehicle Design, funded by the Swedish Innovation Agency Vinnova (Grant Number 2016-05195), and Volvo Cars for financial support.
REFERENCES
Asadi, H. (2016). “A review on otolith models in human perception,” Beh. Brain Res.
pp. 67–76. Borah, J., Young, L. R., and Curry, R. E. (1978). “Sensory mechanism modelling,” Tech. report, Texas.
Analysis of Human Perception Models for Motion Sickness in Autonomous Driving 421
Asadi, H., Mohamed S., Lim C.P. and Nahavandi S. (2017). “Semicircular canal modeling in human perception,” Reviews in the Neurosciences, 28(5), pp. 537–549. Bertolini G, et al (2012). “Is Vestibular Self-Motion Perception Controlled by
the Velocity Storage? Insights from patients with chronic degeneration of the vestibulo-cerebellum. PloS one, 7(6), e36763. Bos, J.E. and Bles, W. (1998). “Modelling motion sickness and subjective vertical mismatch detailed for vertical motions,” Brain Research Bulletin, 47(5), pp. 537–542. Bos, J.E. and Bles, W. (2002). “Theoretical considerations on canal-otolith intera-
ction and an observer model,” Biological Cybernetics, 86(3), pp. 191–207. Braccesi, C., and Cianetti, F. (2011a). “Motion sickness. Part I: development of a model for predicting motion sickness incidence,” Int. J. Hum. Factors Model. Simul., vol. 2, no. 3, p. 163–187. Braccesi, C., Cianetti, F. and Elia, A. (2011b). Motion sickness. Part II: experimental
verification on the railways of a model for predicting motion sickness incidence. International Journal of Human Factors Modelling and Simulation, 2(3), p.188. Fernandez, C. and Goldberg, J.M. (1971). “Physiology of peripheral neurons innervating semicircular canals of the squirrel monkey. II. Response to sinusoidal stimulation and dynamics of peripheral vestibular system.” J Neurophysiol, 34(4), pp. 661–675. Haslwanter T. et. al. (2000). “Three dimensional eye-movement responses to off-
vertical axis rotations in humans,” Exp Brain Res, (134), pp. 96–106. Irmak, T. et al. (2021). “Individual motion perception parameters and motion sickness frequency sensitivity in fore-aft motion”. Exp Brain Res 239, 1727–1745. Kamiji N., Y. Kurata, T. Wada et al. (2007). “Modeling and validation of carsickness
mechanism,” SICE Conference 2007, pp. 1138–1143. Karmali, F., and Merfeld, D. M. (2012). A distributed, dynamic, parallel computational model: the role of noise in velocity storage. Journal of Neurophysiology, 108(2), 390–405. Lim K., Karmali F., Nicoucar K. and Merfeld D. M. (2017). “Perceptual preci-
sion of passive body tilt is consistent with statistically optimal cue integration” J Neurophysiol 117:2037–2052. McCauley, M.E. et al. (1976), “Motion sickness incidence: exploratory studies of habituation, pitch and roll, and the refinement of a mathematical model,” HFR Incorporated, Tech. Report, no. 1733–2. Merfeld DM. (1995) “Modeling the vestibulo-ocular reflex of the squirrel monkey
during eccentric rotation and roll tilt,” Exp Brain Res, (106), pp. 123–134. Merfeld, D.M., et al. (2001). “Neural processing of gravito-inertial cues in humans.
II. Influence of the semicircular canals during eccentric rotation”, J. Neurophysiol, 85(4), pp. 1648–1660. Merfeld, D.M. and Zupan, L.H. (2002). “Neural Processing of Gravitoinertial Cues in Humans. III. Modeling Tilt and Translation Responses”. J. Neurophysiol, 87(2), pp. 819–833. Newman, M.C. (2009). “A multisensory observer model for human spatial orienta-
tion perception” Master Thesis, MIT. Oman C.M. (1990). “Motion sickness: A synthesis and evaluation of the sensory conflict theory,” Can. J. Physiol. Pharmacol., vol. 68, no. 2, pp. 294–303. Pommellet, P.E. (1990). “Suboptimal Estimator for the Spatial Orientation of a Pilot”
Master Thesis, MIT.
Raphan T, Matsuo V and Cohen B. (1979). “Velocity storage in the vestibulo-ocular reflex arc (VOR)” Exp Brain Res 35, 229–248 (1979). Reason, J.T. (1978). Motion Sickness Adaptation: A Neural Mismatch Model.
Journal of the Royal Society of Medicine, 71(11), pp. 819–829. Selva, P. (2009). “Modelling of the vestibular system and nonlinear models for human spatial orientation perception,”Ph.D.Dissertation, University of Toulouse, France. Sivak, M. and Schoettle, B. (2015). “Road safety with self-driving vehicles:
general limitations and road sharing with conventional vehicles,” Rep. No. UMTRI-2015-12. Telban, R.J. and Cardullo, F.M. (2005). “Motion Cueing Algorithm Development: Human-Centered Linear and Nonlinear Approaches,” NASA TechReport CR- 2005–213747. Uefune, T., Wada T. and Sonoda, K. (2016). “Computation of the Vestibulo-Ocular
Reflex for eye closure based on the 6DOF-SVC model.” International Conference on System, Man, and Cybernetics, Budapest. Vingerhoets, R.A.A., Van Gisbergen, J.A.M. and Medendorp, W.P. (2007). “Verticality Perception During Off-Vertical Axis Rotation.” J Neurophysiol, 97(5), pp. 3256–3268. Wada T., Kamij N., and Doi S. (2015). “A Mathematical Model of Motion Sickness
in 6DOF Motion and Its Application to Vehicle Passengers,” arXiv: Neurons and Cognition. Wada T. et al. (2020). “A Computational Model of Motion Sickness Considering Visual and Vestibular Information,” IEEE Int. Conf. Syst. Man, Cybern., pp. 1758–1763. Wada T. (2021). “Computational Model of Motion Sickness Describing the Effects
of Learning Exogenous Motion Dynamics” Front. Syst. Neurosci. 15: 634604.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-36-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional