Gary Burnett, Richard Donkor, Sarah Sharples (2022). A Presence Questionnaire for Understanding the Driving Simulator Experience. In: Advances in Human Aspects of Transportation: Part III. AHFE Open Access, vol. 17. https://doi.org/10.54941/ahfe100757
Presence is seen to be important in the Virtual Reality (VR) domain, as there is often a close link between an individual’s experience of a virtual environment and their subsequent performance/behaviour. Unfortunately, the generic presence questionnaires available in the VR literature have been developed considering tasks not necessarily relevant to the driving situation - for instance, relating to the participant’s ability to move objects, use touch, smell items, etc. This paper describes the evolution of a driving simulator experience questionnaire, designed specifically to enable researchers and practitioners to understand how study participants perceive the driving simulation environment in relation to real-world equivalent situations. The final questionnaire has been informed by 20 interviews and focus sessions, 5 expert reviews, and simulator studies involving 225 people across different fidelity simulators and research institutions. Specifically, 41 items are currently included, believed to be of importance for natural driving. Examples of questions relate to strategic - “I felt as if I had been on a journey”, tactical -“I was compelled to obey the displayed road signs…”, and control – “I had a strong sense of physically controlling the vehicle” elements of the driving task. Items are also included relating to social aspects of driving, e.g. “I was aware that other people were driving cars around me”. In next steps, we intend to evaluate the questionnaire by comparing ratings made by individuals with their subsequent driving behaviour/performance in a simulator.
Simulators are active-safety technologies to investigate and develop solutions to the many road accidents and vehicle design and transport issues. Coupled with the use of prototype or real car cabins and driving controls, visual, auditory and motions systems, advanced technological improvements have enabled the development of close-to- natural driving scenarios which gives researchers and drivers the benefit of intrinsically safe driving and research environments (Kaptein et al., 1996, Törnros, 1998). Modern day simulators are used in investigations which test the effects of distraction tasks on drivers’ performance and situation awareness (Philip et al., 2003), the effect of new in- vehicle systems on driver behaviours (Burnett and Donkor, 2010, Liu, 2003) and many others. In spite of the many benefits of simulators, various concerns have been raised. For any driving simulator, the challenges relate to how the results derived from a simulator show a true reflection of reality, the level of fidelity to consider in certain simulation studies, behavioural realism and simulation experience – feeling of immersion, presence, control, induced motion sickness etc. These concerns have been the major contributing issues to the lack of credibility and the low level of trust some researchers and designers put into the use of simulators, their operations and the results derived (Owsley and McGwin, 2010, Bach et al., 2008).
From a simulation experience perspective, drivers interact with various systems which combine to give the driver a perception of driving in a simulated environment (Allen et al., 2007, Park et al., 2005). These systems constitute what is termed fidelity - also defined as the extent to which simulation imitates the reality of an environment (Gross, Pace, Harmon and Tucker, 1999). Other aspects of fidelity describe the extent to which the simulator replicate the necessary psychological and cognitive functionalities e.g. situation awareness, decision making etc. (Liu et al., 2009). The integration and optimum operation of these systems can influence positively a driver’s experience (Burnett et al., 2007). On the other hand, certain unsuitable situations or experiences e.g. temporal and spatial distortions (Nichols, 1999), pixelated graphics (Owsley and McGwin, 2010) etc. can negatively impact on the driving experience. The fidelity and the operations of a simulator should therefore be considered essential if researchers aim to enhance and positively impact a driver’s experience.
There is also a reason to believe that an individual’s simulation experience can affect validity. Validity in simple terms refers to the extent to which a simulator provides the results expected of it. Validity is a major issue, particularly behavioural validity - the extent to which drivers’ behave in the simulator as they would in the real world, (Blaauw, 1982). Certain driver behaviours observed in simulations (e.g. an empirical investigation being treated as game-play) can unquestionably have negative consequences on the validity of study results. In addition, the inherent artificiality of simulators and the lack of risk have also been suggested as contributing factors to the false sense of safety and responsibility and the unrealistic behaviours seen in studies (De Winter et al., 2012, Ranney, 2011, Donkor et al., 2011). Understanding how these factors can be observed and evaluated can help improve a driver’s experience and the validity of results.
Fidelity, validity and driving experience can be associated with various virtual reality (VR) concepts, since a driving simulator is essentially a specific form of VR. VR factors such as presence, psychological absorption, immersion and many others have been reported to influence a user’s experience in a Virtual Environment (VE). Presence is a well-established construct used to determine a person’s experience in VR (Lombard and Ditton, 1997, Lessiter et al., 2001, Slater et al., 1994, Witmer and Singer, 1998). Presence is linked to constructs such as flow, absorption, immersion and dissociation, which collectively describe a user’s experience in a VE. Presence is often measured with questionnaires, but these contain questions/constructs not necessarily tailored to driving. To evaluate drivers’ simulation experiences, there is a need to relate the factors of fidelity, validity, realism (all integral parts of the simulation), the perceived and observed driving behaviour as well as the VR experience. Understanding and assessing just one of these aspects does not suffice. A composite assessment requires well-grounded assessment tools which are currently non-existent. Traditionally, measuring user’s opinion of VR contexts has been done using interview questions or questionnaires. This paper reports the development of a questionnaire – a driving simulator experience questionnaire (DSEQ) - as a potential assessment tool to evaluate drivers’ subjective experiences within simulated environments.
Virtual Reality Experience Components Presence has been one of the most prominent and useful constructs for investigations which involve human interactions within a virtual environment (VE). Since its inception, various researchers, designers and developers have sought to design, develop and use presence as an experiential quality metric to assess a person’s physical, behavioural, physiological and psychological experience in a VE (Witmer and Singer, 1998, Slater et al., 1994, Lessiter et al., 2001). Presence has been defined and described in various ways. Witmer and Singer (1998) p.1 defined presence as “…the subjective experience of being in one place or environment, even when one is physically situated in another”. Conceptually, presence is coarsely divided into two broad categories – physical and social with co-presence merging these two categories. Physical presence refers to “the sense of being physically located in mediated space, whereas social presence refers to the feeling of being together, or the social interaction within a virtual or remotely located world” (Riva et al., 2003). Social elements of a VE, such as interaction with actors and objects contribute to social presence (Lombard and Ditton, 1997). A range of variables categorised into user characteristics (e.g. user’s perceptual, cognitive and motor abilities, prior experience in VE, readiness to suspend disbelief) and media characteristics (e.g. display medium, the extent of sensory information presented, users’ ability to modify aspects of the environment such as objects, actors and events) have been reported to contribute to an enhanced or worse sense of presence. (Riva et al., 2003, Baños et al., 2004, Lessiter et al., 2001, Slater et al., 1994). User and media characteristics can impact on technological (media) presence and the psychological (inner) presence.
In terms of driving simulators, inner presence influences a driver’s psychological state of mind. For example, drivers’ emotional involvement, motivations and the perception of realism have been reported as impact factors of inner presence (Loomis, 2002). Similarly, media characteristics can be categorised under the fidelity (both physical and psychological) variables. For example, visual field of view, screen resolution, graphical complexity, traffic representation, wind/road noise and others influence how a driver perceives presence in a driving simulator (Lin et al., 2002). Presence has been evaluated with both objective measures e.g. skin conductance, heart and respiration rate, ocular responses (Jang et al., 2002) and subjective approaches. In this respect, a number of questionnaires e.g. Presence Questionnaire-Witmer and Singer (1998), ITC-SOPI-Lessiter et al. (2001) etc. have been developed. These questionnaires have been used in various contexts, but rarely in driving simulation research. In fact, only a few studies (e.g. Burnett and Mowforth (2007) and Scheuchenpflug, Ruspa and Quattrocolo (2003)) have attempted to investigate driver presence and experience in simulated driving. These authors agree that the research community is in need of a standard and generic measurement tool which could be used in driving simulator studies to subjectively measure a driver’s sense of presence.
Immersion is also closely linked to media presence. Defined as “the physical extent of the sensory information provided as function of the enabling technology”, (Kalawsky, 2000), immersion suggest that a user’s perception of a VR experience can be influenced by becoming engaged in the VE while retaining some awareness of the surrounding technology (Banos et al., 2004; Witmer and Singer, 1999). In hardware terms, the capacity of a simulator to induce the feeling of actually being a part of the road environment determines its immersive level. Therefore, immersion could be a good indicator of a driver’s simulation experience. Another associated construct is flow. Flow determines a user involvement and enjoyable experiences when performing activities in a VE. Csikszentmihalyi (2000) describes flow as a “dynamic state” and “the holistic sensation that people feel when they act with total involvement”. For a driver to reach a state of flow, the driving activity should necessarily lead to involvement, facilitate concentration on the distinct features of the drive, provide a clear goal and instant feedback to the driver and enable a sense of control. Such experiences have been reported as the positive inducers which usually results in a driver’s loss of self-consciousness of the real world (psychological absorption), time passing more rapidly and the driver acquiring challenging but enjoyable experiences of being on a journey (Baños et al., 2004). Flow has been measured by using questionnaires such as the Experience Sampling Method (ESM) (Csikszentmihalyi, 1982) including questions about flow experience (‘‘I was so involved in what I am doing; I didn’t see myself as separate from what I am doing”) questions on challenges and skills, mood and motivation, etc.
Lastly, among the potential problems that users experience in VE’s, dissociation is perhaps the most widely discussed. Bernstein and Putnam (1986) pp.727 defined dissociation as ‘‘the lack of normal integration of thoughts, feelings, and experiences into the stream of consciousness and memory”. Wilson (1997) reported that a large number of interacting factors (e.g. temporal delays, environmental stability, sensory bandwidth) lead to an inconsistency between users’ performance and the performance of the system. For most situations, this occurs when there is a lack of interactivity between the participant and virtual world causing postural instability, sickness etc.
Driving Behavior in Simulators The discussion presented above focussed on VR constructs which could influence a driver’s experience in a VE. Driving behaviour is also integral to the driving experience. As previously mentioned drivers’ behaviours that are consistent with real world behaviour can lead to a high level of simulation experience and influence the validity of simulation results. In explaining driving behaviour, a few general questions are of relevance. First, what aspects of behaviour and performance can be ascertained, and secondly what characteristics of the simulated environment are likely to influence drivers’ physical, emotional, psychological and social behaviour.
Ideally, an overall theory or model encompassing all aspects of driving would be beneficial - however, such a model is yet to be developed. A number of driver behaviour models have been developed in the last decade. Although these models, which are mainly conceptual and computational (i.e. models that compute, simulate and predict various aspects of driving behaviour), have successfully enabled researchers to understand the representational behaviour, procedural components and actions of the driver, there is still a great deal of work to be done in achieving a truly integrated driver model that simulates and predicts real-world driver behaviour in simulators. A focus for this paper is the functional and psychological aspects of driving as seen in driving simulations which influence driving experiences.
The hierarchical model of Michon (1985) indicates that at the strategic level, various knowledge-based behaviours are of relevance. For instance, important aspects such as memory, and rules are also known to inform a driver’s plan of route, mode of driving and the goal of driving (Wickens et al., 2004). The outcomes of the plans and decisions from the strategic level can impose certain performance constraints on the tactical (manoeuvring) and control levels. Manoeuvres may require a set of rules and a certain degree of control. Behaviours and actions such as steering, braking and gear changes occur at these levels. This would also depend on a driver’s skill, experience and abilities to evaluate performance at the tactical and strategic levels. In addition, the communal nature of driving posits that social psychological models such as the four-facet model by Groeger (2000) are also applicable to driver behaviour. The four-facet model takes into consideration the cognitive and personality factors that influence the driving task and describes the characteristics that may be measured on an individual level. Groeger explains that in driving, a series of processes can be triggered when there is a change (discontinuity) in a driver’s active goal. These processes comprise a system of forward and feedback links which enable a driver to adjust and adapt in their specific driving experience. It is noteworthy that, although observable simulation behaviours may be linked to experiences in real world driving, a de-motivated driver may exhibit inappropriate or inconsistent behaviour leading to concerns for validity or transferability of results.
QUESTIONNAIRE DEVELOPMENT Literature concerning VR concepts and the driving behaviour models provided useful insights into key factors which could influence drivers’ simulation experiences. Several knowledge elicitation methods were used to identify and consolidate the various VR and driving behaviour aspects of “driving experience”. Task decomposition methods were then used to gather detailed information regarding the driving task from drivers and the perspective of experts. The data collection tasks were used to gather a list of items which could be included in the DSEQ.
A series of qualitative sessions were conducted to comprehensively describe driving and its related activities. In semi-structured interviews, five experts in human factors and driving simulation were asked: (i) what mechanisms work to create realism in driving simulations? (ii) what things are considered important to people to create the feeling of driving in a simulator? (iii) How is motivation, perception and control perceived in the simulator, (iv) what are the main entities involved in driving? Transcriptions and written comments were collated to provide the core components of driving and the sub-component psychological, functional, and physical factors that contribute to the driving simulation experience as seen from the experts’ perspective.
For initial questions development, focus groups and interview sections were then conducted. Fifteen drivers (10 with experience using driving simulators and 5 without simulation experience but related VR experience e.g. car and racing video games) participated in a focus group. Each group of five participants were led by the experimenter to discuss and describe in their own words their individual experiences, their shared understanding and the ways in which they are influenced by others when driving in the simulator or involved in a driving racing game. The major responses were analysed based on Strauss and Corbin (2008) grounded theory using content analysis (Holliday, 2002) to form categories such as psychological experience e.g. sense of driving in a new environment, realistic scenery, physical experiences e.g. responsive or irresponsive controls, sickness and social experiences e.g., recognising and interacting with road users, roadway architecture, rules etc.
An initial set of 30-items for the DSEQ was generated from the previous activities. A pilot version of the questions was administered to 19 drivers to explore the DSEQ basic properties. The result was cronbach’s alpha of 0.65. Additional questions delving further into manifestations of drivers’ experiences with a focus on presence and immersion were included to expand coverage of DSEQ. The revised 41 item version was administered to two set of 20 drivers who were participating in a driving simulation experiment. In this initial study, a large percentage of the questions were clearly understood. For example for the question “The availability of auditory information such as engine noise and road noise improved the experience”, 94% of the participants agreed or strongly agreed while 5% stayed neutral. Other questions such as “I was aware that other people were driving cars around me” and “I was checking all around me during the journey” received a high response. Cronbach’s alpha was 0.77. The results showed that the five-point Likert scale (Strongly disagree - Strongly agree) for all items provided a consistent response option. It also made it easier for respondents to complete the questionnaire and highlighted questions that needed rephrasing due to ambiguity. However, a Rasch model analysis (Rasch, 1960) - suggested that additional items were needed.
QUESTIONNAIRE REFINEMENT – EXPERIMENTAL STUDY As a result of the above activities, a 66-item version of the DSEQ was developed. This version was then administered to 225 respondents across different simulator configurations and research organisations. The sample participants comprised drivers of diverse demographic origins, each of which experienced a simulated drive. Simulators varied according to the level of fidelity (both physical and psychological), the type of road and driving environment and the tasks performed. The content of the display environment also varied in photorealistic features and interactivity.
Participants and Procedure Of the 225 respondents, 40 had participated in two studies by Young et al. (2011) and Birrell and Young (2011) which used the Brunel University medium-fidelity Driving Simulator. Participants in these two studies drove on an urban road and simulated rural driving with a set speed of 60mph while performing various in-vehicle tasks. A further 59 respondents had just used the low fidelity simulator in the MRL Lab at the University Of Nottingham – in which they used a head-up display while driving on simulated motorway, rural road and urban road environments. Sixty subjects, n=20 used the medium fidelity Southampton University Driving Simulator, in a study which involved evaluating the effect of trust in SATNAV voices and a study which involved drivers performing tasks that involved reading, menu interactions etc. A further 24 drivers had participated in a study at the University of Nottingham medium fidelity simulator within the Human Factors research group (Donkor, Burnett and Sharples, 2013) which involved driving on a rural road whilst undertaking various secondary tasks. In this study an emergency braking event was included to consider emotional responses. The rest of the sample participants were recruited in various studies which involved drivers interacting with simulators in many ways e.g. short first time drives, demonstrations, investigation on visual demand of in-vehicle interfaces, simulated- strategic bus drives etc. In all studies, participants completed the questionnaire after the drive.
Preliminary Analyses Global estimates of reliability were computed for the DSEQ. Respondent reliability index were analysed with WINSTEPS. Item reliability and Principal Axis Factors were analysed with SPSS version 20.0. First, the appropriate direction of scoring for each questionnaire item was estimated by observation. A high score on each of the strongly agree or strongly disagree scale was regarded as a high positive or higher negative experience and vice versa. When in doubt about the question, the number of “don’t understand” responses was analysed.
Results
Response Rate The minimum set of rules recommended by Ferguson and Cox (1993) was used to ensure that a sufficient and appropriate subject sample were obtained. The DSEQ met these heuristics as shown in Table 1. The subject-to- variables ratio was 3.4:1 and it was estimated that this ratio would increase as items were removed. Response to the “don’t understand” items recorded during the data analyses were noted and considered during the reliability item analysis.
Table 1: Subject, Variable and Factor ratios for initial administration of DSEQ (recommendations from Ferguson & Cox (1993))
Of the 66 questions (items), 10 questions had a “don’t understand” response, with a range of 3-8 responses. As “don’t understand” responses had to be eliminated, reasons for their occurrences and subsequent elimination were determined statistically in reliability item analysis. Participants who responded “don’t understand” to 3 or more of the questions were eliminated. This process led to the elimination of six respondents.
Reliability Cronbach’s alpha and Rasch’s estimate of person reliability for the 66-item version were 0.80 and 0.70 respectively which showed internal consistency and stability of the data set. Although the 0.80 co-efficient was considered desirable (Kline, 1994), Loewenthal (1996) cautions that such a high reliability coefficient may be indicative of a repetitious questionnaire and as such a questionnaire should also aim to achieve few items and high validity. Cronbach’s alpha was of 0.86 after questions with “don’t understand” responses were eliminated. Item-total correlation coefficient for each of the items was also obtained. Loewenthal (1996) recommend that items with a correlation coefficient below 0.15 should be removed. Fifteen items comprising a combination of low correlation co-efficient were eliminated. Most of these questions had been previously scored with a “don’t understand” response. The overall Cronbach’s Alpha value rose to 0.89, indicating an improved reliability.
Principal Axis Factoring The resultant 51 items were entered into a Principal Axis factoring (PAF) analysis1. Preliminary checks using Kaiser-Meyer-Olkin measure of sampling adequacy showed KMO=0.75 and most KMO values with 0.65 - 0.79. Bartlett’s test of sphericity, ᵪ2 (1275) = 6285.47, p<0.0001, indicated that correlations between items were significantly large for PAF. Using Orthogonal Varimax rotation, a scree plot – a plot indicating the Eigen values of each of the factors extracted- was produced. The scree for the DSEQ is shown in Figure 1.
Factors with Eigen values greater than 1 were initially considered. Tabachnick and Fidell (1989) report that factors extracted are usually within the range of variables divided by 3 and 5 (p. 635). With 51 items it was felt that 10 and above factors was an overestimation. Two independent observers identified two points of inflexion on the Figure 1. These corresponded to 5 and 10 factors.
1 Factor analysis summarizes patterns of correlation by revealing groups of correlated items which are called factors. Eigen values – numerical term used to represent the proportion of variance across items – produces factors resulting from the analysis. An item (a variable) can load onto just one factor or cross-load on other factors. Ideally a minimum of three variables loadings indicate a factor. Human Aspects of Transportation III (2022)
Points of Inflexion
Figure: 1 DESQ Factor Scree Plot
A choice for 9 factors proved most appropriate because one item loaded on a single factor but also highly loaded on all other factors. In selecting the number of item to factors loadings, Igbaria et al. (1994) suggest that an item should be considered if it has a coefficient of 0.50 or higher on a specific factor and a loading no higher than 0.25 on other factors. Ferguson & Cox (1993) propose that a loading of 0.4 should be considered as appropriate in an endeavour to reach high factor saturation (high mean factor loadings for a factor). A correlation coefficient of 0.35 was adopted to select appropriate items loading onto a factor. It was envisaged this modified value (the usual loading coefficient being 0.30) would eliminate the greater number of probable cross-loadings considering the set of items.
In cross-loadings situations, a criterion was chosen to select the item-to-factor loadings. First, an item primary factor must be 0.2 greater than any cross-loading, in addition to having a primary loading above 0.35. Ferguson & Cox (1993) suggest that if the difference between the primary and secondary item-factors loadings is <0.2, then the secondary loading should be rejected to attain a clear and distinct factor structure. However, if >0.2 the item can remain and be assumed to load onto the factor with the highest loading.
Factor Structure Three human factors experts independently assigned names to the nine factors and a high level of agreement was established. Table 2 shows the final factors and their associated items. Eigen values and corresponding percentage of the cumulative variance are also shown. The asterisk in two closing brackets “(*)” shows items that were deemed appropriate to load unto two factors.
Table 2: Factors and Items loadings with respective Eigen values and cumulative variance2 Factor Factor name Items
Table available in the original PDF paper.
2 The star in two closing brackets “(*)” shows items that were deemed appropriate to load on two factors. Human Aspects of Transportation III (2022)
The final version of the questionnaire is available at https://www.survey.bris.ac.uk/nottingham/dseqi/. Participants should respond to the items with a rating on a standard five point Likert scale, where 1=strongly disagree and 5=strongly agree. A total of 41 items were retained following the analysis detailed above.
DISCUSSION AND CONCLUSIONS By using a number of VR, driving behaviour and simulation concepts, the analysis presented has shown that the experience of drivers in simulators comprise a number of factors including the virtual and behavioural experience, sensory experience, realism of car and driving environment, immersion, vehicle control, social experience and break in presence. Several human factors methods, particularly, the knowledge elicitation techniques (literature, interviews, focus groups) produced major items for the questionnaire. For instance, most of the people involved in the focus groups and interviews recognised how social experiences, such as recognising and interacting with road users, road artefacts and road rules and regulations, play a major role in their driving simulator experience. These experiences were captured in questions such as “I was aware that other people were driving cars around me” and “The driving scene depicted could really occur in the real world”. Besides these, other psychological experiences were also captured in the questions such “Driving in the virtual environment seemed consistent with my real world experiences”, “the vehicle I drove behaved in a realistic way” and the “the actions I performed in the car seemed consistent with the real world”.
The first factor Realism of Car and Driving Environment comprises a number of items that have similarities to Sense of Physical space, as reported in Lessiter et al. (2001). These items describe fidelity of the simulator, driver perception of the mechanism of the simulated in a VE and the perception of realism of other components in the VE. This shows that the driving experience is not only related to a user’s sense of being in a vehicle but also within a spatially dynamic driving environment, including an assessment of the interaction between these components i.e. the appeal, realism, believability of the physical and projected environment.
The Virtual and Behavioural Experience factor provides a measure of drivers’ involvement in the simulated environment, their sense of trip, sense of location, and behaviours. This factor indicates that involvement and sense of trip are important elements of a drivers’ assessment of the experience, a point which has been highlighted by Ranney (2011). This factor also shows that the presentation of a virtual driving environment inevitably influences certain behaviours and perceptions of driving in the car. It also relates to the concept of flow and psychological absorption as described by Csikszentmihalyi (2000) p.36. Drivers’ perception of the dynamic state of the vehicle influence the sensation of driving felt when they are total involved in the simulation. It can also be linked to the psychological state of mind described by . Indeed, the DSEQ includes questions about how drivers feel about being on a journey, whether they sense they were driving and how this was shown e.g. physical sensations, movement of the vehicle, the perception of driving on real road, and sense of control over events outside the vehicle. These are likely to be influenced by the fidelity, but are also likely to impact on drivers’ behaviours.
The third factor Sensory experience relates to drivers senses’ when involved in a driving simulator. This factor focuses on the photorealism of the driving environment, sensory modality (use of auditory and kinaesthetic feedback), environment richness and consistency of information and drivers’ awareness of the simulator components (Witmer and Singer, 1998, Riva et al. 2003). These components are also related to the media contents and form. For instance, a simulator with a large field of view, surround sound system and reliable motion feedback system is like to produce an enhanced driving experience compared to one without any of these features.
In addition to this, the fourth factor, Immersion, relates to the driving simulation experience when drivers engage with the driving environment as well as the perception of sensory features of the driving environment and interaction within the environment(Kalawsky, 2000, Lessiter et al., 2001). The primary determinants of Immersion are the combination of user, media content and media forms. Support for this assertion has been well-studied by many researchers (Slater et al., 1994, Lombard and Ditton, 1997, Kalawsky, 2000, Lessiter et al., 2001, Riva et al., 2003, Insko, 2003). With the reliable technological component, a higher degree of immersion can contribute to an improved simulation experience.
Items that comprise the fifth and seventh factors Adaptation and Awareness of Hazards and Attentional demands are more specific and relate to driving experiences where these factors are deemed influential. It was observed that these two factors received high factors loadings due to the nature of simulations e.g. user interacting with in-vehicle devices, head up displays, road direction signs and so on. Nonetheless, these two factors provide a measure of driving experience in terms of assessing drivers’ emotional involvement, motivations and realism (all contributors to impact factors of inner presence) (Loomis, 2002).
Questions that comprise the sixth factor, Tactical Control, relate the driving behaviour and the believability that drivers have to control the simulated vehicle in physical space. Considered in isolation, this factor constitutes key components of the work conducted by Gibson and Crooks (1938), McKnight and Adams (1970) and driver behaviour described by Michon (1985). The questions from this item can be used to judge how drivers behave in relation to abiding by the rules of the road, provide a sense of control, distance and general awareness of events that contribute to the experience. In a combined state, these items can influence a driver’s psychological absorption and social interaction or social presence.
As can be seen in table 2, the Social Context and Realism factor had many double factor loadings. The items of this factor could be used to support other factors - since they can be catered for in the first five factors. The first two questions will be of significance to the naturalness and solidity of the environment. For instance, having trees and traffic on either side of a motorway is likely to enhance the perceived naturalness of the driving environment and in turn, enhance the driving experience. Simple experiments utilising no traffic should also produce differences on this scale and reduce the perceived ecological validity of the simulation experience.
The last factor, Break in presence, show that there is a probability that certain situations can reduce the simulation experience (dissociation). For example, a driver looking around in a desktop simulator and asking if this is a real simulator is likely to produce a higher score on this scale. This will indicate that the fidelity of the simulator is inappropriate as he/she clearly expects more from the experience. Wither and Singer (1998) report on simulator sickness being a contributing factor to a break in presence. Sickness was not addressed in this questionnaire but it can be suggested that content may also affect a self-reported break in presence.
The DSEQ is in many ways similar to previous questionnaires which have been used to assess presence and user experience. It shares similarities in terms of the influence of fidelity, flow, immersion, presence and dissociation. However, it has distinct and unique items which focus on assessing driver simulation and simulator specific experiences. This questionnaire is by no means an all-encompassing measure of a driver simulation experience. It is suggested that these items can be treated as independent items in situations where researchers have an interest in individual items and their effects on the simulation experience or in a combined form when the combined items captures the overall experience.
The results of the analysis conducted shortened the initial questionnaire into a more structured one. However, as shown in table 2, further items could be removed to create a more robust set of items. Removal of items which would not specifically apply to all driving situations e.g. “I reacted to critical situations on the road” can help increase the internal consistency and sensitivity of the questionnaire for investigations which are not relevant to this factor.
The factor structure presented in this paper has been based on responses across a range of simulators of varying fidelity and research protocols. It is the first of its kind and we encourage driving simulator researchers to adopt it within their studies as a means of understanding the experience of their participants. Each of the factors derived from the analysis will be determined by the interaction between the driver, physical fidelity of the simulator e.g. vehicle, roadway, road traffic and environments and the purpose of the simulation, behavioural influences etc. In further development, it will now be important to consider the generalisability of the questionnaire and the reliability of the factors. Moreover, it will be critical to evaluate the questionnaire by comparing ratings made by individuals across a range of different simulation experiences with their subsequent driving behaviour/performance in the simulator. This exercise will aim ultimately to understand the extent to which the DSEQ can predict the validity of a driving simulator study. In addition, it would be of value to consider in quantitative terms what different ratings for factors in the DSEQ mean in absolute terms – i.e. what constitutes a ‘good’ versus ‘bad’ experience for a driving simulator?
REFERENCES
Allen, R. W., Park, G. D., Cook, M. L. & Fiorentino, D. The effect of driving simulator fidelity on training effectiveness. DSC 2007 North America, 2007. Bach, K. M., Jæger, M. G., Skov, M. B. & Thomassen, N. G. Evaluating driver attention and driving behaviour:
comparing controlled driving and simulated driving. Proceedings of 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction, 2008 Liverpool, United Kingdom. 1531514: British Computer Society, 204. Baños, R. M., Botella, C., Alcañiz, M., Liaño, V., Guerrero, B. & Rey, B. 2004. Immersion and emotion: Their impact on the sense of presence. CyberPsychology & Behavior, 7, 734-741. Bernstein, E. M. & Putnam, F. W. 1986. Development, reliability, and validity of a dissociation scale. The Journal
of nervous and mental disease, 174, 727-735. Birrell, S. A. & Young, M. S. 2011. The impact of smart driving aids on driving performance and driver distraction.
Transportation research part F: traffic psychology and behaviour, 14, 484-493. Blaauw, G. 1982. Driving experience and task demands in simulator and instrumented car: A validation study.
Human Factors: The Journal of the Human Factors and Ergonomics Society, 24, 473-486. Burnett, G. & Donkor, R. 2010. Evaluating the impact if Head-Up Display Complexity on peripheral detection performance: a driving simulator study. Advances on Transportation Studies. Burnett, G. E., Irune, A. & Mowforth, A. 2007. Driving Simulator Sickness and Validity: How Important Is It to
Use Real Car Cabins? Advances in Transportation Studies, pp. 23-32. Csikszentmihalyi, M. 1982. Toward a psychology of optimal experience. Review of personality and social psychology, 2, 13-36. Csikszentmihalyi, M. 2000. Beyond boredom and anxiety, Jossey-Bass. De Winter, J. C. F., Van Leeuwen, P. M. & Happee, R. 2012. Advantages and Disadvantages of Driving Simulators:
A Discussion. In: Spink, A. J., F. Grieco, O.E. Krips, L.W.S. Loijens, L.P.J.J. Noldus & Zimmerman, P. H. (Eds.) Proceedings of Measuring Behavior 2012. Utrecht, The Netherlands. Donkor, R. A., Burnett, G. E. & Sharples, S. 2011. Validating driving simulators for distraction research.
Proceedings of the 2nd International Conference on Driver Distraction and Inattention. Gothenburg, Sweden. Ferguson, E. & Cox, T. 1993. Exploratory factor analysis: A users’ guide. International Journal of Selection and Assessment, 1, 84-94. Gibson, J. J. & Crooks, L. E. 1938. A theoretical field-analysis of automobile driving. The American Journal of
Psychology, 51, 453-471. Groeger, J. A. 2000. Understanding driving: Applying cognitive psychology to a complex everyday task, Psychology Press. Holliday, A. 2002. Doing & writing qualitative research, Thousand Oaks, CA, Sage. Igbaria, M., Schiffman, S. J. & Wieckowski, T. J. 1994. The respective roles of perceived usefulness and perceived fun in the acceptance of microcomputer technology. Behaviour & Information Technology, 13, 349-361. Insko, B. E. 2003. Measuring presence: Subjective, behavioral and physiological methods. EMERGING
COMMUNICATION, 5, 109-120. Jang, D. P., Kim, I. Y., Nam, S. W., Wiederhold, B. K., Wiederhold, M. D. & Kim, S. I. 2002. Analysis of physiological response to two virtual environments: driving and flying simulation. CyberPsychology & Behavior, 5, 11-18. Kalawsky, R. S. The Validity of Presence as a Reliable Human Performance Metric in Immersive Environments.
PRESENCE 2000, 3RD INTERNATIONAL WORKSHOP ON PRESENCE, 2000. Citeseer. Kaptein, N., Theeuwes, J. & Van Der Horst, R. 1996. Driving simulator validity: Some considerations.
Transportation Research Record: Journal of the Transportation Research Board, 1550, 30-36. Kline, P. 1994. An easy guide to factor analysis. Rutledge, London and New York, 67-68. Lessiter, J., Freeman, J., Keogh, E. & Davidoff, J. 2001. A cross-media presence questionnaire: The ITC-Sense of Presence Inventory. Presence: Teleoperators & Virtual Environments, 10, 282-297. Lin, J. J., Duh, H. B. L., Abi-Rached, H., Parker, D. E. & Furness, T. A. 2002. Effects of Field of View on Presence,
Enjoyment, Memory, and Simulator Sickness in a Virtual Environment. Proceedings of the IEEE Virtual Reality Conference 2002. IEEE Computer Society. Liu, D., Macchiarella, N. & Vincenzi, D. 2009. Simulation fidelity. Human Factors in Simulation and Training, 61-
73. Liu, Y.-C. 2003. Effects of using head-up display in automobile context on attention demand and driving performance. Displays, 24, 157-165. Loewenthal, K. M. 1996. An introduction to psychological tests and scales, UCL Press Limited. Lombard, M. & Ditton, T. 1997. At the heart of it all: The concept of presence. Journal of Computer-Mediated
Communication, 3, 0-0. Loomis, J. Understanding synthetic experience must begin with the analysis of ordinary perceptual experience.
2002. IEEE, 54-57. Mcknight, J. & Adams, B. 1970. Driver Education and Task Analysis, Vol. 1, (Technical Report). National Highway Safety Bureau, Washington, DC. Michon, J. 1985. A critical view of driver behavior models: What do we know, what should we do. Human behavior
and traffic safety, 485–520. Nichols, S. 1999. Virtual Reality Induced Symptoms and Effects (VRISE): Methodological and Theoretical Issues.
University of Nottingham. Owsley, C. & Mcgwin, G. 2010. Vision and driving. Vision research, 50, 2348-2361. Park, G. D., Allen, R. W., Rosenthal, T. J. & Fiorentino, D. Training effectiveness: How does driving simulator fidelity influence driver performance? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2005. SAGE Publications, 2201-2205. Philip, P., Taillard, J., Klein, E., Sagaspe, P., Charles, A., Davies, W., Guilleminault, C. & Bioulac, B. 2003. Effect
of fatigue on performance measured by a driving simulator in automobile drivers. Journal of psychosomatic research, 55, 197-200. Ranney, T. A. 2011. Psychological Fidelity: Perception of Risk. In: FISHER, D. L., RIZZO, M. & CAIRD, J. K.
(Eds.) Handbook of Driving Simulation for Engineering, Medicine, and Psychology. CRC Press: Taylor and Francis. Rasch, G. 1960. 1980. Probabilistic models for some intelligence and attainment tests. Riva, G., Davide, F. & Ijsselsteijn, W. A. 2003. Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments. IOS Press. Scheuchenpflug, R., Ruspa, C. & Quattrocolo, S. 2003. Presence in virtual driving simulators. In: D. DE WAARD,
K.A. Brookhuis, S.M. Sommer & W.B. Verwey (Eds.) Human Factors in the Age of Virtual Reality. Maastricht, the Netherlands: Shaker Publishing. Slater, M., Usoh, M. & Steed, A. 1994. Depth of presence in virtual environments. Presence-Teleoperators and Virtual Environments, 3, 130-144. Strauss, A. & Corbin, J. 2008. Basics of qualitative research: Techniques and procedures for developing grounded
theory, Sage. Tabachnick, B. & Fidell, L. 1989. Using Multivariate Statistics. Harper Collins. Tuan, PD A comment from the viewpoint of time series analysis. Journal of Psychophysiology, 3, 46-48. Törnros, J. 1998. Driving behaviour in a real and a simulated road tunnel--a validation study. Accident Analysis &
Prevention, 30, 497-503. Wickens, C., Gordon, S. & Liu, Y. 2004. An introduction to human factors engineering, Pearson Prentice Hall. Witmer, B. & Singer, M. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence, 7,
225-240. Young, M. S., Birrell, S. A. & Davidsson, S. Task pre-loading: designing adaptive systems to counteract mental underload. Proceedings of the international conference on Ergonomics & Human Factors 2011, 2011. CRC Press, 168-175
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Aspects of Transportation: Part III
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2099-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional