INTRODUCTION
“In the last 30 years, the number of computerized technologies in vehicles has exponentially increased. Partly this is caused by high speed ICT developments, such as miniaturization, in personalized computer technology; most people will carry at least one computational device with them during the day. For a new generation of drivers cars are an extension of their plugged-in lives, with iPods, DVD players and other gadgets.” (USA Today, 2009).
Over the same period of time, developments in the automotive industry have significantly increased the number of features in vehicles. They increase comfort (climate control), increase fuel efficiency (eco drive) and increase safety (connected car). All these systems, add to the complexity of the driving task itself as a source of distraction, even though a significant number of these systems are designed to support drivers (Advanced Driver Assistance Systems, ADAS). With distraction playing a role in most (near) accidents [1], it is crucial to ensure newly introduced (ADAS) systems do not add complexity and increase distraction.
The common way to ensure new systems do not overload drivers is to test the impact of systems on driver performance, distraction and cognitive workload. Numerous driver observation methods are available and have proven their value in driving simulator testing. However in most studies only a small set of behavioral data is recorded, data are only partially analyzed, and interpretation can be controversial. The most common causes are the very time-consuming procedures needed to combine data from different data acquisition tools and insufficient knowledge of the individual data collection systems.
As research becomes more time-pressured, the significant effort needed to integrate datasets can prevent it from being done at all. When simulators are used for training purposes, the time lag between driving test and review of results should be minimal. This is best solved by real-time data integration and analysis. It also allows adjusting experimental trials based on a driver’s performance to optimize a session.
DRIVELAB COMPONENTS With DriveLab, developed in the Advanced Driver Vehicle Interface in a Complex Environment (ADVICE) project (, we introduce an easy to use, flexible and real-time multimodal driving behavior analysis tool. DriveLab is based on The Observer XT, Noldus’ tool for behavioral analysis for over 20 years (Zimmerman 2009). It integrates a multi-camera remote 3D eye tracker (Smart Eye) and a fixed-base medium fidelity driving simulator (Green Dino). Each system has its own strengths and has proven its value in numerous experiments (Jones at al 2009, Zhang et al 2014, Anand at al 2013, van Leeuwen et al 2013).
To ensure accurate synchronization of all systems the computer clocks, running the systems, are synchronized using the Meinberg NTP client.
Driving simulator The driving simulator consists of a real-world steering column, seat gearbox and foot pedals, complemented by three screens providing a 140 degrees visual field. It comes with a pre-defined set of driving scenarios covering all common driving tasks in various driving conditions. Each scenario can be adapted to fit the experimental requirements. For example the amount of other traffic and the behavior of the traffic can controlled. The simulator is not only used for generating the test environment it is also a source of driver performance data).
Figure 1 Driving simulator with eye tracking in action
SmartEye Pro The 3-camera eye tracker has a big head box and a large tracking area, which allows for natural head motion. The eye tracker support free camera placement for optimal unobtrusive tracking. It uses an open coordinate system, which can be used by other systems.
Noldus Communication Framework The Noldus Communication Framework (NCF) is an implementation of the Advanced Message Queuing Protocol (AMQP) (O’hara, 2007). It is a system that supports real-time scalable and flexible data exchange between the data acquisitions systems (SmartEye Pro and driving simulator) and the data recording systems (The Observer XT). NCF is an open communication framework, designed to easily extend the system with additional measurement systems such as acquisition of psycho-physiological signals or facial expressions.
The Observer XT The Observer XT is used for the collection and analysis of data. It allows an integrated visualization of multimodal data, for example gaze events, driving speed and a video of the driver and the driving environment (fig 2). Its data filtering features enables zooming in on specific situations (e.g. approaching a crossing) and driver characteristics(e.g. age and experience of drivers). The package is completed by a keen set of statistics.
Figure 2 The Observer XT: sample visualization
CONCLUSION
Combined, these systems provide the most commonly used parameters for measuring driver performance, such as headway and lateral position, tracking distraction and inattention, and multiple cognitive workload measures (such as pupil dilation and steering reversal rate).
A unique feature is the matching of gaze point with other traffic, traffic lights and road signs (objects that move around, varying in size in the virtual environment). The fully automated matching results in Regions of Interest (ROIs) with varying size, location and existence, something which usually requires labor-intensive manual annotation. Additionally the distance at which these RoIs are seen is recorded.
REFERENCES
ADVICE project, for more information see: http://www.noldus.com/projects/advice. Anand, S., Terken, J., & Hogema, J. (2013, October). The effect of cognitive load on adaptation to differences in steering wheel force feedback level. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 158-164). ACM. Bouma, H., Baan, J., Borsboom, S., van Zon, K., Luo, X., Loke, B., ... & Dijk, J. (2013, October). WPSS: Watching people
security services. In SPIE Security+ Defence (pp. 89010H-89010H). International Society for Optics and Photonics. Cai, H., & Lin, Y. (2012). An integrated head pose and eye gaze tracking approach to non-intrusive visual attention measurement for wide FOV simulators. Virtual Reality, 16(1), 25-32. Chisholm, S. L., Caird, J. K., Lockhart, J., Fern, L., & Teteris, E. (2007). Driving performance while engaged in MP-3 player interaction: Effects of practice and task difficulty on PRT and eye movements. In Driving Assessment 2007: 4th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design. De Winter, J. C. F., De Groot, S., Mulder, M., Wieringa, P. A., Dankelman, J., & Mulder, J. A. (2009). Relationships between driving simulator performance and driving test results. Ergonomics, 52(2), 137-153. Dingus, T. A., et al., The 100-car Naturalistic Driving Study, Phase II – Results of the 100-Car Field Experiment, 2006, National
Highway Traffic Safety Admin, Report No. DOT HS 810 593 The Observer XT, for more information see: http://www.noldus.com/observer Green Dino, 2008.Dutch simulators[online]. Available from:www.dutchsimulators.nl. [Accessed 28 May 2008]. Grootjen, M., Neerincx, M. A., van Weert, J. C., & Truong, K. P. (2007). Measuring cognitive task load on a naval ship: implications of a real world environment. In Foundations of Augmented Cognition (pp. 147-156). Springer Berlin Heidelberg. Jones, M., Jones, M., Olthoff, T., & Harris, S. (2010, September). The cab technology integration lab: A locomotive simulator for
human factors research. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 54, No. 24, pp. 2110-2114). SAGE Publications. Hansen, K. D., & Andersen, H. H. (2005, September). Measuring the perceptual image quality of a sight effectiveness enhancement system for cars. In Proceedings of the 2005 annual conference on European association of cognitive ergonomics (pp. 159-166). University of Athens. Lasch, A., & Kujala, T. (2012, October). Designing browsing for in-car music player: effects of touch screen scrolling
techniques, items per page and screen orientation on driver distraction. In Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 41-48). ACM. O’Hara, J., “Toward a commodity enterprise middleware,” ACM Queue 5(4), 48-55 (2007). Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013, January). Berkeley mhad: A comprehensive multimodal human action database. In Applications of Computer Vision (WACV), 2013 IEEE Workshop on (pp. 53-60). IEEE. Weevers, I., Kuipers, J., Brugman, A., Zwiers, J., Dijk, E. M. A. G., & Nijholt, A. (2003). The virtual driving instructor: a multi-
based system for driving instruction. van Leeuwen, P. M., Happee, R., & de Winter, J. C. F. (2013). Investigating the Effect of a Visual Search Task for Simulator- Based Driver Training. In 7th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design (No. 65). Zhang, Z., Lu, G., Wang, Y., Tian, D., & Wu, J. (2014). The Effect of Moving Objects on Drivers' Gaze Point at Unsignalized
Intersections with Mixed Traffic. Bridges, 10, 9780784412442-235. Zimmerman, P. H., Bolhuis, J. E., Willemsen, A., Meyer, E. S., & Noldus, L. P. (2009). The Observer XT: A tool for the integration and synchronization of multimodal signals. Behavior research methods, 41(3), 731-735.