INTRODUCTION
The project KOLIBRI was started in April 2011 and lasted until June 2013. The aim of the project was to reduce stops at traffic lights and thus cut down on pollutants such as noise and noxious emissions resulting from avoidable decelerations and accelerations. The project focused on traffic lights outside urban areas (arterial roads). In the project, two test sites were modernized, so the traffic lights could transmit their process data to a central server. Based on this historical data, traffic engineers from TRANSVER GmbH (Munich) were able to estimate the behavior of traffic lights at special times in the future. These estimated traffic light switching times were transmitted to a demonstration car of BMW and smartphones from the Technische Universität München to inform the driver in advance about the state of the next traffic light.
The experiment published in this paper was conducted during an intermediated step in the project on one of the test sites in the north of Munich. The traffic lights were programmed with a fixed coordinated switching scheme (green wave). With the fixed switching scheme, it was possible to program the behavior of the traffic lights into smartphones without the need of the communication infrastructure between the traffic lights and central servers, which were just being implemented at that time. Thus, the smartphones in the test vehicles had enough information about the (exact) time behavior of the traffic lights, the GPS position of stop lines and the velocity and position of the moving car via GPS. From former experiments, we already had a model of the road section for our driving simulator to test early prototypes of the Human Machine Interface. It was also possible to program the green wave into the driving simulator. Thus, we had the opportunity to test the interface on the real road and in the simulator. Therefore, we had the idea to let the KOLIBRI smartphone system be evaluated by the same test subjects (within design) in real and simulated driving.
This idea was originally not part of the project, but due to the possibilities offered by modern smartphones it could be easily implemented even in a short two-year project such as KOLIBRI. This is an example of how fast evolving consumer electronics stand in contrast to long-term developments and the product lifecycles of automobiles or even infrastructure components.
Different institutes have a long tradition in driving simulator (validation) studies, a recommendable review can be found by Blana (1996). A recent overview of this field can be obtained by Mullen et al. (2011) and Ahlström et al. (2012).
The understanding of validities, especially relative and absolute validity, in this paper is that of Mullen et al. (2011). So to cite Mullen et al. (2011): “While absolute validity requires that the two driving environments produce the same numerical values, relative validity is established when the differences between the two environments are in the same direction, and of the same or similar magnitude.”
While former studies mainly found relative validity: “Absolute validity is rarely established in driving simulator studies, […]” Mullen et al. (2011). Ahlström et al. (2012) are one of the rare cases; they achieved absolute validity in real and simulated driving for speed.
Traffic light assistance is also no new topic. There are several projects and papers around dealing with this issue in theory or computer simulations. We were more interested in projects that have a strong link to real experiments. A short overview of different approaches for real traffic light assistance was given in Krause and Bengler (2012). Here we will especially refer to Pauwelussen et al. (2008). They implemented a green wave support system in their driving simulator. As did Wooldridge, Trayford and Doughty, (1984), who tested a traffic light assistant in real traffic. Both projects made thoughts about behavior and human factors engineering.
METHODS
Experimental Design The experiment had a within-design, so every subject drove each condition and every subcondition. Two main blocks, afterwards referred to as conditions, were:
- Driving on a real road (REAL)
- Driving in the simulator (SIM) Each of these conditions was further subdivided into four subconditions:
- Rural road driving, uncoordinated traffic light switching scheme without the KOLIBRI traffic light assistant (NCO)
- Rural road driving, coordinated traffic light switching scheme (green wave) without the KOLIBRI traffic light assistant (GW)
- Rural road driving, coordinated traffic light switching scheme (green wave) with the KOLIBRI traffic light assistant on a smartphone (KO)
- Suburban driving (URB) The subconditions are further described in the following sections.
Procedure The experiment lasted about two hours per test subject. Due to time restrictions, only two persons drove first in the simulator and then on the road. The rest followed the sequence: first real driving, than the simulator. The order of the subconditions was randomized. The direction of driving (north-south or south-north) was also randomized, but controlled between simulator and real driving. The real tracks are about ten minutes from the institute, the test subject drove this distance to accommodate to the real car. For accommodation to the driving simulator, the subjects drove 5 minutes. The experiment was carried out in June/July 2012. The KOLIBRI Human Machine Interface was verbally described to the subjects, while standing still.
Tracks In the conditions NCO, GW and KO, the same road section was used. The road section is a 6.5 km part of the federal road B13 in the north of Munich. On this section, seven traffic-light-controlled intersections were passed. The subject drove straight forward from 48°17'56 N 11°34'36 E to 48°14'48 N 11°36'7E. The road has two lanes per direction separated by guide rails (see Figure 1). The subjects randomly drove the conditions in direction north-south or south-north. The volume of traffic under real circumstances is below 500 vehicles/hour/direction during the day at the time of the experiments. In the simulator, some cars in front and behind were added, that never interacted with the driver (locked in place at a certain distance). The speed limit on these road sections is most of the time 100 km/h and reduced to 70km/h in traffic light areas. The street was modeled for the simulator (see Figure 1). The focus was not on every single visual detail; instead, that stop lines and speed limits were at the right place, in line with GPS positions. The traffic light scheme (green wave) especially designed for the KOLIBRI project was only active at noontime. During the time before and after this period, the traffic lights were uncoordinated and operating on a traffic response scheme by car detectors. For the simulator, the green wave scheme was programmed into the simulated traffic lights. For the uncoordinated scenario in the simulator, an uncoordinated backup scheme of the real traffic lights was set up. On the actual road, every single traffic light is equipped with this backup scheme, which is activated when car detectors are not functioning. As can be seen in Figure 1, the road section has a highway-like appearance.
Figure 1 B13 track Top: Real road (slightly different view angle, camera at radiator grill) Bottom: Simulator track For suburban driving on a real road, a suburban district near the B13 was used (see Figure 2 and 3). The route through this area was approx. 2 km long, mainly with speed limit of 30 km/h. The examiner specified the route and announced turns.
Figure 2 Real track through a suburban area
Figure 3 Real track suburban driving environment For the simulator, an innercity scenario was used (see Figure 4) that was originally implemented for another project (Helmbrecht, 2012). The scenario was not a model of the real suburban district. The distance to travel was 2.3 km, restricted to 50km/h or 30km/h. The track through this city (turns) was communicated to the driver via an automated navigation voice.
Figure 4 Simulator track suburban driving (Helmbrecht, 2012)
Road Vehicle For the experiment on the road, a BMW 520d (year of manufacture 2011, type F10) diesel-engined and with automatic gear was used. The velocity was tracked with a GPS Trip Recorder 860E at 1 Hz. Fuel consumption was manually read from the board computer and reset following each condition. According to personal communication with BMW technicians, the measurement of fuel consumption in the board computer had not been manipulated. The test subject drove the car without the Head Up Display (HUD).
Driving Simulator The static driving simulator of the Institute of Ergonomics consists of three projectors (1400x1050 resolution), which offers an almost 180-degree front view on three 3.4 m x 2.6 m screens. Three other projectors displayed images to screens behind the BMW mockup, visible to the driver via the car mirrors. For driving simulation, the software SILAB V3.0 from WIVW GmbH, Würzburg was used. CarSim V7.11 from Mechanical Simulation, Ann Arbor, was used for the car model to calculate fuel consumption and dynamics. The software was loaded with a 150kW sedan model. An active steering wheel with software from Simotion, Munich provided the driver an adequate feeling for steering. The simulator logged all available data with 120 Hz.
KOLIBRI Traffic Light Assistant – Human Machine Interface (HMI) The traffic lights were operating in the subcondition GW and KO on a coordinated green wave, so the switching times were predefined and could be programmed into a time-synced smartphone. The KOLIBRI traffic light assistant system on a smartphone gave the driver speed recommendations (within the allowed speed limit) to make it to the next traffic light while still green (Figure 5, left) or it would advise that it would not be possible to make the next traffic light while still green (Figure 5, middle). If the driver drove more than 10 km/h too fast, a speed alert was shown (Figure 5, right).
Figure 5 KOLIBRI MMI Left: Speed recommendation. Green area indicates speeds to reach the next traffic light at green. Mid: It is not possible to make the next traffic light at green (Prepare to stop). Right: The vehicle is more than 10 km/h faster than allowed (Speed Alert).
The smartphone was mounted right beside the steering wheel next to the air vents (see Figure 6).
Figure 6 Mounting of the smart phone in the simulator (left) and the vehicle for the field test (right)
NASA-RTLX In the experiment, the well-known task load questionnaire NASA-TLX (Hart and Staveland, 1988) was used in an unweighted respectively equally-weighted version, sometimes called NASA-RTLX (R = raw). The German translation stems from Seifert (2002). The questionnaire was filled in after each experimental subcondition.
Participants Twenty subjects participated in the experiment (19 male, 1 female). No participant quit as a result of simulator sickness. The subjects were between 25 and 35 years of age, the average age was 28 years of age. All subjects had a valid driver's license and drove on average 12,000 kilometers per year (SD: 7,300 km). 90% had driven a simulator before and 80% had specific previous knowledge of the static driving simulator at the Institute of Ergonomics. All participants had experience with an automatic gear. Eight persons were familiar with the KOLIBRI traffic light assistant. Four of these persons had used the assistance system before in other simulator experiments. The other four participated in an experiment conducted in parallel, in which the assistant was also used on the real road.
Statistics Unless otherwise reported, the alpha level is set 0.05.
The critical value for an r-coefficient to become significant on level of p=0.05; df=18 is 0.444. In tables, this value is indicated in light gray.
The level of p=0.01;df=18 results in a critical value for the correlation of 0.561, often connected to the term “highly significant” is shown in tables as dark gray.
The default reported error bar is the 95% Confidence Interval (CI).
RESULTS
Driving Behavior – Speeding
Figure 7 and Table 1 contain information about the metric, i.e. percentage of travelled distance with more than 5 km/h above speed limit. In former studies (Krause and Bengler, 2012; Krause et al. 2014a), we used the percentage of time that the subjects were traveling above the speed limit, in relation to the time when the car is moving (>5 km/h). We assumed that expressing speeding in percentage of travelled distance and with a little tolerance of 5 km/h above speed limit would be slightly more accurate and easier to imagine,.
As can be seen in Figure 1, the subjects drive in all scenarios faster in the simulator than on a real track. This difference is the lowest for the suburban condition (URB). While for the highway conditions (NCO, GW), the quickly travelled, distance is doubled in the simulator compared to real driving.
The comparison of GW with the KOLIBRI driver information system (KO) reveals that in simulated driving, KOLIBRI can reduce the fast travelled percentage of distance from 40% to 22.7%. On the real track, this effect is still present, but due to the lower speeding behavior, KOLIBRI reduced speeding from 21.4% to 14.9%. A one sided t-test for repeated measurement between REAL_GW and REAL_KO would be significant p=0.02 (df=19; t =2,19).
These findings are in line with former KOLIBRI simulator studies (Krause and Bengler, 2012; Krause et al. 2014). These former papers included time-based speeding percentages without a 5 km/h limit tolerance. We reassessed the data from these experiments with the metric used in this paper and found for the data of Krause and Bengler (2012) a reduction from 48.9% speeding distance to 20.8% with the KOLIBRI assistance. And from 42.5% to 22.3% for Krause et al. (2014a). Thus, the effect is stable and reproducible. It is also worth noting that the first KOLIBRI implementations for the simulator experiment (Krause and Bengler, 2012) did not included a speed alert. So this indicates that the main speed reduction of the KOLIBRI system is not an enforcement by negative feedback (warning) of the speed alert, else a positive compliance of the test subjects to the speed recommendations of the KOLIBRI system. The speed recommendations of the KOLIBRI system are always within the allowed speed limit. From the similarity with former experiments with simulator driving only, we also conclude that the sequence in this experiment (simulator driving mainly after real driving) seemed not to change the speeding behavior.
Table 1 contains the correlation coefficients for the experimental conditions. The highest correlation of all conditions is between the use of the KOLIBRI system in real and simulated driving (0.805). Therefore, the use of the KOLIBRI system in the simulator is a good predictor for the individual speeding behavior with the KOLIBRI system on the road (and vice versa). On the other hand, simulated driving with the driver information system KOLIBRI seems to change the behavior of some subjects, which is a good indication. Thus, this condition has a weak correlation to other simulated scenarios. The simulated scenarios have high inter-correlation, except for the before-mentioned KOLIBRI system.
Figure 7 Average percentage (N=20) of travelled distance with more than 5 km/h above speed limit Another interesting finding is that driving in a real suburban area seems a good predictor of speeding for all assessed conditions in the simulator (and vice versa). Real driving with the KOLIBRI system shows similar values. For the correlations between the real driving conditions, there is only one significant finding: Speeding in suburban areas seems correlated with speeding while using the KOLIBRI system.
The coefficients for SIM_NCO x REAL_NCO, SIM_GW x REAL_GW, SIM_KO x REAL_KO and SIM_URB x REAL_URB have a special meaning. These directly connect real and simulated driving. The values all show a positive correlation, but only two are highly significant. Therefore, it is interpreted, that the simulator setup is well suited to test speeding behavior in slow suburban driving and the influence of driver assistance systems like KOLIBRI on a highway. On the other hand, the simulator setup seems to have reduced capability to get information for uninfluenced speeding behavior on a highway.
Table 1 Correlation of percentual distance travelled with more than 5 km/h above speed limit REAL_NC
The used speeding metric revealed values between min. 0.04% and max. 45.03% for driving on the real road and min. 2.07% and max. 68.17% in the simulator. Therefore, we obtained values above 0% even for the most defensive driver under each condition.
Driving Behavior – Accelerations The accelerations from the GPS tracks and driving simulator data are calculated in Figure 8. The calculation used the differential quotient of the GPS velocity (1Hz). The driving simulator data was downsampled to 1Hz and handled the same way. In the figure, all subconditions (regardless of highway or sub-urban) are combined into one histogram. For positive accelerations, the curves show good congruity. For decelerations, the real driving shows a small side peak at around -0.3m/s². This is a typical deceleration for coasting. The side peak around -0.6m/s² for the simulated driving has the same reason (coasting) and the simulated car model would also have a typical coasting value around -0.3m/s², but due to a mechanical or electrical problem with the brake, the simulator applied an additional deceleration of around -0.3m/s². We assessed the data of the former experiments; these did not include the brake problem.
Figure 8 Accelerations from real and simulated driving
Driving Behavior – Fuel Consumption The individual fuel consumption for the subcondition NCO was set to 100%. Thus, a driver with a fuel consumption of 7 l/100 km in NCO and 5.6 l/100 km in GW would have a consumption of 80%. These individual values are averaged for Figure 9. In the simulator, the coordination of the traffic lights could lower fuel consumption by 12% and the use of the KOLIBRI system on the coordinated route by another six percentage points. This reduction of about 18% is in line with findings in Krause and Bengler (2012), where the use of the KOLIBRI system on an uncoordinated route in the simulator reduced fuel consumption by 20%. The fuel reductions for the coordinated scheme in real traffic is visible in both conditions (GW and KO) with around 7%. A repeated ANOVA for the absolute fuel values from SIM with Bonferroni-corrected comparison revealed significance between all subconditions. The same analysis for REAL showed no significance between GW x KO and even NCO x KO (p = 0.055).
The findings from the simulator, i.e. that the KOLIBRI system can lower fuel consumption even on a coordinated green wave road could not be found in real traffic. Table 2 contains the averages and deviations of the absolute fuel values for all subconditions.
Figure 9 Fuel consumption in relation to the uncoordinated subcondition (NCO)
Table 2 Mean and standard deviation of absolute fuel values (liter/100 km) for the subconditions SIM_UR B SIM_NC O
Subjective Ratings The NASA-RTLX values (score between 0 to 100) are reported in Figure 10. The values in simulated driving are always higher than the scores from real driving. This is in line with most of the findings reported in Blana (1996, p.50). Figure 11 shows at the example of subcondition GW, the NASA-RTLX sub-dimensions. These reveal that most of the difference stems from the subscales “Performance” and “Frustration”. Thus, it seems not very satisfying for subjects to drive in a simulator, compared to real road driving. At first glance, the scores for real road driving KOLIBRI nearly double from around 14 (GW) to 26 (KO). In another experiment, we gathered objective data with a Detection Response Task (Knott, 2013), which showed that KOLIBRI did not induce measurable additional cognitive load. Thus, our interpretation of the NASA-RTLX workload scores is slightly different, as might be expected: Driving on a straight two-lane highway in off-peak hours seems to be encountered as boring, with NASA- RTLX values around 14. These under load values are shifted with KOLIBRI to a value of 26 that is similar to driving in a quiet suburban area. Rather than making the boring situation more interesting by e.g., driving faster, the driver probably directs his available attention at the longitudinal control of the vehicle, motivated by KOLIBRI. This simple “gamification” presumably results in a reduced speeding behavior
Figure 10 NASA-RTLX scores for all subconditions
Figure 11 NASA-RTLX subdimension at the example of subcondition GW DISCUSSION & CONCLUSIONS
It is known that a possible countermeasure of test subjects to compensate for an increased workload by an information system, is to reduce the driving speed. So one might argue that the test subjects with KOLIBRI might tend to drive more within the speed limit (see Figure 7), due to an increased workload by the KOLIBRI system. The subjective ratings (see Figure 10) of an increased workload would also support this hypothesis. We conducted an experiment in real traffic to obtain objective data for the workload with the KOLIBRI assistant with a variant of a Detection Response Task: The Tactile Detection Task (TDT). The objective values from this TDT found no increased mental demand for the test subjects (Knott et al., 2013). Thus, it is likely that the reduced speeding can be attributed to the intended purpose of the KOLIBRI system: Guide the driver to the next traffic light and indicate at an early stage whether the traffic light will be red or green.
It is worth noting that traffic engineers designed the green wave coordination by analyzing the driving behavior of uninformed drivers of light vehicles on the road section. Thus, a kind of human factors engineering. The results show that an informed driver with the KOLIBRI system in real traffic does not seem to have an advantage in terms of fuel consumption (see Figure 9) for a modern fuel-efficient car. However, Figure 7 shows that the informed driver can achieve this result with a more relaxed manner of driving, with less speeding behavior. Nevertheless, we assume that diver information on upcoming traffic light conditions are especially useful for uncoordinated traffic lights.
The authors are surprised by the outcome of the acceleration histogram (Figure 8). The simulator changed its characteristics slightly; maybe due to repair activities, aging of components, or even other reasons. A short checklist provided to the examiners by the simulator specialists would be helpful as a quality-management benchmark. This would enable them to quickly perform a rough check of key characteristics of a simulator. Examples: If car model X is active, the car accelerates from a standstill to 100 km/h in typically X seconds. When the car drives100 km/h and the accelerator pedal is released, the car decelerates to 90 km/h in about X seconds. If the steering wheel is suddenly turned by 90° the car needs X seconds to drive a 360° turn.
Maybe one can even think about a more sophisticated technical procedure. So the simulator is checked periodically for altered characteristics in visual, auditory or anthropometric domain.
Driving in real traffic revealed a lower speeding behavior. We consider regulatory issues to be partially responsible for this: In Europe (75/443/EEC) car speedometers are not allowed to show speeds lower than the real vehicle speed. To account for this, car manufacturers have a tolerance level to show a lower speed than the actual one. Therefore, if the speedometer indicates a speed of 100 km/h it may only be 95 km/h, or even 88 km/h. This “detuning” is not implemented in any of our three static driving simulators. It can be seen as a lack of physical or face validity. Ahlström et al. (2012) accounted for the altered speedometer behavior and used an advanced vibration system for the road surface in the simulator and thus achieved even absolute validity for speed in real and simulated driving.
OUTLOOK During the KOLIBRI project, we conducted three additional experiments on the real road for human factors engineering, beside the one reported in this paper. One assessed the mental demand by objective measures (Knott et al., 2013), another the gaze behavior in real traffic (Krause et al., 2014b) and the last, how subjects react to wrong information by the traffic lights assistance system. The data of the last experiment are currently under examination and are yet to be published.
One drawback of the experiment reported here was that the display content (speed recommendation) of the smartphone in real driving situations was not logged. So we could not assign behavior (e.g. decelerating, accelerating, speed level) to the display content. Logging was performed in the last real road experiment, which we are currently analyzing. With the data of the last experiment a between subject comparison of driving behavior in real and simulated driving of former experiments regarding the specific display content of the traffic light assistant will be possible.
Most partners of the KOLIBRI project are also trying to organize a follow-up project, to lead the traffic lights assistance over into regular operations, with an operator model.
In the meantime, the institute has decided to use an electric vehicle as staff car. Therefore, it would be tempting, to validate the simulator in the special field of electric driving.
ACKNOWLEDGMENTS
The project was funded by Bayerische Forschungsstiftung (Bavarian Research Foundation). The authors would like to thank the project partners TRANSVER, BMW Group, Board of Building and Public Works in the Bavarian Ministry of the Interior, especially the Bavarian Road Administration, Center for Traffic Management.
REFERENCES
Ahlström, C., Bolling, A., Sörensen, G., Eriksson, O., & Andersson, A. (2012). Validating speed and road surface realism in VTI driving simulator III (VTI rapport No. 745A). Blana, E. (1996). Driving simulator validation studies: A literature review. ITS Working Paper 480 Council Directive of 26 June 1975 on the approximation of the laws of the Member States relating to the reverse and
speedometer equipment of motor vehicles, European Commission. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload. Amsterdam: North Holland Press; Sole distributors for the U.S.A. and Canada, Elsevier Science Pub. Co. Helmbrecht, M. (2012) Ein vorausschauendes Fahrerassistenzsystem in komplexen Verkehrssituationen. Diploma Thesis at the
Institute of Ergonomics, Technische Universität München Knott, V., Krause, M., & Bengler, K. Einsatz des KOLIBRI-Ampelassistenten im realen Straßenverkehr – Eine kognitive Belastung für den Fahrer? pp.437-440. GfA Press, Dortmund. Krause, M., & Bengler, K. (2012). Traffic Light Assistant - Driven in a Simulator. In R. Toledo-Moreo, L. M. Bergasa, & M. Á.
Sotelo (Eds.), Proceedings of the 2012 International IEEE Intelligent Vehicles Symposium Workshops. IEEE Intelligent Vehicle Symposioum (IV-2012), Madrid, June 3-7, 2012. Krause, M., Rissel, A., & Bengler, K. (2014a). Traffic Light Assistant - What the User Want: AHCI Conference March 2014, Barcelona. IARIA XPS Press. (in press). Krause, M., Knott, V., & Bengler, K (2014b). Traffic Light Assistant – Can Take My Eyes Off of You. In HFES Europe Chapter
Annual Meeting 2014, Turin . (in review). Mullen, N., Charlton, J., Devlin, A., & Bedard, M. (2011). Simulator Validity: Behaviors Observed on the Simulator and on the Road, pp 13-1 – 13-18. Pauwelussen, J., Hoedemaeker, M., & Duivenvoorden, K. (2008). CVIS Cooperative Vehicle-Infrastructure Systems: Deliverable
D.DEPN.4.1b Assess user acceptance by small scale driving simulator research. Seifert, K. (2002). Evaluation multimodaler Computersysteme in frühen Entwicklungsphasen. Ein empirischer Ansatz zur Ableitung von Gestaltungshinweisen für multimodale Computer-Systeme (Dissertation). Technische Universität Berlin, Berlin. Retrieved from http://opus.kobv.de/tuberlin/volltexte/2002/432/pdf/seifert_katharina.pdf Wooldridge, M. J., Trayford, R. S., & Doughty, B. J. (1984). Urban energy saving through dynamic advisory speeds: The
progress of a research project / M.J. Wooldridge, R.S. Trayford, B.J. Doughty: [S.l.] : CSIRO. Division of Energy Technology.