Marcel Baltzer, Eugen Altendorf, Sonja Meier, Frank Flemisch (2021). Mediating the Interaction between Human and Automation during the Arbitration Processes in Cooperative Guidance and Control of Highly Automated Vehicles: Base concept and First Study. In: Advances in Human Aspects of Transportation: Part I. AHFE Open Access, vol. 15. https://doi.org/10.54941/ahfe100647
Publisher statistics will load when you open Metrics.
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER
Mediating the Interaction between Human and Automation during the Arbitration Processes in Cooperative Guidance and Control of Highly Automated Vehicles: Base concept and First Study
With increasingly automated vehicles, the cooperation between the automation and the human becomes crucial. In cooperative guidance and control (CGC) of highly automated vehicles the questions arise how authority, responsibility and control are distributed between driver and automation, how this distribution is obtained and how different states like abilities, availabilities and modes are communicated between driver and automation. If two independently thinking entities start planning and acting together, the emergence of conflicts is inevitable. These conflicts can be solved by applying the method of arbitration. Arbitration works with different modalities as well as on different planning levels. To handle the complex interdependencies in human-automation interaction in the guidance and control of highly automated vehicles, an “interaction mediator” was designed to incorporate a framework of modules, which in turn are designed to be easily extendable. These modules are the “Mode Selection and Arbitration Unit”, enabling a proper distribution of responsibility depending on present abilities and availabilities, the “Manoeuvre Selection and Arbitration Unit”, the “Trajectory Adaption and Arbitration Unit”, and the “Coupling Valve” where control commands depending on the distribution of responsibility and control are coupled to one common control command for the entire ego-vehicle.
With increasingly assisted and automated vehicles, that incorporate more and more abilities to support the driver in the driving task, the cooperation between the automation and the human becomes crucial. In cooperative guidance and control (CGC) of highly automated vehicles (see Flemisch et al., 2014), where a cooperative automation and a human simultaneously perform the driving task, several new questions arise: How are authority, responsibility and control distributed between driver and automation (see Flemisch et al., 2011)? How is this distribution obtained and how are different states, like abilities, availabilities, the active manoeuvre, the current distribution of responsibility and control (mode) etc., communicated between driver and automation? H-Mode, an instance of cooperative guidance and control based on the H(orse)-Metaphor (Flemisch et al., 2003; Kienle et al., 2009; Bengler & Flemisch, 2011), incorporates both a cooperative automation design and a multi-modal interaction design (Flemisch et al., 2014). To create an effective and compatible interaction between driver and automation, the automation was implemented working on levels according to the actions and the plans of a human driver: navigation, guidance (with the sublevels manoeuvre planning and trajectory planning) and control (Löper, Kelsch & Flemisch, 2008). Since the automation and the human interact with each other haptically, visually and acoustically on different planning levels (navigation, manoeuvre, trajectory and control level) an effective management and mediation of interaction between the cooperative partners was developed.
COOPERATIVE GUIDANCE AND CONTROL (CGC) Cooperation in the context of highly automated driving
Cooperation in terms of cooperative guidance and control of highly automated vehicles is described as the “action or process of working together of at least one human and at least one computer on the guidance and control of at least one vehicle” (Flemisch et al., 2014). The use of the term “cooperation” for human-machine systems was suggested by Rasmussen (1983), Hollnagel and Woods (1983), or Sheridan (2002) and adapted for vehicle control by Flemisch et al. (2003). Cooperation in human-machine systems implies that both, human and automation, can be in the physical control loop simultaneously, e.g. in form of “shared control” (Griffith & Gillespie, 2004; Mulder, Abbink, & Boer, 2012) or “cooperative control” (Biester, 2008; Hakuli et al., 2012; Flemisch et al., 2014). When a human and a computer work together at the same time, on the one hand the human can also delegate unwanted control tasks to a partner while still being in the control loop (Rasmussen, 1983), hence improving comfort, and on the other hand the risk of failures or inefficient behaviour (both technical and human) is reduced due to the resulting redundancies in perception and action, hence improving safety and efficiency (Bengler et al., 2012).
Cooperation should be seen as a cluster concept that incorporates certain qualities or phenomena to improve the cooperativeness of control. A cooperative control design includes the following aspects (Flemisch et al., 2014): (a) An automation that has sufficient autonomous abilities, (b) Outer compatibility (compatible interfaces between driver and automation), (c) Inner compatibility (compatible cognition, i.e. planning and action processes), (d) Predictability of abilities and intentions, (e) Dynamic distribution of control, (f) Arbitration of conflicts and (g) Adaptability and adaption as a dynamic balance of flexibility and stability.
To create the ideal mediation of interaction between driver and automation, the above qualities should be considered in the interaction design to improve cooperativeness.
Assistance and Automation in the context of CGC Today, even state of the art assistance systems in cars provide various functions of automation or other ways of supporting the driver. Warning systems, like a Lane Departure Assistant or a Night Vision System, indicate upcoming dangers by perceiving the vehicles’ environment and informing the driver of such through different types of interfaces. Moreover, parts of the driving task can already be automated, as well in the lateral as in the longitudinal driving direction. Since several years most manufactures offer Adaptive Cruise Control systems, which will automatically control the driven speed and the distance to a car driving in front of the own vehicle. Active Lane Keeping Assistance systems control the lateral offset to the current driving lane by tracking the road surface markings through optical systems. A simple combination of these two systems would basically result in a partial automation of the driving task, at least on highways such as the German Autobahn. While most auf those assistance systems are still being activated discretely by the driver, the abovementioned combination leads to new levels of complexity in the field of human machine interaction in the case of (partially) automated driving. In cooperative guidance and control the vehicle’s automation can provide the ability of driving almost autonomously. But the actual degree to which this automation ability is used, i.e. the degree of automation, which can be seen as a state, can vary. Figure 1 shows a one-dimensional scale of automation levels, which ranges from manually driven to fully automatically operated. The term assistance, as used for today’s commercial state of the art systems can be
integrated in the midst area of such an assistance and automation scale.
H-Mode – an implementation of cooperative guidance and control The concept of H-Mode is based on the H(orse)-Metaphor (Flemisch et al., 2003). Similar to the desktop metaphor as design metaphor for computer operating systems, the H-Metaphor is a design metaphor for the cooperation of a human and an automation, comparable to the cooperation of a rider and a horse. H-Mode is an instantiation of cooperative guidance and control of vehicles that can be driven in different degrees of automation. The degree of automation represents the distribution of control between driver and automation that again is represented by different modes. In the current H-Mode 2d 1.1 prototype there are: Tight Rein (assisted/ lowly automated), Loose Rein (partially/ highly automated with the driver in the control loop) and Secured Rein (highly automated with the driver temporarily out of the control loop: temporarily fully automated), see Figure 1. These names are derived from the H-Metaphor representing the control distribution between a rider and horse: In Tight Rein the rider has a firm grip at the reins and controls the horse very directly while being assisted by the horse to stay on course; in Loose Rein the horse receives a high degree of freedom how to continue the ride while the rider still has haptic contact and can intervene very quickly; in Secured Rein the reins are laid down (or secured at the saddle) temporarily leaving all responsibility and control of the ride to the horse with the rider being able to focus on something else. This metaphorical representation of modes can be transferred easily in the cooperative guidance and control domain where the H(orse)-automation has the necessary abilities to receive the corresponding responsibility for its assigned control task and haptic multi-modal coupling (Figure 1, bottom) of driver, automation, vehicle and environment is emphasised.
Human initiates transition
Secured Rein Tight Rein Loose Rein partially/highly highly automated/ assisted/lowly Fully temporarily fully automated Manual automated (driver in the loop) automated
automated
Automation initiates transition
Perception
Perception
Perception
Intention
Intention
Intention
Interaction
Interaction
Action
Action
Action
Interaction
Arbitration
Arbitration
Arbitration
Intention
Intention
Intention
Perception
Perception
Perception
Figure 1. Top: Assistance and automation Scale (after Flemisch et al., 2008) Bottom (left to right): Tight Rein, driver has a strong coupling with the vehicle in its environment; Loose Rein, driver has a weaker coupling with the vehicle in its environment; Secured Rein, driver is temporarily decoupled. (Flemisch et al., 2014) DESIGN SPACE OF COOPERATIVE MOVEMENT
Figure available in the original PDF paper.
As mentioned above, the compatibility between human and automation needs to be ensured for both agents to move cooperatively. Compatibility describes the degree of matching between automation and human and leads to a mutual understanding and interaction (Flemisch et al., 2008). It can be differentiated between “outer” and “inner” compatibility, where the outer compatibility can be achieved through correct interfaces between human, automation and environment, while inner compatibility can be achieved by designing the automation that it matches the human’s cognitive planning and action processes (Bubb, 1993; Flemisch et al., 2008).
Haptic multi-modal interfaces In the current prototype (H-Mode 2d 1.1), there is haptic, visual and acoustic interaction. The haptic interaction mainly consists of tactile interaction via active inceptors, e.g. a haptically active gas pedal (Mulder 2007), a combination of a haptically active gas pedal and a haptically active steering wheel (Flemisch et al., 2012), or a haptically active side-stick (Flemisch et al., 2012; Flemisch et al., 2010). To create a haptical Input/Output device, force sensitive resistors (FSR) were attached to measure how firm the driver grasps the haptical device (Krapf, 2009, Bengler & Flemisch, 2011). In addition capacitive sensors were added to reliably detect if the driver touches the haptic device (hands on/off).
In H-Mode the haptic interaction resource is used as the principal interaction resource and enriched by other modalities like visual or haptic modality, and therefore the term “haptic multi-modal interaction” applies (Flemisch et al., 2010). This focus on haptic interaction is motivated from the fast reaction time, since the direction and therefore specific meaning of the conveyed information is included (e.g. Suzukia & Jansson, 2003; Brandt, Sattel & Böhm, 2007) and the possibility to give continuous feedback improving situation awareness (e.g. Flemisch et al., 2003; Abbink, Boer & Mulder, 2008).
Visual interaction is enabled via a contact-analogue Head Up Display (kHUD), invented by Bubb (1975), where objects on the display merge with objects in the environment (Damböck et al., 2012), e.g. showing the possible driving trajectories as represented in Figure 2, top. Furthermore a Touchscreen is implemented as Head-down Display (HDD) in front of the driver (see Figure 2, left), which combines haptic and visual interaction in one device. Objects on the HDD are represented as a combination of color-codes, symbols and text. Finally for the acoustic interaction simple “beep” sounds in variable volumes and frequencies are employed.
Figure 2. Top: Situation representation in contact-analogue HUD. Left: Situation representation on touch sensitive HDD. Right:
Prototype Setup with HDD and contact-analogue HUD.
Cognition-based automation design Inner compatibility including cognitive compatibility in terms of cooperative guidance and control of highly automated vehicles should include a compatible goal and value system, as well as a compatible representation of movement through time and space (Flemisch et al., 2014). A possibility to achieve these prerequisites is to implement an automation that plans and acts compatible to the driver. A simplified model representation divides the human planning process in four different levels: navigation level, manoeuvre level, trajectory level and control level (Löper et al., 2008 extending Donges, 1982 and Rasmussen, 1983).
On the navigation level the route to arrive at a certain target is planned, representing (long term) planning. The route is the base input for the manoeuvre level that in turn represents the (mid-term) planning how to manoeuvre through traffic. Manoeuvres subdivide the driving task in interconnected spatial and temporal processes (Löper et al., 2008). H-Mode uses basic manoeuvres that have a semantic representation like “turn left”, “follow lane”, “follow vehicle” etc. These manoeuvres serve as input for the trajectory level that represents the short term planning of the driving task. From these generated trajectories control commands can be determined in the control level.
SYSTEM DESIGN OF THE INTERACTION MEDIATION If two independently thinking entities start planning and acting together, the emergence of conflicts is inevitable. On the one hand, conflicts can be helpful to consider aspects of actions that have been previously overlooked. On the other hand, conflicts can be obstructive or even result in deadlocks, e.g. when an automation wants to turn right and the driver wants to turn left: Taking the middle could be bad for both and fatal for the combined human-machine- system. To solve these conflicts, the method of arbitration is applied. Human – machine arbitration includes a structured negotiation between the human and the automation with the intention to reach a common unambiguous decision on how to act in due course of time (Kelsch et al. 2006). ). In order to ensure outer compatibility between human and automation arbitration must be enabled through different modalities (haptic, acoustic and visual). Furthermore to improve inner compatibility between human and automation arbitration takes place on the different planning levels (manoeuvre level, trajectory level and control level) as well.
The negotiation of conflicts is necessary to balance opposing intentions that can be represented by tension poles. Tension poles represent poles of opposing intentions in tension fields (Flemisch, 2000). Depending on the polarity of the tension pole, an action results in a force that will pull the entity’s action towards or push the entity’s action away from this pole. Kelsch et al. (2012) generalised the concept of tension poles to the concept of “action tension”. “Action tension” is defined within a human machine system as “a directed motivation (tension) toward a particular action” (Kelsch et al., 2012).
The idea of tension and poles of action can be used to design escalation schemes that can be incorporated in interaction patterns. An escalation has the task to inform the human about the current state and the direction of change. This can be increasing criticality of the current situation e.g. when approaching another car, as well as informing about the current state of transitioning more control and responsibility to the automation. A balanced escalation design incorporates the tasks to inform, warn and redistribute control under the consideration of performance, safety and acceptance: An escalation that warns too early might result in a cry-wolf effect (Breznitz, 1983; Wickens et al., 2009), whereas an escalation that acts too late might not be able to prevent an accident.
The interaction mediator has the task to supply the interaction necessary for the different arbitration processes between human and automation. The interaction depends on multiple aspects like the current situation and the planning level. Therefore the interaction mediator incorporates a framework of modules, which in turn are designed to be easily extendable. In the following these modules will be elaborated in detail.
Mode Selection and Arbitration Unit The Mode Selection and Arbitration Unit (MSAU), an extension of the MSU designed in HAVEit (Hoeger et al. 2011), facilitates the arbitration of the distribution of responsibility and control between human and automation, depending on their respective abilities. In the current H-Mode prototype the human has the main authority to initiate the transition from one mode to another. The automation only initiates transitions to lower degrees of automation if its abilities do not match the current situation’s demands. The different modes are visualised in the HDD as a combination of two symbols, the human and the cognitive H-automation, with different colours and the name of the current mode (Figure 2, left). Because it is of major importance to prevent a control deficit or control excess, the mode selection and arbitration unit was implemented as a state machine. As depicted Figure 3 every mode represents a state of control distribution. A transition from one mode to another can be initiated on multiple ways and will be explained in the following.
power off power on
Ai [failure corrected]
Failure
Failure Ai [failure]
MRM / Emergency Normal driving
Fluid active, Di [hands off] Fluid active, Di [loose grip] Di [switch SR]
Activation impossible
Activation impossible Di [any switch] Activation impossible
Di [any switch]
Di [any switch]
Secured
Fluid active, Di[tight grip]
Ai [emergency]
Tight Rein Loose Rein
Fluid active Di[loose grip] Rein
Ai [LR preconditions not longer fullfilled] Di[hands on]
[driver attentive]
[driver attentive]
Emergency
Brake
Di [switch TR]
|| [driver drowsy]
Di[switch LR] Ai[SR preconditions not longer fullfilled] Ai [driver distracted] || [driver drowsy]
Ai [standstill?]
Ai [No response by driver] Di [tight grip]
Ai [emergency]
MRM
Figure 3. State machine of the Mode Selection and Arbitration Unit in H-Mode 1.1 (extension of Hoeger et al. 2011)
Figure available in the original PDF paper.
The driver can change between different automation modes. First, the human can press a button to change the automation mode, or more precisely, he/she has the possibility to select his/her request to initiate a transition from the current automation mode to another by pressing the respective button, e.g. Tight Rein (see Figure 2, left). Secondly, in emergency situations the driver can over-steer, over-brake or over-accelerate which results in the initiation of emergency transitions transferring a high degree of control back to the driver. Finally the driver can take a faster or looser grip to change between the automation modes Tight Rein, Loose Rein and Secured Rein. To enable the change between different automation modes depending on how tight the driver grasps the haptic device fluid transition must be enabled, e.g. at the start of the drive, by pressing the respective button (see Figure 2, left). This arbitration of control is quite intuitive for most humans (see evaluation) since humans intuitively use larger forces to have a larger impact with the current action. If Secured Rein (temporarily fully automated) is available, which could be on an especially certified road e.g. with no unexpected road works, a so called “secure lane” similar to the eLane concept proposed in CityMobil (Toffetti et al., 2009), the driver can initiate a transition to Secured Rein by simply taking his/her or her hands off the haptic device. By grasping the haptic device loosely (hands on) the driver initiates a transition to Loose Rein. When grasping the haptic device tightly the driver initiates a transition to Tight Rein. The current prototype has the necessary grasping force thresholds implemented as a hysteresis, in order to simplify the control task in terms of accuracy and reduce physical stress while driving with a tighter grip.
If the driver wants to receive more control, which is represented by a transition to a lower degree of automation, he better does this only if he is able to perform the control task in the requested mode. This includes mode awareness and thereby awareness of his/her responsibility to control. Furthermore a physical contact (“interlocked transition”) with the haptic device is necessary and hence the ability to control. If the automation detects insufficient driver activity (e.g. mode is Secured Rein and the driver presses the Tight Rein button while hands are off) the automation creates a warning message and halts the transition request to a lower degree of automation to prevent a possible control deficit.
The transition from one automation mode to another incorporates an interaction pattern that escalates from the moment of initiation to the moment of finalisation. The state of the current transition is communicated on the haptic and on the visual interface.
Figure 4 shows a visualisation of a fluid transition from Tight Rein to Loose Rein that the driver initiated simply by loosening his grip at the haptic device. In the first phase, one very transparent arrow pointing from Tight Rein to Loose Rein appears and the mode Loose Rein appears in a light blue. Furthermore a vibration with low frequency is initiated in the haptic device. At phase two of the transition process a second arrow appears, and the vibration frequency is increased. This continues until the transition is finished and the vibration stops after a short high frequent vibration burst. If a mode becomes unavailable during the transition phase or a transition was aborted by either partner, the arrows are mirrored and reduced back to zero supported by decreasing frequency (aborting transition back to a lower degree of automation) or increasing frequency (aborting transition back to a higher degree of automation).
Figure 4. Fluid transition from Tight Rein to Loose Rein.
If the transition is aborted due to the unavailability of the selected mode and hands on is necessary, e.g. the transition from Loose Rein to Secured Rein was initiated by the driver by taking his/her or her hands off the haptic device and the Secured Lane is ending, an acoustic warning sound supported by the visual request to take hands back on the haptic device would start, because an haptic interaction would be futile.
The arbitration processes in the different planning and action levels depends on the amount of control the human possess in the current situation. On the one hand if the driver is temporarily out of the control loop, he does not influence any driving tasks. On the other hand if the driver has a high amount of control the automation needs to adapt to the humans actions. Therefore the arbitration processes in the “Manoeuvre Selection and Arbitration Unit”, “Trajectory Adaption and Arbitration Unit” and the “Coupling Valve” depend on the current automation mode.
Manoeuvre Selection and Arbitration Unit In most driving situations more than one manoeuvre might be possible. For example on a highway a car can follow another car, or it can change the lane and overtake the other car. To decide what manoeuvre is preferable, a valential approach can be implemented. A valential is the combined rating of potential and valence (Löper, Kelsch & Flemisch, 2008). To do so the vehicle’s automation identifies all manoeuvres that are possibly performed in the current situation with respect to the overall system state (potential). For each manoeuvre positive and negative aspects are evaluated (valence). Based on the values of potential and valence, a combined valential-score can be determined for each manoeuvre. The automation suggests performing the manoeuvre with the highest valential.
The manoeuvre selection and arbitration process depends on the current automation mode and can be seen as a higher level driving task. In Secured Rein the automation primarily selects the manoeuvres depending on the highest valential. The driver receives a visual feedback by seeing the according trajectories displayed at the contact- analogue HUD. If the automation is confident enough (sufficient and safe information is available and current situation allows automated lane changes) it also initiates automated overtaking manoeuvres when approaching another vehicle. In Loose Rein the driver selects his/her preferred manoeuvre with gestures of steering, accelerating or braking, which is then interpreted to the best fitting manoeuvre, e.g. steering left for the manoeuvre “change to left lane”. The trajectory to the selected manoeuvre is displayed in the contact-analogue HUD, so the driver can react if the automation misinterpreted the driver’s intentions. In Tight Rein the automation interprets the control commands from the driver and assists him to follow the interpreted manoeuvre which is mainly the “follow lane” or “follow vehicle” manoeuvre.
Trajectory Adaption and Arbitration Unit The automation generates trajectories depending on possible manoeuvres to which it can adjust. The current trajectory that corresponds to the currently selected manoeuvre is displayed on the contact-analogue HUD and coloured in the corresponding mode colour (see Figure 2, top), dark violet in Secured Rein, blue in Loose Rein and light grey in Tight Rein. Depending on the current mode, the driver can adapt the selected trajectory differently.
In Secured Rein the target speed is selected by the automations preferred speed, but still can be adjusted on the HDD by the driver. The automation adapts its speed to changed speed limits or to the surrounding traffic. In Loose Rein the driver is in charge of the higher level driving task: Manoeuvre changes or trajectory adaption like adapting the lateral offset of the lane’s centre, speed and/or time gap to the vehicle in front. Adaption can be initiated by natural use of the steering device (for lateral offset adaption) and acceleration device (for speed/ time gap adaption) combined in one haptic device if using a side-stick or separated when using a combination of steering wheel and gas pedal. According to the adaption the automation will give a visual feedback by changing the displayed trajectories on the contact-analogue HUD. In Tight Rein the automation adapts to the values it assumes the driver aims at (target offset, target speed or target time gap) by interpreting his/her control commands and assisting the driver to adjust to these values.
Coupling Valve As previously presented, cooperative guidance and control includes dynamic distribution of control which can be seen as dynamic coupling (Flemisch et al., 2010). The term “Coupling Valve” is a metaphorical representation of control comparable to a valve that controls the pressure/state of a hydraulic system. The driver perceives the current control distribution by the impact of his or her actions on the vehicle in the environment. Being in a highly automated mode the driver makes gestures to perform the (higher level) driving task whereas in an assisted mode he needs to steer and accelerate by himself and is only supported by the automation to a certain degree (e.g. 30% of the necessary steering angle is provided by the automation). This difference in terms of activity and impact is defined as coupling. More accurately: The coupling defines to what amount driver, automation, vehicle and environment are integrated in the control loop. This includes the measuring (perception), the defined control variable that is independently defined by driver and automation (intention) and the resulting cooperative control variable (action) that directly impacts the steering and acceleration of the vehicle in its environment.
Depending on the current mode or the criticality of the current situation, driver, automation, vehicle and the environment are coupled differently. In “normal” situations, i.e. situations where the vehicle is not heading into danger and the automation has no necessity to overtake control, the coupling of driver, automation and vehicle depends on the current arbitrated mode In Tight Rein (Figure 1, bottom left) the automation brings only a small amount of control into the system, which includes assisting forces via the haptic input device to stay on a given trajectory the automation adjusts to. In Loose Rein (Figure 1, bottom middle) the control distribution is nearly mirrored: the automation has the majority of control to stay on a given trajectory where the driver adjusts the trajectory or changes to a different trajectory due to initiated manoeuvres. In Secured Rein (Figure 1, bottom right) the driver has no haptical coupling and is only coupled by perceiving visual and acoustic signals from the vehicle in its environment or from the acoustic and visual information on the HDD or HUD prepared by the automation if he is not distracted. Therefore the driver is temporarily out of the control loop and takes no part in the driving task.
In very critical situations the automation initiates the decoupling of the driver, e.g. when approaching an obstacle with high relative speed in Tight Rein or during lane departure and no action is initiated by the driver to improve the situation. To arbitrate this decoupling, an escalating interaction pattern applies (see Figure 5). When approaching an obstacle (see Figure 5, left), the automation will initiate informing/warning double ticks via the haptic resource and escalate via vibrations and an increased counter force away from the obstacle (tension pole with repulsing polarity) until the driver is completely decoupled and the automation controls the evade or brake manoeuvre. This is comparable to the lane departure avoidance via a virtual gravel trap (see Figure 5, right), where the tension pole with negative polarity lies off road. The tension pole with attracting polarity lies in the trajectory which the automation prefers, and driving towards it reduces the amount of decoupling until the driver is completely coupled again.
Figure 5. Interaction pattern: Red: direction towards tension pole with repulsing force, Green: direction towards preferred track with attracting force. Left: Middle term repulsor-attractor - dynamic decoupler (4-stepped) „collision avoidance“ Right: Short term repulsor-attractor - dynamic decoupler (3-stepped) „virtual gravel trap“)
Figure available in the original PDF paper.
EVALUATION The interaction mediator was implemented in a prototype (H-Mode 2d 1.1) with the abovementioned functionalities and qualitatively evaluated in a driving simulator with a projector screen had a horizontal FOV of about 80 degrees and where a haptically active grip sensitive side-stick was employed as control device. The driving scene was a section of a three-lane highway with other cars.
In total 20 people (10 female (f), 10 male (m)) participated in two test series. Participants were aged between 19 and 34 years with an average age of 24.5 years (SD=3.8,Mf=24.4,Mm=24.5) and held a driving license. The two-hour study included a preliminary questionnaire regarding user characteristics such as socio-demographic factors. The second part particularly aimed at user participation in the design process by using the theatre-technique (Schieben et al., 2009). The last part was a two-stage evaluation with a 10-minute-training in between. The participants rated items, such as perceived safety, on a 7-point semantic differential scale.
The results of the last part are most valuable in terms of evaluating aspects such as system performance as participants gained high familiarity with and understanding of the system (Meier et al. 2013a, Meier et al. 2013b).
The transition from "Tight Rein“ (assisted) to "Loose Rein“ (highly automated) is… good reliable controllable comprehensible very/7 quite/6 rather/5 neutral/4
Figure 6: Evaluation of the transition from Tight Rein to Loose Rein
Figure available in the original PDF paper.
The participants started each drive in the simulator in Tight Rein. In the fluid transition mode and in Tight Rein (Assisted) the first transition is Tight Rein to Loose Rein that participants rated quite positive (see Figure 6). Regarding the mode awareness, participants comprehended the transitions between automation modes quite well (M=5.3, SD=1,75; 1=“highly disagree”, 7= “highly agree”) and were generally mostly aware of the mode they were driving in (M=5.35, SD=1.93; 1= “highly disagree”, 7= “highly agree”). Moreover, the critical point in time for changing from one mode to another was chosen very well (M=3.95, SD=0.76; 1= “too late”, 7= “too early”).
The distribution of control between me and the vehicle...
The control distribution was perceived in an unexpected manner (see Figure 7). Participants felt most comfortable with the control distribution of Loose Rein where the human is responsible for the higher level driving task (on the manoeuvre and trajectory level) and the automation is responsible for the control variables, i.e. steering and accelerating (M=6.1, SD=0.85; 1=”very uncomfortable”, 7=”very comfortable”). Due to the thinking aloud method we can refer this to aspects such as the perceived higher safety compared to Tight Rein on the one hand and on the other hand to the enjoyment of the higher interaction compared to Secured Rein.
The active support of H-Mode for speed adjustment ...
The support of the automation is quite well accepted, visible in the results for the perceived active support of H- Mode, e.g., for speed adjustment (see Figure 8). These results are to be interpreted in favour of the trajectory and arbitration unit which is supported by other results such as the overall perceived steering during lane change (M=5.63, SD=0,76; 1=”very poor”, 7=”very good”).
CONCLUSIONS
The current prototype of the interaction mediator has been tested and qualitatively proven to work as supposed and has been accepted quite well (Meier et al., 2013a). The complex distribution of control in cooperative guidance and control of highly automated vehicles in three discrete modes has been successfully accomplished. The transitions from one mode to another, as an efficient way to dynamically change the distribution of control, have also been proven to be robust. In terms of usability the subjects knew in what mode they currently were and were successful in adapting the automation’s driving behaviour like speed, lateral offset etc. to their individual needs. In addition, the way how to drive in the highly automated mode Loose Rein, e.g. to choose driving manoeuvres with naturalistic steering and accelerating/braking, was easily understood and applied, resulting in a high perceived comfort.
Although the qualitative evaluation of the current prototype (H-Mode 2d 1.1) was quite supportive, the interaction schemes and patterns need detailed improvement, e.g. experiments to determine the transition speed from one mode to another or the decoupling scheme when approaching another vehicle.
REFERENCES
Abbink, D. A., Boer, E. R., & Mulder, M. (2008). Motivation for continuous haptic gas pedal feedback to support car following.
In Intelligent Vehicles Symposium (pp. 283-290). Institute of Electrical and Electronics Engineers (IEEE). Bengler, K., & Flemisch, F. (2011). Von H-Mode zur kooperativen Fahrzeugführung - Grundlegende ergonomische Fragestellungen. 5. Darmstädter Kolloquium: kooperativ oder autonom? Bengler, K., Zimmermann, M., Bortot, D., Kienle, M., & Damböck, D. (2012). Interaction Principles for Cooperative Human-
Machine Systems. Information Technology, 54(4), 157-163. Biester (2008). Cooperative Automation in Automobiles (Doctoral Dissertation). Brandt, T., Sattel, T., & Böhm, M. (2007). Combining haptic human-machine interaction with predictive path planning for lanekeeping and collision avoidance systems. In Proceedings of the 2007 Intelligent Vehicles Symposium (pp. 582-587). Institute of Electrical and Electronics Engineers (IEEE). Breznitz, S. (1983). Cry-wolf: The psychology of false alarms. Hillsdale, NJ: Lawrence Erlbaum. Bubb, H. (1975). Untersuchung über die Anzeige des Bremsweges im Fahrzeug (Doctoral Dissertation) Bubb, H. (1993). Systemergonomische Gestaltung. In H. Schmidtke (Ed.), Ergonomie. München: Carl Hanser. Damböck, D., Weißgerber, T., Kienle, M., Bengler, K., (2012). Evaluation of a Contact Analog Head-Up Display for Highly
Automated Driving. In Neville A. Stanton (Ed.) Advances in Human Aspects of Road and Rail Transportation (pp.3-12). Boca Raton: CRC Press. Donges, E. (1982). Aspekte der Aktiven Sicherheit bei der Führung von Personenkraftwagen. Automobil-Industrie 27(2), 183-
190. Flemisch, F.O. (2000). Pointillistische Analyse der visuellen und nicht- visuellen Interaktionsressourcen am Beispiel Pilot- Assistenzsystem (Doctoral Dissertation). Flemisch, F., Bengler, K., Bubb, H., Winner, H., & Bruder, R. (2014). Towards a Cooperative Guidance and Control of Highly
Automated Vehicles: H-Mode and Conduct-by-Wire. Ergonomics. Online, doi: 10.1080/00140139.2013.869355. Flemisch, F., Meier, S., Baltzer, M., Altendorf, E., Heesen, M., Griesche, S., Weißgerber, T., Kienle, M., Damböck, D. (2012).
Fortschrittliches Anzeige- und Interaktionssystem für die kooperative Führung hochautomatisierter Fahrzeuge: Ausgewählte Ergebnisse mit H-Mode 2D 1.0. In 54. Fachausschusssitzung Anthropotechnik. Deutsche Gesellschaft für Luft- und Raumfahrt. Flemisch, F., Heesen, M., Hesse, T., Kelsch, J., Schieben, A., & Beller, J. (2011). Towards a dynamic balance between humans and automation: authority, ability, responsibility and control in shared and cooperative control situations. Cognition, Technology & Work, 14(1), 3-18. Flemisch, F. O.; Heesen, M.; Kelsch, J.; Schindler, J.; Preusche, C., & Dittrich, J. (2010). Shared and cooperative movement
control of intelligent technical systems: Sketch of the design space of haptic-multimodal coupling between operator, co- automation, base system and environment. 11th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design, and Evaluation of Human-Machine Systems
Flemisch, F. O., Kelsch, J., Löper, C., Schieben, A., & Schindler, J. (2008). Automation spectrum, inner / outer compatibility and other potentially useful human factors concepts for assistance and automation. In D. de Waard, F.O. Flemisch, B. Lorenz, H. Oberheid, and K.A. Brookhuis (Eds.), Human Factors for assistance and automation (pp. 1-16). Maastricht: Shaker. Flemisch, F. O., Adams, C. A., Conway, S. R., Goodrich, K. H., Palmer, M. T., & Schutte, M. C. (2003). The H-Metaphor as a
Guideline for Vehicle Automation and Interaction. NASA Langley. Report No. NASA/TM-2003-212672. Griffiths, P., & Gillespie, R.B. (2004). Shared control between human and machine: haptic display of automation during manual control of vehicle heading. Haptic Interfaces for Virtual Environment and Teleoperator Systems. IEEE 12th International Symposium on HAPTICS '04. Hakuli, S., Bruder, R., Flemisch, F., Löper, C., Rausch, H., Schreiber, M., & Winner, H. (2012). Kooperative Automation. In H.
Winner, S. Hakuli & G. Wolf (Eds.), Handbuch Fahrerassistenzsysteme - Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort (pp. 641-650). Wiesbaden: Vieweg+Teubner. Hoeger, R., Zeng, H., Hoess, A., Kranz, T., Boverie, S., Strauss, M., Jakobsson, E., Beutner, A., Bartels, A., To, T.-B., Stratil, H., Fürstenberg, K., Ahlers, F., Frey, E., Schieben, A., Mosebach, H., Flemisch, F. O., Dufaux, A., Manetti, D., Amditis, A., Mantzouranis, I., Lepke, H., Szalay, Z., Szabo, B., Luithardt, P., Gutknecht, M., Schömig, N., Kaussner, A., Nashashibi, F., Resende, P., Vanholme, B., Glaser, S., Allemann, P., Seglö, F., & Nilsson, A. (2011). Deliverable D61.1Final Report. HAVEit: Highly automated vehicles for intelligent transport. Kelsch, J., Flemisch, F.O., Löper, C., Schieben, A. & Schindler, J. M. (2006). Links oder rechts, schneller oder langsamer?
Grundlegende Fragestellungen beim Cognitive Systems Engineering von hochautomatisierter Fahrzeugführung. In M. Grandt & A. Bauch (Eds.), Cognitive Systems Engineering in der Fahrzeug- und Prozessführung (pp. 227-240). Bonn: Deutsche Gesellschaft für Luft- und Raumfahrt. Kelsch, J.; Heesen, M.; Hesse, T. & Baumann, M. (2012). Using human-compatible reference values in design of cooperative dynamic human-machine systems. Contribution to 30th European Annual Conference on Human Decision-Making and Manual Control (EAM 2012). Braunschweig: German Aerospace Center (DLR), Institut of Transportation Systems. Kienle, M., Damböck, D., Kelsch, J., Flemisch, F.O., & Bengler, K. (2009). Towards an H-Mode for highly automated vehicles:
Driving with side sticks. Proceedings of the First International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Krapf, S. (2009). Entwicklung einer Lösung zur Anfassdruckmessung an einem Joystick (Term paper, Technische Universität München) Meier, S., Altendorf, E., Baltzer, M., Flemisch, F., (2013a). Partizipative Interaktions- und Automationsgestaltung teil- bis
hochautomatisierter Fahrzeuge: Ausgewählte Ergebnisse explorativer Nutzerstudien zu H-Mode 1.1, In E. Brandenburg, L. Doria, A. Gross, T. Günzler & H. Smieszek (Eds.) Proceedings of the 10. Berliner Werkstatt Mensch-Maschine-Systeme: Grundlagen und Anwendungen der Mensch-Maschine-Interaktion (pp. 461-468), Berlin: Zentrum Mensch-Maschine- Systeme. Meier, S., Altendorf, E., Baltzer, M., Flemisch, F. (2013b) Influences of Socio-Demographic Factors, Personality Traits and Car Usage on Cooperative Guidance and Control. In J. Terken, A. Riener, R. Schroeter & S. Osswald (Eds.) Adjunct Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 99-100). Eindhoven: AutomotiveUI. Mulder, M. (2007). Haptic Gas Pedal Feedback for Active Car-Following Support (Doctoral Dissertation). Mulder, M., Abbink, D.A., & Boer, E.R. (2012). Sharing Control With Haptics: Seamless Driver Support From Manual to
Automatic Control. Human Factors, 54(5), 786-798. Rasmussen, J. (1983). Skills, Rules and Knowledge; Signals, Signs, and Symbols, and Other Distinctions in Human Performance Models. IEEE Transactions on Systems, Man, and Cybernetics, 3(3), 257-266. Schieben, A., Heesen, M., Schindler, J., Kelsch, J. & Flemisch, F. (2009). The theater-system technique: Agile designing and
testing of system behavior and interaction, applied to highly automated vehicles. Proceedings of the First International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp 43-46). New York, NY: ACM Press. Suzukia, K., & Jansson, H. (2003). An analysis of driver's steering behaviour during auditory or haptic warnings for the designing of lane departure warning system. JSAE Review, 24(1), 65-70. Toffetti, A., Wilschut, E. S., Martens, M. H., Schieben, A., Rambaldini, A., & Flemisch, F. (2009). CityMobil: Human factor
issues regarding highly automated vehicles on eLane. Transportation Research Record: Journal of the Transportation Research Board, No. 2110 (pp. 1-8). Washington, D.C.: Transportation Research Board of the National Academies. Wickens, C.D., Rice, S., Keller, D., Hutchins, S., Hughes, J., & Clayton, K.. (2009). False Alerts in Air Traffic Control Conflict Alerting System: Is There a ''Cry Wolf'' Effect? Human Factors, 51(4), 446-462.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Aspects of Transportation: Part I
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2097-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional