Gesa Praetorius, Jonas Lundberg, Anders Johannesson, Karl Johan Klang, Gustaf Söderholm, Magnus Bang (2025). Evaluation of a Digital Assistant concept for Vessel Traffic Service Operators. In: Advances in Human Factors of Transportation. AHFE Open Access, vol. 186, pp. 622–632. https://doi.org/10.54941/ahfe1006552
Vessel Traffic Service (VTS) is an information service for merchant vessels in areas with high traffic loads or in proximity to ports. VTS is provided by VTS operators (VTSOs) that synthesize a large amount of information from different screens and different information sources to choose when and what to transmit to the traffic in the VTS area. As recent technical advances have paved the way for new ways of interaction and collaboration between human operators and automation, the focus for future system design has shifted from operator in or on the loop to human-automation teaming. This study builds on these recent advances and presents findings from an evaluation of a concept for an advanced decision support tool, a Digital Assistant, for VTSOs. The digital assistant presents information and offers different options for interacting, such as acknowledging information or delegating certain tasks. Four expert users evaluated the system in Wizard of Oz demonstration. Overall, the users deemed the concept as having potential and being helpful in high workload situations. They interacted with and partially delegated tasks to the assistant system but also raised concerns about not always agreeing with the automation’s suggestions, nor understanding what information these were based on. While the approach seems promising, future system iterations should consider the timing of the interaction between operator and automation, and the access to information to enable VTSOs to trace the automation’s decision making and increase its transparency.
On July 14, 2009, the chemical tanker Maria M was bound for an anchorage outside in the port approach to Gothenburg to bunker. The bridge team reported to the Vessel Traffic Service (VTS) that they would cross the reporting line into the service’s area in approximately 30 minutes. The Vessel Traffic Service Operator (VTSO) was monitoring the VTS area and providing information to the traffic in the area. At 22:06, the VTSO plotted a radar track for a vessel which showed an unusual movement. As no transponder data was available, the VTSO assumed that it was a fishing
Evaluation of a Digital Assistant Concept for Vessel Traffic Service Operators 623
vessel. Approximately 30 minutes later, the bridge team contacted the VTS to disclose that Maria M had run aground outside the VTS area. While the investigation concluded that the grounding was caused by several factors, such as a lack of knowledge of the vessel’s bridge equipment and deficiencies in teamwork, it was also concluded that the VTSO should have reacted upon the information available to him despite the fact that the vessel was outside of the VTS area, and despite her missing transponder data (SHK, 2009). This incident highlights one of the foremost problems within maritime shore-based services. While the information available to VTSOs may be rich and their experience may support them in identifying cues for deviations, the amount of information is vast, often forcing the VTSO to integrate and evaluate bits and pieces at the same time as advising and informing other traffic in a large area. In the case above the transponder data from the Automatic Identification System (AIS) disclosing the vessel’s name, her destination, draught and so forth, was missing, i.e. the RADAR plot showed a vessel’s movement, but it might not have been clear to the VTSO that this was the chemical tanker, especially not given the unusual maneuvering and current position. The integrated information system provided the information, yet it did not help in supporting the operator’s decision making in this situation.
Through recent years, technological advancements have made it possible to rethink the role of decision support systems in traffic management services. Moving away from a focus on technical capabilities, current research endeavors emphasize the collaborative aspects of human-technology interactions and have adopted a human-automation teaming approach. The current study is in line with this paradigm exploring how VTSOs can be supported by self-explanatory and adaptive automation. While the current VTS decision support systems already offer a multitude of systems and sensors that are merged, this study takes a novel approach to understand how operator and automated system can work jointly to facilitate safe and efficient traffic movements within dedicated areas. Drawing up on the Digital Tower Assistant (DiTA) concept (Westin et al., 2020), a dedicated glyph display has been designed and evaluated through a Wizard of Oz approach (Benyon, Turner and Turner, 2005).
VESSEL TRAFFIC SERVICE (VTS) Vessel Traffic Service (VTS) is an information service for merchant vessels over 300 dwt, or over 45m in length. VTS is provided in areas that are challenging to navigate in and areas with high traffic density to promote safe, fluent and environmentally friendly traffic movements (IALA, 2024). VTS is provided through VHF communications by shore-based operators, VTS operators (VTSOs) in a determined area of responsibility, a VTS area. Within the VTS area, merchant vessels receive information on upcoming traffic meetings and other aspects that may impact on their ability to navigate safely. While VTS is defined through the legal framework of the International Maritime Organization (IMO), the service is implemented locally by the Competent Authority and may therefore differ from country to country
(IALA, 2024). In this study, the focus has been on VTS operations as carried out with Swedish waters.
Earlier research in the VTS domain has focused on the service’s role in maritime traffic management (e.g. van Westrenen & Praetorius, 2014), on its contribution to safety and efficiency (Praetorius and Hollnagel, 2014) and how to uncover the complexity of everyday work (Relling et al., 2020). However, in recent years, an increasing number of publications have addressed the need for user-centered design (Crestelo et al., 2022) as well as the potential for advancing decision support (Stach, Koch and Constapel, 2024; Xiao et al., 2023) (Kao et al., 2020) available to operators. However, while there has been a push towards higher degrees of automation and the integration of intelligent system support for VTSOs, designing adaptive and self-explanatory automation with a focus on human-automation teaming has not yet been explored in research studies.
Supporting VTS Decision Making In this study, operator support has been developed as a “glyph”. The glyph concept with communication though dialogue windows is founded on the Joint Control Framework (Lundberg and Johansson, 2021) and the Reduced Autonomy Workspace (RAW) concept presented by (Nylin, Johansson Westberg and Lundberg, 2022), in terms of providing an event horizon with a latest valid time for a decision to be made. The dialogue window presents information through text on various levels of autonomy in cognitive control (LACC). The content of the text-based messages is inspired by Simon (1973) and should be brief and well-structured to alleviate decision making in a time-critical situations. In the dialogue window there is also a glyph representing a changing situation on LACC level 6 such as a sudden speed reduction or loss of AIS signal, indicating a need for further assessment of potential consequences for the situation.
Figure 1: Dialogue window with graphical representation and information text. This representation allows only for information to be acknowledged by the operator.
Figure 1 depicts one of the developed dialogue windows. The information glyph is intended to provide information that is important for the VTSO, while not requiring immediate action. The information provided is that a vessel has lost her AIS signal, and the operator is informed about an estimated
Evaluation of a Digital Assistant Concept for Vessel Traffic Service Operators 625
position based on when the signal was lost. The symbol on the left-hand- side represents the signal loss and the duration of for how long the glyph will be displayed unless the operator chooses to “Acknowledge” it. Pushing “Acknowledge” will close the dialogue window.
METHODOLOGY
The evaluation was conducted in a dedicated simulation laboratory at Linköping university. A Wizard of Oz strategy (Benyon et al., 2005) was employed. The aim was to identify how the operator perceives the support provided by the automation and how decision making is affected by presenting an option to delegate certain tasks, such as communication with a vessel, to the automation. The simulation was combined with semi- structured interviews (Patton, 2014) to explore operator perceptions and requirements for human-automation collaboration. The interviews were held after the simulation and served as a debriefing, as well as a means to gain a deeper understanding for how the operators experienced the interaction with the system. Four VTS operators from two different VTS centers in Sweden were sampled purposively. Three of them had previously taken part in project-related workshops. The participants were between 41 and 48 years old and had a background as navigational officer (on average 5.7 years). Their operational experience ranged from 6 to 18 years as VTSO, with a median of 10 years of duty.
Apparatus The Wärtsilä Navi-Habour Vessel Traffic Management System was used as VTS simulator for the evaluation. The simulator setup consisted of three small screens and one large central display, each providing a view of the maritime traffic (Fig. 2). To enhance this setup, screen capture technology was used to replicate the entire displays in real-time. An external computer captured the video feed from the simulator, as a middleman, and fed back the video to the displays. A software was applied on top of this captured feed, adding a layer of glyphs graphically. The glyphs were triggered remotely by an external computer, controlled by one of the simulator pilots. Tobii Glasses 3 were used to record eye-tracking, and a border of ARUCO-codes was overlayed on each display to capture the eye-gaze on each screen together with the glasses.
Test Scenarios Prior to the evaluation, two expert workshops were held to identify situations that increase VTSO workload and stress, that are experienced as risky, and that are occurring frequently enough so that support by an automation could become meaningful within everyday work settings. Details about the workshop design and participants are described in (Lundberg et al., 2024).
The scenarios used in this evaluation are summarized in Table 1 and were based on the situations identified and explored in the expert workshops. The first scenario served as familiarization in which the operator successively was presented with all the glyphs and type of situations that would occur during
the evaluation. Scenario 2–4 all addressed specific situations and presented a potential escalation from informing the operator to the automation acting. The final scenario was 30 minutes long. The aim was to enable an interaction between operator and automation in a longer timeframe and without isolating one type of situation.
Figure 2: Simulator setup.Table 1: Overview of the evaluation scenarios.
Table available in the original PDF paper.
Scenario Length (min) Aim Situations 1 16 Familiarization Deviation from route Traffic meeting/congestion Loss of AIS signal 2 6 Escalation from information to acting (deviation from intended route)
Deviation from intended route (information) Heading for shallow waters Deviation from intended route (need to act/ automation acts) 3 5 Escalation from information to action (traffic congestion)
Traffic congestion (information) Reducing speed Traffic congestion (suggestion for action/automation acts) 4 12 Reporting & Loss of AIS signal
Unanticipated speed reduction Passing reporting point Loss of AIS signal 5 30 Glyph concept evaluation, interaction with no guidance from research team
Deviation from intended route Heading towards shallow waters Attention guidance Traffic congestion Reduced speed Passing reporting point Dragging anchor
Evaluation of a Digital Assistant Concept for Vessel Traffic Service Operators 627
Procedure The evaluation was conducted with one operator at a time. Before entering the simulator, the participant was briefed in a separate room. The briefing contained a short description of the concept, the overall structure of the evaluation, and the rights related to participating in a scientific study. This also offered an opportunity for the participant to ask for clarification or other information. In the end of the briefing, informed consent was obtained in writing.
After the briefing, the participant entered the simulator and received a short introduction to the system setup before the familiarization started. After the familiarization, the consecutive scenarios were used to test different interactions and means of collaboration between the operator and digital assistant. During the scenarios, the participants were encouraged to think aloud to capture the participants’ impressions and perceptions coupled to certain events and situations. Trigger probes were used when a participant stopped verbalizing thoughts and decisions. During the simulations, VHF communication from vessels to the VTSO was enacted by one of the researchers when required. The simulation took approximately 90 minutes. Afterwards a debriefing interview was held in a separate room. The interviews took 35 minutes on average.
Analysis For this article, only notes, video recordings and interview transcripts were considered. The data was analyzed iteratively focusing on the interactions between automation and operator and the operators’ perceptions. Notes and interviews were coded in MaxQDA following directed content analysis approach (Hsieh and Shannon, 2005).
RESULTS
Overall, the participants experienced the simulation to be ecologically valid, but discussed that the system to work with, e.g. screen set-up, and the number of events during a short time frame differed from their everyday work settings. The scenarios represented situations that would normally occur, but maybe not in the same density. It was also emphasized that routine work heavily relies on verbal cues on the VHF radio and local expert knowledge about traffic patterns in the supervised area. For two of the four participants, the VTS area in the simulation was not their usual area of operation thus it might have impacted on their performance.
With regards to their own performance, the experts felt that they were able to act professionally and within the frame of what would normally be considered as normal to good performance for a VTSO. It was mentioned though that being exposed to a novel VTS area, i.e. not the area that one normally works with, made it harder to identify anomalies, as well as traffic patterns which could be considered normal or deviating.
Perception of the Automation The current design is experienced as taking attention from the operator tasks as the glyphs are displayed in another screen and not anchored to context, i.e. the geographical area or specific vessel(s), where the situation occurs. The operator is thus forced to switch from a monitoring task on the overview screen to reading a text box. This switch of focus may draw attention from what is important to early identify anomalies.
The content of the messages was perceived as relevant and important, and short and concise. The glyphs were further recognized to be “good to have” if time to react is limited. However, not all participants agreed to the meaningfulness of the suggested solutions presented by the automation. It was also highlighted that several glyphs at once and certain types of information were rather hindering than supporting the decision-making process.
“I got something when there was a situation in the Danafjord. She is dragging and then suddenly you get that you have not had a look at the area east of Tistlarna in 20 minutes. So then one feels that I don’t care about that. This can be good and bad” (P2).
It was also mentioned that it is important for the automation to understand the work context including the operator’s taskload and workload to ensure that the glyphs do not distract but help. Thus, notifications from the system need to be time- and context-sensitive, e.g. are displayed when relevant and consider whether the operator needs to prioritize another task at the current moment in time.
Interaction With the Traffic In the current approach, the interaction with the traffic can either be through the dialogue window, i.e. letting the automation send information, or by radio communication.
“How do you know that someone has understood? That is the hard thing here. You get a better notion of that if you talk. If it is the wrong person, they may just push acknowledge independently of whether they have understood” (P1).
While the operators expressed that some of the information could potentially be useful, they issued concern that interacting with traffic, especially in situations with a certain urgency, such as when heading towards shallow waters or towards a traffic congestion, communicating through the glyph interface would not be an option. Instead, all participants emphasized that VHF communication is the most effective and safest way to establish contact and alert a vessel of a deviation or a developing situation.
Distribution of Work The degree to which participants chose to delegate certain tasks to the system differed. As mentioned above, there was a consensus among participants that verbal interaction provides a more direct means of interaction offering a
Evaluation of a Digital Assistant Concept for Vessel Traffic Service Operators 629
multitude of cues, e.g. whether the bridge team has received and understood a certain advice, to VTSOs. However, it was also perceived that suggestions on how to resolve a situation would be valuable in a very stressful situation.
“Well, it was rather the case that it delegated tasks to me. Do you want this information or not. It was not the case that I could tell it ‘do this now”’(P3).
Rather than teaming with the automation, the system was experienced as notifying the participant and not collaborating as can be seen in the quote above. The interaction was always initiated by the system and the current iteration did not enable the operator to send own queries or tasks towards the digital assistant. With regards to the potential to collaborate with the automation rather than receiving notifications, the participants highlighted the importance of trust into the system’s ability to recognize important information and tailor glyphs to the current context. Trust is built over time and through experience of working with the system. This also includes the need to better understand how the automation reasons and why certain solutions are presented as in the glyph interface. System features that would increase the likelihood of a more collaborative work included the ability of speech recognition and ability of analyzing VHF communication for deviations, as well as being able to understand and interpret the context, i.e. traffic patterns, information about intended routes and likely future developments.
One of the participants also raised the fact that the ongoing development towards autonomous ships may open new possibilities for a digital agent in VTS operations. It might be of benefit of an autonomous ship is handled by a digital agent ashore.
Design Suggestion Based on the experiences from the simulation, the participants generated design suggestions to increase the perceived usability of a digital assistant in everyday VTS work. These can be clustered into 3 categories: closed loop of information exchange, enhanced visibility and enhanced functionality.
All participants emphasized that it was hard to understand whether a certain message sent to a vessel through the glyph interface had been received and understood by the bridge teams. VHF communication was preferred to ensure a closed loop of information exchange. Thus, the representation of the glyph needs to be enhanced so that different stages of information exchange, i.e. sent, received, acknowledged, can be communicated to the VTSO. One suggestion was a change in color coding, or an updated message to the VTSO to ensure a closed loop between shore and ship.
Another aspect raised was the placement and overall visibility of the glyph representation. Participants highlighted the need to integrate the dialogue window within all screens of the system so that they would not only be displayed in the periphery. To ensure that the displayed glyphs do not interfere with operators’ work, they should be placed at a relevant position within the information system. One solution could be to anchor
these to the highlighted event, e.g. showing them close to the vessel or the geographical area they concern. Another option could be to integrate the glyphs through a mouse-over concept so they would appear but remain hidden not to obstruct or hide important information from the operator. A window with a message history could then serve as a memory aid with a list of previous and current interactions and delegation between operator and automation.
To enhance the functionality of the digital assistant system, the context awareness of the automation needs to be increased. During the evaluation, the system was mostly perceived as issuing notification while not being context sensitive. Voice recognition and identifying relevant cues within the traffic picture were mentioned as desired functionality. Being able to initiate the interaction rather than being notified was mentioned as a desired feature. This may also include the ability to modify messages before these are sent to vessels, as well as to delegate tasks or request specific support or information.
DISCUSSION
This study aimed to explore how an advanced decision support, a Digital Assistant, is perceived by VTS operators. Overall, the experts within the study were positive towards the design and perceived the system as supportive. The identified situations were relevant and the messages from system to operator concise and informative. However, while the participants delegated certain tasks to the automation, they emphasized the need for closed loop communication between ship and shore and the importance of receiving cues from the verbal interaction with the traffic. These findings are in line with earlier research (e.g. Praetorius and Hollnagel, 2014; Costa, Lundh and MacKinnon, 2018) that identified VHF communication as an important source of information for VTSOs. It offers direct feedback on if and how information has been received and serves as indication about the state of the bridge team, which can help to adapt the service to a specific vessel.
Context awareness and sensitivity towards the operator’s current work situations, e.g. taskload and workload, were mentioned as an important aspect in the collaboration with the system to ensure that information is presented when needed and without interfering with current work processes. To collaborate with the automation as a team, or a joint cognitive system (Hollnagel and Woods, 2005), work experience over time is needed to understand how the system learns and what information its decisions are based on. Only if the operator understands and trusts what is provided by the automation, collaboration and delegation of tasks on higher levels of the LACC (Nylin et al., 2022) become possible.
The current study has evaluated a first concept for operator-automation collaboration in VTS operations. The number of participants was limited but still generated meaningful feedback for future system iterations. In a next step, operator-system interactions will be analyzed with the JCF (Lundberg and Johansson, 2020) including combing think aloud and interview data with eye-tracking and screen recordings.
Evaluation of a Digital Assistant Concept for Vessel Traffic Service Operators 631
CONCLUSION
This article has presented results from a first concept evaluation for advanced decision support in the VTS domain. While the participants were overall positive, the evaluation highlighted the need for further iterations to increase the flow of interaction between operator and automation. To support collaboration, the digital assistant needs to show an increased context sensitivity and offer the opportunity for operators to initiate queries and joint tasks. Means to compensate for missing cues, such as in VHF communication, need to be explored further.
ACKNOWLEDGMENT
This study has been financed by the Swedish Transport Authority as part of the Följsam Automation 2-project (TRV 2020/138317). The authors would also like to express their gratitude to the experts who participated in the evaluation. Thank you!
REFERENCES
Benyon, D., Turner, P., & Turner, S. (2005). Designing Interactive Systems. People, Activities, Contexts, Technologies. Pearson Education Limited. Costa, N. A., Lundh, M., & MacKinnon, S. N. (2018). Non-technical communication factors at the Vessel Traffic Services. Cognition, Technology and Work, 20(1). Crestelo Moreno, F., Roca Gonzalez, J., Suardíaz Muro, J., & García Maza, J. A.
(2022). Relationship between human factors and a safe performance of vessel traffic service operators: A systematic qualitative-based review in maritime safety. Safety Science, 155. Hollnagel, E. and Woods, D. D. (2005) Joint Cognitive Systems. Foundations of Cognitive Systems Engineering. Boca Raton, FL: CRC Press. Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content
analysis. Qualitative Health Research, 15(9). IALA. (2024). VTS MANUAL. Kao, S.-L., Hsueh, C.-K., Chou, C.-C., & Yuan, T.-Y. (2020). A Decision-making Support System for Automatically Determining the Route Priority of Vessels Entering/ Exiting the Ports. Transportation Journal, 59(4), 335–368. Lundberg, J., & Johansson, B. J. E. (2021). A framework for describing interaction
between human operators and autonomous, automated, and manual control systems. Cognition, Technology and Work, 23(3). Lundberg, J., Nylin, M., Praetorius, G., Jansson, A. A., & Bång, M. (2024). Modelling operator control work across traffic management domains: Implications for interaction design. Cognition, Technology and Work, 26(2). Nylin, M., Johansson Westberg, J., & Lundberg, J. (2022). Reduced autonomy workspace (RAW)—an interaction design approach for human-automation cooperation. Cognition, Technology and Work, 24(2). Patton, M. Q. (2014). Qualitative Research & Evaluation Methods: Integrating
Theory and Practice. SAGE Publications. Praetorius, G., & Hollnagel, E. (2014). Control and resilience within the maritime traffic management domain. Journal of Cognitive Engineering and Decision Making, 8(4).
Relling, T., Lützhöft, M., Hildre, H. P., & Ostnes, R. (2020). How vessel traffic service operators cope with complexity–only human performance absorbs human performance. Theoretical Issues in Ergonomics Science, 21(4). SHK. (2009). Kemikalietankfartyget MARIA M - IMO-nr 9301885 - IBSK -
grundstötning den 14 juli 2009. Simon, H. A. (1973). The structure of ill structured problems. Artificial intelligence, 4(3–4), pp. 181–201. Stach, T., Koch, P., & Constapel, M. (2024). A Formalism for Scalable Maritime
Traffic Monitoring and Explainable Anomaly Detection and Resolution at Vessel Traffic Services. Journal of Physics: Conference Series, 2867(1). van Westrenen, F., & Praetorius, G. (2014). Maritime traffic management: A need for central coordination? Cognition, Technology and Work, 16(1). Westin, C., Boonsong, S., Josefsson, B., & Lundberg, J. (2020). Building Trust in
Autonomous System Competence–the DiTA Digital Tower Assistant for Multiple Remote Towers, an Early Concept Evaluation. In 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC) (pp. 1–9). IEEE. Xiao, Z., Fu, X., Zhao, L., Zhang, L., Teo, T. K., Li, N., Zhang, W., & Qin, Z. (2023).
Next-Generation Vessel Traffic Services Systems - From “Passive” to “Proactive.” IEEE Intelligent Transportation Systems Magazine, 15(1), 363–377.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Factors of Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-62-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional