Felix - Marcel Petermann, Malene Liavaag, Julie Karine Schmidt Solberg, Ole Andreas Alsos, Erik Styhr Petersen (2022). Increasing the Usability of Audio Alerts with Voice Instructions on Ship’s Bridges. In: Human Error, Reliability, Resilience, and Performance. AHFE Open Access, vol. 33, pp. 16–21. https://doi.org/10.54941/ahfe1001563
The goal of this study was to determine how audio alarms with voice instructions affect the time required for the cause of an alarm message to be recognised by operators on a ship's bridge compared with presently used alarm signals. [Method]Four distinct alert messages with speech cues was created for this study. These were evaluated in a ship simulator where a team of two operator was directing a high-speed catamaran in sheltered waters. At irregular intervals, the operators were presented with four different error messages to which they had to respond. The intervention group (n=12) was given alarm messages with speech cues and conventional alerts with signal tones. All participants performed identical navigation scenarios, and the times required by operators to discover and identify the alarms and correct the problem was measured. In a post-testing interview a subjective estimation of the test participants were collected. [Results]The preliminary findings indicate that a combination of sound and voice commands (1) gives more information about what is wrong, (2) are faster to locate and respond to, (3) have a higher degree of memorability, (4) have a higher degree of user satisfaction, and (5) increases the users situation awareness. The test participants who just received the alarm tones took longer to locate the source of the problem since they had to go around the bridge to discover the precise location because the tones are only distinguishable to a limited extent. Some people were irritated by the auditory tones, while others just disregarded them for a short period of time. Voice instructions were seen to be useful by the test subjects, especially when many warning messages were obtained at the same time, according to their subjective evaluation. The voice commands assist in determining the severity of the problem and its location, as well as determining which issues require immediate attention. [Conclusion]In this study, we compared traditional alerts with the use of augmented alerts with voice instructions in a ship simulator with a crew of nautical students (N = 12) playing out a complex navigation scenario. We found that alerts with voice instructions were easier to learn and understand, and they made it faster and easier to locate the equipment that caused the alert, thus allowing the crew to direct more attention to navigational procedures.
Ship bridges and remote-control centers (ROC) for highly automated or auto- nomous ships have one thing in common: they both serve as the central hub around which all the information onboard a vessel is channeled. Additionally, this includes alert messages, which may be generated by a variety of compo- nents at multiple moments with different sounds and intensities along the process. The ability of operators on both conventional and remotely moni- tored ships to respond promptly and effectively in the event of an emergency is critical. Looking at ship bridges and remote centers in general, visual cor- tex overload and unassigned alerts are the two most serious issues to contend with. The distinction between a ship’s bridge and a ROC is that the latter lacks critical situational awareness impressions, such as restricted viewing angles, missing audible cues, and no discernible vessel movement. The goal of this study is to propose improved alerts for use on ship bridges and ROCs, and to measure their effectiveness and efficiency, as well as user satisfaction and learnability, using (1) quantitative data from simulator-based user tests (time- on-task) and (2) qualitative data from pre- and post-user test interviews. At last, we provide guidance for the design of audio alerts on ship bridges and ROCs and suggest further research on the use and design of voice commands.
Increasing the Usability of Audio Alerts with Voice Instructions on Ship’s Bridges 17
BACKGROUND
Ships are becoming more sophisticated as new technology is added, and each piece of equipment may have its own alert system. Thus, the ship’s bridge is often inundated with several audible and visual notifications (Magli and Zec, 2019). The present auditory signals only carry a limited amount of information, while visual cues require the navigator to be near to the source. Walking around the bridge to look at a screen to acknowledge or mute an alert may be hazardous while navigating or doing other critical duties. The same problems may occur in ROCs since they are often designed as conventional ship bridges.
Alerts onboard a ship should be designed to comply with IMO (Internati- onal Maritime Organization) rules such as BAM (bridge alert management) and CAM (central alert management). These rules aim to give the naviga- tor a uniform look across all suppliers’ equipment. The IMO’s set of rules includes four degrees of severity, with emergency being the most severe. Each alert type has its own set of visual and auditory rules (Deutsches Institut für Normung, 2019).
In research on audible alerts, Kim et al. (2020) conducted a study to inve- stigate the effects, of regular, abstract alerts as opposed to two different kinds: voice alerts and audio icons, called earcons, which are sounds associated with the alerts. For example, a collision warning would be followed by a crash sound as the audio icon. The authors found that the earcons and voice alerts led to a faster identification of the root cause, whereas the normal, beeping alerts were perceived as less explanatory (Kim et al., 2020). However, many of the alerts are overstated or annoying (Magli & Zec, 2019), and locating the source of the warnings might be difficult for the crew. Magli´c et al. (2020) proposed introducing a call management system on the bridge, with audible cues to communicate the seriousness of the calls.
Situation awareness is defined as the observation of items in the environ- ment within a range of time and space, the interpretation of their significance, and the projection of their status in the near future (Endsley, 1995). Situa- tion awareness is critical for navigators, who must perceive what is going on around them, understand the possible repercussions, and act on them. Alerts need to be designed to support the navigators’ situational awareness, and dangerous situations can arise if they do not understand the significance of the alert.
To summarize, alerts that fulfil regulatory criteria do not convey sufficient information to assist the user on the bridge or in the ROC. On the contrary, the individual alerts and operations on the ship’s bridge are very complex. It is therefore necessary to improve the design of alerts so that they can provide more information to the navigators/operators so that they can increase their situational awareness. In the next section, we will describe how we design new alerts and how we evaluate them.
METHODOLOGY
In this study, we followed a user-centered design approach utilizing several methodologies within the double diamond design framework (Ball, 2019),
Alert design 1 (A1) Alert design 2 (A2) Ordinary alert with alternating three brief beeping sounds
Alternating three brief beeping sounds and voice commands notifying of the problem and the affected equipment.
Figure 1: The simulator used in the study.
e.g., interviews with stakeholders, affinity diagrams, mind mapping, and user journeys. This explorative phase helped to identify essential themes and incor- porate difficulties and pain points from the navigator’s perspective and their actual needs regarding alerts and notification on the ship’s bridge. It is not in the scope of this study to go into details of the method and results of this phase, but the process led to the design of alerts (within the IMO and IEC regulations) that were augmented with additional information as shown in Table 1.
To evaluate the designed alerts for “Effectiveness, Efficiency, Satisfaction, and Learnability” (Nielsen, 1993), we applied a mixed method appro- ach, blending ethnographic simulator-based research with interviews and quantitative data collection.
The experiments were conducted in a Kongsberg ship bridge simulator (Figure 1) with nautical students (N = 12), who had a minimum experience of 2 years on a ship and 1 year in a simulator. Participants were paired up, one piloting the vessel and one to keep watch at monitor and instruments.
The evaluation had the following phases: (1) a brief where the partici- pant was informed about the experiment and permission for audio and video recording was acquired, (2) a pre-study interview gathering demographic and experience information about the participants, (3) a simulation where parti- cipants navigated a high speed vessel with a challenging navigation scenario in highly trafficked narrow water while difficulties were simulated at certain intervals and the corresponding designed alerts were triggered, and (4) a post evaluation interview, where their experience and subjective opinion of the efficacy of the newly designed alerts were studied.
Increasing the Usability of Audio Alerts with Voice Instructions on Ship’s Bridges 19
Table 2. Time-on-task to identify the cause of an alert for both the normal alert (A1) and the voice command augmented alerts (A2).
During each experiment run, the participants went through a test scenario that consisted of four alert circumstances (gyro lost, GPS lost, radar lost, and log failure), alternating between traditional and augmented alert patterns. The alerts were sent in order A2-A1-A1-A2 to one test group and order A1- A2-A2-A1 to the other. As a result, the impact of various alert designs could be explored. Time-on-task was measured to determine if one alert design was effective. The times were measured from when the alert occurred to the crew was able to detect the corresponding equipment. This data was used in conjunction with the responses to the post-session questionnaire to make comments about the satisfaction and learnability of the newly introduced alert designs.
RESULTS
By analyzing the simulation observations and the user interviews done at the end of the user testing, we found that with traditional alert designs (A1), all users moved around the ship’s bridge to locate the actual source of the alert. Additionally, during the following interview, all test participants reported that they did not know what the underlying problem was until they disco- vered its cause on the screens. According to some, they were able to identify various frequent alerts only based on the sound signal, e.g. “Gyro lost”. They did note, however, that some of the alert sounds were the same or almost identical, making differentiation difficult.
On the other hand, all test subjects said that the alerts with voice com- mands (A2) enabled rapid identification of the alert’s cause, allowing the user to focus on the issue at hand rather than spending time searching for the cause first. Further, with voice commands, they could recognize the alerts without any prior training. One thing that some users still overlooked was the urgency of the particular alert since it was possible for many alerts to occur simultaneously.
When we examined the time necessary for users to determine the reason for the alert, we discovered that alerts augmented by voice commands (A2) required less time for GPS, radar, and log. There was little variation in the
gyro alert. However, it should be pointed out that, given the limited sample size, the figures stated here do not constitute a reliable assertion.
However, when the quantitative and qualitative findings were examined, it was evident that both sets of findings supported one another. One exception is the small variation in the gyro’s alert message. This is an alert message that the test subjects were already acquainted with and recognized from the tone, and the alert was placed more in the users’ primary line of sight. Other alerts were not as obvious or were positioned in areas other than the primary emphasis.
CONCLUSION
In this study, we compared traditional alerts with the use of augmented alerts with voice instructions in a ship simulator with a crew of nautical students (N = 12) playing out a complex navigation scenario. We found that alerts with voice instructions were easier to learn and understand, and they made it faster and easier to locate the equipment that caused the alert, thus allowing the crew to direct more attention to navigational procedures.
Given its small sample size and scope, this study may be seen as a founda- tion for further exploration. Our next step is to design alerts using earcons and haptic feedback, connect the alerts to visual cues in the HMI, and eva- luate this with a larger sample size, both in a simulated setting and on a real ship bridge.
We believe that good alert design will be especially important in the design of shore-based control rooms or remote operation centres, where the crew do not have the same feeling of the ship as when being onboard and looking through the windows.
ACKNOWLEDGEMENT
We thank Andreas Madsen for assistance with the simulator experiments and the nautical students participating in the study. The study was funded by the Norwegian Research Council through SFI AutoShip (grant no. 309230).
REFERENCES
Ball, J. (2019). The Double Diamond: A universally accepted depiction of the design process. [online] Design Council. Available at: https://www.designcouncil.org.uk /news-opinion/double-diamond-universally-accepted-depiction-design-process. Deutsches Institut Für Normung (2019). DIN EN IEC 62923-1, Maritime navigation
and radiocommunication equipment and systems - bridge alert management. Part 1, Operational and performance requirements, methods of testing and required test results (IEC 62923-1:2018) = Navigations- und Funkkommunikationsgeräte und -Systeme für die Seeschifffahrt - Brücken-Alert-Management. Teil 1, Betriebs- und Leistungsanforderungen, Prüfverfahren und geforderte Prüfergebnisse (IEC 62923-1:2018). Berlin Beuth Verlag Gmbh Juni. Endsley, M.R. (1995). Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), pp. 32–64.
Increasing the Usability of Audio Alerts with Voice Instructions on Ship’s Bridges 21
IMO (2010). Adoption of Performance Standards for Bridge Alert Management (BAM); Resolution MSC.302(87). [online] IMO. Available at: https://wwwcdn.imo.org/localresources/en/KnowledgeCentre/IndexofIMOR esolutions/MSCResolutions/MSC.302(87).pdf [Accessed 10 Dec. 2021]. Kim, I., Lee, S. and Youn, I. (2020). Adopting the audible alert system for the
electronic chart display and information system for improvement of early navigati- onal situation awareness. Journal of International Maritime Safety, Environmental Affairs, and Shipping, 4(4), pp. 177–186. Magli´c, L., Valˇci´c, S., Gundi´c, A. and Magli´c, L. (2020). Voice Communication Systems Impact on Navigating Officers. Journal of Marine Science and Engineering, 8(3), p. 197. Magli´c, L. and Zec, D. (2019). The Impact of Bridge Alerts on Navigating Offi-
cers. Journal of Navigation, 73(2), pp. 421–432. Nielsen, J. (1993). Usability Engineering. Amsterdam: Morgan Kaufmann.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Error, Reliability, Resilience, and Performance
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-09-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional