Leander Pantelatos, Ole Andreas Alsos, Mina Saghafian, Øyvind Smogeli, Asun Lera Stclair (2024). A Mixed-Method Approach for Assessing User Perspectives in Autonomous Urban Ferries. In: Advances in Human Factors of Transportation. AHFE Open Access, vol. 148, pp. 718–728. https://doi.org/10.54941/ahfe1005264
Autonomous urban ferries may become a new solution for future urban public transport. The perception of users and their attitudes plays a vital role in the acceptance and use of autonomous technologies. In this method article, we present a mixed-method approach to assess users’ perspectives on autonomous urban ferries. The method is based on citizen engagement and consists of four sessions with a broad range of representative public transport users. The sessions are (1) information and table discussions, (2) an immersive virtual mixed-reality simulation of a ferry trip in a full-scale ferry mock-up, (3) a ferry trip onboard a working autonomous prototype ferry, and (4) reflective table discussions. The method collects rich and multidimensional data which can be used to inform researchers and practitioners on how to study passengers’ perspectives on public autonomous transport solutions.
Autonomous technologies are increasingly present in the maritime sector due to their promising benefits such as cost efficiency, environmental sus- tainability, and operational safety (Reddy et al., 2019). Battery powered zero-emission autonomous urban passenger ferries can serve as an efficient solution in future public transport in urban areas. Given that around 90% of urban areas are coastal (Climate Change | UN-Habitat, no date), together with the urban population expected to increase, there is a need for more efficient modes of transport to ensure the quality of life in urban spaces (Goerlandt, 2020) – a task autonomous ferries potentially can fulfil.
An autonomous ferry differs from a conventional ferry in that it can – with the help of different sensors such as lidar, radar, and cameras – navigate and maintain a safe course at a certain Level of Autonomy (LoA) (Smogeli, 2023). Prototypes such as the milliAmpere2 (Alsos et al., 2022) are currently operated at Level 2 – onboard supervised autonomy implying a human safety
A Mixed-Method Approach for Assessing User Perspectives in Autonomous Urban Ferries 719
host aboard who intervenes if necessary (Smogeli, 2023). Within the field of autonomous transportation, a critical point with respect to the LoA is abandoning the human safety host aboard which has been outlined in several studies (Goerlandt, 2020; Goldbach et al., 2022).
Besides the technological development required to reach the implemen- tation of higher LoAs, investigating the user perspectives is important to understand if and how the general public would accept autonomous ferries. Safety is a prominent aspect to be researched in the field of autonomous trans- portation. Even if an autonomous ferry would be objectively considered safe, it might not be perceived as such. Some studies have investigated safety per- ceptions of autonomous ferries already such as (Goerlandt, 2020; Munim et al., 2022), but restricted to survey-based and interview methods only. This approach is valuable for gaining insights into the perceptions of participants towards autonomous ferries but limits the possibility of obtaining first-hand experiences of passengers interacting with the technology.
Simulation of different kinds is another method commonly used to study passenger perceptions of trust and safety towards autonomous transportation (Pan et al., 2017; Hock et al., 2018). Simulations can enhance the experience based on their type and their level of realism. They can help people immerse in an experience, that otherwise would remain abstract and theoretical at best. Nevertheless, simulations often have hard-coded scenarios and lack the characteristics of an open system with unexpected scenarios and unplanned social encounters and influences.
Real-world testing allows for piloting the autonomous ferry within the intended context and specifically investigates how environmental, socio- technical, and human factors influence user perceptions of autonomous transportation. On a general level, the need for public trials of autonomous transportation is also evident in policy documents both in the US (U.S. Department of Transportation, 2018) and the EU (EU Comission, 2018). In research, examples of real-world testing of AVs can be found (Mouratidis and Cobeña Serrano, 2021; Stålhane, Myklebust and Haug, 2021; Goldbach et al., 2022).
All these methods have advantages and limitations in capturing the safety perception of AVs. Besides safety perceptions, societal and sustainability perspectives would also influence the acceptance of autonomous ferries. However, these are difficult to capture with only some of the methods described. Therefore, it is necessary to take a holistic approach and com- bine these methods and triangulate data in an exploratory approach in order to provide rich insights.
The mixed-method approach outlined in this paper is a product of the TRUSST project—Assuring Trustworthy, Safe, and Sustainable Trans- port for All. TRUSST aims to develop an integrated assurance framework, grounded in interdisciplinary and socio-technical perspectives. The collabo- rative project involves DNV as a risk management and assurance provider, the Norwegian University of Science and Technology (NTNU), and Zeabuz, a spin-off company from NTNU specializing in autonomy infrastructure for autonomous ferries.
As the context of autonomous ferries is still novel in society, this work pro- poses a method based on citizen engagement with several sessions in order to gain a deeper understanding of the public perception of autonomous fer- ries. A discussion on opportunities and limitations of applying the method concludes the paper.
DETAILS ABOUT THE METHOD The approach was framed using a set of methodologies to engage with cit- izens. This qualitative approach is inspired by citizen engagement, where citizens engage with governmental and/or industrial actors in policy-making and technology development (Olphert and Damodaran, 2007). Engaging with citizens enables end users to contribute with experiences, and (local) knowledge and provides an opportunity for learning and skill sharing. Pos- itive outcomes can be improved quality and effectiveness of solutions as well as increased social inclusion, ownership and faster technology uptake (Olphert and Damodaran, 2007). In this approach, the citizen engagement approach was used to explore user perspectives in autonomous ferries and to define a set of recommendations for further development of technology and assurance. The structure of the engagement activities was based on thematic analysis method (Braun and Clarke, 2006).
Recruiting Participants and Complying With Research Ethics Due to practical considerations of conducting several sessions, the recom- mended participant limit was found to be 20. As the study is set in a specific context, emphasis on recruiting from the specific context was prioritized, focusing on residents already using public transport. A recruitment profile, including residency postal codes, transport usage, gender, age, disabilities, education, and ethnicity, guided the process. A third-party company handled recruitment, and participants received vouchers as incentives.
To adhere to ethical standards, participants were fully informed about the study’s purpose, data collection, processing, storage, and voluntary nature. Anonymization was ensured through code-only identification. The informed consent form and data management plan were approved by the Norwegian Centre for Research Data (SIKT) under project number 37623.
Establishing Four Sessions of Immersive and Collaborative Nature The method is based on a citizen engagement consisting of four sessions as depicted in Figure 1. The sessions consisted of (1) information and table dis- cussions, (2) an immersive virtual mixed-reality simulation of the ferry trial between destinations in the area of operation (Figure 3), (3) real ferry tri- als with the autonomous prototype milliAmpere2 within the real context (Figure 2), and (4) reflective table discussions.
The sessions focused on the topics of safety, sustainability, and societal impact. The study emphasized capturing the participants’ initial perceptions but also understanding how these would change after gaining more knowl- edge and immersion in the overall concept. The sessions were arranged with
A Mixed-Method Approach for Assessing User Perspectives in Autonomous Urban Ferries 721
some time in between to obtain immediate feedback as well as reflected answers.
Figure 1: The four sessions of the mixed-method approach (Pantelatos et al., 2023).
Session 1: Information and Table Discussions Before the first session, participants were briefed on basic information and shown pictures of the autonomous urban ferry concept by Zeabuz. The first questionnaire (WS1-A) was handed out at the beginning of the first session to gauge the initial perceptions of the participants. Informative presentations on future possibilities for urban spaces, technology, and a visualized user journey followed. It was emphasized that the material presented was not a representa- tion of how a potential implementation would look but rather concepts and ideas for discussion. After the presentations, participants were divided into four groups for table discussions. The discussion was guided by large work- sheets containing a set of questions listed in Appendix 1. Participants wrote their individual reflections on post-its for each question and shared them on a large sheet, from where a discussion arose in the group. A facilitator accom- panied each group taking notes and keeping time. Each question ended with a presentation of the discussion in plenary, during which participants had the chance to ask questions to the different stakeholders present. Concluding the session, participants filled out a second questionnaire (WS1-B) to mea- sure how their perceptions had changed after the session having gained more knowledge and further perspectives.
Session 2: VR Mixed-Reality Simulation During the second session, participants were offered a mixed-reality simu- lation of milliAmpere2 in the canal. The simulation was conducted using a full-size mock-up of milliAmpere2, making it a tangible virtual reality lab fur- ther described in Alsos et al. (2022). Three different scenarios were simulated (Appendix 1). A facilitator followed each participant during the simulation and took notes on their behaviour, actions, and comments using the think- aloud protocol (Nielsen, 1994). After the simulations, participants filled out a questionnaire (VR) to capture their immediate thoughts and perceptions. The session concluded with a focus group interview moderated by a facilita- tor. The interview was semi-structured, and the facilitator had an interview guide at hand. The session was conducted in smaller groups (5–7 partici- pants) spread over three different dates, where the participants could choose a preferred date.
Figure 2: Ferry trial with the milliAmpere2 in the canal in Trondheim.
Session 3: Ferry Trial During the third session, participants experienced a real ferry trial with the prototype milliAmpere2 (Figure 2). The trials were spread across three dates in smaller groups and consisted of two parts. In the first part, a “normal” ferry crossing without any obstacles was conducted. In the second part, a leisure boat demonstrated crossing traffic and caused milliAmpere2 to stop. Two safety operators onboard monitored the autonomous system during all the trials and answered questions the participants had. Two facilitators noted the participants’ behaviour, actions, and comments. As in the VR session, a questionnaire (FERRY) was handed out, and a semi-structured focus group interview concluded the session.
Session 4: Reflective Table Discussions In the final session, participants were divided into smaller groups and asked a set of open-ended questions to reflect on and give recommendations for the further development of the autonomous ferry concept. The table discussions were conducted in the same manner as in session 1 (Appendix 1) and were followed by the plenary presentation of the recommendations discussed in the groups. At the end of the session, participants filled out a final questionnaire (WS2).
COLLECTING QUESTIONNAIRE, INTERVIEW AND VIDEO DATA Throughout the sessions, several different data samples were collected, as summarized in Table 1.
Table 1. An overview of the data collected in the different sessions.
Questionnaire (VR) Video Facilitator notes Interview notes
Questionnaire (FERRY) Video Facilitator notes Interview notes
Questionnaires Five different questionnaires were handed out over the four sessions, each consisting of a quantitative and qualitative part. Some questions were
A Mixed-Method Approach for Assessing User Perspectives in Autonomous Urban Ferries 723
repeated through several or all the questionnaires to investigate if and how the different sessions would influence the participants. The questions were developed by the research team building on similar studies within autonomous transportation such as Goerlandt (2020), Stålhane, Myklebust and Haug (2021), and revolved around the use of public transport, interest in and experience with autonomous transportation, perceived safety, and usage. In the quantitative part, a 5-point Likert scale or “Yes” or “No” was used. In the qualitative part, open-ended questions on thoughts, needs, expecta- tions, and concerns related to the autonomous ferry were asked. A detailed overview of the questions can be seen in Appendix 1.
Worksheets During the table discussions, participants used large-format worksheets (DIN A1) with open-ended questions for discussion. The questions allowed for individual reflection and freedom of thought before starting a group discus- sion, where different perspectives were built on and expanded. A detailed list of the questions can be found in Appendix 1.
Interview Guides Semi-structured group interviews were conducted as part of the method’s explorative nature. An interview guide was created for the debriefing inter- views in sessions 2 and 3, with group sizes varying between 4–7 participants. The interview guide started with an easy question to make the participants comfortable before asking more in-depth questions of interest to the matter of investigation. A detailed list of the questions can be found in Appendix 1.
Video During the immersive sessions (2 and 3), cameras were installed in front of the rig and on the side to capture the movement of the participant in two dimensions. The VR view of the participant was also recorded as depicted in Figure 3. During the ferry trial in session 3, cameras were installed under the mast and in the front and back of the ferry.
Figure 3: Video footage from the VR mixed-reality simulation.
ANALYSIS The collected material underwent semantic thematic analysis (Braun and Clarke, 2006), incorporating written content from questionnaires, work- sheets, facilitator and interview notes, and video analysis for triangulation. The sessions, spanning immersion, collaboration, and discussion over time, yielded immediate feedback and perceptions, together with retrospection. Thematic analysis of combined data samples generated hypotheses on factors affecting acceptance and the impact of perceived safety, including infor- mation, the presence and role of a safety host, and ferry design features. Consolidating diverse data through thematic analysis allowed for an in- depth exploration of first-hand interactions with technology and reflections over time. The combination of individual thoughts and group discussions enriched the analysis, providing insights into perceptions of implementing autonomous urban ferries in a specific context.
DISCUSSION
The proposed and novel mixed-method approach builds upon citizen engage- ment with a set of immersive and reflective sessions. The approach col- lects rich and mixed data that allows for triangulation during the analysis. Nevertheless, there are several points we would like to discuss further.
Opportunities With the Approach Research on user perspectives of autonomous public transportation has been conducted using a variety of methods. Methods in studies on the closely related field of autonomous buses range from surveys, and telephone interviews (Goldbach et al., 2022) to field studies with physical proto- types in specific contexts (Salonen and Haavisto, 2019; Mouratidis and Cobeña Serrano, 2021). The use of physical prototypes often involves surveys or interviews to gather immediate participant responses, as demonstrated by Stålhane, Myklebust and Haug (2021), who combined focus groups and questionnaires before and after bus trials for more detailed insights.
In contrast, research on user perspectives on autonomous ferries is still limited. Existing studies rely on online/telephone surveys and interviews (Goerlandt, 2020; Munim et al., 2022), potentially lacking immersion in the ferry concept. The unique aspects of being on board an autonomous ferry, exposed to wind, waves, and currents, necessitate immersive trials to obtain deeper insights into user perceptions. The introduction of autonomous fer- ries has broader implications for a city, requiring a holistic approach that considers various perspectives besides safety and security.
The proposed mixed method approach emphasizes three key dimensions. Firstly, the chronological order of sessions progresses from theoretical to increasingly realistic scenarios, allowing participants to reflect on the overall concept and track changes in their thoughts and perceptions. This enhances the validity of the data. For instance, a study following this approach (Pantelatos et al., 2023) found that the participants’ views on the importance of a human host onboard changed between the immersive sessions with a vir- tual milliAmpere2 and the real milliAmpere2, leading to recommendations
A Mixed-Method Approach for Assessing User Perspectives in Autonomous Urban Ferries 725
for a transparent ferry superstructure and a human host at quay areas in the reflective session. Familiarity with the context and prototype provided trust- worthiness, yielding valid recommendations to ensure safety and security for further exploration.
Secondly, conducting sessions with spaced intervals gives participants time for individual reflection, potentially minimizing group influence on opinions. The mixed methods research design and session sequence provide valuable insights into the impact of time on forming perceptions of a novel concept.
Thirdly, the approach captures both individual and group perspectives through questionnaires, individual note-taking, and table discussions. While individual methods ensure independent expression, group discussions facili- tate in-depth exploration of diverse topics, including safety, trust, job security, and broader societal and environmental implications on the specific context.
The proposed mixed method approach produces multidimensional data for triangulation, including surveys, interview notes, personal notes, observa- tions, video observations, individual interviews, and group interviews. This broad exploratory method, while specific to a context, serves as a founda- tion for more targeted studies and explores the feasibility of implementing a ferry system. The analyzed data can be highly relevant for academia, the local municipality, and providers of autonomous infrastructure.
Limitations of the Approach Besides the mentioned opportunities, there are limitations to consider. Con- ducting the approach as in Pantelatos et al. (2023), indicated that the discussions in later sessions became somewhat repetitive. Furthermore, the group dynamics could potentially introduce positive bias as participants would meet over several sessions, especially if the hosts are directly involved in the development of the ferry. The approach can thereby be seen to be optimized further. The resource-intensive nature of the approach necessi- tates close collaboration among various stakeholders, such as researchers, technicians, facilitators, maritime authorities, harbour authorities, and the municipality etc. Furthermore, it limits the number of participants.
In the current setup 15 participants required 5 facilitators and techni- cal crew in the immersive sessions. A more streamlined setup allowing for a higher number of participants would be more likely to reach theoretical saturation within the individual context. In turn this could give more gen- eralizable conclusions of overarching themes such as environmental impact and societal aspects.
Despite these limitations, the explorative nature of the research, both in terms of subject as well as the method, makes us believe that there is ground to build on this method. The rich data the method provides can be used for informed decision-making within the specific context and can guide further in-depth research.
Acknowledging limitations to the approach, the authors believe that the complexity of introducing autonomous ferries requires rich and multidimen- sional data of user perceptions in order to research and develop solutions benefiting society.
APPENDIX 1 Session 1 Information and Table Discussions
ACKNOWLEDGMENT
This research was funded by the Research Council of Norway (RCN) through various projects: TRUSST (project number 313921); MAS (326676); MIDAS (331921), and SFI AutoShip (309230). The citizen engagement activities were developed in collaboration with Mission Publiques (missionspubliques.org), a professional citizen engagement consultancy. The authors wants to thank
A Mixed-Method Approach for Assessing User Perspectives in Autonomous Urban Ferries 727
everyone participating and being involved in the citizen engagement, as well as the reviewers for valuable comments and suggestions.
REFERENCES
Alsos, O. A. et al. (2022) ‘NTNU Shore Control Lab: Designing shore control centres in the age of autonomous ships’, Journal of Physics: Conference Series, 2311(1), p. 012030. Available at: https://doi.org/10.1088/1742-6596/2311/1/012030. Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative
Research in Psychology, 3(2), pp. 77–101. Available at: https://doi.org/10.1191/ 1478088706qp063oa. Climate Change | UN-Habitat (no date). Available at: https://unhabitat.org/topic/cl imate-change (Accessed: 2 February 2023). EU Comission (2018). Available at: https://eur-lex.europa.eu/legal- content/EN/TXT/?uri=CELEX%3A52018DC0283 (Accessed: 29 January 2024). Goerlandt, F. (2020) ‘Maritime Autonomous Surface Ships from a risk governance
perspective: Interpretation and implications’, Safety Science, 128, p. 104758. Available at: https://doi.org/10.1016/j.ssci.2020.104758. Goldbach, C. et al. (2022) ‘Towards autonomous public transportation: Attitudes and intentions of the local population’, Transportation Research Interdisciplinary Perspectives, 13, p. 100504. Available at: https://doi.org/10.1016/j.trip.2021. 100504. Hock, P. et al. (2018) ‘How to Design Valid Simulator Studies for Investigating User
Experience in Automated Driving: Review and Hands-On Considerations’, in Pro- ceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. AutomotiveUI ’18: 10th International Con- ference on Automotive User Interfaces and Interactive Vehicular Applications, Toronto ON Canada: ACM, pp. 105–117. Available at: https://doi.org/10.1145/ 3239060.3239066. Mouratidis, K. and Cobeña Serrano, V. (2021) ‘Autonomous buses: Intentions to use, passenger experiences, and suggestions for improvement’, Transportation Research Part F: Traffic Psychology and Behaviour, 76, pp. 321–335. Available at: https://doi.org/10.1016/j.trf.2020.12.007. Munim, Z. H. et al. (2022) ‘Public Perception on Safety of Autonomous Ferry in
the Norwegian Context’, in 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). 2022 IEEE International Con- ference on Industrial Engineering and Engineering Management (IEEM), Kuala Lumpur, Malaysia: IEEE, pp. 1027–1032. Available at: https://doi.org/10.1109/ IEEM55944.2022.9989557. Nielsen, J. (1994) Usability engineering. Morgan Kaufmann. Olphert, W. and Damodaran, L. (2007) ‘Citizen Participation and engagement in the Design of e-Government Services: The Missing Link in Effective ICT Design and Delivery’, Journal of the Association for Information Systems, 8(9). Available at: https://doi.org/10.17705/1jais.00140. Pan, X. et al. (2017) ‘Virtual to Real Reinforcement Learning for Autonomous Driv-
ing’. arXiv. Available at: https://arxiv.org/abs/1704.03952 (Accessed: 29 January 2024). Pantelatos, L. S. et al. (2023) ‘The Role of a Human Host Onboard of Urban Autonomous Passenger Ferries’, in. ACHI 2023: The Sixteenth International Con- ference on Advances in Computer-Human Interactions, International Academy, Research and Industry Association (IARIA).
Reddy, N. P. et al. (2019) ‘Zero-Emission Autonomous Ferries for Urban Water Transport: Cheaper, Cleaner Alternative to Bridges and Manned Vessels’, IEEE Electrification Magazine, 7(4), pp. 32–45. Available at: https://doi.org/10.1109/ MELE.2019.2943954. Salonen, A. O. and Haavisto, N. (2019) ‘Towards autonomous transportation.
Passengers’ experiences, perceptions and feelings in a driverless shuttle bus in Finland’, Sustainability, 11(3), p. 588. Smogeli, Ø. (2023) ‘Autonomous Urban Passenger Ferries—A New Mobility Mode in Need of Appropriate Regulation’, in T. M. Johansson et al. (eds) Autonomous Vessels in Maritime Affairs: Law and Governance Implications. Cham: Springer International Publishing (Studies in National Governance and Emerging Tech- nologies), pp. 187–212. Available at: https://doi.org/10.1007/978-3-031-24740- 8_10. Stålhane, T., Myklebust, T. and Haug, I. S. (2021) ‘Trust and Acceptance of Self-
Driving Busses’, in Proceedings of the 31st European Safety and Reliability Conference (ESREL 2021). Proceedings of the 31st European Safety and Reli- ability Conference, Research Publishing Services, pp. 2194–2201. Available at: https://doi.org/10.3850/978-981-18-2016-8_147-cd. U.S. Department of Transportation (2018) ‘Preparing for the future of transportation: Automated vehicle 3.0’. Available at: https://www.transportation.gov/av/3 (Accessed: 29 January 2024).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Factors of Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-24-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional