Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Comparative Study on User Experience of ATM Cardless Withdrawal Services

1 Master Program of Innovation and Design, Taipei Tech., Taipei,10608, Taiwan ↗
2 Doctoral Program in Design, College of Design, Taipei Tech., Taipei, 10608, Taiwan ↗
3 Department of Industrial Design, Taipei Tech., Taipei,10608, Taiwan
Publication year
Digital Object Identifier10.54941/ahfe1001700

Abstract

In response to the development of financial technology, the financial industry has already widely used self-service technology to ensure consumers’ convenience when conducting transactions. Banks have then successively applied cardless services to ATMs to provide advanced and convenient transactions for consumers. However, the use of cardless services has not been universalised. Therefore, this research compares the usability of well-known ATM cardless withdrawal services and explores why it has not been universalised. This study evaluates the usability and user experience design of the top three ATMs in Taiwan. The evaluation requires three methods. Firstly, we request users to perform two experimental task operations. The tasks include cardless service applications and cardless cash withdrawals. We also conducted behavioural mapping while users were performing their tasks to obtain usability insight. Next, we use the Single Ease Questionnaire, System Usability Scale, and Net Promoter Score to obtain users' thoughts on user behaviour, usability issues, difficulty, and satisfaction while operating the ATM cardless service interface. Lastly, we conduct a semi-structured interview to obtain insights through user evaluation, advantages and disadvantages of the interface.As a result, we found that Bank A had the best usability performance among the three banks, with a SUS score of 81 (B class) and an NPS score of 30. Furthermore, Bank A provides two entrances in the cardless service application, while Banks B and C only provide one entrance. The results show that various entrances align with the user's cognition of the operation process and can effectively reduce the time of operational tasks and the number of errors. However, in the cash withdrawal task, the information architecture of Bank A's banking app for cardless withdrawal is a layer lesser than Bank B and C. Also, users only need three steps to obtain the cardless withdrawal serial number in Bank A's banking app. In comparison, Bank B need six steps, whereas Bank C need seven steps. Therefore, short information architecture and process can effectively improve usability and users' satisfaction.However, in the cardless services application task, Bank's A application entrance is placed at a level beyond the user's expected cognition, resulting in a gradual decline in user confidence during the operation. In addition, users are unaware that bank cards are required as an authentication tool required for applying cardless services. As a result, most users feel confused whenever they are required to provide their bank card during the application process. Also, users are unable to complete tasks cognitively in the shortest process. Furthermore, all three ATM systems did not provide clear guidance steps at the appropriate time. Finally, even if Bank A's cardless withdrawal has the best performance, most users still prefer traditional password authentication for ATM transactions.The research results help understand users' operation behaviour of ATMs' current cardless service interface as a reference for subsequent design improvements.

INTRODUCTION

In today’s world, financial institutions are using mobile device authentication technology to develop cardless services. Since 2017, after the Taiwan’s Fina- ncial Supervisory Commission has approved cardless ATMs, various banks have developed cardless services, including biometric identification and com- munication accounts. Cardless services, however, rely primarily on mobile banking app serial numbers to verify users’ identities. The 2021 ATM & Self-Service Software Trends report indicates that 39% of respondents intend to move to cardless or appointment-based payment methods in the future. Moreover, 18% of the respondents also indicated that they would improve ATMs customer experiences in the future (All & Harper, 2021). Hence, it is evident that ATMs have to be user-friendly to improve customer experience.

According to Coventry et al. (2003), users that successfully completed their first task report higher satisfaction levels, thereby increasing their willingness to use the system again. Zhang (2019) found that forgetting an ATM card is a leading reason for using cardless services. Despite the positive experi- ences, most users still prefer an ATM card. According to De Luca et al. (2010), operation time and steps of ATM interface affect users’ satisfaction. The researchers also found that identity authentication only took up a small proportion of the total operation time and recommended keeping it within 10%. In fact, 90% of the operating time is spent browsing the interface’s main menu or waiting for the ATM to operate.

© 2022. Published by AHFE Open Access. All rights reserved. 133

This research evaluates the cardless service interfaces of three banks based on the most ATMs in Taiwan, including the operation process, interface information, and information architecture (IA). The result will serve as the basis for future interface optimizations, thus increasing the willingness to use cardless withdrawals in the future.

METHODOLOGY

This research evaluates the cardless service interfaces of three banks: Bank A, Bank B, and Bank C (see Figure 1). Furthermore, it evaluates the usability of the three banks’ cardless service primary authentication method, mobile banking app serial numbers.

In this experiment, 30 participants aged 20 to 39 years old and have no prior experience with cardless services were recruited. Participants were ran- domly assigned to one of the three banks. All experiments were conducted online through a remote desktop software. The ATM and mobile banking app interfaces will be presented at their actual size. Each participant will then perform two tasks: (1) You have recently learned that cardless services are a trend in FinTech. Therefore, please activate X bank’s cardless service; (2) You realized that you left your wallet at home on your way for a meal. For this reason, you will need to withdraw NTD 1,000 through the X bank’s cardless service using mobile banking app serial number.

There are four phases to the experiment: (1) The participants are asked to fill out basic information and ATM-related experiences; (2) Then, they are asked to complete specific scenario tasks; (3) After completing both tasks, the participants were asked to fill out the System Usability Scale (SUS) and the willingness questionnaire; (4) Finally, we conducted a semi-structured interview to understand participants’ evaluation and insights of the interface for future design improvements.

RESULTS

As a result, we found that Bank A was the best performing bank among the three banks in terms of task operation time, error rate, and SUS value. Conversely, Bank C had the worst performance (see Table 1).

For task 1, “Activation of Cardless Service”, Bank A, B, and C has recorded error rates of 20%, 23%, and 66.9%. In addition, all of the participants were unaware that the ATM card is a primary identity authentication tool, causing confusion when asked to insert the card during the operation. On the home- page of the cardless service function on the ATM, Bank A offers two types of user flow (see Figure 2). Conversely, Bank B and C only provide one user flow on the homepage. As a result, Bank A’s participants completed the task using the sub-route, reducing their errors and operational time. However, they expected to locate the activation entrance in the second step. Neverth- eless, the entrance was located in the fourth step, causing participants’ to lose confidence as the number of steps increased. Conversely, participants in Bank B and C (see Figure 3a and 3b) expect to find the activation entra- nce by clicking on the “Press here for cardless transaction” button. However,

Comparative Study on User Experience of ATM Cardless Withdrawal Services 135

Figure 1: ATM Interface of Bank A, B and C.
Original PDF Figure 1:
Table 1. Average operation time, error rate and SUS score of three banks in two tasks.

Table available in the original PDF paper.

Task 1 Task 2 Bank A Bank B Bank C Bank A Bank B Bank C

117s 266s 586s 88s 292s 148s

Average operation time (seconds) Average error rate 20% 23% 66.9% 2.8% 51.11% 7.27%

Figure 2: Cardless service application user flow of Bank A ATM.
Original PDF Figure 2:

the entrance will only appear after the participants inserted their ATM card, which increases the task time and error rate for Bank B and C. Additionally, Bank C has the most complicated activation process. Participants have to swi- tch between the ATM and the mobile app twice during the activation process. Further, Bank C did not outline all the steps on the instruction page. There- fore, 50% of the participants were unaware of which devices to switch and how to carry on the task, resulting in a task time and error rate that were double the rate of Banks A and B.

For task 2, “Cardless Withdrawal”, Bank A had an error rate of 2.8%, Bank B had a rate of 51.11, and Bank C had a rate of 7.27%. Bank A’s mobile banking app for cardless withdrawal IA has a layer lesser than Bank B and C. Users require only three steps to obtain the cardless withdrawal

Figure 3: ATM Homepage of Bank B and C.
Original PDF Figure 3:

serial number. However, Bank B and C’s mobile banking app requires betw- een 6 and 7 steps to complete. Participants’ interviews indicate that a simpler IA and process can improve the operation process’ usability and satisfaction. Though Bank B and C are only a step apart, the average operation time is nearly 150s. We discovered that all participants in Bank B had at least made an error on the navigation view of the mobile banking app. 90% of partici- pants expected to find the “Cardless Withdrawal” keyword in the navigation view. However, the IA was too deep and did not meet the users’ usage habits. Further, Bank B also require users to input the withdrawal amount to verify their identity. However, most participants in interviews mentioned that this step could be misconstrued as a way to amend the withdrawal amount. In addition, 50% of the participants attempted to enter other amounts, failing their identity verification and having to start again, proving that withdrawal amounts could confuse the purpose of identity verification.

All participants took more time to complete their cardless withdrawal tasks despite instruction issues. Bank A’s instructions failed to catch the subjects’ attention as the instructions below the illustration led participants to beli- eve it was irrelevant to their current task, resulting in 60% of them ignoring the instructions (Figure 4a). In task 2, Bank B’s ATM requested a withdra- wal serial number that did not provide a way to obtain one, leading to 80% of the subjects not knowing how to proceed (see Figure 4b). On the other hand, Bank C provided instructions on obtaining a serial number on a page before inputting the withdrawal serial number, resulting in 50% of partici- pants ignoring the instructions at first but returning to the previous page for guidance (see Figure 4c).

Based on the SUS questionnaire, Bank A scores (M = 77.25, SD = 16.22), Bank B scores (M = 31.5, SD = 21.42), and Bank C scores (M = 20, SD = 15.59). Therefore, Bank A (77.25) scored B and met the usability stan- dard (68). However, Bank B (31.5) and C (20) failed the usability test and were rated F. Thus, there is still room for improvement regarding the cardless withdrawal interface of the ATMs of the three banks.

DISCUSSION

In this study, participants were classified according to their ATM experi- ence. The frequency of operating an ATM in a month was divided into

Comparative Study on User Experience of ATM Cardless Withdrawal Services 137

Figure 4: (a) Withdrawal serial number input of Bank A, (b) Withdrawal serial number input of Bank B, (c) Cardless withdrawal instruction page of Bank C.
Original PDF Figure 4:
Table 2. Average number of errors per participants and SUS score of three banks in both task.

Table available in the original PDF paper.

SUS score Frequency of operating ATMs per month Average number of errors per participants Task 1 Task 2

Inexperienced (1-2 times) 5.17 2.5 38.54 Experienced (3-4 times) 4.06 1.72 45 High experienced (5 times and above) 2.61 1.23 46.8

Figure 5: Total number of errors of three banks in different ATM experience in Task 1 and 2.
Original PDF Figure 5:

inexperienced (1-2 times), experienced (3-5 times), and high-experienced (five or more). Among participants, 40% were inexperienced subjects, followed by 33.33% experienced subjects and 26.67% high-experienced subjects. The study investigated how subjects’ experiences with ATMs affected task errors and SUS values. As a result, ATM task performance and SUS performance improved with experience (see Table 2) Also, results showed that inexperienced participants were most likely to make mistakes. For instance, in Task 1, the inexperienced participants made 70 errors, the experienced participants made 42 errors, and the high- experienced participants made 18 errors (see Figure 5a). On the other hand, in Task 2, the inexperienced participants made 30 errors, the experienced par- ticipants made 16 errors, and the high-experienced participants made nine errors (see Figure 5b).

Figure 6: Total number of errors of three banks in different stages in Task 1 and 2.
Original PDF Figure 6:

This study further found that in task 1, the error rate encountered by the three banks during the “getting to the activation entrance” phase was greater than the “verification setup” phase (see Figure 6a). According to inexperie- nced participants of Banks B and C, “cardless service” is one of the essential functions of ATMs. Nevertheless, it was placed at a difficult location where inexperienced subjects were unable to find the correct entrance. Similarly, in task 2, the error rate experienced by the three banks during the “obtaining the withdrawal serial number” phase is higher than the subsequent “identifi- cation verification” phase (see Figure 6b). The inexperienced participants of Banks A and B missed the instructions for obtaining the serial number for cardless withdrawals. Meanwhile, Bank C provided instructions on the pre- vious page inadvertently ignored by the inexperienced, resulting in them not knowing how to proceed.

Furthermore, this study examined participants’ ATM experiences and their scores on SUS items. As per James R. Lewis (2009) research, items Q4 (I think that I would need the support of a technical person to be able to use this system) and Q10 (I needed to learn a lot of things before I could get going with this system) from the SUS questionnaire can be classified as learnability factors. In Q4, Bank A had an average score of 1.6, Bank B had 3.4, and Bank C had 3.6. 30% of subjects scored below the Q4 average in Bank A, 50% in Bank B, and 60% in Bank C. In Q10, Bank A has an average score of 1.5, Bank B has an average score of 3.3, Bank C has an average score of 4.0. 30% of subjects scored below the Q10 average in Bank A, 50% in Bank B, and 80% in Bank C. It is apparent that the cardless withdrawal interface of Bank C is more difficult to learn than the interfaces of banks A and B.

In this study, we added Q11 (I think cardless withdrawal is helpful for me) and Q12 (I prefer traditional password verification over cardless withdra- wals) to the SUS questionnaire to compare with Q1 of the SUS. According to the results, high-experienced consumers are more likely to use cardless withdrawals in the future. Meanwhile, 80% of Bank A’s participants sco- red higher than the average SUS score (68), and 70% thought it was helpful to conduct a cardless withdrawal. However, 50% of the participants prefer traditional passwords to cardless withdrawals. The participants conside- red the traditional password verification more convenient and faster than

Comparative Study on User Experience of ATM Cardless Withdrawal Services 139

the cardless withdrawal, resulting in only using the cardless withdrawal in emergencies.

CONCLUSION

Based on the results of this study, Bank A had the best performance out of the three. Bank A provides two entrances for its cardless service, which allows for a shorter activation and withdrawal process, reducing the operational time and number of errors. However, when activating for cardless services, all participants did not know that a bank card must be inserted to verify their identity. Additionally, the study found that inexperienced participants made the most errors. Meanwhile, during the “getting to the activation entrance” phase of task 1 and the “identity verification” phase of task 2, the inexperie- nced made the most mistakes. The three banks lack instructions on activating cardless service in task 1 and obtaining the serial number in task 2. Finally, results from the SUS score show that Bank C has lower learnability than Bank A and B. Lastly, the SUS result shows that Bank C has lower learnabi- lity than Bank A and B. A complex application process for Bank C, frequent switchovers between ATMs and mobile banking apps, and inaccurate instru- ctions increase the difficulty of tasks. In addition, participants asserted that cardless withdrawals are not more convenient and faster than traditional pas- sword verification. As a result, they prefer traditional password verification over cardless withdrawals. These findings provide a basis for future interface optimization, hoping to improve the willingness to use cardless withdraw- als in the future by considering the differences in the situation between users with different operating experiences at ATMs.

REFERENCES

All, A., & Harper, T. (2021). 2021 ATM & Self-Service Software Trends. Networld Media Group. 36–37 Coventry, L., De Angeli, A. and Johnson, G., 2003, April. Usability and biometric

verification at the ATM interface. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 153–160). De Luca, A., Langheinrich, M. and Hussmann, H., 2010, July. Towards understanding ATM security: a field study of real world ATM use. In Proceedings of the sixth symposium on usable privacy and security (pp. 1–10). Lewis, J.R. and Sauro, J., 2009, July. The factor structure of the system usability scale.

In International conference on human centered design (pp. 94–103). Springer, Berlin, Heidelberg. Zhang, Z., 2019. ATM Cardless Customer Experience in Banking Digitalization, Taking Cathay United Bank, CTBC Bank, Taishin International Bank For Example. National ChengChi University MBA Program, 1–63.

Cover of the proceedings volume Usability and User Experience

Explore the complete digital volume

Usability and User Experience

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-15-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →