Chen Jinghong, Xin Hu (2024). Mobile Phone Accessibility Solution for People with Upper Limb Dysfunction. In: Human Factors in Design, Engineering, and Computing. AHFE Open Access, vol. 159, pp. 869–877. https://doi.org/10.54941/ahfe1005653
With the rapid development of smartphones and digital technology, it has become the norm for people with upper limb dysfunction to use smartphones to access the Internet. This study applies the methodology of qualitative research, combining semi-structured in-depth interviews and observation strategies, to examine the daily lives of people with upper limb dysfunction in detail, and to compare and analyze existing solutions, aiming at exploring better solutions for smartphone accessibility, and proposing rationalized suggestions for future development, in order to promote the development of cell phone use experience for people with upper limb dysfunction. In this study, the upper limb dysfunction population is firstly divided into six categories according to the cause of the injury, and the research object does not include the people who have no arm or complete loss of upper limb function due to various reasons. The study first focused on existing smartphone accessibility solutions, focusing on a number of features including voice recognition technology, screen reader technology, "accessibility menu" buttons, magnification, high contrast and color adjustment, etc. The results show that although existing accessibility solutions provide a certain level of support for people with upper limb dysfunction, they do not provide the same level of support for people with upper limb dysfunction, but they do not provide the same level of support for people with upper limb dysfunction. The results show that although existing accessibility solutions provide a certain degree of support for people with upper limb dysfunction, there are still a number of limitations, including, but not limited to, shortcomings in technical applicability, user satisfaction and personalized design. Future accessibility technologies should focus more on personalization and ease-of-use design, and could incorporate mechanisms such as eye-tracking technology and head movement control to meet the needs and abilities of different users. The findings of this study provide a useful solution to the digital connectivity problem for people with upper limb dysfunction, and also provide important references and insights for the continued development of accessible technology and social progress.
functions such as keeping in touch socially, accessing information, economic transactions and entertainment. However, for people with disabilities, there may be many barriers to using smartphones. According to China’s sixth national population census and the second national sample survey of persons with disabilities, as of 2010, the total number of persons with disabilities in China was approximately 85.02 million, of which 24.72 million were persons with physical disabilities, accounting for 29%, and among persons with physical disabilities, those with upper limb dysfunction faced particularly significant barriers to the use of smartphones. 2012 saw the State Council issue the first special policy on the construction of a barrier-free environment in China, with the aim of providing a barrier-free environment for persons with disabilities. In 2012, the State Council issued China’s first special policy regulations on accessibility construction, the Regulations on Barrier- Free Environment Construction, which clearly states that manufacturers of telecommunications terminal equipment should provide technologies and products that can be connected with barrier-free information exchange services (Wen Jiabao, 2012). At present, cell phone system designers have incorporated accessibility modes for assisted operation in order to ensure that the disabled population has a better experience when using smartphones, but the design of the accessibility modes is more targeted at the hearing- impaired and visually impaired population, and does not provide much help to the physically disabled, especially the upper limb dysfunctional population; therefore, providing the upper limb dysfunctional population with an accessible smartphone solution can help them to Therefore, providing accessible smartphone solutions for people with upper limb disabilities can help them better participate in social activities, significantly improve the quality of life of people with upper limb disabilities, and at the same time reduce the digital divide, making society more fair and inclusive.
Figure 1. Domains of human systems integration. (Adapted from U.S Air Force, 2005).
Figure available in the original PDF paper.
CLASSIFICATION OF PEOPLE WITH UPPER LIMB DYSFUNCTION Upper extremity dysfunction encompasses conditions of limited or impaired movement of the hand, wrist, and forearm up to the shoulder, which may stem from congenital factors or acquired factors such as disease or trauma. They may experience challenges with muscle strength, dexterity, and coordination, such as the inability to easily perform screen swipes, button taps, or other movements that require meticulous hand coordination, which directly affects their ability to operate a smartphone. Upper extremity dysfunction can have a variety of causes and is often categorized based on etiology, site of involvement, and degree of loss of function. Based on the cause of the impairment, it can be divided into the following six categories:
1. Dysfunction due to neurological disorders: e.g. post-stroke, cerebral palsy, multiple sclerosis, peripheral nerve injury, etc. 2. Diseases of the musculoskeletal system: including fractures, arthritis, rotator cuff injuries, muscle or tendon tears, etc.
A Mobile Phone Accessibility Solution for People With Upper Limb Dysfunctions 871
3. Congenital defects: such as congenital limb hypoplasia, congenital armlessness or short limbs, etc. 4. sequelae of trauma or surgery: for example, car accident injuries, contracture after burns, functional limitation after surgery, etc. 5. Circulatory diseases: e.g. limb swelling after deep vein thrombosis, limiting movement. 6. Inflammatory diseases: e.g. synovitis, myositis, etc.
This study does not include people with armlessness or complete loss of upper limb function due to various reasons.
CHALLENGES FACED BY PEOPLE WITH UPPER LIMB DYSFUNCTION In order to help people with upper limb dysfunction to better use smartphones, cell phone system manufacturers have developed a number of accessibility solutions. These solutions include voice recognition, haptic feedback and text conversion technologies, which are designed to provide alternatives to traditional touch-screen interactions, making it easier for people with upper limb dysfunction to use smartphones. Through these technologies, people with upper limb dysfunction can use their smartphones more autonomously and conveniently to fulfill their daily needs and improve their quality of life. Despite the significant progress of existing accessibility technologies, there are still many limitations in practical applications, such as long time holding, fine motor control, and interface icon is too small, which need continuous improvement and research.
RESEARCH METHODOLOGY This study utilizes the methodology of qualitative research, combining semi-structured in-depth interviews and observation strategies to carefully examine the daily lives of people with upper limb dysfunction. The study focuses on the difficulties encountered by this group in using smartphones, as well as their needs and experiences of accessible technology applications. By analyzing the smartphone usage patterns and preferences of these users, the potential development path of mobile accessibility models is explored.
The study conducted semi-structured in-depth interviews between September 2023 and November 2023 with 20 residents living in Mainland China and Macau. These participants were active smartphone users and had varying degrees of upper limb dysfunction. The study sample showed diversity in gender, age and educational background, covering 8 females and 12 males, with an age span from 15 to 58 years old, an educational level from no qualifications to postgraduate studies, and experience in smartphone use ranging from 2 to 13 years. Because participants were recruited through snowball sampling, there was no pre-determined gender and age ratio for the interviewees.
The interview process ensured informed consent from the participants and was guided by the principle of theory saturation. After each interview, the researcher carefully organized and analyzed the content, adjusting
872 Jinghong and Xin
subsequent interviews based on the results of the analysis until the new data no longer introduced new themes or ideas (see Table 1 for details).
Table 1. Basic profile of respondents (M for male F for female).
Table available in the original PDF paper.
Respondent No.
Age Brief Description of Disability
Careers Education Attainment Smartphone Use (Years)
10
M1 23 Missing fingers on the right hand
profession university undergraduate course three-year college 4 M2 19 Missing fingers on left hand schoolchildren high school education 7 M3 36 Limited function of the fingers of the left hand brand ambassador
schoolchildren junior high school 3
M4 15 Restricted hand and finger function
three-year college 10
M5 40 Limited function of the fingers of the left hand
business employee
9
out of work vocational secondary school
M6 42 Absence of the palm of the right hand with limited function of the fingers of the left hand
7
M7 32 Loss of right forearm profession university undergraduate course
4
M8 51 Loss of left forearm out of work vocational secondary school profession three-year college 6 M9 43 Missing fingers on the right hand
schoolchildren university
9
M10 22 Limited function of the fingers of the left hand
undergraduate course profession three-year college 9
M11 37 Restricted hand and finger function
bachelor’s degree 12
M12 33 Missing fingers on left hand business employee
schoolchildren high school education 4
F1 18 Restricted hand and finger function
three-year college 7
F2 32 Restricted hand and finger function
brand ambassador
5
F3 57 Missing fingers on left hand out of work not have 2 F4 41 Missing fingers on the right hand
profession vocational secondary school
7
F5 38 Restricted hand and finger function
brand ambassador
vocational secondary school
7
F6 58 Missing palm of right hand out of work high school education 3 F7 24 Restricted hand and finger function
business employee
university undergraduate course F8 39 Missing palm of left hand profession three-year college 10
EFFECTIVENESS OF CURRENT CELL PHONE ACCESSIBILITY SOLUTIONS FOR PEOPLE WITH UPPER LIMB DYSFUNCTION In the current mainstream cell phone operating systems, the design of the accessibility mode function mainly focuses on the use experience of the visually impaired and hearing impaired groups. After organizing the results of the interviews with the interviewees, we have derived the usability suggestions of the upper limb dysfunctional people on the existing accessibility mode function (see Table 2 for details).
A Mobile Phone Accessibility Solution for People With Upper Limb Dysfunctions 873
Table 2. Existing accessibility features of cell phones and perceptions of interviewed users.
Table available in the original PDF paper.
Existing Accessible Technologies
Technical Function Is it Effective for People With Upper Limb Dysfunction validity
Speech recognition technology
Control your smartphone with voice commands null
screen reader technology
Describing what is displayed on the screen via cell phone voice validity
“Accessibility menu” button
An accessibility button is always displayed at the top of the phone screen and contains frequently used functions null Zoom function Partial zoom in on the phone screen
null
High contrast and color adjustments
Adjusting the phone’s color display mode
SPEECH RECOGNITION TECHNOLOGY Speech recognition technology, also known as Automatic Speech Recognition (ASR), is the conversion of lexical content from human speech into computer- readable input, such as binary codes or sequences of characters (Hou Mang & Hu Xiaohong, 2018). Speech recognition technology allows users to control their smartphones through voice commands, thus avoiding the need for manual operations. For example, users can make phone calls, send text messages, search for information, or operate apps through voice commands, as typified by Apple’s Siri and Google’s Google Assistant. This feature is a very effective aid for people with upper limb dysfunction, effectively reducing the frequency of manual operations.
SCREEN READER TECHNOLOGY Screen reader technology can describe the content displayed on the screen, including text, icons and other interface elements, through voice feedback, so that users can understand what is happening on the screen through hearing, Apple’s VoiceOver and Android’s TalkBack are typical representatives of screen reader technology. Screen reader technology is mainly targeted at visually impaired people, and is not very helpful for people with upper limb dysfunction, and according to the feedback from the respondents, they do not use this function in their daily use.
“ACCESSIBILITY MENU” BUTTON The “Accessibility Menu” button always displays an accessibility button at the top of the phone’s screen, which the user can tap to open a window containing frequently used functions such as “Lock Screen”, “Power”,
874 Jinghong and Xin
“Volume”, “Screenshot”, etc. Some mobile phone systems also allow users to customize the functions included in the accessibility screen to suit their specific needs and reduce operating difficulties. Users can click on this button to open a window containing frequently used functions such as “lock screen”, “power”, “volume”, “screenshot”, etc. Some cell phone systems also allow users to customize the functions included in the accessibility interface to suit their specific needs and to alleviate operational difficulties. The “Accessibility Menu”button is designed to reduce the number of steps required to complete a function, and is an effective aid for people with upper limb dysfunction by reducing the number of steps required by their hands when using a cell phone.
ZOOM FUNCTION The Magnify feature is designed to help users with poor vision read and view screen content more easily. Users need to pinch the screen with two or more fingers to use this feature, which magnifies text, images, and other interface elements on the screen to make details more visible. The magnification feature is also intended for the visually impaired and requires complex gestures to activate, making it unsuitable for people with upper limb impairments.
HIGH CONTRAST AND COLOR ADJUSTMENTS In high contrast mode the background of the screen display will be darkened and the text and icons will become brighter, improving readability by increasing the contrast between the text and the background. At the same time, the high contrast mode helps to reduce eye fatigue when using the screen for long periods of time.
The Color Adjustment option allows users to change the way colors are displayed on the screen to suit different visual needs. For color-blind users, these settings can adjust colors to make colors that are normally difficult to distinguish easier to identify. Color adjustments may also include grayscale modes, which in some cases can reduce visual distractions on the screen.
High contrast and color adjustments are designed to enhance the experience of users with specific visual needs, and to reduce visual fatigue for normal users, but are less relevant to enhancing the experience of people with upper limb dysfunction.
DISCUSSION
By compiling the results from the target interview population, we found that existing solutions for cell phone accessibility features have limited effectiveness for people with upper limb dysfunction and, based on the descriptions of the interviewees and the level of development of the existing degree of technology, we propose the following recommendations for the development of a cell phone accessibility model:
1. Eye tracking is a technology that monitors and analyzes the focal position and movement of a user’s eyes. This technology tracks the position of the eyes and the direction of vision in real time by using a camera or special sensors within the cell phone to be able to pinpoint which part of
A Mobile Phone Accessibility Solution for People With Upper Limb Dysfunctions 875
the screen an individual is looking at (Zhou, Qiong & Huang, Yaxian, 2023). This technique would allow users to perform actions such as selecting, scrolling, and navigating through eye movements. This approach not only improves the naturalness of the interaction, but it is also an effective interaction alternative for those users who have limited movement of their upper limbs. Communication with the interviewees resulted in the following more common usage scenarios: Navigation and selection: eye-tracking technology allows users to navigate the smartphone interface, select an application or activate a specific function using only the movement of their eyes.
Text input: Together with the on-screen keyboard, users can use their eyes to select letters for text input.
Scrolling and Zooming: By tracking the vertical or horizontal movement of the eyes, users can scroll pages or zoom in and out of images and documents.
2. Head movement control is an accessibility technology that allows users to operate smart devices through the movement of their head.Head movement control technology typically utilizes the front-facing camera of the device to track the movement of the user’s head and translate it into navigation or selection commands. This control mechanism is particularly suited to users who are unable to interact using their hands or other body parts. Head movement control not only enhances the accessibility of the device, but also provides a more intuitive and natural way for users to interact. After communicating with the interviewees, the following common usage scenarios were identified: Screen navigation: Users can move their head to navigate around the smartphone screen, selecting applications or adjusting settings.
Selection and activation of functions: specific movements of the head can be set to tap, select or activate elements on the screen.
Text input assistance: In combination with an on-screen keyboard and predictive input technology, head movements can assist users with text input.
3. A more simplified and intuitive user interface is recommended to reduce the reliance on complex gestures or multi-step operations. The simplified interface should be designed to reduce the cognitive load on the user while maintaining the necessary functionality and ease of use. This simplification of the interface will especially benefit users who face operational barriers, allowing them to interact with the device more efficiently. In order to make smartphones more accessible to people with upper limb impairments, the “Simplified Interface Layout” feature enabled in the accessibility mode should follow the following design principles: Large icons and buttons: Using large icons and buttons makes it easier for users to click and reduces misuse. Icons and buttons should be clear, easily recognizable, and directly related to their function.
Clear, high-contrast text: Using large, high-contrast text ensures that it is easy to read, even in different lighting conditions. Avoid fancy fonts or complex text typography.
Simplified menus and navigation: Place the most frequently used functions in a prominent location and reduce menu levels and options. Use simple and intuitive navigation, such as clear forward and backward buttons.
876 Jinghong and Xin
Minimize on-screen content: Present the most essential functions and information on the screen, avoiding overstuffing the screen with too many elements. Provide a clear screen layout to avoid users feeling confused or overwhelmed.
Voice input and feedback: Provide convenient voice input functionality that enables users to search and enter commands by voice. Provide clear voice feedback so that users can get an instant response when operating.
Customization and Adjustability: Allow users to adjust the font size, color scheme and layout according to their needs. Provides easy-to-access setting options so that users can easily adjust the interface.
Touch-friendly design: Considering users with limited hand movement, the design should minimize the aspects that require fine manipulation. For example, avoid small sliders or elements that require precise touch.
By implementing these design principles, people with upper limb dysfunction can be made more convenient and comfortable when using smartphones. A simplified interface layout not only improves ease of use but also reduces cognitive load, thus enhancing the overall user experience.
CONCLUSION
Since the popularization of smartphones in China in 2013, the accessibility of cell phones has made great progress, but there is still much room for improving the experience of this special group of people with upper limb dysfunction. Although existing accessibility technologies such as voice recognition and “accessibility menu” buttons provide a certain degree of support for people with upper limb dysfunction, through user interviews and surveys, we found that many solutions, although technically feasible, have failed to gain high acceptance among users due to cumbersome operation or single functions that cannot meet individualized needs. The solution is highly recognized by the users. Therefore, the development of specific accessibility features for people with upper limb dysfunction can make it easier for them to complete their daily life and work tasks when using cell phones, thus improving their quality of life and self-confidence. In order to realize these possibilities, we need to continue to promote technological innovation and strive to build a more inclusive and friendly digital society, so that everyone can enjoy the convenience and opportunities brought by digital technology.
REFERENCES
Chen, Li-Fang & Huang, Chen-Feng. 2011 A new species of the genus Lepidoptera (Lepidoptera, Lepidoptera) from China. (2023). A Study on the Current Situation of Smartphone Use among the Elderly in Ethnic Areas under the Perspective of “Digital Divide”. Journal of Journalism Research (20), 43–45. Hou, Mang, Hu, Xiaohong & Zhao, Hangtao. (2018). Application of online speech
recognition technology in smart home. Information and Computer (Theoretical Edition) (24), 118–120. Jin, Yan, Ziqi Liu & Chongwu Bi. (2022). A study on the aging adaptation of APP in the context of information accessibility. Modern Intelligence (08), 96–106.
A Mobile Phone Accessibility Solution for People With Upper Limb Dysfunctions 877
Li, Dongxiao & Xiong, Mengqi. (2017). After “Accessible”: research and reflection on the accessible communication of new media. Zhejiang Journal (06), 199–206. Ni, Jia & Li, Fangyu. (2017). Design of a cell phone application for elderly dietary
health based on information accessibility. Industrial Design (01), 93–94. Ni, Q. & Zhou, G. C.. (2015). Navigation method of Android cell phone based on accessibility design. Journal of Hefei University of Technology (Natural Science Edition) (04), 490–494. Wei, S. Z., Liu, L. H. & Wu, Q. Q.. (2022). Smartphone literacy and digital inclusion
of middle-aged and elderly people. Library Research (03), 119–128. Weng, Yani, Zhang, Xiaodong, Rong, Yilong, Sun, Yuan, Zhao, He & Ye, Ya Fei.
(2022). Reflections on Barrier-Free Information Exchange in a Digital Society. Beijing Planning and Construction (02), 23–27. Wen, Jiabao. (2012). Barrier-free environment construction regulations. Wang, D. Ding. (2021). Visual accessibility in app interface design. Art & Design (Theory) (07), 88–90. Yang, Miao & Li, Ziyu. (2023). Research on aging design of smartphone interface.
Screen Printing (18), 65–67. Zhou, Qiong, Huang, Ya-Xian & Xu, Ya-Ping. 2011 A study on the use of eyetracking technology in the development of false information. Application of eye- tracking technology in the screening of false information. Library Forum 1–12. Zhao, Na. (2022). Mutualization: Technology Empowerment and Media Life of the
Visually Impaired: An Example of Smartphone Usage by the Visually Impaired. Media (06), 71–73+75.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Design, Engineering, and Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-35-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional