Jesse Leaman, Hung La, Bing Li (2024). An Update on International Robotic Wheelchair Development. In: Human Factors in Robots, Drones and Unmanned Systems. AHFE Open Access, vol. 138, pp. 47–56. https://doi.org/10.54941/ahfe1005007
Disability knows no boarders, so the development of assistive technology is an international effort. This review is a follow up to our previous comprehensive review (Leaman 2017) and a recent mini-review (Sivakanthan 2022). The transition from Power Wheelchair to Robotic Wheelchair (RW) with various operating modes like, Docking, Guide Following, and Path Planning for Autonomous Navigation, has become an attainable goal. Thanks to the revolution in Aerial Drones for the consumer market, many of the necessary algorithms for the RW software have already been developed. The challenge is to put forward a system that will be embraced by the population they are meant to serve. The Human Computer Interface (HCI) will have to be interactive, with all input and output methods depending on the user’s physical capabilities. In addition, all operating modes have to be customizable, based on the preferences of each user. Variables like maximum speed, and minimum distance to obstacles, are input conditions for many operating modes that will impact the user’s experience. The HCI should be able to explain its decisions in order to increase its trustworthiness over time. This may be in the form of verbal communication or visual feedback projected into the user’s field of view like augmented reality.Given the commitment of the international research community, and the growing demand, a commercially viable RW should become reality within the next decade. This will have a positive impact on millions of seniors and people with disabilities, their caregivers, and the governments paying for long-term care programs. The RW will pay for itself by reducing the number of caregiver hours needed to provide the same level of independence. The RW should even positively impact the economy since some users will have the confidence to return to work, and many will be able to participate in social events.
The demand for assistive mobility technology is greater than ever, in part because the population of survivors continues to grow, but also because the stigma surrounding disability is getting smaller. Millions of people around the world could benefit from a power wheelchair (PW) to get from one place to another. The first PWs had a seat with two electric motors and a joystick, but many potential PW users find it difficult or dangerous to use a traditional
joystick to navigate, so they rely on a companion to use the joystick to pro- pel their chair. A Smart Wheelchair (SW) is a PW that has been upgraded with a collection of environmental sensors, computers, and lights. A Robotic Wheelchair (RW) is one evolutionary step further because it also includes arti- ficial intelligence (AI) to process the sensory data in real-time, and additional motor controllers so that the data can be used for autonomous navigation. The transition to RWs, that cooperate with the user, is at least as important as that from manual to powered wheelchairs. It could be even more important, since it would mark a paradigmatic shift, rather than merely a technological improvement (Pineau, 2011). The RW will help people with severe disabili- ties today and in the future, lead safer, more independent, productive lives. According to Future Market Insights, the demand for RWs is anticipated to grow significantly in the next decade (2022–32).
Leaman (2017) is a review of the international SW research effort, with multiple groups developing prototypes to test a new input method, or perfect challenging operating modes. Other groups perform human trials to study the best ways to make people feel comfortable to use the new technology. We found that there was still a need for a RW upgrade kit that includes all the technology to convert a standard PW to a RW. The reason we believe an upgrade kit is better than a stand-alone RW is that users would not have to trade in their trusty PW, a platform they have gotten comfortable with.
Our proposed RW upgrade kit is the culmination of 28 years of research by Dr. Leaman (JL), who has been paralyzed from the shoulders down since 1996. The first upgrade was a rear-view camera, and remains a vital safety feature in the 2024 edition. The dash-cam video provides the point of view of the RW user. We encourage users to submit video clips capturing the accessi- bility of popular destinations and events in their neighborhood. An infrared scanner can be used to build a point-cloud map of the RW’s surroundings, however, 3D maps can also be generated from at least 2 video feeds. With the sensors and computers in place, several operating modes become possible (See Table 1).
Following this introduction, Section 2 discusses the background and meth- ods used for our review, Section 3 lays out the future of RWs, and finally, Section 4 is the conclusion.
BACKGROUND & METHODS We pick up our review where the mini-review of Sivakanthan et al., 2022 (S22) left off. In particular, the RW taxonomy in S22 was very useful for organizing RW research into groups. After conducting a simple Google search for the key phrase “robotic wheelchair” we found that 19 of 34 references were webpages. The biggest difference between our review covering 2005–15 and the current review is that a larger fraction of the references are now websites. Two of those are online stores and four are online news magazines. The rest are web pages dedicated to RW research groups, including years of research and often multiple prototypes.
An Update on International Robotic Wheelchair Development 49
Figure 1: Evolution of Dr. Leaman’s smart wheelchair.
See Figure 2 for images of the latest RW prototypes and concepts. The first row features: 1) the University of Pittsburgh’s MEBot; 2) the Adven- tus, a collaboration between the University of Toronto and the University of Washington, 3) a RW from Saudi Arabia, and 4) the latest RW from Switzer- land. The second row features: 1) an autonomous RW by MIT, 2) the BallBot by the University of Illinois developed with a $1.5 Mil. NSF Grant; 3) the Centaur a SW designed for the masses by former Ford executives, and 4) the Mobius a follow-up to the iBot. The third row features: 1) a concept model of
a RW for toddlers by Cornell, 2) a Romanian RW, 3) a RW by Northwestern University, and University of Michigan’s RW, called the Vulcan. The fourth row of Figure 2 features: 1) a RW by Spanish researchers; 2) a Japanese RW, 3) a RW designed to play tennis, and 4) a RW with a robotic arm for assisting with daily living activities.
Figure 2: Examples of robotic wheelchairs from around the world.
RW Upgrade Kit The RW is a combination of hardware electronics and custom software. The PW Platform is the foundation on which various sensors and input/output methods are mounted. Dr. Leaman has been upgrading his PW since 1998 when he was a NASA intern. See Fig. 1 for pictures of the evolution of his PW to SW.
Platform The specific configuration of the seat, arm/foot rests, lateral supports, head- rest, and recliner should be chosen by the user’s physical therapist. We find
An Update on International Robotic Wheelchair Development 51
that PW users prefer to use the platform best suited for their particular capa- bilities. A robotic arm mounted on the PW can even help open doors and play tennis. If combined with an Exoskeleton, the robotic arm can help with upright mobility (Hernandez-Ramos, 2022).
Wheels: The number of wheels and their location varies from one PW make/model to another. Front wheel drive and Center wheel drive are most popular, but there are also Stair climbers and even Tracks. Center wheel drive allows for a very small turning radius but will start by turning first. The Front wheel drive on the other hand starts by going straight, but has a larger turning radius.
Input Methods: When a joystick is not an option, the RW user can activate single switch controls via sip & puff, or head tracking. There is also voice command, but that doesn’t work very well in noisy environments. Vision- Language-Action AI research is an important input modality. The ‘holy grail’ of input methods remains the brain computer interface (BCI), but current electroencephalograph (EEG) technology is not yet accurate enough to safely operate a vehicle like a PW, however the folks at Neuralink are currently conducting human trials of an implanted BCI that shows a lot of promise.
Output Methods: There have to be some electronics included in the kit for the RW to communicate with its user and the rest of the world. This includes speakers and lights, plus a computer monitor with or without touchscreen. Finally, there are wearable monitors for an augmented reality or full virtual reality experience.
Human Computer Interface The Human Computer Interface (HCI) software should be more than just a collection of individual operating modes, even if they are fully customizable by the user (See Tab. 1). The ideal HCI is simple and elegant, with a drop down menu and fillable pop up windows. Not long after customization the user should feel like the RW fits like a glove. The HCI will have several oper- ating modes that have to be developed and tested in human trials. We had to adapt various algorithms related to autonomous navigation, localization, and mapping by mobile robots to the PW platform (Hung, 2017). There are many researchers taking up the challenge, Alonso (2021), Bakouri (2022), and Kutbi (2023). The future of the HCI is explainable augmented reality projection in the field of view.
Mobile Office: Arguably, the most practical mode is as a mobile work station. The combination of mount-n-mover and laptop/tablet gives the PW user a level playing field with their able-bodied peers. In fact, some might say that not having to worry about the weight of the computer is actually an advantage. JL uses his laptop for Command Line Coding; Charts, Tables and Spreadsheets; Letters, Resumes, and Contracts; Typesetting for Conference Proceedings, Grant Applications, and Business Plans; as well as Capturing and Editing Video.
Figure 3: Situational awareness algorithm builds a 3D map of the RW’s environment.
Emergency Signaling: In case of an emergency the RW will send a mes- sage, blink lights, and broadcast a distress call. This basic mode can detect emergencies, like tipping back, or falling over, by monitoring the inertial mea- surement unit (IMU) data. Future upgrades will use bio-sensors like pulse, blood oxygen, blood pressure, and forehead temperature to detect user stress levels. This mode should be standard, running behind the scenes at all times.
Mapping: Maps should begin with a 2D representation of the PW’s imme- diate surroundings in order to achieve situational awareness. By combining data from an IMU, and wheel encoders, with the point cloud from an infrared scanner, we can perform EKF localization, (See Fig. 3) [Hung, 2013; Lim, 2014; and Nguyen, 2016]. Then we use object detection to locate all the static objects in the environment to build up an increasingly accurate map. These serve as reference points for previously discovered mobile objects, and makes it much easier to detect and track multiple new objects (Pereira, 2022). This works adequately well for one room at a time, but for larger spaces the mem- ory, and processing requirements are quite high. One solution is to stream the data to the cloud for processing and storage, but that requires a strong inter- net signal, which is hard to guarantee anywhere in the world. One thing you can do with an accurate 3D map is to model the stability and comfortability of the RW user if they try to navigate a particular location The maximum tilt angle tolerable should be adjustable. The scanners and object recognition can be used for other applications as well. A person with a visual impairment can potentially navigate their environment more efficiently than a traditional
An Update on International Robotic Wheelchair Development 53
cane. A series of clicks or beeps conveys range information, and the camera can identify street signs to aid in navigation.
Collision Avoidance: Some tasks are at odds with the behavior expected from a more general collision-avoidance mode. While passing through a doorway the system must allow the wheelchair to come close to objects to pass through narrow door frames, at the expense of travel speed. It is important to have a robust system, but it must be adjustable to each user’s individual comfort level. Minimum distance, beep amplitude, and beep frequency are all adjustable.
Guide Following: When this mode is engaged, the RW locks its gaze on a person or another RW, and follows them until receiving a command to stop. The guide should wear something distinctive so the RW is less likely to lose track of them. Adjustable parameters in this mode include the minimum and maximum distance to the guide, as well as the default guide and their recognizable features.
Docking: In this mode the RW takes the latest environmental data and knowledge of user preferences, to find the best parking spots inside their homes, office, or on public transportation. This mode can be customized by choosing favorite parking spots and the maximum speed to use on approach.
Path Planning: All of the aforementioned operating modes can work with, or without, a motor controller, but all are needed for safe autonomous navi- gation. The HCI has Object Tracking and Situational Awareness that rapidly builds 3D maps of its surroundings. New RW users interested in autonomous navigation should register their system, and take a wheelchair skills test to certify their ability to safely operate the RW. Users should have the ability to switch between modes and choose from destinations on a map. Variables like maximum speed and acceleration should be adjustable.
col_1, …, col_j, freq_es, mag_es Mapping Maximum Distance d_max Mode 2D, 3D, Explore Collision Avoidance Frequency & Magnitude freq_ca, mag_ca Min. Distance, Mode d_min, Warn, Auto Guide Following Min. & Max. Separation sep_min, sep_max Formation Line, Pair, Diagonal Docking Favorites Fav_1, …, Fav_l Mode Directions, Auto Path Planning Max. Velocity & Accel. v_max, a_max Map Mode Home, Work, Guide, Auto
CONCLUSION
In this paper we review the international Robotic Wheelchair (RW) research and development effort up until 2024. The ideal RW is intuitive, upgrad- able, and easy to maintain. We have been testing the hardware and software functionality of the Human Computer Interface part of our RW upgrade kit, making sure that sensors are in the right locations, providing the quality of data needed for situational awareness and 3D mapping. The RW should become a standard PW upgrade, so that users can lead safer, more efficient, independent lives.
RWs are supposed to benefit the greatest possible population, so PW users should be able to upgrade their own chair, and not have to buy a brand new wheelchair platform. There are dozens of groups around the world working on their unique RW prototype, but there are in our view only two RWs that are close to being commercially viable, the Adventus and the Mobius. There are as of yet no complete RW upgrade kits. Given the enthusiasm of the assistive technology community, we are optimistic that a commercially viable RW is only a few years away.
REFERENCES
A. Brohan and et al. Rt-2: Vision-language-action mod- els transfer web knowledge to robotic control. In arXiv preprint arXiv:2307.15818, 2023. Adventus Robotics. https://www.adventusrobotics.com. online, Sept. 2023 Center for Robotics and Biosystems at McCormick School of Engineer- ing. https://robotics.northwestern.edu/research/topics/robot- platforms/robotic- wheelchair.html. online, Sept. 2023. Computer Science and Artificial Intelligence Laboratory. https://www.csail.mit.edu/
node/5962. online, Sept. 2023. D. Cojocaru and et al. The design of an intelligent robotic wheelchair supporting people with special needs, including for their visual system. Healthcare, 2022. Element14. https://sg.element14.com/robotic-wheelchair. online, Sept. 2023. Farnell an Avnet Company. https://uk.farnell.com/robotic-wheelchair-applications.
online, Sept. 2023. Future Market Insights. https://www.futuremarketinsights.com/. reports/roboticwheelchair-market online, Sept. 2023. H. Luo, Z. Yang, Peng Yin, Johnell O Brooks, and Bing Li. Modeling and prediction
of user stability and comfortability on autonomous wheelchairs with 3-d mapping. IEEE Transactions on Human-Machine Systems, 52(6):1216–1226, 2022. H. M. La, N. Gucunski, K. Dana, and S-H. Kee. Development of an autonomous bridge deck inspection robotic system. Journal of Field Robotics, 34(8):1489–1504, 2017. H. M. La, R. S. Lim, and et al. Mechatronic systems design for an autonomous robotic system for high-efficiency bridge deck inspection and evaluation. IEEE/ASME Transactions on Mechatronics, 18(6):1655–1664, 2013. J. Haffner and EPFL. https://actu.epfl.ch/news/control-for-wheelchair- robots-to-
mitigate-risk-of-/. online, Sept. 2023. J. Leaman and H. M. La. A comprehensive review of smart wheelchairs: Past, present, and future. IEEE Transactions on Human- Machine Systems, 47(4):486–499, 2017.
An Update on International Robotic Wheelchair Development 55
J. Leaman and H. M. La. ichair: Intelligent powerchair for severely disabled people. In ISSAT International Conference on Modeling of Complex Systems and Environments (MCSE), Da Nang, Vietnam, June 8–10, 2015. J. Leaman and H. M. La. The intelligent power wheelchair upgrade kit. In
2020 Fourth IEEE International Conference on Robotic Computing (IRC), pages 416–421, 2020. J. Leaman, Z. Yang, Y. N. Elglaly, H. La, and B. Li. Embodied-ai wheelchair framework with hands-free interface and manipulation. In 2022 IEEE International Conference on Systems, Man, and Cybernet- ics (SMC), pages 40–45, 2022. J. Photopoulos. https://ec.europa.eu/research-and-innovation/en/horizon-magazine
/robotic-arms-and-temporary-motorisation-next-generation-wheelchairs. Sept. 2023. J. Reed, and et al. https://centaurrobotics.com. online, Sept. 2023. J. Williams. https://www.wcia.com/news/u-of-i-professor-creates-hands-free-roboti c-wheelchair/. online, Sept. 2023. L. V. Nguyen and H. M. La. Real-time human foot motion localization algorithm
with dynamic speed. IEEE Trans. on Human-Machine Systems, 46(6):822–833, 2016. M. Ehrlich and et al. Adaptive control of a wheelchair mounted robotic arm with neuromorphically integrated velocity readings and online-learning. Frontiers in Neuroscience, 2022. M. Kutbi and et al. Egocentric computer vision for hands-free robotic wheelchair
navigation. Journal of Intelligent & Robotic Systems, 2023. M. Bakouri. Develop- ment of voice control algorithm for robotic wheelchair using nin and lstm models. Computers, Materials & Continua, 2022. M. Kutbi and et al. Usability studies of an egocentric vision-based robotic wheelchair.
ACM Transactions on Human-Robot Interaction, 2020. M. Zolotas and et al. Head-mounted augmented reality for explainable robotic wheelchair assistance. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2018. M. A. Hernandez-Ramos and et al. Design of a biomechatronic device for upright
mobility in people with sci using an exoskeleton like a stabilization system. Applied Sciences, 2022. Meinig School of Biomedical Engineering. https://www.bme.cornell.edu/spotlights/r obotic-wheelchair-one-three-year-olds-mobility-limitations. online, Sept. 2023. Mobius Mobility LLC. https://mobiusmobility.com.online, Sept.2023. P. Foster, C. Johnson, and B. https://web.eecs.umich.edu/ kuipers/re- search/wheelchair/. Sept. 2023. Patient Registry. Create a generalized brain interface to restore autonomy to those
with unmet medical needs today and unlock human potential tomorrow. In https://neuralink.com, 2023. Pineau, R. West, A. Atrash, J. Villemure, and F. Routhier. On the feasibility of using a standardized test for evaluating a speech- controlled smart wheelchair. Int. J. Intell. Control and Syst., pages 124–131, 2011. R. Alonso and et al. An abstraction layer exploiting voice assistant technologies for
effective human—robot interaction. Applied Sciences, 2021. R. Cooper. https://www.innovation.pitt.edu/wp- content/uploads/2017/09/id-3480mebot.pdf. online, Sept. 2023. R. Pereira and et al. Sort and deep-sort based multi-object tracking for mobile
robotics: Evaluation with new data association metrics. Applied Sciences, 2023.
R. S. Lim, H. M. La, and W. Sheng. A robotic crack inspection and mapping system for bridge deck maintenance. IEEE Transactions on Automation Science and Engineering, 11(2):367–378, 2014. S. Sivakanthan and et al. Mini-review: Robotic wheelchair taxonomy and readiness.
air.html. online, Sept. 2023. Y. Jiang and et al. General robot manipulation with multimodal prompts, 2023. Z. Zaidi and et al. Athletic mobile manipulator system for robotic wheelchair tennis.
IEEE Robotics and Automation Letters, 2023.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Robots, Drones and Unmanned Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-14-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional