INTRODUCTION
Given the growth of the older adult population worldwide, dementia is an increasingly serious health issue. Alzheimer’s disease (AD) is the most com- mon form of dementia and is associated with memory loss and cognitive impairments that affect daily life. Approximately 5.8 million older adults in the U.S. are living with AD (Alzheimer’s Association, 2020). According to the National Institute on Aging, the most common early symptoms of AD are problems with recent memory. During the progression of AD, patients may experience symptoms such as memory loss as well as cognitive and functional impairments.
© 2022. Published by AHFE Open Access. All rights reserved. 33
As a result, people with AD often require high levels of care and assistance to maintain daily activities (e.g., preparing meals and using transportation). The majority of care provided to a person living with AD or other forms of dementia is from a family caregiver, representing 18.6 billion hours of unpaid care valued at $244 billion (Alzheimer’s Association, 2020). The long dura- tion, time-intensive nature of caregiving imposes high burdens on caregivers, leading to medical problems such as increased depression and stress, decli- nes in physical health, issues in maintaining employment, and financial stress (Schulz, Sherwood, 2008).
People with AD often exhibit repetitive questioning, which can increase strain and burden on their caregivers (Hamdy, Lewis, Copeland, Depelteau, Kinser, Whalen, 2018). Social robots with the capability of conversation hold promise in alleviating this burden (Lima, Wairagk, Gupta, Baena, Barnaghi, Sharp, Vaidyanathan, 2021).
A multi-disciplinary course to create a social robot for Alzheimer’s and dementia was formed by faculty at the University of Tennessee-Knoxville’s College of Engineering, College of Education, Health, and Human Sciences, and the College of Architecture and Design. Undergraduate and graduate students in the three colleges participated in the semester-long course. At the completion of the course, students produced two social robot prototypes to be user tested for feasibility and acceptability. Estimated to be less than $300, the socially assistive robots were designed to be more financially viable for a wider distribution among the public, also addressing the current healthcare disparity in the United States.
The semester-long course consisted of three phases: researching phase, design and prototype phase, and a testing phase. This study posed two rese- arch questions: How can students understand the functional problems and needs associated with AD? And, how can different disciplines work together to create a social robot?
DEVELOPMENT OF A CLASS The class was led by an assistant professor in graphic design, with mentor- ship from a professor in biomedical, aerospace, and mechanical engineering. A graduate research assistant from biomedical, aerospace, and mechani- cal engineering with a research background in social robots and dementia contributed. One undergraduate student in mechanical engineering and two undergraduate students in computer science were paid as undergraduate rese- archers. Two graduate and one undergraduate architecture students were enrolled in the class and contributed to the project. Two undergraduate graphic design students were also enrolled.
The Research Faculty provided students with appropriate literature and opportunity for discussions to help them understand the complexities surrounding dementia and Alzheimer’s Disease. Through the published journal articles and class- room discussions, students better understood the functional needs of the AD patients and also identified needs for AD caregivers. In all students read
Designing a Multi-Disciplinary Class to Create a Social Robot for Alzheimer’s 35
twenty journal articles that included keywords: Human-Robot Interaction, Socially Assistive Robots, Assistive Technology Devices, Mild Cognitive Impairment, Alzheimer’s, Dementia, Quality of Life, Artificial Intelligence, Caregiving. The first three weeks of the course were spent reading, discussing, and evaluating published literature.
Two separate focus group meetings were held where the class met with a participatory team of community partners that consisted of neurologists, healthy older adults, engineers, product designers, nursing staff, as well as ADRD patients and their caregivers. These were both very essential to the progress of the design and functions of the social robot design because they experienced firsthand the wants and needs of AD patients from the care- giver and health expert perspective. Reading published journal articles and direct feedback from caregivers and people with early onset dementia and Alzheimer’s informed the group.
Additionally, students learned from research through observation. Functi- ons and interactions of several humanoid robots currently housed the Biome- dical Engineering lab were studied exclusively. Both robots, built by Softbank Robotics, function similarly. Pepper is a full-sized social humanoid robot able to recognize faces and basic human emotions. It is optimized for human inte- raction and has the ability to engage with people through conversation and touch screen.
Nao is a humanoid robot with very similar functions to Pepper, but is much smaller and doesn’t have a touch screen. The problem with both robots is that they are extremely expensive and cost prohibitive for most of the public. They have the ability to be programmed to be social companions and to help complete cognitive assessments, but that is not their main purpose.
Both Pepper and Nao were brought to the focus group meetings and the class was able to observe their interactions with the participatory team to bet- ter inform them of the features the participatory group thought were essential. The goal of the class was to use features of both robots that specifically address the needs of patients with AD and dementia, and create a low-cost version.
The Student Roles The class met weekly to identify the role and design of the social robot. Faculty and students from each discipline brought their own individual exper- tise, which was invaluable. The goal of the semester-long course was to create two uniquely different body’s to be tested. Results from the user testing expe- rience would inform students which design style was preferred. With the participatory focus group team, students worked to identify what the user tasks should be. Mechanical engineering and architecture students worked on the build of the robot’s physical body – identifying the size, shape, and scope of the design of the body, creating the housing of the internal computing components and the casing for the touch-screens. Computer science students devised the user tasks, programmed movement (version 1) and the LED lights (version 2). Graphic design students created the UI for the touch-screen inter- face for both versions. All teams worked independently and together across
various parts of the project. By mid-semester two body protypes were cre- ated and the social robot was named FRED (the Friendly Robot to Ease Dementia).
Their research suggested that FRED must be friendly in appearance. Other goals were that FRED needed to contain all its hardware and electrical parts within the body, have the potential to move, emit sound, be made inexpen- sively through 3D printing techniques, contain sources of light, can be easily editable for future revisions, and have a location for a screen from which the user can interact with. The professors helped guide the students in their individual roles and responsibilities, and for 3 hours each week the team met, reconvened, and checked in with one another.
After the specific tasks of FRED were agreed upon, the computer science students programmed them. The class decided that FRED should be con- trolled via voice and/or touch screen. Graphic design students created a user interface for a touch-screen interaction.
The platform used for developing FRED was Android. Since FRED requi- red several devices to function, such as a microphone, speakers, and a screen, the computer science students thought it was important to start development on a platform that provided adequate support for these automatically. A phone or tablet was used because these components are all natively suppor- ted on Android devices. Moreover, the Android platform is flexible for many applications, has great support for external libraries, and provides a prede- termined avenue for distributing software through the Google Play Store. FRED’s external components, like the face and the motor, are all controlled by an Arduino which communicates with the phone through Bluetooth.
Android Studio was used for the development environment for the phone application because it provided a simple interface for GUI development. Additionally, Kotlin was chosen as the language for the project because it was easier to handle Android functions than Java with its type inference and simpler syntax. Ideally the project will continue to use Kotlin, since there were no significant roadblocks inherent to the language or Android itself, and it can handle many technical aspects for developers.
FRED used the SpeechRecognizer object that was built into the Android SDK. Its cloud-based functions record audio constantly and send arrays of audio bytes to a server that performs all of the computations.
The 3D printed body of FRED containing servo motors for rotation (ver- sion 1) and LED ring face (version 2) was controlled by an Arduino with Bluetooth/Wi-Fi capabilities. The Arduino then communicated with the Sam- sung phone via BLE (Bluetooth Low Energy). The Arduino servers as the peripheral. In a BLE connection, the peripheral functions in the same manner as a server, providing the data for devices to which it is connected. Conver- sely, the phone served as the client, or the device receiving said data. The BLE connection was initiated as soon as the app was launched. Once the app is open, the phone immediately begins searching for the robot. The app will not open to the main menu until it makes a connection with the robot. Like the phone, the body of the robot will be connectable as soon as it is turned on.
FRED’s voice was implemented using the Android TextToSpeech object. This object simply takes strings, text, as input and outputs the same text as
Designing a Multi-Disciplinary Class to Create a Social Robot for Alzheimer’s 37
speech. Though the current voice sounds moderately robotic, there are plans to improve this in the future by fine tuning different features of the text to speech including, pitch, voice speed, and new voices.
The graphic design students worked on creating the visual assets for the touch-screen interactions. The students prototyped several versions of the interactions – experimenting with different color combinations, typeface choices, and illustration styles using Adobe XD. After feedback from the class and instructors, students decided on a color combination to address accessi- bility through appropriate contrast, a typeface choice and size that was easy to read, and a simple and friendly illustration style. Two final design versions were completed –one version with a touch screen that had eyes and a mouth that moved as FRED was engaging with the user to look as if he were talking (version 1), and the second, where the touch screen just had text (version 2). The design students saved the assets as PNG files to share with the computer science students, who would then implement them into the programming.
The mechanical engineering student took on one design and the archite- cture students took on another. Our mechanical engineering student had a lot of knowledge with 3D builds and printing. He created version 1 of FRED. The digital representation of the figure was made using Fusion360 and Soli- dworks. Designs generated within CAD were mainly done, but not limited to, using features such as extrudes, lofts, revolves, fillets, cuts, mates for movement, etc. Completed designs were exported as STL files so they may be 3D printed out in the Innovation Collaboration Studio (ICS) found in the engineering side of UTK’s campus.
Version 1, is roughly 8” tall, with a rounded base, a rectangular tablet screen casing that holds the phone screen, and two small LED lights above. It was printed using a gray PLA filament. The tablet screen has a mouth built into the UI, so when the robot is talking the mouth is moving. On this version, the shape is quite abstract with the tablet screen being the main emphasis. Ver- sion 1 has a servo motor, which allows the robot tablet face to twist around 180 degrees to engage with the user when its talking and answering questions. The architecture students had previous build experience designing for 3D and experimenting with different filament materials for the 3D printers. They conceptualized and designed FRED (version 2) using Rhinoceros (a 3D mode- ling software). Each individual component was constructed using a series of Boolean operations that either added to or subtracted from more simple geo- metries such as cones and cubes. This process provided a taxonomy of parts that fit together like the pieces of a puzzle. The more eccentric geometries (i.e., the head) were derived from spheres and then manipulated using Rhino’s Sub-D commands, ultimately lending itself to FRED’s curvaceous formal lan- guage. Completed designs were exported as STL files and they were printed out in the FabLab, an innovative fabrication space part of the College of Architecture and Design.
FRED version 2 is a stationary robot where the tablet is built into the chest cavity with a sliding door for easy access and stands approximately 12” high. The main emphasis on this robot is the face, which has 93 colored LED lights that pulsate and emit colors to show engagement and interaction when responding to users. The UI on the tablet on version 2 shows the text
version of what the robot is saying – there are no identifiable eyes or mouth with this version.
Version 2 of FRED was designed to have a subtle presence within a room. Different filaments were experimented with and the team ultimately landed on several iterations of filament material – a light wood filament, translucent and a pearl white filament. These types of filaments were selected based off color choice, the soothing feel of natural looking materials, and a way for the lights to be shown more easily.
All students were certified with the Collaborative Institutional Training Initiative (CITI) to allow them to participate in the user testing experience. Users will complete pre-surveys with demographic data and history on care- giving experience. They will then complete 6 tasks that the team identified as essential responsibilities of FRED, and a post user-testing survey where they will rate their experiences and preferences with version 1 and version 2 of FRED. Ultimately, the goal of the user testing will be to see the feasibility and acceptability of social robots to help aid in the caregiving experience – and to evaluate the preference of the design style of both versions. Through the user testing, the team was able to identify areas of improvement. Results of the surveys and user testing will allow for better design and development of social robots to help older adults and dementia patients in their activities of daily living.
The Challenges There were complications when designing the parts for 3D printing which mainly arose from a misjudgment of dimensioning and tolerancing with both versions. Translating the initial drawing into a 3D digital model was time consuming. Ensuring that all the electrical components for FRED would fit perfectly in the robot body alongside all interlocking parts was a difficult task. The beds of the 3D printers were only so wide and tall, which restricted how large of a printed part could be printed at a time. This required the creation of the robot body to be split into several parts, all of which must be fitted into one another to form the complete body. The inconsistencies between the rigging would result in loose fitting and parts that failed to stay together. Trial and error revealed that a tolerance of 0.03 inches was best suited for most printed parts done out of the ICS and the FabLab. Similarly, electrical equipment such as screens, wires, and Arduinos are just some of the components used that needed to be fitted into the robot body. Ultimately, not enough interior space led to parts being seen outside the body which is undesirable for user interaction and robot aesthetics.
Making the walls thin enough for cavities to be formed while not compro- mising the structural stability of the robot was another challenge. Sometimes this led to scaling up the size of the robot body to accommodate. The downside was longer print times and required more filament. Moreover, unpredicted errors during printing like the nozzle clogging, extended the time schedule for parts to be printed.
Printing the parts took anywhere from 4 to 24 hours. Sanding the prints for plastic burrs and physical modifications of the parts, like adding addi- tional holes, were done in the ICS woodshop using the tools. The entire
Designing a Multi-Disciplinary Class to Create a Social Robot for Alzheimer’s 39
process of the design, to the printing, to the assembly process was generally repeated several times. This was due to needing to fix unforeseen problems with the actual parts when coming off the printer or receiving new feedback and critiques in weekly meetings that pointed out potential flaws or areas of improvement.
The benefit of having cloud-based speech recognition that results were returned quickly, and these calculations did not take up computing resources on the device itself. One consequence of this, was reliance on a stable internet connection. The functions became sluggish since the device was constantly sending audio data to the server.
One problem with using an Android device for development is that these devices may lose support, or the manufacturer may stop producing them. Another problem is the limited size options with the screen.
Another noted challenge in the course was limited contact time. The course met once a week for 3 hours. This was not nearly enough time for such a com- prehensive project. A disadvantage to a class that only meets once a week is that students tend to put off the work until the last minute, meaning a lot of our deadlines kept getting pushed back. The class timeline shifted signi- ficantly by the end of the semester, and by the completion of the semester, students only conducted a small run of user testing among college-age stu- dents. The results from the user testing validated the need for more robust technical changes. We believe moving from an Android system to a Rasp- berry pi system will be necessary to maintain better control with functions including sound and voice recognition.
The Future The next step for the platform is to integrate the Android software with a Raspberry Pi in order to reduce cost and modularize the platform. Plans for future designs include a robot that can readily communicate and respond to a user. When designing for an older audience, a larger screen size would be ideal. There are more screen size options that pair with a Raspberry pi system. Adding external speakers and connecting to a Raspberry pi system versus using built-in speakers from a cellphone will ensure that noise is emitted and received crystal clear. The view of the tablet screen will be unobstructed and at a good angle. And lastly, that the robot is visually appealing to interact with while hiding all the circuitry.
Faculty will develop a dialogue management system based on machine lear- ning to allow the robot’s dialogue system to adjust to an appropriate rate of asking a follow-up question and the difficulty level of the question. The design of follow-up question may also distract the person with AD from repe- titive behaviors, which is a suggested caregiving strategy, and stimulate their brain activities through the conversation with different levels of difficulty.
Undergraduates and graduate students in nursing will be added to the pro- ject later to help with the addition of the cognitive assessment tool. Future research includes further implementation of various cognitive assessment and cognitive training programs using the social robot to improve the quality of life for people living with AD.
CONCLUSION
In all, the course was successful in many ways. A lot was accomplished in a 16-week course that only met one day a week. Having a team of students and faculty from a variety of disciplines allowed for a unique mix of expertise needed to contribute to something so robust. At the completion of the course, students were able to produce two social robot prototypes ready for user tested for feasibility and acceptability. The class identified major changes that needed to be made before conducting further testing. Very careful attention was made in designing the features of FRED to help assist caregivers and people with AD with tasks of daily living. The final cost for both robots were kept under $300, making FRED accessible for wide distribution among the public, while also addressing the current healthcare disparity in the United States.
ACKNOWLEDGMENT
The authors would like to acknowledge additional team members Fengpei Yuan, Zach Cary, Juan Casem, Brandon Gardner, Addie-Grace Hopper, and Cory Blanknship, who contributed to the making of FRED.
REFERENCES
Alzheimer’s & Dementia. The Journal of the Alzheimer’s Association. PP. 391–460. (March 2020). Volume 16, Issue 3. https://doi.org/10.1002/alz.12068. Hamdy, R. C., Lewis, J. V., Copeland, R., Depelteau, A., Kinser, A., Kendall-Wilson, T., & Whalen, K. (2018). Repetitive Questioning Exasperates Caregivers. Geron- tology & Geriatric Medicine, 4, 2333721417738915. https://doi.org/10.1177/ 2333721417738915. Lima, M., Wairagkar, M., Gupta, M., Baena, F., Barnaghi, P., & Sharp, D., Vaidyana-
than, R. (2021). Conversational Affective Social Robots for Ageing and Dementia Support. IEEE Transactions on Cognitive and Developmental Systems. PP. 1–1. 10.1109/TCDS.2021.3115228. Schulz, R., & Sherwood, P. R. (2008). Physical and Mental Health Effects of Family Caregiving. The American Journal of Nursing, 108 (9 Suppl), 23–27. https://doi.org/10.1097/01.NAJ.0000336406.45248.4c.