Yanzhen Jason Yue, Yi Chen, Chee Seng Chong, Xiangming Landy Lan (2025). Design and Evaluation of a Social AI Bot for Empathetic Support in Online Dementia Healthcare Community. In: Human Interaction and Emerging Technologies (IHIET 2025). AHFE Open Access, vol. 197, pp. 558–565. https://doi.org/10.54941/ahfe1006750
Social & Cognitive Computing Department, Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore 138632, Singapore ↗
Dementia caregiving exerts heavy emotional, psychological, and social challenges on caregivers, leading caregivers to feel isolated and overwhelmed. Online health communities have become vital platforms for dementia caregivers to seek support, advice, and empathy from others. However, many help-seeking posts go unanswered or receive responses lacking sufficient emotional support. Because of advances from Artificial Intelligence (AI) and large language models (LLMs), there are some academic literatures that have examined solutions for healthcare backgrounds previously. However, very few scientific investigations explored social AI bot's functionality for providing emotional support replies on dementia caregiver online communities. For this current investigation, we aim to develop and introduce a LLM-based social AI bot for dementia online healthcare communities and explore its efficacy for providing supportive and empathy replies towards help-seeking post submissions from dementia caregivers. Empathy scores and sentiments scores were calculated for comparisons between actual user replying post and social AI bot replying post. It was portrayed from analysis results that replying post generated from social AI bot have considerably high empathy scores and sentiments scores than actual user replying post. The scientific investigation findings enhanced our understanding on how social AI bot can be effective for providing proper empathy support for dementia caregivers.
Dementia is a major health concern with often devastating consequences for persons with dementia (PWD), their families, and society at large (Possin et al., 2019; Tan et al., 2020). With the increasing life expectance, the number of people suffering from dementia globally is projected to increase to 132 million by 2050 (WHO, 2017). Dementia cost (including direct medical and social care costs, and costs of informal care) were estimate at US$ 818 billion globally in 2015, and it is estimated that the cost of caring for people with dementia worldwide will rise to US$ 2 trillion in 2030 (WHO, 2017). The heavy care stress by family caregivers has been widely documented
Design and Evaluation of a Social AI Bot for Empathetic Support in Online 559
(Chan 2010; Hilal et al., 2013; Yap et al., 2005). In general, caregiving for community-dwelling PWD is long-term and tedious, often resulting in intensive strains on caregivers. Family caregivers experience changing challenges during their care journey of the PWD, such as emotional and physical burnout, disruption in work routines, and financial strains. Previous studies have identified multi-factorial needs of dementia caregivers and care recipients, including accessible and suitable services/resources, emotional and social support, self-care and stress management, etc. (Vaingankar et al., 2016; Jennings et al., 2015).
In recent years, online healthcare communities (OHCs) have become prevalent and helpful platforms for dementia caregivers to seek social support, share experiences, and access peer advices (Yin et al., 2024). Online healthcare communities provides dementia caregivers with a virtual space to connect, share and interact with other caregivers. Those platforms usually can offer various and relevant content on caregiving strategies, symptom management, and emotional coping shared by other caregivers in similarly situated conditions (Bateman et al., 2017). Dementia caregivers visit online healthcare communities not only to obtain practical information (e.g., medication management tips or strategies to handle various behavioural issues of PWD), but also to seek emotional support and share personal experiences, frustrations, and emotional burdens with others in similar situations. Dementia caregivers use these platforms to share their daily struggles and frustrations that may not be easily acknowledged in their offline lives. The expressions by dementia caregivers is often met with empathetic feedback, emotional resonance, and encouragement from other members, which helps to reduce feelings of loneliness and emotional suppression (Kaliappan et al., 2025). The sense of community and mutual understanding in dementia-focused forums has been found to enhance caregivers’ emotional well-being in their caregiving journey (Etxeberria et al., 2021).
However, many help-seeking posts go unanswered or receive responses lacking sufficient emotional resonance. How to ensure that caregivers receive personalized and empathetic responses remain significant challenges in these platforms. User disengagement due to lack of proper replies and sufficient interactions in online healthcare communities raises concerns about the continuous use of OHCs by relevant users and the sustainability of OHCs to better support dementia caregivers.
To improve the emotional support of OHCs for caregivers, researchers and designers have begun to explore integrating conversational AI agent which may have the potential to offer empathetic engagement when human users are unavailable or unresponsive. With the advancements in artificial intelligence (AI) and large language models (LLMs), more social media platforms (e.g., Weibo, Twitter) have integrated AI-based social bot (i.e., conversational agents) to provide timely engagement interactions (Gao et al., 2024; Yuan et al., 2019). Social AI bots, in the form of an AI-driven conversational systems, are designed to simulate human-like interactions and foster social engagement (Zhao et al., 2023). Social AI bots are usually developed to engage with online community users and provide social support based on users’ needs. Prior research has studied the potential impact of social AI
bots on reduction of loneliness and provision of mental health support (Gaffney et al., 2019). Yet, studies on the application of social AI bot in dementia online healthcare communities and its effectiveness to provide empathetic support are rare. There is still a lack of understanding towards the effectiveness of social AI bot in delivering emotionally sound responses to help-seeking posts made by dementia caregivers.
In the present study, we aims to develop a social AI bot (named “Sparky”) with ChatGPT for dementia online healthcare community. In addition, we will assess whether the social AI bot-generated replies demonstrate an acceptable level of empathy and positive sentiment compared to real user replies.
METHODOLOGY
In the present study, the aim is to design, develop and evaluate a social AI bot for dementia caregivers on online healthcare communities. Specifically, we developed a social AI bot and integrated it into a dementia caregiving online forum for assessment with user-generated content.
Developing Social AI Bot “Sparky” To support dementia caregivers with emotional and social support in an online healthcare community, our study designed and developed an online forum using NodeBB (an open source, Node.js based forum software platform). In addition, we also developed an LLM-based social AI bot named “Sparky” which was integrated into the online forum. The social AI bot was developed using OpenAI’s ChatGPT-4o architecture and deployed within the online healthcare community forum that we developed to facilitate the interactions among dementia caregivers. “Sparky” was engineered to facilitate responsive and compassionate interactions in two specific user scenarios: (1) when a caregiver initiates a new thread post seeking for help, and (2) when a caregiver makes a replying post directed at the AI bot.
The social AI bot “Sparky” was embedded into the community forum’s infrastructure through a middleware layer that connects the frontend web forum to the OpenAI ChatGPT-4o APIs via a secure backend. Its operation is designed to respond in two primary interaction contexts:
1. New Thread Post Initiation: When a user creates a new post thread that includes caregiving-related concerns or help-seeking cues, the system parses the content and initiates a response from Sparky. 2. Reply-Directed Interaction: If a user directly replies to a post generated by our social AI bot “Sparky”, the system interprets this as a request for continued AI interaction. The reply is processed and routed through the LLM. A follow-up response is generated and posted as a continuation of the thread. These subsequent interactions allow caregivers to engage in more in-depth conversations or reflective dialogue with our developed social AI bot.
The system continuously monitors the online forum activities and automatically triggers AI-generated post responses when either a new thread
Design and Evaluation of a Social AI Bot for Empathetic Support in Online 561
is posted or a reply is directed at “Sparky”. When an eligible post is detected, the system calls for the OpenAI GPT-4o model, which processes the post’s content and context to generate an empathetic reply to the help seeking posts made by the dementia caregivers. The generated response is then submitted to the forum via the NodeBB API. In this way, the social AI bot “Sparky” can interact directly in the caregiver community environment with dementia caregiver users.
The architecture and workflow of Sparky is illustrated in Figure 1.
Figure 1: System workflow of the social AI bot “Sparky”.
An example of the prompt engineering passed to the LLM model for our developed social AI bot may look like the following: “Your name is Sparky, you are on a forum which is designed to help caregivers of their loved ones with dementia. Write a warm, empathetic, and conversational reply to a new forum post. The reply should make the original poster feel seen, supported and appreciated for sharing. Keep the tone supportive, approachable and concise.
Output your reply content quoted with <output></output> tags. Here is the original post: {My mom didn’t recognize me today. I feel completely broken and unsure how to keep going.}” An example of the interaction between dementia caregiver user and “Sparky” is illustrated in Figure 2.
Evaluation of the Social AI Bot To evaluate the effectiveness of the developed social AI bot “Sparky”, we randomly sampled 200 help-seeking posts from a publicly available online dementia support forum. For each help-seeking post, we extracted the first reply posted by a real user (i.e., in total 200 real user replies). Furthermore, we used the 200 help-seeking post as input to pass to our Social AI bot “Sparky” to generate corresponding social AI bot replies (i.e., 200 AI bot
replies). Our study statistically compared the level of empathy for both the real user replies and the social AI bot replies. In addition, we also applicated sentiment analysis to quantify the emotional polarity expressed in both two types of replying (i.e., 200 real user replying post vs. 200 AI bot replying post).
Figure 2: Example interaction between dementia caregiver users and social AI bot “Sparky”.
Specifically, for empathy classifier analysis, our study utilized the bdotloh/roberta-base-empathy model available on the Hugging Face model hub. This model is a fine-tuned version of RoBERTa-base, trained on annotated datasets containing emotional support conversations, enabling it to estimate empathy scores with high contextual sensitivity. We implemented this model using the transformers library’s pipeline function with a text classification task configuration. Each reply was assigned an empathy score ranging from 0 to 1, with higher values indicating stronger empathetic expression.
Meanwhile, we conducted sentiment analysis to quantify the emotional polarity expressed in the reply texts. Our study adopted the TextBlob library, a lightweight Python-based NLP tool that provides a lexicon-based approach to sentiment scoring. Each reply was assigned a sentiment polarity score ranging from –1 to +1, where –1 denotes strongly negative sentiment, 0 indicates neutrality, and +1 signifies highly positive sentiment. The polarity score was computed by applying TextBlob’s sentiment analysis module to the raw text of each reply.
The summary of the empathy scores and sentiment scores for real users’ replying posts and social AI bot’s replying posts are shown in Table 1.
Design and Evaluation of a Social AI Bot for Empathetic Support in Online 563
Table 1: Summary of empathy scores and sentiment scores.
Table available in the original PDF paper.
Scores Replying Type Mean N Std. Deviation Empathy Score Real User 0.9823 200 .0195 Social AI Bot 0.9928 200 .0007 Sentiment Score Real User 0.0827 200 .2064 Social AI Bot 0.1961 200 .0975
Paired t-test analysis was conducted to statistically investigate the differences between the social-AI-bot-generated and real-user-generated replies using the above two key indicators: empathy scores and sentiment scores.
Table 2: Paired T-test results.
Table available in the original PDF paper.
Mean Std. Deviation t df Significance –0.0105 .0194 –7.631 199 p < 0.001
Pair 1 Real User Empathy Score – Social AI Bot Empathy Score
–0.1134 .2217 –7.231 199 p < 0.001
Paire 2 Real User Sentiment Score – Social AI Bot Sentiment Score
The paired t-test analysis showed that the social AI bot’s (“Sparky”) replies have significantly higher empathy scores and sentiment scores compared to real user generated replies. The analysis results imply that that the social AI bot can match or exceed the emotional quality of support commonly offered by real users in online healthcare communities for dementia caregivers.
CONCLUSION AND DISCUSSION Our study for this investigation explored design, development, and an assessment of a social AI bot from ChatGPT-4o. A social AI bot “Sparky” was designed to provide answers with supportive empathy in post thread that were generated from dementia caregivers. “Sparky” was also integrated into an online healthcare community forum for dementia caregivers. Integrating social AI bot into online forum design allowed seamless and appropriate interaction between dementia caregivers on the online healthcare community. The findings from our analytic assessment results have multiple important implications for how LLM-based social agents can function in online healthcare communities environments where individuals are most emotionally vulnerable.
The results of our study were that social AI bot based on LLM could potentially provide considerable help on an emotional level for dementia caregivers. Furthermore, study work of ours provided evidence that social communication among people on online healthcare communities could be facilitated by AI-based agents, where peer support on an ongoing timely basis is limited or slowly provided.
In addition, analytical results also indicated higher empathy scores and sentiment scores for responding posts generated by social AI bot. This would presumably imply that higher positive responses can be provided by social AI bot on the online healthcare community for facilitating caregivers at an acceptable empathy level for responding posts. The findings for our study are particularly significant in light of the prevalent emotional burden among dementia caregivers. For example, social AI bot can potentially fill emotional gaps of peer interaction by suitably identifying caregivers’ emotional states and delivering encouraging advices on online dementia healthcare communities. Overall, findings from our current study proved feasible for utilizing a LLM-based social AI bot for providing empathetic support on dementia caregiving communities.
Our study also uncovered recommendations for designers and developers of systems to embrace proper LLM-enabled social AI bot for digital healthcare and caregiving support. Specifically, findings from our study hinted at how social AI bot can be created such that it can provide sustained presence on the online healthcare forums and trigger user interactions at appropriate intervals, thereby leading to heightened engagement for dementia caregivers.
Future research can also investigate the possible long-term impact of social AI bot on caregivers’ well-beings and behavioural engagement. It can also be of benefit to investigate how dementia caregivers’ diverse needs can shape AI-human exchange in virtual healthcare communities.
REFERENCES
Bateman, D. R., Brady, E., Wilkerson, D., Yi, E. H., Karanam, Y., & Callahan, C. M. (2017). Comparing crowdsourcing and friend sourcing: A social media-based feasibility study to support Alzheimer disease caregivers. JMIR Research Protocols, 6(4), e56. Chan, S. W. (2010). Family caregiving in dementia: The Asian perspective of a global problem. Dementia and Geriatric Cognitive Disorders, 30(6), 469–478. Etxeberria, I., Salaberria, K., & Gorostiaga, A. (2021). Online support for family
caregivers of people with dementia: A systematic review and meta-analysis of RCTs and quasi-experimental studies.Aging & Mental Health, 25(7), 1165–1180. Gaffney, H., Mansell, W., & Tai, S. (2019). Conversational agents in the treatment of mental health problems: Mixed-method systematic review. JMIR Mental Health, 6(10), e14166. Gao, Y., Zhang, M. M., & Lysyakov, M. (2024). Is social bot socializing? Evidence
from a microblogging platform. In ICIS 2024 Proceedings (Paper 12). Hilal, S., Ikram, M. K., Saini, M., Tan, C. S., Catindig, J. A., Dong, Y.
H., Lim, L. B., Ting, E. Y., Koo, E. H., Cheung, C. Y., Qiu, A., Wong, T. Y., Chen, C. L., & Venketasubramanian, N. (2013). Prevalence of cognitive impairment in Chinese: Epidemiology of dementia in Singapore study. Journal of Neurology, Neurosurgery & Psychiatry, 84(6), 686–692. Jennings, L. A., Reuben, D. B., Evertson, L. C., Serrano, K. S., Ercoli, L., Grill, J., Chodosh, J., Tan, Z., & Wenger, N. S. (2015). Unmet needs of caregivers of individuals referred to a dementia care program. Journal of the American Geriatrics Society, 63(2), 282–289.
Design and Evaluation of a Social AI Bot for Empathetic Support in Online 565
Kaliappan, S., Liu, C., Jain, Y., Karkar, R., & Saha, K. (2025). Online communities as a support system for Alzheimer disease and dementia care: Large-scale exploratory study. JMIR Aging, 8, e68890. Possin, K. L., Merrilees, J. J., Dulaney, S., et al. (2019). Effect of collaborative
dementia care via telephone and internet on quality of life, caregiver well-being, and health care use: The Care Ecosystem randomized clinical trial. JAMA Internal Medicine, 179(12), 1658–1667. Tan, G. T. H., Yuan, Q., Devi, F., Wang, P., Ng, L. L., Goveas, R., Chong, S. A., & Subramaniam, M. (2020). Dementia knowledge and its demographic correlates amongst informal dementia caregivers in Singapore. Aging & Mental Health, 1–9. Vaingankar, J. A., Chong, S. A., Abdin, E., Picco, L., Jeyagurunathan, A., Zhang, Y.,
Sambasivam, R., Chua, B. Y., Ng, L. L., Prince, M., & Subramaniam, M. (2016). Care participation and burden among informal caregivers of older adults with care needs and associations with dementia. International Psychogeriatrics, 28(2), 221–231. World Health Organization. (2017). Global action plan on the public health response to dementia 2017–2025. Yap, L., Seow, C., Henderson, L., & Goh, Y. (2005). Family caregivers and caregiving
in dementia. Reviews in Clinical Gerontology, 15, 263–271. Yin, Z., Stratton, L., Song, Q., Ni, C., Song, L., Commiskey, P., Chen, Q., Moreno, M., & Fazio, S., Malin, B. (2024). Perceptions and utilization of online peer support among informal dementia caregivers: Survey study. JMIR Aging, 7, e55169. Yuan, X., Schuchard, R. J., & Crooks, A. T. (2019). Examining emergent
communities and social bots within the polarized online vaccination debate in Twitter. Social Media + Society, 5(3). Zhao, W., Zhao, Y., Lu, X., Wang, S., Tong, Y., & Qin, B. (2023). Is ChatGPT equipped with emotional dialogue capabilities? arXiv preprint arXiv:2304.09662. https://arxiv.org/abs/2304.09662
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET 2025)
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-73-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional