Nazka Leosala, Paul Rainer De Villa, Lorelie Grepo, Joshua Phillip Reyes, Kaye Domingo (2026). A Usability Evaluation of the Lusog-Isip Mental Health Mobile Application. In: Usability and User Experience. AHFE Open Access, vol. 232, pp. 37–46. https://doi.org/10.54941/ahfe1008033
Lusog-Isip is the first mental health and self-care mobile application developed specifically for Filipinos in response to the increasing prevalence of depression, substance use disorders, and mood disorders among Filipino individuals aged 18 to 24. From its launch in 2021 until April 2023, the application received average user ratings of 3.3 on Android and 3.6 on iOS, based on a five-point scale. User feedback during this period frequently cited usability concerns; however, no formal usability evaluation had been conducted since the application launch. To address this gap, a usability study was conducted involving 21 participants aged 18 to 24. Data were collected using the Post-Study System Usability Questionnaire (PSSUQ), the After-Scenario Questionnaire (ASQ), and ten open-ended questions. Lusog-Isip earned an overall PSSUQ score of 2.13, better than the 2.82 benchmark, indicating that users are generally satisfied with the app's usability. However, the results of the ASQ revealed that users had a hard time accessing the modules. They were also unsatisfied with the time consumed during signup and in accessing the modules. Further, respondents did not strongly agree that enough support information was provided for accessing and troubleshooting errors in the modules. While the findings indicate that the app’s usability is satisfactory, improvements in navigation and accessibility are still necessary to enhance overall user experience and satisfaction.
Lusog-Isip is the first-ever mobile application for mental health and self-care developed for Filipinos. It was launched in 2021 by the United States Agency for International Development (USAID) RenewHealth and the Department of Health of the Philippines (DOH) in response to the 2020 Philippine Special Initiative for Mental Health of the World Health Organization, which revealed that at least 3.6 million Filipinos suffer from depression, substance use disorders, and mood disorders. The goal of the mobile application is to help the said population address the mentioned mental health-related issues through assessment of their well-being and coping strategies, offering evidence-based tools and materials, and providing a list of online or nearby mental health and psychosocial support services (Department of Health, n.d.).
From its launch in 2021 until April 2023, Lusog-Isip has been downloaded by more than 10,000 Android and iOS users. The average user-rating of Android users is only 3.3 out of 5 stars, with 1 being the lowest and 5 being the highest. Meanwhile, iOS users, on average, rated the mobile application a 3.6 out of 5. Based on the study done by Alqahtani and Orji (2020), the average user rating, which quantitatively shows the overall satisfaction of the users, is between 4 and 5 out of 5 for most mental health applications. However, for Lusog-Isip, 43.40% of the ratings received from Android users are 3 and below, 33.3% of which are having issues with usability. Similarly, for at most 50% of iOS users, the mobile application only received a 3, with 36.8% having the same concern.
Currently, mobile applications are promising mental health interventions (Chandrashekar, 2018). Considering the current user rating of Lusog-Isip, its potential implication for the overall satisfaction of the users, and its prospective contribution to the mental health domain in the country, there is still a dearth of published evaluations done on the mobile application. Existing studies only dealt with the acceptance and intention of Filipinos to use them and their effect on the psychosocial well-being and coping strategies of the users. Little to no effort has been exerted to conduct a summative usability testing of the application.
This study primarily aims to assess the usability of the Lusog-Isip mobile application through summative evaluation. Specifically, it strives to measure the satisfaction of users in using the app by employing the Post-Study System Usability Questionnaire (PSSUQ), gain knowledge on how easy it is to use the app using the After-Scenario Questionnaire (ASQ), and provide qualitative insight into the overall usability of the application. The study ran from April to June 2023 and involved 21 participants, all of whom are Filipinos aged 18 to 24 years old residing in the Philippines.
LITERATURE REVIEW
Lusog-Isip is the first mobile app centered on mental health and self-care that is tailored fit for Filipino users (Healthy Pilipinas, n.d.). It was developed under the United States Agency for International Development (USAID) RenewHealth Project together with the help of the Department of Health of the Philippines (DOH) and was launched on October 15, 2021. It aims to provide convenient self-help tools and individualized mental health resources to Filipinos, especially when face-to-face consultations with health professionals are inaccessible (University Research Co., 2021), which was the case during the COVID-19 pandemic.
Some of the evidence-based tools and materials the Lusog-Isip app delivers to the user include workbooks, exercises, audio guides, journaling, mood tracking, and self-care reminders. Users are also given a list of mental health and psychosocial support services, both online and nearby, should they find the mentioned resources insufficient (University Research Co., 2021). These features sufficiently agree and are in line with the findings of Drissi et al. (2020) regarding the common functionalities employed by mental health mobile apps in the market.
A Usability Evaluation of the Lusog-Isip Mental Health Mobile Application 39 In a pilot evaluation conducted by Hechanova-Alampay et al. (2022), it was found that the Lusog-Isip app has improved the well-being and coping strategies of its users. The Healthcare Smartphone App Evaluation Tool (HSEAT) results on user evaluation showed that the subjects had a positive response to the app’s design, objectivity, wording, understandability, consistency, security, and quality of the content.
However, HSEAT is primarily used in evaluating mobile applications in general and thus, cannot give detailed information on specific aspects that users are having problems with. Hence, the positive feedback on the understandability and interface design of the Lusog-Isip app back then may not be the case years after. It is also worth noting that the pilot testing was conducted in 2021; 2023 data show that users are generally unsatisfied with the mobile application.
In the analysis of user reviews of 106 mental health applications by Alqahtani and Orji (2020), usability emerged as the primary aspect valued by users. The same study also reported that the average user rating for mental health applications is 3.93 out of 5. However, the average rating of Lusog- Isip since its launch until April 2023 is only 3.3 and 3.6 out of 5 stars on Google Play and iOS App Store, respectively. In addition, of those who rated the apps, 33.3% of Android users and 36.8% of iOS users have complaints concerning the usability of the app. These highlight the need for a usability study on the Lusog-Isip application to identify areas for improvement.
METHODS
Instruments This study employed the After-Scenario Questionnaire (ASQ) and Post- Study System Usability Questionnaire (PSSUQ) to evaluate the user-perceived satisfaction and ease of use of the Lusog-Isip mobile application. In addition, pre– and post–exploration open–ended surveys were administered to gather qualitative insights from participants.
The ASQ is a standardized instrument designed to evaluate user satisfaction after completion of specific user scenarios (Lewis, 1991). It consists of 3 items assessing the ease of task completion, time required to complete the task, and satisfaction with support information.
The PSSUQ is a post-test usability questionnaire that assesses user satisfaction (Sauro and Lewis, 2016). It has three versions, of which the third was used in this study. It contains 16 items grouped into three subscales: system usefulness (items 1-6), information quality (items 7-12), and interface quality (items 13-15). An overall usability score can be obtained from the mean rating of items 1-16. Fruhling and Lee (2005) defined system usefulness as the effectiveness of the support within the application and the effort needed in performing tasks. Information quality is defined as the ease of understanding of information and feedback presented, such as error messages, status updates, and on-screen instructions. Interface quality gauges the satisfaction of user expectations in terms of design and functionality.
Both instruments use a 7-point Likert scale ranging from 1 (“Strongly Agree”) to 7 (“Strongly Disagree”), with lower scores indicating more positive usability perceptions.
Participants Twenty-one (n = 21) students from the University of the Philippines Diliman aged 18 to 24 were recruited for the study using convenience sampling. Recruitment of participants was done through face-to-face and online means.
Study Procedure The study was conducted in three parts. In the first part, the participants were briefed about the purpose, objective, and study procedure. Those who expressed interest in participating in the study were asked to sign an informed consent form. Before proceeding to the second part, they answered a 4-item pre–exploration open-ended survey.
For the second part, each participant independently performed 23 component tasks on Lusog-Isip. After each task, they rated the ease of task completion, time required to complete the task, and satisfaction with support information, and recorded their comments in the ASQ. Each participant was observed by one member of the research team while they accomplished each component task, noting user behavior, difficulties encountered, and task completion.
In the third part, the participants rated the overall system usefulness, information quality, and interface quality using the PSSUQ. Afterwards, they answered a 6-item post–exploration open-ended survey.
Data Analysis Mean ASQ scores were computed per scenario to evaluate perceived ease of task completion, time required to complete the task, and satisfaction with support information of each scenario. Mean subscale PSSUQ scores were calculated to assess usability across system usefulness, information quality, interface quality, and overall usability. Results were interpreted using the PSSUQ benchmarks from Sauro and Lewis (2016).
Qualitative responses from the open-ended surveys were analyzed using thematic analysis following the procedure described by Nowell et al. (2017).
RESULTS AND DISCUSSION
Pre–Exploration Open-Ended Survey Only one of the participants had used other mental health applications two to five months prior to the study, indicating that almost all participants first used Lusog-Isip during their respective usability tests.
In terms of what they look for in a mental health application, the common themes that emerged are low cost, usability (i.e., ease of use in terms of navigation), and features (e.g., learning materials, performance tracker, assessments, contact with mental health professionals, etc.). These are consistent with the findings of Alqahtani et al. (2021) and Bakker et al. (2016).
After Scenario-Questionnaire (ASQ)
Table 1 shows the results of the ASQ. 95.24% of the participants completed Tasks 7, 9, 16, and 23. Meanwhile, only 52.38% completed Tasks 18 and 19. Ease of use scores revealed light difficulty in reading the User Agreement and
A Usability Evaluation of the Lusog-Isip Mental Health Mobile Application 41 Data Privacy Statement (Task 3) and accessing the modules (Tasks 18 and 19). These tasks, along with Tasks 4 and 5, also received poorer ratings for the time required, indicating that participants found the signup process and answering of the FCSS questionnaire relatively time-consuming. In addition, participants perceived that support information is insufficient when accessing the modules (Tasks 18 and 19).
Table 1: Summary of ASQ results.
Table available in the original PDF paper.
Component Task Mean E Rating
Mean T Rating
Mean SI Rating
Mean ASQ Rating
Completion Rate (%)
1. Click “Get Started” 1.00 1.00 1.00 1.00 100
2. Choose a language 1.00 1.00 1.10 1.03 100
2.00 2.14 1.62 1.92 100
3. Read User Agreement and Data Privacy Statement
4. Signup 1.81 2.00 1.43 1.75 100
1.29 2.00 1.38 1.56 100
5. Take the Filipino Coping Strategies Scale (FCSS) questionnaire
1.10 1.10 1.43 1.21 100
6. Answer the WHO-5 Well-being Index questionnaire
1.48 1.29 1.52 1.43 95.24
7. Go to the Homepage. What is your tolerance percentage?
1.10 1.00 1.38 1.16 100
8. Answer “How are you feeling today?”
1.29 1.14 1.19 1.21 95.24
9. Edit your banner to see all kinds of content that change 3x a day
10. Go to My Self-Care Tools 1.62 1.33 1.52 1.49 100
11. Access My Mood Journal 1.14 1.05 1.05 1.08 100
1.29 1.14 1.19 1.21 100
12. Add an entry to My Mood Journal
13. Open My Self-care Toolkit 1.19 1.24 1.19 1.21 100
1.24 1.19 1.52 1.32 100
14. Answer the questions in My Self-care Toolkit
1.38 1.33 1.29 1.33 100
15. Add person to contact in case of emergency
1.85 1.70 1.75 1.77 95.24
16. Look for mental health and wellness service providers
17. View achievements 1.19 1.05 1.48 1.24 100
2.67 2.87 2.93 2.82 52.38
18. Go to the Modules. What is the first module?
19. Open the GINHAWA module 2.93 3.60 2.47 3.00 52.38
1.29 1.24 1.52 1.35 100
20. Turn off notifications from the app
21. Change the language 1.00 1.05 1.29 1.11 100 (Continued)
Table 1: Continued.
Table available in the original PDF paper.
Component Task Mean E Rating
Mean T Rating
Mean SI Rating
Mean ASQ Rating
Completion Rate (%)
22. Edit your nickname 1.05 1.00 1.10 1.05 100
23. Logout 1.00 1.19 1.24 1.14 95.24 E = Ease of task completion, T = Time required to complete the task, and SI = Satisfaction with support information
The comments of the participants in the ASQ provided further context in the lower ratings observed in these tasks. For Task 3, several participants described the User Agreement and Privacy Statement as text-heavy. Similarly, for Task 4, participants noted that the sign-up process required excessive information. Some participants also reported the absence of an immediate error prompt when a chosen username is already taken. In Task 16, participants expressed difficulty locating contact details of mental health professionals within the application. For Tasks 18 and 19, most comments revolved around the inaccessibility and loading delays of the modules.
The observations of the researchers while participants performed the component tasks further corroborated these findings. During Task 4, some participants verbally expressed frustration regarding the length of the sign-up form and displayed signs of impatience when required to re-enter all information after encountering username errors. In Task 16, participants navigated multiple tabs before finding the correct tab. In Tasks 18 and 19, observable signs of irritation were noted due to slow loading times.
Post-Study System Usability Questionnaire (PSSUQ)
Table 2 shows the PSSUQ ratings in comparison with the benchmarks from Sauro and Lewis (2016). Mean subscales ratings are 2.00, 2.34, 1.90, and 2.13 for system usefulness, information quality, interface quality, and overall usability, respectively. Since the mean ratings are better than their respective benchmarks, 2.80, 3.02, 2.49, and 2.82, it can be inferred that users are generally satisfied with the usability of the Lusog-Isip mobile application.
Table 2: Summary of the PSSUQ results.
Table available in the original PDF paper.
Subscale Item Mean Ratings Benchmark
1.86
1. Overall, I am satisfied with how easy it is to use this system.
2. It was simple to use this system. 1.62
2.19
3. I was able to complete the tasks and scenarios quickly using this system.
2.00 2.80
System Usefulness
4. I felt comfortable using this system. 1.67
5. It was easy to learn to use this system. 1.86
2.81
6. I believe I could become productive quickly using this system.
(Continued)
Table 2: Continued.
Table available in the original PDF paper.
Subscale Item Mean Ratings Benchmark
3.29
7. The system gave error messages that clearly told me how to fix problems.
2.67
8. Whenever I made a mistake using the system, I could recover easily and quickly.
1.90
2.34 3.02
Information Quality
9. The information (such as online help, on-screen messages, and other documentation) provided with this system was clear.
10. It was easy to find the information I needed. 2.38
1.67
11. The information was effective in helping me complete the tasks and scenarios.
2.14
12. The organization of information on the system screens was clear.
13. The interface of this system was pleasant. 1.62 1.90 2.49
14. I liked using the interface of this system. 1.71
Interface Quality
2.38
15. This system has all the functions and capabilities I expect it to have.
Items 1-15 Overall 2.13 2.82 16. Overall, I am satisfied with this system. 2.29
Notably important also is the mean rating for item 7, which is poorer than its corresponding benchmark, 3.29 to 3.02. This reflects the lack of adequate feedback to support users in case a mistake is made while using the app.
Post-Exploration Open-Ended Survey The majority of the participants expressed overall positive perceptions towards Lusog-Isip. The most liked aspect of the application is the interface, followed by the available assessments, overall ease of use, language, and self- care tools. Other areas the participants expressed appreciation for include the availability of the Filipino language version, the purpose of the app, and the presence of a mental health service database.
In contrast, the least liked elements include issues in accessibility, identification, navigation, input, and content. Important features in the app, such as the modules were often inaccessible or slow to respond, in addition to its lengthy content. Participants also voiced confusion as the icons and buttons within the app lack proper labeling, causing them to make unnecessary clicks. Some participants reckon as well that it requires unnecessary personal information upon sign-up.
Most participants indicated that they would most likely continue using the app for features such as the self-care toolkit, modules, mood tracker, constant notifications, and educational content. Other factors mentioned that would encourage them to continue were a pleasant interface and the addition of labels to icons.
CONCLUSION AND RECOMMENDATIONS Users are generally satisfied with the overall usability of the Lusog-Isip mobile application. However, several interface enhancements must be made to facilitate smoother navigation and improve the accessibility of features, specifically the modules. The proposed improvements could not only enhance the overall usability, but also the usage and adoption rate of the mobile application.
Modules It was observed that the inaccessibility or unresponsiveness of modules is due to the fact that the menu consists of a list of high-quality PDF files requiring a reliable internet connection to load. Instead of PDF files, learning materials should be app pages-information that is already integrated into the app. In addition, it is expected that the developers will conduct debugging or any appropriate methods that will provide an improvement in the accessibility of this feature.
Figure 1: Current and proposed module interface.
Icons Many of the participants were not able to make up the connections between the icons and their corresponding uses, resulting in navigation and identification issues. Thus, it is suggested that labels be provided for each icon to reduce ambiguity.
Figure 2: Current and proposed icon labels.
A Usability Evaluation of the Lusog-Isip Mental Health Mobile Application 45 Sign-Up
During signup, there was no immediate prompt informing users that their desired username is unavailable, requiring them to repeat the whole signup process. An error prompt must immediately appear once the user entered a username that is already taken by other users. They must also be able to edit individual form inputs without redoing the whole signup process.
Figure 3: Current and proposed username prompt.
Redesigning the interface with the recommended changes aims to improve the usability of the Lusog Isip app.
Limitations The usability evaluation conducted was limited to the 18 to 24 age group, mostly college students recruited from convenience sampling. It is recommended that further evaluation be done with older age groups, as the Lusog Isip app is intended for the Filipino population. Testing the effect of redesigning the interface is also recommended to validate the improvement in the app’s usability.
ACKNOWLEDGMENT
The authors would like to express their gratitude to the study participants for their participation.
REFERENCES
Alqahtani, F., & Orji, R. (2020). Insights from user reviews to improve mental health apps. Health Informatics Journal, 26(3), 2042–2066. https://doi. org/10.1177/1460458219896492 Alqahtani, F., Orji, R., & Winn, A. (2021, February 26). Co-Designing a Mobile App
to Improve Mental Health and Well-Being: Focus Group Study. NCBI. Retrieved April 20, 2023, from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7954656/ Bakker, D., Kazantzis, N., Rickwood, D., & Rickard, N. (2016, March 1). Mental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments. NCBI. Retrieved April 20, 2023, from https://www.ncbi. nlm.nih.gov/pmc/articles/PMC4795320/ Chandrashekar, P. (2018, March 23). Do mental health mobile apps work: Evidence
and recommendations for designing high-efficacy mental health mobile apps. mHealth, 4, 6. 10.21037/mhealth.2018.03.02 Department of Health (n.d). DOH, USAID RenewHealth to Launch Lusog-Isip Mobile App for Mental Health | Department of Health website. doh.gov.ph. https://doh.gov.ph/press-release/DOH-USAID-RenewHealth-to-Launch-Lusog- Isip-Mobile-App-for-Mental-Health Drissi, N., Ouhbi, S., Janati Idrissi, M. A., & Ghogho, M. (2020). An analysis on
self-management and treatment-related functionality and characteristics of highly rated anxiety apps. International Journal of Medical Informatics, 141, 104243. https://doi.org/10.1016/j.ijmedinf.2020.104243 Fruhling, A., & Lee, S. (2005). Assessing the reliability, validity and adaptability of pssuq. AMCIS 2005 Proceedings. https://aisel.aisnet.org/amcis2005/378 Healthy Pilipinas. (n.d.). Lusog-Isip Online. Healthy Pilipinas. Retrieved April 22,
2023, from https://selfhelp.cbdr.org.ph/ Hechanova-Alampay, M. R., Angeles, P. L., Tuliao, A., Hilario, E., Pagente, A. F., & Narra, C. V. (2022). The development and pilot evaluation of a mental health mobile app in the Philippines. Mental Health and Social Inclusion. https://doi. org/10.1108/mhsi-04-2022-0024 Lewis, J. R. (1991). Psychometric evaluation of an after-scenario questionnaire
for computer usability studies: The ASQ. ACM SIGCHI Bulletin, 23(1), 78–81. https://doi.org/10.1145/122672.122692 Lewis, J. R. (1992). Psychometric evaluation of the post-study system usability questionnaire: The PSSUQ. Proceedings of the Human Factors Society Annual Meeting, 36, 1259–1260. Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic Analysis:
Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods, 16(1). https://doi.org/10.1177/1609406917733847 Sauro, J., & Lewis, J. (2016). Chapter 8 - Standardized usability questionnaires. In Quantifying the User Experience: Practical Statistics for User Research (pp. 185–248). Elsevier Science. https://doi.org/10.1016/B978-0-12-802308-2.00008-4 University Research Co. (2021, December 22). Mental Health on the Move in the
Philippines – Meet the Lusog-Isip App - URC. Urc-chs.com. Retrieved April 22, 2023, from https://www.urc-chs.com/news/mental-health-on-the-move-in-the- philippines-meet-the-lusog-isip-app/
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Usability and User Experience
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 979-8-950676-08-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional