Shuning Li, Biju Cheriyan, Srinija Kasaraneni, Jason Mengjie Mao, Thankam Thyvalikakath (2025). Development and Usability Assessments of a Patient Medical Summary Application for Dental Clinicians. In: Usability and User Experience. AHFE Open Access, vol. 194, pp. 135–140. https://doi.org/10.54941/ahfe1006675
Accessing up-to-date patient medial information is essential for dental clinicians (DCs) to provide high quality clinical care. The objectives of this study were to develop a Fast Healthcare Interoperability Resources (FHIR) based patient medical summary (PMS) application to retrieve information from HIEs and to assess the usability of the PMS application. We followed the user-centred design method which included multiple iterations of developments and usability assessments. Forty-eight of the 50 participants were able to complete reviewing 6 patients’ medical histories during the interview, while the other two completed 5. The system usability scale (SUS) score for the first round was 82±12; while it was 76±16 for the second round of assessment. The lower SUS score in the second round reflected a major change in the study participant population. Even though the SUS score dropped, we noticed that the revisions corresponding to the improvements identified in the first iteration received very positive feedback. User-centred design and usability assessment can help to develop applications which have better information organization and display, and easy for clinicians to interact during decision-making. The PMS application was well received by the DCs and can be a valid option for DCs to access patient medical history.
Accessing up-to-date patient medial information is essential for dental clinicians (DCs) to provide high quality clinical care. However, majority of the dental practices still rely on patient reported medical history forms followed by optional medical consultation to collect patient medical information (Li et al., 2024).
With the support of several federal policies and the financial incentives established by the Centers for Medicare and Medicaid Services, community
and regional health information exchanges (HIEs) have expanded significantly since 2009 (Rahurkar et al., 2021). HIEs provide an option for retrieving patients’ medical information for DCs (Li et al., 2022a). However, previous studies showed that poor usability was a major hindrance to a successful adoption and implementation of HIE applications (Everson, 2017, Schleyer et al., 2019).
OBJECTIVES The objectives of this study were to develop a Fast Healthcare Interoperability Resources (FHIR®) based patient medical summary (PMS) application to retrieve information from HIEs and to assess the usability of the PMS application.
METHODS
We followed the user-centred design method which included multiple iterations of developments and usability assessments. Each usability assessment interview session had two parts: (1) following think-aloud protocol (Thyvalikakath et al., 2014) to review 3–6 patients’ medical histories using the PMS application and (2) answering questions and filling out the system usability scale (SUS) questionnaire (Brooke, 1996).
The initial version of the PMS application was developed based on high- priority information items identified from literature reviews and preliminary studies (Li et al., 2022b, Li et al., 2024) and was used for the 1st round of usability assessment. We have started the participants recruitment with clinical faculty, dental residents, and 3rd and 4th year dental students from one dental school, then expanded to a nationwide participant search. This round was terminated when achieving data saturation with no new information being extracted from the interviews.
The 1st revised version was developed based on the 1st round interview findings and was used for the second round of usability assessment. The 2nd round assessment participants were recruited nationwide. They were all actively practicing dentists or faculty members at dental schools. During the recruitment, we also considered their dental profession (general dentist, dental specialist, or hygienist), primary clinical settings, and geological locations to ensure a more diversified group of participants. We addressed findings from this round of the interviews and developed the final revision of the PMS.
Data Analysis After each round of the interviews, the transcripts of the recorded interview sessions were annotated and analysed. The SUS score was calculated by using the method introduced in Brooke’s book (Brooke, 1996).
Data were summarized using frequencies and percentages for categorical variables and mean and standard deviations (SDs) for continuous variables. Associations between characteristics of the participants (age, gender, dental profession, years of experience, primary clinic type, and primary clinic size)
Development and Usability Assessments of a Patient Medical Summary 137
and their SUS scores were examined using Linear regression/ANOVA. P<.05 were considered statistically significant.
Figure 1: User-centred design process for the patient medical summary (PMS) application.
RESULTS
Fifty participants were recruited for informant interviews during the two rounds of usability assessment. The demographics of the interview participants were shown in Table 1.
Table 1: Characteristics of interview participants.
Table available in the original PDF paper.
Characteristics Values Total, n 50 Age, mean (SD) 44.6 (13.5) Gender, n (%)
Male 17 (34) Female 33 (66) Profession, n (%) * Dental Student 8 (16) Faculty 16 (32) General Dentist 21 (42) Hygienist 6 (12) Resident 4 (8) Specialist 6 (12) Years of Experience, n (%) <=5 (including students) 14 (28) 6–10 7 (14) 11–20 8 (16) 21–30 10 (20) 31+ 11 (22) Primary Clinic Type, n (%) **
Academic 30 (60) Private 17 (34) Public 2 (4) Continued
Table 1: Continued Characteristics Values
Table available in the original PDF paper.
Primary Clinic Size, n (%) ** Small (1–2 Dentists) 14 (28) Medium (3–20 Dentists) 16 (32) Large (21+ Dentists) 8 (16)
* Percentage total is more than 100 due to the presence of multiple professions for one participant. ** Percentage total is less than 100 due to missing inputs.
The SUS score for the 1st round assessment was 82±12. Four major improvements were identified through the analysis of the 1st round interview transcripts (Figure 2): (1) group medical conditions based on body systems for easy review; (2) display the full list of allergies to avoid missing information; (3) add colour coded icons for all lab results for fast information gathering; and (4) provide more details for hospital encounters to build the connects among different categories of information.
Figure 2: The screenshots from the three rounds of designs and their differences.
The SUS score for the 2nd round of assessment was 76±16. The lower SUS score reflected a major change in the study participant population. Twenty-three of the 26 participants in the 1st round were from the same academic institution, while the 24 participants in the 2nd round were recruited nationwide. Even though the SUS score dropped, we noticed that the revisions corresponding to the improvements identified in the 1st iteration received positive feedback. Another three major improvements were identified through the analysis of the 2nd round interview transcripts (Figure 2): (1) add buttons for predefined sets of labs to support quick access to most recent (in the last 12 months) lab results or abnormal lab results; (2) add search buttons for labs, medications, medical conditions, and procedures sections for fast gathering of important information; and (3) provide normal range information for lab results to support interpretation and decision making.
Development and Usability Assessments of a Patient Medical Summary 139
The seven major improvements identified in the two rounds of assessments reflected DCs information and navigation needs. Most relevant information should be highlighted, and shortcuts should be provided for easy and quick access. Time is critical in clinical care. The PMS application should minimize the time for DCs to access the information they need for clinical decision making. Information should also be provided in a cohesive and connected manner to avoid information fragmentation which may cause longer time to locate information and prone to errors and missing information.
Associations between characteristics of the participants (age, gender, dental profession, years of experience, primary clinic type, and primary clinic size) and their SUS scores were presented in Table 2 and no significant impacts identified. DCs, no matter their personal and profession differences, have a common agreement on the overall usability of the PMS. However, we did notice significant differences for some individual questions in the SUS (Table 3). For the question “I found the system unnecessarily complex”, participants from larger clinics tended to give higher score. For the questions “I felt very confident using the system”, participants in the middle of their career (11–20 years of experience group) had the lowest confidence using the system.
Table 2: Impacts of participants’ demographics on their SUS scores.
Table available in the original PDF paper.
Characteristics SUS, Mean (SD) p-Value Gender 0.35 Male 76.6 (16.8) Female 80.7 (13.0) Profession 0.80 Dental Student 84.1 (14.4) Faculty 77.1 (21.6) General 77.7 (12.4) Hygienist 77.5 (16.4) Resident 90 (0) Specialist 79.2 (13.9) Years of Experience 0.23 <=5 (including students) 82.1 (12.9) 6–10 86.8 (6.73) 11–20 71.2 (11.3) 21–30 75.8 (20.3) 31+ 80.0 (13.9) Primary Clinic Type 0.29 Academic 79.9 (15.6) Private 80.4 (11.8) Public 60.0 (10.6) Primary Clinic Size 0.81 Small (1–2 Dentists) 76.1 (13.5) Medium (3–20 Dentists) 80.9 (11.2) Large (21+ Dentists) 80.0 (21.4)
Table 3: Impacts of participants’ demographics on individual SUS questions.
Table available in the original PDF paper.
SUS Questions Characteristics Median (IQR) I felt very confident using the system.
Years of Experience <=5 (including students) 5 (4-5) 6–10 5 (5–5) 11–20 3.5 (2.75–4) 21–30 4 (4–4.75) 31+ 4 (4–5) I found the system unnecessarily complex.
Primary Clinic Size Small (1–2 Dentists) 1.5 (1–2) Medium (3–20 Dentists) 2 (1–2) Large (21+ Dentists) 2.5 (2–3)
CONCLUSION
User-centred design and usability assessment can help to develop applications which have better information organization and display, and easy for clinicians to interact during decision-making. The PMS application was well received by the DCs and can be a valid option for DCs to access patient medical history.
REFERENCES
Brooke, J. 1996. Sus-A Quick and Dirty Usability Scale. Usability Evaluation In Industry, 189, 4–7. Everson, J. 2017. The Implications and Impact Of 3 Approaches to Health
Information Exchange: Community, Enterprise, and Vendor-Mediated Health Information Exchange. Learn Health Syst, 1, E10021. Li, S., Felix Gomez, G. G., Xu, H., Rajapuri, A. S., Dixon, B. E. & Thyvalikakath, T. 2024. Dentists’ Information Needs and Opinions On Accessing Patient Information Via Health Information Exchange: Survey Study. Jmir Form Res, 8, E51200. Li, S., Rajapuri, A. S., Felix Gomez, G. G., Schleyer, T., Mendonca, E. A.
& Thyvalikakath, T. P. 2022a. How do Dental Clinicians Obtain Up-to-Date Patient Medical Histories? Modeling Strengths, Drawbacks, and Proposals for Improvements. Front Digit Health, 4, 847080. Li, S., Williams, K. S., Medam, J. K., Patel, J. S., Gonzalez, T. & Thyvalikakath, T. P. 2022b. Retrospective Study of the Reasons and Time Involved for Dental Providers’ Medical Consults. Front Digit Health, 4, 838538. Rahurkar, S., Vest, J. R., Finnell, J. T. & Dixon, B. E. 2021. Trends in User-Initiated
Health Information Exchange in the Inpatient, Outpatient, and Emergency Settings. J Am Med Inform Assoc, 28, 622–627. Schleyer, T. K. L., Rahurkar, S., Baublet, A. M., Kochmann, M., Ning, X., Martin, D. K., Finnell, J. T., Kelley, K. W., Team, F. D. & Schaffer, J. T. 2019. Preliminary Evaluation of the Chest Pain Dashboard, A Fhir-Based Approach for Integrating Health Information Exchange Information Directly into the Clinical Workflow. Amia Jt Summits Transl Sci Proc, 2019, 656–664. Thyvalikakath, T. P., Dziabiak, M. P., Johnson, R., Torres-Urquidy, M. H., Acharya,
A., Yabes, J. & Schleyer, T. K. 2014. Advancing Cognitive Engineering Methods to Support User Interface Design for Electronic Health Records. Int J Med Inform, 83, 292–302.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Usability and User Experience
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-70-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional