Ben Schooley, Akanksha Singh, Sarah Floyd, Stephan Pill, John Brooks (2022). Direct Weighting Interactive Design of Patient Preferences for Shared Decision Making in Orthopaedic Practice. In: Healthcare and Medical Devices. AHFE Open Access, vol. 51, pp. 108–116. https://doi.org/10.54941/ahfe1002105
Patients need the ability to accurately and efficiently communicate their preferences across outcome domains to their healthcare providers.1-7 No existing system provides an efficient and timely approach to collect and communicate patient preferences across outcome domains to support shared decision making (SDM) in orthopaedic practice.2-4,8-19 The overarching goal of this research is to design, build, and test an app that collects baseline patient preferences and health status across orthopaedic outcomes and reports this information to the provider for use in patient care. A core component of the app is a Direct-Weighting (DW) preference assessment approach, originated from our prior research, and applied in a touchscreen based interactive design. It is envisioned that patients will use the app after scheduling a first visit to a surgeon for a new orthopaedic condition. Direct weighting (DW) approaches calculate patient-specific preference weights across outcomes by asking patients to disperse portions of a hypothetical “whole” across outcomes in a manner that reflects a patient’s preferences.20 DW has low respondent burden but it requires respondents to make “implicit” comparisons which may be difficult to conceptualize.20 However, the DW approach has become generally accepted in the quality-of-life literature and it has been shown that patients dividing up pieces of a “pie” across quality-of-life domains yields valid representations of patient preferences across the domains.20-22 However, the DW approach has not been validated with specific clinical scenarios using a clinically focused set of outcomes or by using a mobile software app. Drawing on prior research, we iteratively design and develop the app with input from prior DW research, informaticians, and clinicians. We use a qualitative approach to pilot test the app with 20 first-time visit patients presenting with joint pain and/or function deficiency. Participants were interviewed about their outcome preferences for care, used the app to prioritize outcome preferences, answered interview questions about their experience using the app, and completed a mHealth App Usability Questionnaire (MAUQ). Interview questions focused on the utility and usability of the mobile app for communicating with their provider, and capability of the app to capture their outcome preferences. Results validated five core preference domains, with most users dividing their 100-point allocation across 1-3 domains. The tool received moderate to high usability scores. Patients with older age and lower literacy found the DW approach more difficult in terms of allocating 100 points across 5 domains. Suggestions for DW interface interaction improvement included instantiation of a token/points oriented DW preference scoring methodology rather than a 1-10 sliding scale approach for improved preference weighting cognition and SDM with a provider. As more patient reported outcome (PRO) apps hit the marketplace across a broad spectrum of health conditions, these results provide evidence for a DW approach and interactive design for patients to communicate their treatment preferences to their providers.References:1.Baumhauer JF, Bozic KJ. Value-based Healthcare: Patient-reported Outcomes in Clinical Decision Making. Clin Orthop Relat Res. 2016;474(6):1375-1378.2. Slim K, Bazin JE. From informed consent to shared decision-making in surgery. J Visc Surg. 2019;156(3):181-184.3. Damman OC, Jani A, de Jong BA, et al. The use of PROMs and shared decision-making in medical encounters with patients: An opportunity to deliver value-based health care to patients. J Eval Clin Pract. 2020;26(2):524-540.4. Sorensen NL, Hammeken LH, Thomsen JL, Ehlers LH. Implementing patient-reported outcomes in clinical decision-making within knee and hip osteoarthritis: an explorative review. BMC Musculoskelet Disord. 2019;20(1):230.5. Kamal RN, Lindsay SE, Eppler SL. Patients Should Define Value in Health Care: A Conceptual Framework. J Hand Surg Am. 2018;43(11):1030-1034.6. Charles C, Gafni A, Whelan T. Decision-making in the physician-patient encounter: revisiting the shared treatment decision-making model. Social Science & Medicine. 1999;49(5):651-661.7. Niburski K, Guadagno E, Mohtashami S, Poenaru D. Shared decision making in surgery: A scoping review of the literature. Health Expect. 2020.8. Selten EM, Geenen R, van der Laan WH, et al. Hierarchical structure and importance of patients' reasons for treatment choices in knee and hip osteoarthritis: a concept mapping study. Rheumatology (Oxford). 2017;56(2):271-278.9. Kannan S, Seo J, Riggs KR, Geller G, Boss EF, Berger ZD. Surgeons' Views on Shared Decision-Making. J Patient Cent Res Rev. 2020;7(1):8-18.10. Briffa N. The employment of Patient-Reported Outcome Measures to communicate the likely benefits of surgery. Patient Relat Outcome Meas. 2018;9:263-266.
The orthopaedics clinical literature broadly acknowledges that the effects of different treatments are likely heterogeneous across patients. (Broekman, Carriere, & Bredenoord, 2016; Floyd, Thigpen, Kissenberth, & Brooks, 2020; Weissman, Kelz, & Lee, 2017). Consequently, optimal treatment decisions in orthopaedics are rarely “one-size fits all” and providers must help individual patients choose treatments aligned with each patient’s clini- cal circumstances and preferences. (Baumhauer & Bozic, 2016; Chhabra, Sacks, & Dimick, 2017; Kamal, Lindsay, & Eppler, 2018). The ability of orthopaedic patients to accurately and efficiently communicate preferences across outcome domains to their providers is vital for shared decision making (SDM) so patients can receive the treatment that best suits them (Damman et al., 2020; Kamal et al., 2018; Slim & Bazin, 2019).
Direct Weighting Interactive Design of Patient Preferences for Shared Decision Making 109
Despite the clear benefits that the communication of patient preferences to providers could have for orthopaedic patients, (Surabh Bhatt et al., May 2020; Christensen et al., 2018; Hurley et al., 2020) barriers in current orth- opaedic practice workflows exist. Clinical measures commonly collected in EMR systems fail to capture the range of outcome domains valued by patients (pain, function, quality of life, etc.). (Surabhi Bhatt et al., 2020; Jayakumar & Bozic, 2020). Thus, no existing system provides an efficient and timely appro- ach to collect and communicate patient information on patient preferences to support SDM in orthopaedic practice. (Chhabra et al., 2017; Damman et al., 2020; Kannan et al., 2020; Selten et al., 2017; Sorensen, Hammeken, Thomsen, & Ehlers, 2019).
Yet, patients need the ability to communicate their treatment outcome pre- ferences accurately and efficiently to their healthcare providers. (Baumhauer & Bozic, 2016; Slim & Bazin, 2019). Treatment outcome preferences may include the patient’s prioritized desire for their treatment to reduce short term or long-term pain, get back to work as soon as possible, keep treatment costs low, or regain lost mobility. The overarching goal of this research is to design, build, and test a mobile app that collects baseline patient preferences across orthopaedic outcomes and reports this information to the provider for use in patient care. A core component of the app is a Direct-Weighting (DW) prefe- rence assessment approach, originated from our prior research, and applied in a touchscreen based interactive design. It is envisioned that patients will use the app prior to their first visit to an orthopaedic surgeon for a new orthopaedic condition or injury. Direct weighting (DW) approaches calcu- late patient-specific preference weights across outcomes by asking patients to disperse portions of a hypothetical “whole” across outcomes in a manner that reflects a patient’s preferences. (Browne, O’Boyle, McGee, McDonald, & Joyce, 1997). DW has low respondent burden but it requires respondents to make “implicit”comparisons which may be difficult to conceptualize (Bro- wne et al., 1997). However, the DW approach has become generally accepted in the quality-of-life literature and it has been shown that patients dividing up pieces of a “pie”across quality-of-life domains yields valid representations of patient preferences across the domains. (Browne et al., 1997; Stiggelbout, de Vogel-Voogt, Noordijk, & Vliet Vlieland, 2008; Wettergren, Kettis-Lindblad, Sprangers, & Ring, 2009). The DW approach has not been validated with specific clinical scenarios using a clinically focused set of outcomes or by using an interactive user experience embodied in a mobile software app. Dra- wing from prior research, we iteratively design and develop the app with input from informaticians, and clinicians and patients.
RESEARCH APPROACH AND METHODS We use a multi-method research approach to design, build, and evaluate a DW patient preference app with 23 first-time visit patients presenting with joint pain and/or function deficiency. We first identified five outcome domains that were the result of prior research by the research team. These included: Q1. Reduces my long-term pain after treatment, Q2. Improves my function and ability to engage in my regular activities, Q3. Limits my
Figure 1: Patient preference app.
out of pocket treatment costs, Q4. Minimizes the time required for trea- tment and rehabilitation, Q5. Limits the pain and discomfort I feel during treatment. We then incorporated these five domains into the design of an android application to be presented to new patients in a regional orthopa- edic clinic and research center. We applied a DW interaction method from input and feedback from orthopaedic researchers, surgeons, and experience design researchers.
We designed a mixed-method evaluation to study patient preferences in a DW approach, in which patients were asked to A. use the patient prefere- nce app, B. participate in a 30-minute interview, and C. complete a usability survey. Participants were interviewed about their outcome preferences for care, used the app to prioritize outcome preferences, answered interview que- stions about their experience using the app, and completed a usability and utility survey. Interview questions focused on the utility and usability of the mobile app for communicating with their provider, and capability of the app to capture their outcome preferences.
Patient Preference App We designed a prototype of an interactive mobile application containing a patient preferences direct weighting (DW) survey and preference visualization features (see Figure 1). Screen 1 allows the test subject to be identified as an anonymous participant of the study. Screen 2 explains the DW task to the user. Screens 3 and 4 illustrate the user’s direct weighting interaction.
We utilized the identified patient preferences and adopted a 100-point bucket weighting design in which the patients were required to distribute
Direct Weighting Interactive Design of Patient Preferences for Shared Decision Making 111
Table 1. Participants’ demographics.
Table available in the original PDF paper.
Called Interviewed Total 100 23 Gender Female 55 17 Male 45 6 Age Average Age 53 57 18-30 Years 9 1 31-40 Years 15 2 41-50 Years 12 4 51-60 Years 27 3 61-70 Years 31 10 71-80 Years 6 3 80 Plus Years 0 0
and assign a total of 100 points into five treatment preferences. Screen 5 is a visual confirmation of the assigned weights and screen 6 is a confirmation of completion.
Evaluation Setting The research setting for this study was a large orthopaedic clinic in the Sou- thern U.S. affiliated with a large integrated medical system. We adopted a purposive, criterion sampling strategy where a research coordinator conta- cted 100 new patients or patients visiting the clinic for new orthopaedic conditions in the age range of 18-80 years, with one or more of the follo- wing orthopedic conditions: shoulder, hip, elbow, knee, foot, hand, back and neck. Sampling occurred until qualitative data saturation was achieved. Twenty-nine (29) patients agreed to participate, with six (6) canceling prior to the interview, resulting in 23 total patients who participated in the app evaluation. Demographics of those patients who were contacted and partici- pated in the evaluation are shown in Table 1. The evaluation was conducted 30 minutes prior to the regular patient check-in time of each participants’ orthopaedic appointment.
Data Collection Data was collected in three parts. First, in an in-person setting, each partici- pant was handed an Android device and used the mobile app to input their treatment preferences using the DW method incorporated into the app. Par- ticipants awarded a total of 100 points spread across preference outcome categories. Next, participants were asked a series of questions during a qua- litative interview to understand their perceptions of the app and the direct weighting approach. Finally, participants were asked to complete a 6-item survey on app usability, patient-provider communication, patient’s intention to use the app in the future, and perceptions about the treatment preference outcome domains represented in the app. The survey was designed using a
4 point Likert scale [4 = Strongly Agree, 3 = Somewhat Agree, 2 = Some- what Disagree and 1 = Strongly Disagree]. Other data collected during the interview included: date and time of interview, participant age range, gender. Interviews were audio recorded and then transcribed using pseudonyms in place of identifying information. No other patient identifiers were collected in the interviews. Each patient participant was provided with a $30 gift card as an incentive for their time and participation.
Analysis The data from in-app patient preferences was analyzed for average weights, std. deviation, maximum and minimum weight for each preference as well as maximum variation across all cases. The survey responses were analyzed for mean scores for each of the six evaluation constructs. Thematic analysis of the interview transcripts was conducted by using a peer analysis methodology in NVivo software. For this, two researchers independently conducted an inductive analysis of data to create preliminary codebooks and reconciled these codebooks to summarize emergent themes. We used grounded theory hypotheses to guide our analysis.
RESULTS
The results are reported for each of the three data collection and analysis methods: app utilization, survey responses, and qualitative interviews. First, all 23 participants entered patient preferences into the app. The most fre- quently weighted category was long-term mobility improvement (M = 33.6) followed by long term pain reduction (M = 28.3), limiting treatment pain/di- scomfort (M = 14.7), limiting time (12.5), and limiting costs (10.8). To assess heterogeneity in preferences, we analyzed the extreme scoring for each que- stion. The maximum weights that were given to each question were - Q1: 45, Q2: 70, Q3: 20, Q4: 30 and Q5: 45. The minimum weights that were given to each preference were - Q1: 5, Q2: 0, Q3: 0, Q4: 0 and Q5: 0. Results vali- date five core preference domains, with most users dividing their 100-points across 1-3 domains.
Results from the survey showed overall positive results and the tool recei- ved moderate to high usability scores. Patient Participants agreed the most with ‘This exercise was easy to complete’ (M = 3.85, SD = 0.37), and ‘The directions were easy to understand’ (M = 3.75, SD = 0.44). They modera- tely agreed with ‘After reading the directions, I felt like I knew what to do’ (M = 3.7, SD = 0.47). ‘The list of concerns captured the important things to consider in selecting a treatment’ (M = 3.7, SD = 0.57). ‘The answers to this exercise will help me to talk with my doctors about my condition’ (M = 3.65, SD = 0.59) and ‘I would be willing to do a similar exercise (Where I assign points to different treatment factors) for other health issues, so that I can discuss treatment choices with my doctor’ (M = 3.6, SD = 0.75).
Finally, thematic analysis of qualitative data resulted in four significant themes.
1. Direct Weighting Approach and Usability. Participants discussed chal- lenges and benefits of the interactive DW approach. In general, patients
Direct Weighting Interactive Design of Patient Preferences for Shared Decision Making 113
discussed that the use of the 100-point constraints in the DW mechanism felt unfamiliar for a survey-based instrument, though the DW scoring also served its purpose to help patients compare and contrast different preference types and bring clarity to their thoughts about their preferences. One patient discus- sed her thought process while trying to figure out how to distribute points across categories, “...because I read all five first, and then I went and said, okay, if I had to rate this, I’m going to put this as 60, because this is the most important to me. But then I knew that I was going to have to start altering that what was most important to me down and use my points to then kind of discern what was my least im-portant and what was my most important, and then determine how that fit into the whole graph.” In general, patients found the mechanism to be thought provoking yet also required more effort than expected while applying math. The older age participants generally found the DW approach more difficult in terms of allocating 100 points across 5 domains. Suggestions for DW interface interaction improvement included instantiation of a token/points oriented DW preference scoring methodology rather than a 1-100 sliding scale approach for improved preference weighting cognition and SDM with a provider.
2. Patient Provider Communication. Participants noted the clarity and the enhancement the use of a preference app brings to their communication with their provider. Participants noted ease in communicating difficult topics such as money constraints, as a participant noted, “...then also, if it’s in the app, then it’s in the patient’s mind too, to discuss even if the doc doesn’t bring it up...”. Participants also noted that the app brings about their most impor- tant concerns to the table such that the discussion with the provider during the visit revolves around that concern, thus bringing more focused treatment options specific to their needs. A participant stated One patient said, “And so if it’s available to the doctor before you get to the visit, they already know what you need.” Participants also agreed that communicating their treatment preferences with their doctor is improved with the use of the app, saved time during the visit and helped them focus their communication with the providers.
3. Outcome Domain Completeness. Participants found the list of five outcome domains adequate, sufficient and complete to convey their prio- rities associated with choosing a treatment. For example, one participant said, “Function, pain, cost, time and discomfort. Yeah, those are the big ones that hit me off the top of my head.” The list of outcome domains were said to have been specific enough to be distinctly different from one another and well understood. In addition, the use of the app survey instrument hel- ped patients to get understanding and clarity about their priorities associated with treatment choice. As noted by another participant, “...These [categories] actually, when I first looked at it, I was like, oh God, I’ve got to rate these, and they’re going to be so close and similar that it’s going to be hard to rate them. But actually, these were very specific.” Overall, participants found the list of outcome domains to be complete. Some participants also expressed the need to have a discussion with their doctor about the treatment trade- offs that would help them accomplish the best mix between their multiple
preferences, and also, mechanisms to measure treatment success against the preference indications made in the app.
4. Patient Trust. The patient preference app in general enhanced patient’s trust in the treatment, in the provider, and with the clinic. One participant said, “I think it enhances my view of the practice in general.” Participants noted that the preference app provided a reassurance to them that the pro- viders care about their concerns and want to understand their treatment priorities. This increased feeling of trust led patients to feel more confident in the treatment choices, shared decision making during the patient visit, and eventually treatment success and satisfaction.
DISCUSSION AND CONCLUSIONS The mix of weights assigned by each patient differed across all patients indi- cating that a high degree of preference variation exists. We concluded that participants had distinct treatment priorities which was adequately captured in the selected five domain options. Pain alleviation, both during the trea- tment and in the long-term, was the most heavily weighted preference across all patients. However, cost of treatment and time taken during treatment were also found to be important to many patients. The heterogeneity in the respon- ses indicates that patients care about personalized treatment and justifies the need for a preference communication tool such as this.
In general, average mean scores for all survey questions leaned heavily towards Agree or Somewhat Agree indicating positive perspectives towards the app in terms of usability, acceptance, patient-provider communication and completeness of treatment preference outcome domain. For the survey results, highest scores (strongly agree) were given to the question on the ease of use of the app procedure validating the simple and efficient design of the app from the patient’s perspective. Lower scores were given to the question on interest in using a similar app for other healthcare conditions, indicating that the DW interaction could be considered more broadly.
For the qualitative results, patients found the DW application to largely provide a simple and beneficial tool for communicating patient preferences with their doctors, for building trust in the treatment and to participate in shared decision making with their providers. Patients validated the suf- ficiency and completeness of the treatment preference outcome domains, highlighting that the preference app captures the most important priorities adequately.
As patient preferences become more integrated into the care process for patients across a broad spectrum of health conditions, these results provide evidence for a DW approach and interactive design for patients to com- municate their treatment preferences to their providers, and further need for evaluation of this approach across healthcare domains and regions as a valuable component of patient-centered engagement and quality care.
REFERENCES
Baumhauer, J. F., & Bozic, K. J. (2016). Value-based Healthcare: Patient- reported Outcomes in Clinical Decision Making. Clin Orthop Relat Res, 474(6), 1375–1378. doi:10.1007/s11999-016-4813-4
Direct Weighting Interactive Design of Patient Preferences for Shared Decision Making 115
Bhatt, S., Davis, K., Manning, D. W., Barnard, C., Peabody, T. D., & Rothrock, N. E.
(2020). Integration of Patient-reported Outcomes in a Total Joint Arthroplasty Program at a High-volume Academic Medical Center. JAAOS Global Research & Reviews, 4(5), e20.00034. doi:10.5435/JAAOSGlobal-D-20-00034 Broekman, M. L., Carriere, M. E., & Bredenoord, A. L. (2016). Surgical innovation: the ethical agenda: A systematic review. Medicine (Baltimore), 95(25), e3790. doi:10.1097/MD.0000000000003790 Browne, J. P., O’Boyle, C. A., McGee, H. M., McDonald, N. J., & Joyce, C. R.
(1997). Development of a direct weighting procedure for quality of life domains. Qual Life Res, 6(4), 301–309. Retrieved from http://www.ncbi.nlm.nih.gov/pub med/9248312 Chhabra, K. R., Sacks, G. D., & Dimick, J. B. (2017). Surgical Decision Making: Challenging Dogma and Incorporating Patient Preferences. JAMA, 317(4), 357–358. doi:10.1001/jama.2016.18719 Christensen, D. L., Dickens, J. F., Freedman, B., Mauntel, T., Owens, B. D.,
Potter, B. K., . . . Grp, M. (2018). Patient-Reported Outcomes in Orthopa- edics. Journal of Bone and Joint Surgery-American Volume, 100(5), 436–442. doi:10.2106/Jbjs.17.00608 Damman, O. C., Jani, A., de Jong, B. A., Becker, A., Metz, M. J., de Bruijne, M. C., . . . van El, C. (2020). The use of PROMs and shared decision-making in medical encounters with patients: An opportunity to deliver value-based health care to patients. J Eval Clin Pract, 26(2), 524–540. doi:10.1111/jep.13321 Floyd, S. B., Thigpen, C., Kissenberth, M., & Brooks, J. M. (2020). Association of
Surgical Treatment With Adverse Events and Mortality Among Medicare Bene- ficiaries With Proximal Humerus Fracture. JAMA Netw Open, 3(1), e1918663. doi:10.1001/jamanetworkopen.2019.18663 Hurley, V. B., Wang, Y., Rodriguez, H. P., Shortell, S. M., Kearing, S., & Savitz, L. A.
(2020). Decision Aid Implementation and Patients’ Preferences for Hip and Knee Osteoarthritis Treatment: Insights from the High Value Healthcare Collaborative. Patient Prefer Adherence, 14, 23-32. doi:10.2147/PPA.S227207 Jayakumar, P., & Bozic, K. J. (2020). Advanced decision-making using patientreported outcome measures in total joint replacement. Journal of Orthopaedic Research, 38(7), 1414–1422. doi:10.1002/jor.24614 Kamal, R. N., Lindsay, S. E., & Eppler, S. L. (2018). Patients Should Define Value
in Health Care: A Conceptual Framework. J Hand Surg Am, 43(11), 1030–1034. doi:10.1016/j.jhsa.2018.03.036 Kannan, S., Seo, J., Riggs, K. R., Geller, G., Boss, E. F., & Berger, Z. D. (2020).
Surgeons’ Views on Shared Decision-Making. J Patient Cent Res Rev, 7(1), 8–18. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/32002443 Mitchell, H. L., & Hurley, M. V. (2008). Management of chronic knee pain: a survey of patient preferences and treatment received. BMC Musculoskelet Disord, 9, 123. doi:10.1186/1471-2474-9-123 Selten, E. M., Geenen, R., van der Laan, W. H., van der Meulen-Dilling, R. G.,
Schers, H. J., Nijhof, M. W., . . . Vriezekolk, J. E. (2017). Hierarchical structure and importance of patients’ reasons for treatment choices in knee and hip oste- oarthritis: a concept mapping study. Rheumatology (Oxford), 56(2), 271–278. doi:10.1093/rheumatology/kew409 Slim, K., & Bazin, J. E. (2019). From informed consent to shared decision-making in surgery. J Visc Surg, 156(3), 181–184. doi:10.1016/j.jviscsurg.2019.04.014
Sorensen, N. L., Hammeken, L. H., Thomsen, J. L., & Ehlers, L. H. (2019). Implementing patient-reported outcomes in clinical decision-making within knee and hip osteoarthritis: an explorative review. BMC Musculoskelet Disord, 20(1), 230. doi:10.1186/s12891-019-2620-2 Stiggelbout, A. M., de Vogel-Voogt, E., Noordijk, E. M., & Vliet Vlieland, T.
P. (2008). Individual quality of life: adaptive conjoint analysis as an alterna- tive for direct weighting? Qual Life Res, 17(4), 641-649. doi:10.1007/s11136- 008-9325-6 Weissman, J. S., Kelz, R. R., & Lee, C. N. (2017). Appropriateness, Health Care Reform, and the Surgeon: Perspectives From the Surgical Outcomes Club. JAMA Surg, 152(9), 813–814. doi:10.1001/jamasurg.2017.1572 Wettergren, L., Kettis-Lindblad, A., Sprangers, M., & Ring, L. (2009). The use,
feasibility and psychometric properties of an individualised quality-of-life instru- ment: a systematic review of the SEIQoL-DW. Qual Life Res, 18(6), 737–746. doi:10.1007/s11136-009-9490-2
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Healthcare and Medical Devices
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-27-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional