Yuan Yin, Yurong Yu (2022). How self-report affects digital health-related behavior change. In: Healthcare and Medical Devices. AHFE Open Access, vol. 51, pp. 15–23. https://doi.org/10.54941/ahfe1002093
This study aims to identify how self-report method affects digital health-related behavior change and further detect its potential broader application. Two groups of 10 participants engaged in a 28-day behavior change program with one group using self-report and the other group not. After the experiment, the group using self-report participated in a semi-structured interview to report their experiences. The initial experiment showed that self-report did not affect behavior; this indicated that self-report is an appropriate method to collect behavior change data. However, the semi-structured interviews suggested that self-reporting brought benefits and encouraged users to further improve their behavior. The paper, thus, summarized that perceived effects of self-report in behavior change are stronger than actual effects of self-report.
The popularization of digital devices such as smart phones, computers, and dashboards made behavior change more closely tied with digital interventi- ons, so the digital behavior change intervention (DBCI) has emerged (Yardley et al., 2016). DBCI refers to interventions employing digital technologies to help change behaviors (Yardley et al., 2016).
Previous effort has been made to study what factors could motivate beha- vior changes, such as self-determination (Hardcastle et al., 2015, Patrick and Williams, 2012), cooperation(Conroy et al., 2014), and award (Yang et al., 2015). Some methods were applied such as “Ecological Momentary Asses- sment (EMA)”, paper diary, and “Short Message Service (SMS) to study these factors. However, when using self-report to identify behavior change perfor- mance, self-report as a method may affect DBCI too, because it triggers the human introspection and further affects behavior change.
As one of self-report methods, EMA can act as not only a DBCI, but also a DBCI effectiveness testing tool. However, how EMA triggers beha- vior change has still been understudied. As Biddle et al. (2001) has found, when comparing retrospective measures and real-time measures, they realized that real-time EMA may be an additional motivation, accelerating behavior change. Minamiet al. (2014) found that EMA could affect smoking by affe- cting subjects’ emotion. Additionally, the difficulty in evaluating the EMA data quality (Liao et al., 2016), external events (Burke et al., 2017, Runyan
et al., 2013), gender (Liao et al., 2016), age (Liao et al., 2016), the frequency of EMA record (Norman et al., 2007) and the type of behavior (Rooksby et al., 2014) also affect the EMA’s effect. Without learning how EMA, or more broadly, self-report, impacts the behavior change mechanism, it is hard to understand how to maintain long-term behavior change.
In response to this research gap, we aim to detect what role EMA, or more broadly, self-report plays in affecting behavior change. By using a case study of sleep cycle change, we reported how perceived effects and actual effects of self-report along with DBCI can in return affect health related behavior change. This study eventually provides academic and designer community a better understanding of behavior change intervention by using self-report.
METHODOLOGY
Participants 20 Chinese (aged between 20 and 25) were selected as participants (Mairs and Mullan, 2015, Patel et al., 2017). They are divided into two groups: 10 participants who used self-report to record their behavior change (experi- mental group) and 10 participants who do not who use self-report to record their behavior change (control group). Both groups used the same EMA digi- tal application, “Sleep Cycle,” to record their bedtime. Since all participants are Chinese, we ask all participants to use their native language (Chinese) in their diary and interviews, to better express themselves. The results were translated and reported in English.
Protocol Controlled-Experiment For all participants, before the study started, an information sheet and a con- sent form were given to and signed by each participant. In the information sheet, they filled their current and target bedtime and wake-up time, the length of their necessary sleep, the aim to change bedtime and the attitude towards changing bedtime. Then, the participants were asked to download “Sleep cycle.” A demonstration was given to show how to press the start button when the participants decide to sleep and how to end it when they wake up.
For participants in experimental group, they were asked to download the application “Meiriji.” They were asked about their planned methods to write the diary in “Meiriji.” Participants in the experimental group could write the diary at any time in the day as they wish.
The participants then participated in a 28-day experiment. The length of this experiment is determined by a commonly agreed theory in psychology, that one habit can be changed if it lasts for 28 days (Pirolli et al., 2017). Every day, when participant decided to go to bed, they would press the start button. When they woke up, they would end the record. For those who in the experimental group, there is an alarm of “Write a diary” to remind them. They were told to keep a diary whenever they want in “Meiriji”, but at least once per day.
How Self-Report Affects Digital Health-Related Behavior Change 17
The participants acknowledged that they voluntarily took part in the rese- arch and would not receive money for their participation. All participants were asked to start the experiment on the same day.
Semi-Structured-Interview A semi-structured interview was conducted with the experimental group within 48 hours after they finished the 28-day experiment. The interview lasted 15 to 30 minutes. It provided an opportunity for participants to share their experience and feedback of using self-report to achieve behavior change. Interview questions focused on the participants’ experiences of using the self-report; involving how easy or difficult it was to keep a diary related to behavior change; in what degree participants agree that writing a diary could reflect self-evaluation on behavior changes; at what level do participants think self-report would affect behavior change.
The control group did not participate in a semi-structured interview, because they did not keep a track of their exact behavior change, so their report is likely to be unreliable, and researchers are not able to identify if their statement is correct.
RESULTS
In this section, we will report whether there is a significant relationship betw- een using EMA and behavior change, and how users find EMA as a self-report method.
The Relationship Between EMA and Bedtime Change After the experiment, researchers exported the bedtime data from the users’ phones. The bedtime data and target bedtime data was transited into numeric values for analysis (table 1).
The experimental group mean bedtime is 11.74 (SD = 0.59), while the control group mean is 12.41 (SD = 0.62). We then calculated the difference of original bedtime and target bedtime (Table 3). The control group mean difference is 0.82 (SD = 0.68), while the experimental group mean is 0.43 (SD = 0.46). The time effect on behaviour change was analysed by One-Way ANOVA. From the results, it could seen that there is no significant effect of time effect and the performance of behavior change, as F(3,559) = 3.6, p= 0.013 (<0.05). not find a significant time effect, as F(3,559) = 3.6, p= 0.013 (<0.05). The self-report effect on behavior change is shown in Table 3. We did not find a significant effect of using EMA on the performance of behavior change, as F(1,559) = 61.788, p= 0.00 (<0.05).
Theme Analysis Thematic analysis was applied to the Chinese transcripts to ensure to capture the key contents in the text. NVivo is used to code. The coding is an inductive content analysis. Then the themes were translated into English together with participants’ quotes.
18 Yin and Yu
Table 1. Relationship between real data and translated data.
How-Self-Report-Affect-Behavior-Changing There are four codes on how self-report affect behavior changing: less cheating, review, emotion and remind internal motivation.
Less Cheating Self-report is a good way to reduce cheating for a sense of achievement because it increases the cost of cheating like making faking sentence and time cost. Two people explained the reason from the emotional results like guilt and shame. P9 mentioned that “I could cheat in the diary, but I would feel guilty about it. I did not want to feel guilty because of this minor issue.” P2 had a different thought on cheating but gave another similar reason on not cheating, “I think cheating is acceptable. However, the reason why I did not cheat is that I know if there is some incoherence between my diary and data, everyone would know I was cheating. It would be a shameful thing.” However, some people said that there is no need to cheat naturally and the self-report cannot reduce cheating because the cheating can be easily achi- eved from blurry sentence in diary. Therefore, self-report is a good way to reduce fake data. Notably, this is based on the interview and a comparatively small data set, and thus more research is needed.
Review Participants have mentioned self-report is a good way to review what they have accomplished. Few reasons have been given. Firstly, the review that has been written before can recall what participants have done. At the same time, the review process can trigger the introspection. P4 mentioned that, “It’s interesting. I can find some pattern when I read the diary before. I find a loop of my sleep time. I tend to sleep after 12am on Sunday. Maybe because I still think it is a weekend.” This introspection may trigger the adjustment of the behavior. P6 pointed out, “Diary could help me adjust my bedtime plan. Like, if I know these days, I am going to stay up late, I will analyze what happened and try to make the target bedtime later or rearrange my daily schedule.” Also, the introspection may simulate behavior change. P6 explained that, “When I keep a diary, I’d like to see what happened before. If I see I have written that I felt sorry for staying up late, I would sleep earlier tomorrow.” Sometimes self-report may be not a good way to help behavior change if there are too many negative statements. Too much negative review may trigger
How Self-Report Affects Digital Health-Related Behavior Change 19
negative emotions and barrier the behavior change. P5 mentioned “When I saw the diary, I wrote ambitiously, like ‘I will go to bed early tomorrow’. The same sentences made me know I could not change my bedtime. I just gave it up and relieved myself.” P7 also pointed out that, “Sometimes I could not remember what I have written means. In other words, the diary is useless. It could not trigger my memory, which is frustrating.” However, not all people like to go through the diary because the poor interface operation (P2); P2 commented, “The application I was required to use does not show what I have written directly. I wanted to read what I have written but I did not want to click.”
In summary, most participants tended to go through what they have writ- ten, and it is a good way for people to recall what happened, generate feedback and help them adjust their bedtime. However, the effect of the recall varies depending on individual attitude.
Related-Emotion Effect When using self-report, participants experienced different emotions, inclu- ding guilt, stress, sadness, and happiness. As for guilt, it is a negative emotion. Although some participants (P4, P5, P9) admit that they may generate guilty when they did not change their behavior as planned, the guilty will not affect the behavior change. This indicated the guilty is not a perceived effect that strong enough to change the behaviour. P5 explained that, “Guilt is for me, but not affect others. As it does not matter, I guess that is why it did not change my bedtime.” P4 tried to explain from a different perspective, “I think guilt works instantly, which did not last until my sleep time, thus it did not change my behavior.”
As for stress and happiness, they are different emotions. Although the two emotions were generated at different stages, both emotions are proved to have increased participants’ motivation to change their behavior (P1, P3, P6, P8, P10). For example, P10 mentioned that “I felt a little bit stressful when I knew the experiment. After all, I hoped I could make it.” While P6 mentioned that “I felt happy that I made it. It gives me more motivation to keep it.”
As for sadness, it is a controversial emotion. Some participants think it is a positive emotion because it could motivate people (P3, P5, P9). P3 explained, “I felt sad when I knew I did not do it for a long time. But that is the reason why I need to sleep early.” However, results of some participants refute it because they think the sadness will enlarge the negative emotions (P2, P4, P8). For example, P5 refuted this view and pointed that “I felt sad when I did not go to sleep by the target time. I could not control myself. I just wanted to give up.”To summarize, guilt has no significant effect on behavior changing in this study, while stress and happiness increase people’s motivation. Sadness’ effect remains controversial.
There are two more negative emotions arising from using EMA: being bothered and being bored. If the two negative emotions caused by using self-report will affect the behavior change is still unsure.
Bothered affect generated because of the repeated writing diary behavior (P2, P7). P7 mentioned that “I have too many things one day. I cannot remem- ber keeping a diary. This is not my daily behavior and I know I will only do
20 Yin and Yu
this action for 28 days. I don’t think it worth remembering. So, I have to use an alarm to remind me every day. Rather than hard, it is more likes a bother thing.”
Another emotion is boredom. It is an emotion that will bring negative effect on behavior changing. As P7 mentioned, “Diary is boring. But if it is a game, I will like it.” There are few reasons promoted. Firstly, it is the unsuitable interface. As P1 mentioned “Self-report is a boring way. I liked it at the first 3 days because of its Chinese Style. But then I did not want to see the diary application again. The dark blue is a colour for my mother”. Another reason of boredom is that the diary keeping is not a habit for participants. P9 explained, “Boring. It is not a common thing I would do. I have not kept a diary for nearly 20 years. I feel shame to see what I have written.”
Remind Internal Motivation Self-report could remind internal motivation because it helps people clarify target aims by daily feedback(P3, P6, P10). P3 mentioned that, “I want to sleep before 11pm, which benefits my health. By writing a diary every day, I am reminded my aim every day. It drove me to go to bed early.” P6 agreed with this and summarized that “Diary seems like an internal pressure on me to manage health by showing the fact that I do not sleep at the time I planned again and again.”
Efficiency Participants have three views on the efficiency of self-report: effective (P1, P4, P9), no relation (P2, P6) and useless (P2, P7, P10). Some participants think efficiency of self-report is effective but when they were asked to explained why they hold this idea, they cannot give an answer (P1, P4). When further asked to give an example on efficiency of self-report, participants may also not be able to give answer. This may be because the efficiency of self-report is a progressive change instead of a huge sudden change and hard to be clear awared (P4).
Since the efficiency of self-report are not clear awared, not all people hold the same view on efficiency of self-report is effective. Some participants thus mentioned efficiency of self-report may not related with behavior change and the behavior change may be caused by outside factors (P2, P6). For example, P6 pointed out, “Maybe it is efficient. I cannot make this conclusion because outside factors could also change my bedtime. Like when I read news about the negative effects of staying up late, I would go to bed earlier.”
An even more different view is that self-report is useless on behavior chan- ging (P2, P7, P10). These participants only considered self-report as a data recording approach and have not efficiency on behavior change. For exam- ple, P7 refuted the benefits of self-report from usability, “Phone is used to make life easy. Keeping a diary should not be done in an app. There are so many applications that can record your life like Facebook or Twitter. I mean you do not need a diary to help you record your behavior.”
Therefore, most people admitted that self-report would help them change behavior subjectively. However, they could not express the benefits in a detai- led way because the change has been progressive but not sudden. Some
How Self-Report Affects Digital Health-Related Behavior Change 21
participants believed that self-report is not related to behavior change or not sure if it could change the behavior, because the external factors were not controlled. The rest of the participants thought self-report is useless, as they regarded self-report only as a record or they disagreed with the use of a diary.
DISCUSSION
Comparison Between Controlled Study and Interview The results displayed above indicated that EMA, as a self-report method, will not affect the performance of behavior change and therefore, it could be a suitable method to report real-time behavior change every day. EMA can be considered as a typical and standard self-report method in some levels, obtaining the most typical and generic features of self-report, so the conclu- sion is expanded from EMA in this study to self-report in general (Dunton et al., 2014, Shiffman, 2000).
A polarized set of opinions of participants were witnessed, that some found it useful, while others disagree. This difference can be explained among those who think self-report is helpful but could not point some detailed impro- vement. For example, P1 mentioned “I do not know (where it helps me). Maybe it is a long-term change. I cannot give you an example. But I know it helped me.”
Participants were also asked if they would keep using this method in the future to help their behavior change, in a broader range of contexts. Intriguin- gly, most of them would not, because they felt the self-report had no effect or the effect was not significant. Although these people also mentioned some positive effects on self-report, the positive effect are more on other behaviors, such as willingness to use self-report method, less cheating, reviewing previ- ous diary, triggering positive emotion and reminding internal motivation. In other words, the self-report may trigger some actions that could indirectly affect behavior change. Although self-report could affect these actions, this effect was not strong enough to affect the performance of behavior change. To be more specific, the affects (including guilt, happiness, boredom, both- ering, stress and sadness) generated from using self-report, only existed for a short time and may not affect the behavior changing process. Also, self- report could make people review what has happened previously, but it is not strong enough to change the performance. Moreover, some improvements are “faked” or imagined.
It is also interesting that although many participants think self-report works, in fact, there is no significant effect been found in this study. From previous results and discussion, we found the initial experiment showed that self-report did not, in fact, affect behavior. This is offered as evide- nce that self-report is an appropriate method to collect data about behavior change. However, the semi-structured interviews suggested that self-reporting in fact brought numerous benefits, including discouraging cheating, encoura- ging reflection among users, and encouraging users to further improve their behavior. The two results answered the research question of “what role self- report-based approach plays in health-related behavior change.” Rather than an actual performance improvement, the self-report is more likely to have
22 Yin and Yu
a perceived effect on behavior change, such as increasing the confidence and beliefs of a person in behavior changing process. This result indicated that perceived effects of self-report are far stronger than actual effects of self-report.
Limitations and Future Research There are some limitations on this research. They will be discussed as follows:
Although the study has promoted the conclusion that self-report will not affect the behavior change. The quantitative results involve participants who think self-report are a hard method and an easy method together. This might be the cause of no significant behavior change. Also, the results may be affe- cted by other external interference factors, such as, age, position, sleeping time, target sleeping time, and the level of willingness to change the behavior. The co-effect of these factors is uncertain and may affect the reliability of the results.
It is also worthy pointing that the results are only directly applicable for Chinese students using smart-phone to change sleeping behavior. We expan- ded our conclusion on the basis of the results to a broader context from EMA to self-report, from sleeping time to digital health-related behavior changes, from Chinese people to other cultured people, from applications in smart phones to represent DBCI. These representations may bring limitations to the study and reduce the reliable about the results. The other health-related behavior change, the other cultural participants, and the other self-report methods should also be considered in future research.
REFERENCES
Burke, L. E., Shiffman, S., Music, E., Styn, M. A., Kriska, A., Smailagic, A., . . . French, B. (2017). Ecological momentary assessment in behavioral research: addressing technological and human participant challenges. Journal of medical Internet research, Volume 19 No.3. Conroy, D. E., Yang, C.-H., & Maher, J. P. (2014). Behavior change techniques in top-
ranked mobile apps for physical activity. American journal of preventive medicine, Volume 46 No.6. pp. 649–652. Dunton, G. F., Dzubur, E., Kawabata, K., Yanez, B., Bo, B., & Intille, S. (2014).
Development of a smartphone application to measure physical activity using sensor-assisted self-report. Frontiers in public health, Volume 2 No.12. Hardcastle, S. J., Hancox, J., Hattar, A., Maxwell-Smith, C., Thøgersen-Ntoumani, C., & Hagger, M. S. (2015). Motivating the unmotivated: how can health behavior be changed in those unwilling to change? Frontiers in psychology, Volume 6. Liao, Y., Skelton, K., Dunton, G., & Bruening, M. (2016). A systematic
review of methods and procedures used in ecological momentary assessments of diet and physical activity research in youth: an adapted STROBE checklist for reporting EMA studies (CREMAS). Journal of medical Internet research, Volume 18 No. 6. Mairs, L., & Mullan, B. (2015). Self-monitoring vs. implementation intentions: A comparison of behaviour change techniques to improve sleep hygiene and sleep outcomes in students. International Journal of Behavioral Medicine, Volume 22 No. 5. pp. 635–644.
How Self-Report Affects Digital Health-Related Behavior Change 23
Norman, G. J., Zabinski, M. F., Adams, M. A., Rosenberg, D. E., Yaroch, A. L., & Atienza, A. A. (2007). A review of eHealth interventions for physical activity and dietary behavior change. American journal of preventive medicine, Volume 33 No. 4, pp. 336–345. Patel, P., Kim, J. Y., & Brooks, L. J. (2017). Accuracy of a smartphone application in
estimating sleep in children. Sleep and Breathing, Volume 21 No. 2, pp. 505–511. Patrick, H., & Williams, G. C. (2012). Self-determination theory: its application to health behavior and complementarity with motivational interviewing. International Journal of behavioral nutrition and physical Activity, Volume 9 No. 1. Pirolli, P., Mohan, S., Venkatakrishnan, A., Nelson, L., Silva, M., & Springer, A.
(2017). Implementation intention and reminder effects on behavior change in a mobile health system: a predictive cognitive model. Journal of medical Internet research, Volume 19 No. 11. Rooksby, J., Rost, M., Morrison, A., & Chalmers, M. (2014). Personal tracking as lived informatics. Paper presented at the Proceedings of the SIGCHI conference on human factors in computing systems. pp. 1163–1172. Runyan, J. D., Steenbergh, T. A., Bainbridge, C., Daugherty, D. A., Oke, L., & Fry,
B. N. (2013). A smartphone ecological momentary assessment/intervention “app” for collecting real-time data and promoting self-awareness. PloS one, Volume 8 No. 8. Shiffman, S. (2000). Real-time self-report of momentary states in the natural environment: Computerized ecological momentary assessment. The science of self-report: Implications for research and practice. pp. 277–296. Yang, C.-H., Maher, J. P., & Conroy, D. E. (2015). Implementation of behavior
change techniques in mobile applications for physical activity. American journal of preventive medicine, Volume 48 No. 4. pp. 452–455. Yardley, L., Spring, B. J., Riper, H., Morrison, L. G., Crane, D. H., Curtis, K., . . .
Blandford, A. (2016). Understanding and promoting effective engagement with digital behavior change interventions. American journal of preventive medicine, Volume 51 No. 5, pp. 833–842.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Healthcare and Medical Devices
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-27-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional