Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Design and Evaluation of a Device for Ecological Momentary Assessment with Workers in a Garment Factory

1 Fraunhofer Portugal AICOS, Porto, Portugal ↗
2 Centre for Psychology at University of Porto, Porto, Portugal ↗
3 Faculty of Psychology and Educational Sciences of the University of Porto, Porto, Portugal ↗ ↗
4 NST Apparel, Vilela, Portugal ↗
Publication year
Digital Object Identifier10.54941/ahfe1001796

Abstract

Chronic sick leaves in Europe are strongly related to mental health issues. Ecological Momentary Assessment (EMA) enables collecting data on mental health, but it is challenging to implement in fast-paced and controlled work environments. We describe the design of an EMA device for workers in a garment factory, along with individual data visualisation, which were evaluated in naturalistic setting with eight workers for one week during work hours to understand whether and how participants would engage with the device. Participants easily used the device at work and appropriated it in different ways, mainly by extending its declared function. Confrontation with individual data visualisations led to episodes of self-discovery and participants suggested extending the use of the device to more operators to get a picture of workers’ wellbeing. However, participants expressed doubts about data validity.

INTRODUCTION

Prevalence of mental health disorders in Europe is high and Portugal is one of the European countries with the highest prevalence, standing at 23% (NOVA Medical School, 2013). Depression and anxiety are responsible for up to 50% of chronic sick leaves in Europe (World Health Organization - Regional Office for Europe, 2021). There are currently different digital tech- nologies and companies providing services for employers to assist employees in maintaining mental health. However, in factory shopfloors there have been scarce studies examining employees’ mental health (Tsutsumi et al., 2009). Digital technology can enable passive data collection on working conditi- ons and employees’ physiological signs to infer dimensions of psychosocial, environmental, and ergonomic risk exposure at work. However, when placed over employees’ bodies it be-comes intrusive. Self-report is an alternative, but,

© 2022. Published by AHFE Open Access. All rights reserved. 185

applied asynchronously, it is prone to recall bias and poor ecological validity (Doherty, Balaskas and Doherty, 2020). Therefore, real-time self-report is expected to provide more reliable data for the analysis of mental health at work. These approaches are known as Ecological Momentary Assessment (EMA) in psychology, Experience Sampling Methods (ESM) in HCI or, in some cases, as Patient Reported Outcomes Measures (PROM) in clinical studies.

We have designed an electronic EMA device for factory operators. In this case study, we report on the human-centred process of designing for appropriation, which resulted in a portable device that allowed users to self-report Pressure, Pain and Social Support. The device was evaluated in a longitudinal study in a naturalistic setting with eight factory operators, who used the device continuously for one work week. On the following week, each user was individually shown visualisations of their data and they were inter-viewed about their experience with the device.

To the best of our knowledge, our case study describes, for the first time, how industrial operators used an electronic device for real-time self- reporting of body and mental states. This is our first contribution. As a second contribution, we identify design aspects that facilitated appropriation of a self-reporting device, along with an understanding of how the appropriation process unfolded.

RELATED WORK

The widespread use of smartphones made it a convenient medium for EMA(Doherty, Balaskas and Doherty, 2020). The method has been exten- sively used with white-collar workers by prompting participants to answer questions on their smartphones at specific points in time, as well as to collect passive data, such as step count. There are also accounts of wearable devi- ces being used for passive data collection on factory or healthcare workers (Heikkilä, Honka and Kaasinen, 2018; L’Hommedieu et al., 2019). A search and comparative analysis for portable and wearable devices designed for non- white-collar workers retrieved a single product which used EMA – the IBM Maximo Worker Insights – by prompting the workers with a post-shift survey. We could find one example of a long-term diary study to report on wellbeing by factory workers (Bellingan et al., 2020), but no accounts of a portable ele- ctronic EMA device being used in a factory shopfloor to report on body and mental states that our study could refer to. Nevertheless, there are accounts of how blue-collar workers appropriate tools (Cross, 2012) and how perso- nal characteristics have implications in self-tracking (Heikkilä, Honka and Kaasinen, 2018).

How users appropriate designs to meet other purposes than pre-defined use has been explored for years, ranging from theory-generation to elicitation of design guidelines (Salovaara and Tamminen, 2009). Appropriation is also associated with a spectrum, from reconfiguring the technology while using it for the same ends, to using it for other ends than those envisioned by designers (Dourish, 2003). The appropriation process encompasses adoption, adapta- tion and incorporation in practice (Dourish, 2003; Dix, 2007). In Dourish’s

Design and Evaluation of a Device for Ecological Momentary Assessment with Workers 187

examples, as in other accounts of appropriation processes (Kohtala, Hyysalo and Whalen, 2020), user-information interaction plays a strong role.

In the case study we report, it is the physicality of objects that plays a prominent role. Here, concepts of function that emerged primarily in indu- strial design are relevant, as shown next. An object has a declared function (Munari, 2008) – the primary end to which it was designed. When used with no qualifier, ‘function’ usually refers to utilitarian function. There are other types of functions: Löbach (2001) identified the practical (declared function), symbolic (how objects resonate with users’ psychosocial and cultural frames of reference) and the aesthetic (sensory sensations). In the function complex, Papanek (2006) identified use (declared function), method (how the product is created), association (psychosocial and cultural frames of reference), aesth- etics (elegance), need (genuine human needs) and telesis (use of appropriate resources regarding time and context).

Akrich (1998) analysed how users changed the declared function of desi- gns and suggested a typification of: displacement, adaptation, extension and detour. In displacement, declared function and object configuration are main- tained, but the target of the function is changed, such as using a hair drier to dry varnish. An adaptation maintains declared function, but configura- tion is changed, as in improving accessibility of a product. An extension adds functions to the declared function, often by changing configuration. A detour implies changes in declared function and in configuration. All the author’s examples focus on practical functions. We will discuss how participants in our study engaged in an extension with slightly different contours than those of Akrich.

THE CONTEXT Context of use and user group are expected to influence how products are appropriated (Kohtala, Hyysalo and Whalen, 2020), as well as how users engage with EMA (Doherty, Balaskas and Doherty, 2020). Therefore, we take a moment to de-scribe garment manufacturing in Portugal and the factory where our device was designed and evaluated. Textile/garment manufacturing is one of the main industries in Portugal, accounting for 20% of employ- ment within the manufacturing industry and representing 4% of the GDP (Lourenço, 2019).

Our study took place in the Northern region of the country (Lourenço, 2019), where most companies are located. Operators are usually women and their work interdependent, as a garment piece moves from one workstation to the next. Operators tend to be grouped under line managers and associ- ated with mainly one machine, with occasional uses of other machines. The business works on cost-per-minute and, as in other countries (Chakravarti, 2011), garment manufacturing is commonly associated with challenging phy- sical and psychosocial demands. The factory where our study took place is a garment manufacturer with circa 300 workers working a single shift, where each sector is led by a line manager. The line manager is respon- sible for daily productivity goals. Fridays are usually order delivery days and can, thus, be felt like more stressful days in some sectors. The factory

Figure 1: Workstations and objects documented during periods of observation.
Original PDF Figure 1:

produces garment pieces for high-end clients, to whom fault tolerances are very low.

DESIGN OF THE EMA DEVICE The design of the device began after a 4-week period of observations, inte- rviews, and contextual inquiry with workers (Figure 1). We designed the device to be portable (e.g., to be used inside a pocket) because operators may change workstations and we did not force device placement over the mach- ine to respect operator-machine bonding which we witnessed during initial fieldwork. We also designed two accessories: one to clip the device on clothes and another to glue it on a surface. Some operators work while seated and others while standing, therefore, the device was designed to allow different ways of using it. The battery was designed to last for five working days, so that operators would not have to charge it during the week. We sought to include aesthetic values shared by operators (e.g., neatness) and to make the use discreet (including being used hidden from sight to protect privacy).

We selected three endpoints to avoid the burden of having to memorize many states, to reduce interaction time, to avoid overwhelming operators and to enable the design of a small-scale device. We combined negative and positive valence states based on the hypothesis that operators would feel con- strained to interact with the device if others knew they were always going to report negative valence states. Button position was inspired by three fingers resting on the device and had the three buttons on the top surface along with initials P, A and D which, in Portuguese, stood for Pressure, (Social) Support and Pain. These endpoints were selected based on a prior phase of user rese- arch and validated by operators. Opposite to the buttons, there was a full legend (Figure 2). We established that information on endpoints should be redundant using visual and tactile means (each button is identified by posi- tion, texture, initial and legend on the back) to reduce memorability burden and allow to use the device hidden from sight.

The device’s electronic components collected and recorded button presses. Electronics were built around the microcontroller Adafruit GEMMA M0, which can be directly powered by a battery and exposes three GPIOs to which the push buttons can be connected. The state machine was optimized for a monitoring cycle, for transferability of data to human-readable format, clock synchronization, and battery optimization.

Design and Evaluation of a Device for Ecological Momentary Assessment with Workers 189

Figure 2: Prototype of device, including legend sheet (centre) and data visualisation.
Original PDF Figure 2:

For data visualisations, we chose one-hour intervals, since these were reference-intervals for operators, who report hourly productivity by hand as part of their job. We chose common forms of temporal representati- ons of data: line charts for weekly view, scatter chart for hourly view and comparable concentric circles for absolute values.

METHODS AND PARTICIPANTS The study received approval by an ethics committee and all participants signed informed consent forms, which, given the sensitive nature of the end- points and the small sample, included the guarantee that data on reported endpoints would not be shared, not even in aggregated format. Eight opera- tors (all women, average age 39.4, SD = 12.0) took part in a longitudinal naturalistic evaluation of the device. All performed sewing operations, but belonged to different sectors, which stood at different phases of the process of building a garment piece.

Participants used the EMA device for five consecutive working days. We chose the duration of one week as a compromise between what would be pos- sible to prototype in terms of battery duration and accuracy, combined with enough days to capture changes in production, deadlines for dispatchment and what we hypothesised would be the mini-mum time for participants to begin appropriating the device. The following week, we conducted indivi- dual, semi-structured 2:1 interviews. The first part of the interview covered how the week with the device had been, how the device was used and whe- ther there were any usability issues, whether operators were aware of their body and mental states and reported them, and whether there were any social issues in the use of the device. In the second part of the interview, we handed an A4 sheet with the operator’s individual graphs of the week and sought to understand whether information was understandable, meaningful, and which potential scenarios and/or consequences of future device use opera- tors envisioned. Interviews were recorded, transcribed verbatim and analysed inductively using Thematic Analysis (Braun and Clarke, 2006).

RESULTS

Device Use It was very easy for participants to use the device and memorise what each button stood for. None used the paper legend. Participants using the device

in the pocket always placed the device in the same position and memorised the order of the buttons, not texture. Initials below each button were useful and consulted in case of need. Participants reported lack of clear feedback to confirm a button press.

With one exception, participants stated that being aware of one’s states was easy. The difficulty arose when deciding whether what they were feeling was sufficient to justify pushing the button. The person who reported trouble in understanding how she was feeling resorted to a colleague to help her decide.

Criteria to report pain could be a sudden rise in intensity or the moment when the person took pain killers. There was also a difficulty in deciding whether to report pain if it was deemed unrelated to work. Support is almost always associated with the line manager. Reports on pressure were associated with external or internal factors. Stress appeared as a relevant mental state – either as a suggestion for a fourth button or as a synonym for pressure. Participants commented the work week was calm, joking that, in stressful weeks, ‘the buttons would even spring out’ or that we ‘would be left with no buttons’.

Variability in participants’ criteria to report a certain state was present in this small sample. Therefore, it is likely that, when increasing the number of operators using the device, the research team will have to adopt one of two options: define, together with operators, which would be the criteria to push each button in order to enable aggregated data or eliminate the idea of aggregated data and assume the resulting individual data are only meaningful for idiographic analysis, in line of what was highlighted by Doherty et al. (2020).

Three participants changed object position during the week. The final posi- tion was found early in the week. Two used it in the chest pocket, one in the side pocket and the remaining five on the machine (Figure 3). The single cri- terion for selecting the final position was finding the most practical place: a place which would enable a quick interaction with the device, while not interfering with work. Participants who placed the device on the machine also highlighted that, with the object in plain sight, they were less likely to forget about it.

Participants took care of the device by placing it in the locker overnight, covering it at the end of the workday or bringing it home. The latter happened only on the first day. In one case, the participant brought it home with her, but the next days she forgot to do it. In another case, the participant wanted to bring it home, but was afraid of breaking it when detaching it from the machine and decided to leave it there.

Self-Discovery Participants recognised themselves in the individual graphs. There were two episodes of self-discovery. For one of the participants self-discovery came in the form of a surprise because, even though she was aware that she had pushed a certain button, she was not aware she had pushed it so often: ‘This makes me think’, she said. First, she attributed the number of entries to false positives and later she went through the weekdays in her head and concluded

Design and Evaluation of a Device for Ecological Momentary Assessment with Workers 191

Figure 3: Operator using the device inside the pocket (two pictures to the left) and devices glued to sewing machines.
Original PDF Figure 3:

that she might have really pushed the button that many times. For another participant, self-discovery came in the form of identifying a pattern: a certain button was always pushed at roughly the same hour of the day.

There was one case in which the readings of the device were completely wrong, and the participant immediately noticed it. Another participant told us she was not surprised when we highlighted in her graph that she had pushed the same button twice in a row. There is an inevitable discussion about to what extent people will believe the device’s readings and if it is pos- sible that, at times, they will trust the device readings more than they trust their memory.

Added Functions Participant A: I liked using the device. Interviewer: You did? Participant A: Yes. It’s different, I don’t know. We’re feeling something and we can push the button. We don’t need to tell anyone. It was like we off-loaded what we were feeling into the device. …… Participant B: On a busy week, I agree the device should be next to us [laughs]. But that’s me joking. I like joking around. (...) But it could help us a little bit… …… Participant B: And thank you for sharing this with us, because this way we learn about stuff, and we know that we can… There are things that help us.

None of the participants declared feeling embarrassed or constrained in any other way to use the device. The device was a motivator of social interactions among colleagues and with managers. For some participants, the device accu- mulated or could accumulate other functions. For Participant A the device could be used as an alternative to externalising one’s feelings. Another parti- cipant also envisioned this function, although she did not share whether she had used it in that way. Participant B’s first quote illustrates another function that emerged in interviews: using the device as a shield from line managers, regardless of their social relationships being qualified as good. On several occasions, the device was an enabler of social interactions among research participants, colleagues, and line managers. It triggered discussions about what the research project was, what the device was meant for, and it was

also at the centre of tongue-in-cheek menaces to line managers. Participant B’s second quote happened after she saw her own data and right before she left the interview room. It is not clear whether she was referring to the device in sight, to the data visualisation or both.

Speculation on Future Use Having more operators take part, participants said, could provide a picture of collective wellbeing. However, participants questioned validity of self-report data, as they speculate that not all operators tell the truth even to occupa- tional physicians. While operators often commented they knew the results of button presses were to researchers’ eyes only, some occasionally dropped sentences suggesting some data might leak. Therefore, we cannot know whe- ther operators were being truthful in their self-reporting. Opinions differed regarding who should have access to data. To some, aggregated data could be visible to decision makers inside the company, if operators saw changes for the better based on this exercise. A contrary opinion was that if data, even if aggregate, were to be shared inside the company few would want to take part.

DISCUSSION

The exercise of self-reporting using an EMA device and self-confrontation with individual data seem to have been meaningful and valuable to operators: participants engaged in the evaluation, discussed their criteria for pushing or not pushing buttons, suggested to extend the use of the device to fellow operators, and shared suggestions to improve the device. The device enabled a right to expression in a context where this is traditionally limited. Flexibility offered by design was harnessed by participants, who decided where it would be more comfortable for them to use it and who, in some cases, experimented with different ways of using the device. Flexibility thus enabled appropriation (Dix, 2007).

The declared function of the device was to collect data on body and mental states with the intention of making it visible. There was an expectation among researchers that, in the future, aggregated data (together with other sources of information) could inform how to improve working conditions. Howe- ver, what some participants did, in Akrich’s (1998) terms, was to extend the original function of the device as an anti-stress device of sorts and as a kind of protection shield. The device kept configuration, but the realm of functi- ons was extended. Interestingly, one of the extended functions (anti-stress object) was practical, but the other was not. The device-as-shield, even if used mockingly by the participants, transforms this into a communication device signalling a message to others about its existence.

In both extensions of use, the device acquires new functions, but the inten- tion is the same: to achieve better working conditions in the eyes of the operators. An anti-stress object allows operators to unload a burden off their chest, which is expected to make them feel better. In the second case, the device shields operators from external pressure – one of the endpoints. By appropriating the device and extending its meanings, operators achieve

Design and Evaluation of a Device for Ecological Momentary Assessment with Workers 193

immediate and direct action which the device’s declared function could not achieve on its own.

LIMITATIONS There are important limitations in our study. First, the evaluation of the EMA device emerged after a long process of user research during which rap- port had been established with the participating operators. Despite useful, it might also have introduced courtesy bias. From the comments, we can infer that some participants might have not been truthful in their self-report for fear of consequences. This highlights the importance of having EMA data be complemented with interviews, observations or other qualitative meth- ods. On the other hand, it affirms the importance of understanding what could lead workers to be untruthful in EMA. Furthermore, continuous invo- lvement in a design process might have coerced operators into taking part in this evaluation. Finally, sample size and the short duration of the evaluation do not allow us to capture nuances regarding different operators, habitus for- ming around EMA nor whether EMA changes depending on calm versus busy weeks.

CONCLUSIONS AND FUTURE WORK How designers design EMA devices – not only in their configuration, but also on the process of design itself – and how users appropriate the devices have implications in how EMA is performed. It was possible to introduce a device for EMA within a fast-paced and highly controlled work environment. The meaning and value of an EMA de-vice for operators were relevant. This was shown through engagement in the process, self-reflection over own data and appropriation of the device in the form of an extension to its declared function to serve the same intention: to promote wellbeing at work.

The design emerging from this process was a result of context, researchers and workers involved. We cannot fully anticipate whether the device can be transferred to different contexts without adaptations. Future work should address longer evaluation periods and larger samples to explore interaction between appropriation, data validity and implications for workers.

ACKNOWLEDGMENT

Project OPERATOR (NORTE-01-0247-FEDER-045910) leading to this work is co-financed by the ERDF - European Regional Development Fund through the North Portugal Regional Operational Program and Lisbon Regi- onal Operational Program and by the Portuguese Foundation for Science and Technology, under the MIT Portugal Program (2019 Open Call for Flagship projects). Author roles: Study design, data collection, data analysis, device design, paper writing (ACB). Data analysis, data visualisations, section 4 (CC). Device design, literature re-view, data collection, section 4 (RM). Ele- ctronics design, section 4 (GL, CR). User research (LC, DS, SM). Logistics and participant recruitment (SBA).

REFERENCES

Akrich, M. (1998) ‘Les utilisateurs, acteurs de l’innovation’, Éducation permanente, Paris: Documentation française, pp. 79–90. Bellingan, M. et al. (2020) ‘Capturing the psychological well-being of Chinese

factory workers’, International Journal of Operations & Production Management, 40(7/8), pp. 1269–1289. doi:10.1108/IJOPM-06-2019–0492. Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative Research in Psychology, 3(2), pp. 77–101. doi:10.1191/1478088706qp063oa. Chakravarti, L.Z. (2011) ‘Material Worlds: Denim on the Globalized Shop Floor’,

Textile, 9(1), pp. 62–74. doi:10.2752/175183511X12949158771473. Cross, J. (2012) ‘Technological intimacy: Re-engaging with gender and technology in the global factory’, Ethnography, 13(2), pp. 119–143. doi:10.1177/1466138111410621. Dix, A. (2007) ‘Designing for appropriation’, in Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 2. Swindon, GBR: BCS Learning & Development Ltd. (BCS-HCI ‘07), pp. 27–30. Doherty, K., Balaskas, A. and Doherty, G. (2020) ‘The Design of Ecological Momen-

tary Assessment Technologies’, Interacting with Computers, 32(3), pp. 257–278. doi:10.1093/iwcomp/iwaa019. Dourish, P. (2003) ‘The Appropriation of Interactive Technologies: Some Lessons from Placeless Documents’, Computer Supported Cooperative Work (CSCW), 12(4), pp. 465–490. doi:10.1023/A:1026149119426. Heikkilä, P., Honka, A. and Kaasinen, E. (2018) ‘Quantified factory worker: desi-

gning a worker feedback dashboard’, in Proceedings of the 10th Nordic Confere- nce on Human-Computer Interaction. New York, NY, USA: Association for Com- puting Machinery (NordiCHI ‘18), pp. 515–523. doi:10.1145/3240167.3240187. Kohtala, C., Hyysalo, S. and Whalen, J. (2020) ‘A taxonomy of users’ active design engagement in the 21st century’, Design Studies, 67, pp. 27–54. doi:10.1016/j.destud.2019.11.008. L’Hommedieu, M. et al. (2019) ‘Lessons Learned: Recommendations for Imple-

menting a Longitudinal Study Using Wearable and Environmental Sensors in a Health Care Organization’, JMIR mHealth and uHealth, 7(12), p. e13305. doi:10.2196/13305. Löbach, B. (2001) Design Industrial: Bases Para a Configuração dos Produtos Industriais. 1st edn. São Paulo: Blucher. Lourenço, M. (2019) ‘Indicadores da Central de Balanços do Banco de Portugal sobre

o sector Têxtil e Vestuário’. 21.o Forum da Indústria Têxtil. Available at: https: //atp.pt/wp-content/uploads/2020/01/Banco-de-Portugal.pdf. Munari, B. (2008) Das Coisas Nascem Coisas. Edições 70. NOVA Medical School (2013) Estudo Epidemiológico Nacional de Saúde Mental - 1.o Relatório. Lisbon: NOVA Medical School. Papanek, V. (2006) Design for the Real World: Human Ecology and Social Change.

2nd edn. Thames & Hudson. Salovaara, A. and Tamminen, S. (2009) ‘Acceptance or Appropriation? A Design- Oriented Critique of Technology Acceptance Models’, in Isomäki, H. and Saariluoma, P. (eds) Future Interaction Design II. London: Springer, pp. 157–173. doi:10.1007/978-1-84800-385-9_8.

Design and Evaluation of a Device for Ecological Momentary Assessment with Workers 195

Tsutsumi, A. et al. (2009) ‘Participatory intervention for workplace improvements on mental health and job performance among blue-collar workers: a cluster ran- domized controlled trial’, Journal of Occupational and Environmental Medicine, 51(5), pp. 554–563. doi:10.1097/JOM.0b013e3181a24d28. World Health Organization - Regional Office for Europe (2021) Depression in

Europe: facts and figures. Available at: https://www.euro.who.int/en/health-top ics/noncommunicable-diseases/mental-health/news/news/2012/10/depression-in- europe/depression-in-europe-facts-and-figures (Accessed: 26 July 2021).

Cover of the proceedings volume Affective and Pleasurable Design

Explore the complete digital volume

Affective and Pleasurable Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-17-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →