Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Understanding user needs of personalisation-based automated systems with development and application of novel ideation cards

1 Human Factors Research Group, University of Nottingham, Nottingham, UK ↗
2 School of Computer Science, University of Nottingham, Nottingham, UK ↗
Publication year
Digital Object Identifier10.54941/ahfe1005032

Abstract

Personalisation is a commonly utilised technology in socially focused online platforms. It has gathered widespread usage through its ability to match a system to the needs of users through their data. This allows systems to be more user-friendly or effective in their use cases. Due to its widespread usage, it will not be long before the technology becomes just as common in the workplace. Currently, there is a limited number of existing personalisation systems in workplace settings, thus, one of the main challenges is gathering user needs. To gather these user needs, a set of Personalisation Design Cards (PDCs) were developed. When these cards are played, they allow end users to design their own theoretical personalisation systems. By containing a selection of information about personalisation systems, data types and potential use cases (in this case, manufacturing), participants can craft systems and evaluate different personalisation approaches. The PDCs were utilised in codesign workshops. Participants indicated that they preferred “dynamic” systems and system shortcuts, while they held reservations about compensation, data accuracy and specific data types. The workshops also enabled the evaluation of the cards.The implications of the research are the creation of the Personalisation design cards, which have shown an ability to engage users in the design process and allow industry organisations to examine how potential systems could function early in the design process.

INTRODUCTION

Personalisation is a powerful way of using personal data to improve the expe- rience for users. As a general abstraction, it will use user data, process this data and then produce an output. The input data can be any form of user data either directly from a user (Montgomery and Smith, 2009) or from other users (Yu, 1999). Due to personalisation systems’ reliance on data, the user requirements process can be difficult. If the computing systems rely on users giving away data, users may be more inclined to want to understand how this data may be being used (Sailaja et al., 2019). To add, existing work has described users who don’t want to engage with these types of systems as ones

© 2024. Published by AHFE Open Access. All rights reserved. 8

Understanding User Needs of Personalisation-Based Automated Systems 9

who want to understand how data is used (Awad and Krishnan, 2006). Thus, there is the question of how and when users wish information about the pro- cess to be displayed to them (users should not need a large commitment to use any system (Norman, 1998, p. 71)). These are the types of problems that are often resolved in user needs research.

As there are limited personalisation systems, it becomes difficult to gather information directly from users about prior usage of workplace personalisa- tion systems. However, by utilising an ideation card methodology, potential users can design their own personalisation systems. These user-generated sys- tems can then be analysed to form user requirements. Thus, this work had two aims. The first aim was the development of a set of cards for participa- tory design (like Lucero and Arrasvuori, 2010), which would allow systems designers to understand the needs of their personalisation system users. The second aim examined how participants would design personalisation sys- tems. This can relate to their user requirements. These aims also highlight the contribution of this work. To understand the participatory design and ideation card approach, it is worth exploring existing research.

LITERATURE REVIEW

Participatory Design itself is a set of methods that “explore the conditions for user participation in the design and introduction of computer-based systems at work” (Kensing and Blomberg, 1998). Abstractly, participatory design often leans heavily on being an iterative process defined by Brandt, Binder and Sander’s (2012, p. 150) Tell-Make-Enact process. In overview, this process involves the discussion between participants (tell), the creating of artefacts (make) and the use of said artefacts (enact). The foundation has been set by the PICTIVE method (Muller, 1991). The aim of this method is defined as “the creation of the design of the interface” and involves the groups of people working in tandem to produce low-fidelity prototypes (this can be done using the termed Design Objects (Muller, 1991)). These basic principles form the blueprint of many other participatory design methods.

Ideation Cards are an increasingly popular research method in which par- ticipants utilise cards to generate insights into the topic of the cards (or activities). The cards themselves are often themed on a particular topic, such as Mixed Reality (Wetzel, Rodden and Benford, 2017) and this has a knock-on effect on the design. Lucero and Arrasvuori (2010) present a set of PLEX cards (the authors describe the name PLEX as based on a prior work’s “Pleasurable Experiences Framework”). Beyond the card’s physical design, the way in which they are used differs between card sets. Halskov and Dalsgård (2006) have a comparatively unstructured approach to their card rules. After a discussion of each card, they suggest an open approach in which participants can use any cards in the design phase. In contrast, Wetzel, Rodden and Benford (2017) utilise their cards in a more traditional game sense. They present various sets of “game rules” which can change the way the cards are used but still provide a structure for participants to follow.

METHOD

Participants In total, ten participants were utilised during the study, split into three groups (in a 3-3-4 split). These were mostly female (F = 7, M = 2, No Entry =1), were majority between the ages of 25–34 (34−45 = 2, 25–34 = 6, 18–24 =1) and mostly had master’s degrees (master’s degree = 8, bachelor’s degree = 1, PhD = 1). They were recruited through email, email lists at the University of Nottingham, word of mouth (including physical and digital variants) or both. It can be implied that all participants were associated in some format with the University of Nottingham. The sample and group assignments were based on convenience and participant availability. Participants were provided with an honorarium in the form of a £20 Amazon Voucher on the studies competition.

Materials

The workshops used printed design ideation cards as the stimuli, one set for each participant. The “Design Objects” provided were similar to the original work (Muller, 1991) and included basic stationery: pens and paper. Video equipment was used to capture the workshops visually (a video camera and/or smartphone) and audio equipment auditorily (either the laptop con- nected to a microphone or a laptop’s built-in microphone). The original and revised cards (created after the workshops, with mat) and rules are available online.1

Initial Development of the Personalisation Design Cards The study’s design rests upon the development of a novel set of Ideation Cards (PDCs). These cards were based loosely on the Mixed Reality Game Cards designed by Wetzel, Rodden and Benford (2017) (henceforth termed MRG cards) to avoid replicating prior card design work. This is notable in the design, in which both sets of cards feature: a coloured border to repre- sent the type of card, a title and a description and an image is included on both sets of cards. Wetzel, Rodden and Benford (2017) describe the image in their MRG cards as “an additional source of inspiration”. As participants may not understand personalisation terminology, the card’s image can clar- ify what the descriptive words cannot (Figure 1). The cards are designed to be printed on regular paper, as per informal discussion with a developer of MRG Cards, providing the rough appearing ‘prototype’ cards should visu- ally afford (Norman, 2013, p. 11) participants the capacity to edit the cards. A similar idea is found in the cards designed by (Halskov and Dalsgård, 2006) that includes an “empty box for comments”.

1OneDrive Link: https://uniofnottm-my.sharepoint.com/:u:/g/personal/joshua_duvnjak_nottingham_ac_ uk/EfqWYkyzi0FOizukrOnMggoBfSnoqiwkn1USxIpMhvP59w?e=u5IxMs

Understanding User Needs of Personalisation-Based Automated Systems 11

Figure 1: The “teleoperation” card’s image features a puppet dog, showing a person operating the dog.
Original PDF Figure 1:

The PDCs adapt the MRG’s Opportunity, Question and Challenge cards (Wetzel, Rodden and Benford, 2017) into Task, Automation and Data. The Data cards describe personal data types that can be used by participants, similar to the Opportunity MRG cards, which provide additional framing to the game. Some data cards were also chosen to represent types that may be more controversial than other types (Marinescu et al., 2022). MRG’s question cards were removed, with Task and Automation being MRG- style Challenge cards. This was to create a foundation for the participants in which to build their system without overloading them with too many potential systems designs, in line with common participatory design theory (Sanders and Stappers, 2008). The Automation cards are based on an early taxonomy, which indicates there may be five categories of personalisation systems. This was to allow participants to design different kinds of systems and understand how these types of systems differ in practise. The Task cards attempt to represent specific fields in the manufacturing industry while simul- taneously are open enough to allow different interpretations of what the user could be doing in each situation.

Procedure At the workshop, participants were requested to sit at a table with other participants. They were provided printed ethics documentation and demo- graphic surveys to complete alongside the card sets. Participants were informed about the contents of the workshops and data collection and allowed to ask questions. When the workshop formally started, participants had flexibility in how they wished to participate (but an effort was made to make sure every participant completed the game at least once). The PDC was used in two sessions. In summation, the rules were as follows: Pick up one blue personal data, yellow manufacturing task and green personalisation automation card). Design and brainstorm (Yao et al., 2019) a system using the cards. Draw an additional blue data card and refine or create a new sys- tem (this step was repeated twice). Explain your system to other participants. Once an approximately 30-minute period had been completed and a short break was held. Images were taken of the participants’ paper systems and the cards (except in one case where images were taken shortly into the second workshop). The participants then restarted the game with new sets of cards

for the second session. The workshop was then completed and participants were allowed to leave.

RESULTS

Thematic Analysis Participant data was transcribed both automatically with manual correc- tions to parts deemed relevant (these relevant parts were used in analysis). The analysis method was a significantly adapted form of the Braun and Clarke (2006) Thematic analysis, which had elements of the “process” to be adjusted, rearranged, or removed. Of the changes, the most notable were the lack of thematic maps arrangement, instead opting for a list which was then devised into main and sub-themes and image material was included as part of the thematic analysis and themes that were replicated in different workshops were considered of greater prevalence. The quotes/transcripts pre- sented herein will have certain sections removed or clarified to retain reading fluidity. There is a discussion as to whether more participants would produce greater levels of results, but it did appear like there was a level of saturation (and thus, themes were identified across groups).

Theme: “Dynamic” Systems The first theme located was that of “dynamic”systems. The word “dynamic” comes from a participant (4) (although their system does not partake in this theme). Dynamic in this analysis refers to how systems can create “shortcuts” or utilise experience/demographic data as part of a changing personalisation process.

The first sub-theme identified was that of Dynamic systems based on user experience. A participant’s (1) drawn material (Figure 2) shows a user is iden- tified to have a level of experience with a system and in turn, alters how much control the automation has over the system. In instances of “high” experience, users have more control over the system than those with “low” experience. The participant states how this could make “low” users safer while also allowing “high” users to complete more complex work. A quote from a different participant (5): “Do You want like a simple and minimalis- tic interface for someone who is a beginner” describes a very similar system. The former uses the “task experience” card as the data type, where this uses assumably general experience. It appears that experience in various forms is considered a usable data type by the participants.

While only briefly mentioned by participants (3 and 7), there seems to be an idea that you can “show shortcuts to people who are experienced” (participant 7). This is the second identified sub-theme. In what format this would take is not directly specified by the participants. There are existing systems which utilise a similar approach. For example, a participant (3) refers to the Python programming language, which may link to the idea of shortcuts but there is no explicit use for their system. In academic literature, Billsus et al. (2002) describe an approach for “mobile technology” which rearranges options to order choices by “most frequently accessed”.

Understanding User Needs of Personalisation-Based Automated Systems 13

Figure 2: A cropped image of a participant’s (1) system from the workshop. The top-left of the image shows a line labelled with low and high task experience. This diagram indicates that the “controllers” system be more or less involved in relation to the users task experience.
Original PDF Figure 2:

The final sub-theme identified within the theme of Dynamic Systems was the use of demographic data. One participant said, “[…] people of different nationalities and they come to work and they have instructions. So you could give people instructions in their own language” (participant 6). This is a clear use of demographic data (the nationality of the worker) and leads to an effect of changing the interface’s language. This would be a “swapper” type system (from cards Personalisation categories), and this is very similar to another participant (3) the idea of age data “linking into more text-based or more graphic based” along with another type of functionality.

Theme: System Usage Concerns The next identified theme was the concerns around the usage of these systems. The sub-themes brand out this in categories of concerns. These were concerns about certain data types (similar to Data Usage found in Marinescu et al. (2022)), the accuracy of conclusions drawn and how these systems could affect compensation.

The first identified sub-theme was the concern around the usage of certain data types. Participants (1 and 9 respectively) referred to certain data types as “insidious”and “intrusive”. This was in response to the data types: heart rate and fNIRS. These data types were included to provoke this kind of thought, as previous work indicated that “personal data” was a theme in which one participant was concerned about certain data (Marinescu et al., 2022). The response from some of these participants clearly show concern, even in a card game situation which it could be suggested has a bias towards being “light- hearted”. In both cases, the participants did try to find “beneficial”(Figure 3) aspects towards the data types.

The next identified sub-theme related to the accuracy of conclusions drawn from the data. One participant in a discussion with other group members stated.

“[…] fatigue and performance maybe not necessarily linked because I would I would feel like an older person maybe being more tired to do the task but they would probably do it faster because of experience.” (participant 5) From the quote, it is possible to understand the general idea. Participants had thoughts that the data may not connect to the conclusions drawn (from the context, this could be an effect on pay), from the fatigue data. This could indicate a distrust of data analysis on the part of the participant, and this could be a wider factor as the other selected quote suggests a similar idea but refers to how “unique” (participant 9) aspects can affect the data and may lead to incorrect conclusions being drawn.

The last sub-theme was that of compensation. As Aforementioned, partici- pants raised points about “worker pay” (participant 6) and how systems like the ones described in the workshops, could affect this. One of the examples provided by a participant (4) suggested that tasks could be allocated based on difficulty (and difficult tasks pay more) and connected fatigue data, further noting that if you were provided low-difficulty tasks, you would receive less pay for this. A different case can be seen from a different participant (9) who said “[…] Are you gonna say I’m too tired to do this today for your boss to turn around and say, okay, okay, you don’t do it, you don’t get paid?”. This quote may be suggesting that rather than employers allocating tasks, users can utilise their data to talk to their employer. They then suggest that an employer may not wish to pay for unworked hours.

Workshop Observations: Card Design, Game Rules, Use of Real-World Examples In the workshops, there were findings -while not directly fitting into the thematic analysis- that provide insight into the use of the PDCs.

A weakness in card clarity was identified by the writing on some of the cards to indicate the type (such as “data”), with another approach being to create a “mat” which outline the type of cards in a deck. In addition, paper cards allowed some participants to edit them in the workshops (they were informed about card editing).

One participant (1) did change two of the personal data cards slightly, but only small edits to the card itself. It was mentioned by two participants (1, 4) that the image for the “Controllers” cards image may be incorrect.

The game rules are a point of discussion as the three groups took slightly different approaches. Generally, participants appeared to naturally work as a group during the workshops. The different groups also varied in the structure of the workshops in terms of how closely they followed the rules.

During the workshops, participants used real-world examples to help illus- trate their points. For example, one participant (5) referred to a “automatic cashier the supermarket”, presumably the self-service checkout. They have explained a type of data that could be affected in the context of this system, which is to help explain their personalisation system.

DISCUSSION

From the workshops, one of the prominent findings was the using of expe- rience to adjust automated systems and another related to the preference

Understanding User Needs of Personalisation-Based Automated Systems 15

for avoiding of using certain data types. The concept of using experience is not novel. An existing work has examined “expert users” vs “inexpert users”, finding that the former would want a more involved automation in comparison to the latter (Schiaffino and Amandi, 2004). Contrastly, the sys- tems designed by our participants suggested that high-experience users would receive a less involved system. One potential missing factor is the conflicting nature between trust in automation and experience. The idea that trust in automation would have an effect similar to that described by Schiaffino and Amandi (2004) was discussed by participants (3, 7). Thus, systems designers should be aware of this type of interaction and make sure to include only currently in-use personal data types. Data privacy for certain data types is a topic that participants referred to in the negative during the current study. This is not a new phenomenon, with a survey study finding that only a minority is unwilling to provide all their “behavioural data for personali- sation” (Yamamoto and Yamamoto, 2020). Further, a related study found concerns with the use of “biometrically monitoring” techniques (Marinescu et al., 2022), which was replicated in the current study in relation to the use of heart rate data. There appears to be a line in which people will feel as if a data type is acceptable to be used and when it is not.

One finding related to the accuracy of certain personalisation analyses and participants suggested this could affect one’s perception in one’s place of work. In the case of the participants, it related to compensation from one’s employer. The type of inaccuracy can be identified as being similar to the concept of “distributive injustice” (Yeung, 2018). Although referring to “customers” rather than employees, the work echoes present the idea that some people will be discriminated against based on “a commercially rational form of social sorting” (Yeung, 2018). The identified concerns could also be in a similar vein to the problems with “categorisation” or potential errors in data analysis leading to negatives for users (Monzer et al., 2020). In the current study, a suggestion from a participant (4) about the potential for “unique” data points to skew the data. There is a connection for compensa- tion and the concept of “value trade-off” (Sailaja et al., 2019). The potential here is that following existing logic in “value trade-off”-like situations (Awad and Krishnan, 2006; Sailaja et al., 2019), employees will not want to give away personal data that will bring them lost wages (negative value). Further, existing literature shows many people would put a high value on the type of personal data in a similar category as discussed by participants (Skatova et al., 2013). Thus, for certain types of data, the value trade-off can never be fully reached in the current work climate. This is pointed out by partici- pants who discuss potential employer reactions to situations that arise from the data.

The cards appeared to be a promotor of thought into the design of per- sonalisation systems. For example, one outcome of the thematic analysis was that of using experience for the data type in personalisation systems. This was one of the data cards present in the study. Participants using the cards in their designs is to be expected and is echoed in other ideation card-based work (Halskov and Dalsgård, 2006; Wetzel, Rodden and Benford, 2017). If the cards were utilised one may make an assumption the cards were relevant,

a previous ideation card study found that certain cards were not used and they had no evidence as to the cause (Halskov and Dalsgård, 2006). In the case of our personalisation cards, participants noted that certain cards were not used due to the concerns over data usage rather than relevance. The cards did, however need revising.

LIMITATIONS One limitation is the effect of accidentally becoming ‘the complete partici- pant’ (termed by Robson, 2011, p. 320) due to one answering participant questions. This was minimized by providing abstract answers. Another lim- itation was that certain participants may have had less input than others and this could affect the data. To reduce this, the researcher attempted to make each participant complete one round of the game individually. Another limitation could be the number of participants (N = 10) in comparison to other research methodologies but existing “participatory design” studies (Demirbilek and Demirkan, 2004; Ahmed et al., 2019) have similar levels of participants (the former had thirteen and the latter had nine). Further, while the work found certain insights, these findings may not represent the broader scope of people within the manufacturing sector.

CONCLUSION

In conclusion, the current work examined how end users would design sys- tems through the development of Personalisation Design cards and following workshops. These workshops show how participants would like personalisa- tion systems designed. While there was room to improve the cards, they were successful in allowing the participants of the study to design systems. Future work in this area could look at expanding the card set, either by including more cards or changing the task cards to represent different industries.

ACKNOWLEDGMENT

This work was supported by the Engineering and Physical Sciences Research Council [grant number EP/S023305/1]. The Third author was supported by the Made Smarter Innovation Centre for People Led Digitalisation EP/V062042/1.

REFERENCES

Ahmed, R. et al. (2019) ‘Visualization of Cardiac Implantable Electronic Device Data for Older Adults Using Participatory Design’, Applied Clinical Informatics, 10(04), pp. 707–718. Available at: https://doi.org/10.1055/s-0039-1695794. Awad, N. F. and Krishnan, M. S. (2006) ‘The Personalization Privacy Paradox:

An Empirical Evaluation of Information Transparency and the Willingness to Be Profiled Online for Personalization’, MIS Quarterly, 30(1), pp. 13–28. Billsus, D. et al. (2002) ‘Adaptive interfaces for ubiquitous web access’, Communications of the ACM, 45(5), pp. 34–38. Available at: https://doi.org/10.1145/506218. 506240.

Understanding User Needs of Personalisation-Based Automated Systems 17

Brandt, E., Binder, T. and Sanders, E. B.-N. (2012) ‘Tools and techniques’, in Routledge International Handbook of Participatory Design. Routledge. Available at: https://doi.org/10.4324/9780203108543.ch7. Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative

Research in Psychology, 3(2), pp. 77–101. Available at: https://doi.org/10.1191/ 1478088706qp063oa. Demirbilek, O. and Demirkan, H. (2004) ‘Universal product design involving elderly users: a participatory design model’, Applied Ergonomics, 35(4), pp. 361–370. Available at: https://doi.org/10.1016/j.apergo.2004.03.003. Halskov, K. and Dalsgård, P. (2006) ‘Inspiration card workshops’, in Proceedings

of the 6th ACM conference on Designing Interactive systems - DIS ‘06. The 6th ACM conference, University Park, PA, USA: ACM Press, p. 2. Available at: https: //doi.org/10.1145/1142405.1142409. Kensing, F. and Blomberg, J. (1998) ‘Participatory Design: Issues and Concerns’, Computer Supported Cooperative Work (CSCW), 7(3–4), pp. 167–185. Available at: https://doi.org/10.1023/A:1008689307411. Lucero, A. and Arrasvuori, J. (2010) ‘PLEX Cards: a source of inspiration when

designing for playfulness’, in Proceedings of the 3rd International Conference on Fun and Games - Fun and Games ‘10. the 3rd International Conference, Leuven, Belgium: ACM Press, pp. 28–37. Available at: https://doi.org/10.1145/1823818. 1823821. Marinescu, A. et al. (2022) ‘The future of manufacturing: Utopia or dystopia?’, Human Factors and Ergonomics in Manufacturing & Service Industries, p. hfm.20976. Available at: https://doi.org/10.1002/hfm.20976. Montgomery, A. L. and Smith, M. D. (2009) ‘Prospects for Personalization on the

Internet’, Journal of Interactive Marketing, 23(2), pp. 130–137. Available at: http s://doi.org/10.1016/j.intmar.2009.02.001. Monzer, C. et al. (2020) ‘User Perspectives on the News Personalisation Process: Agency, Trust and Utility as Building Blocks’, Digital Journalism, 8(9), pp. 1142–1162. Available at: https://doi.org/10.1080/21670811.2020.1773291. Muller, M. J. (1991) ‘PICTIVE–an exploration in participatory design’, in Proceed-

ings of the SIGCHI conference on Human factors in computing systems Reaching through technology - CHI ‘91. the SIGCHI conference, New Orleans, Louisiana, United States: ACM Press, pp. 225–231. Available at: https://doi.org/10.1145/ 108844.108896. Norman, D. A. (1998) The invisible computer: Why good products can fail, the personal computer is so complex, and information appliances are the solution. Cambridge, Mass: MIT Press. Norman, D. A. (2013) The design of everyday things. Revised and expandes editons.

Cambridge, MA London: The MIT Press. Robson, C. (2011) Real world research: A resource for users of social research methods in applied settings. 3. ed. Chichester: Wiley. Sailaja, N. et al. (2019) ‘The Living Room of the Future’, in Proceedings of the 2019

ACM International Conference on Interactive Experiences for TV and Online Video. New York, NY, USA: Association for Computing Machinery (TVX ‘19), pp. 95–107. Available at: https://doi.org/10.1145/3317697.3323360. Sanders, E. B.-N. and Stappers, P. J. (2008) ‘Co-creation and the new landscapes of design’, CoDesign, 4(1), pp. 5–18. Available at: https://doi.org/10.1080/ 15710880701875068.

Schiaffino, S. and Amandi, A. (2004) ‘User – interface agent interaction: Personalization issues’, International Journal of Human-Computer Studies, 60(1), pp. 129–148. Available at: https://doi.org/10.1016/j.ijhcs.2003.09.003. Skatova, A. et al. (2013) ‘Perceived risks of personal data sharing’, Proc. Digital

Economy: Open Digital [Preprint]. Wetzel, R., Rodden, T. and Benford, S. (2017) ‘Developing Ideation Cards for Mixed Reality Game Design’, Transactions of the Digital Games Research Association, 3(2). Available at: https://doi.org/10.26503/todigra.v3i2.73. Yamamoto, Y. and Yamamoto, T. (2020) ‘Personalization Finder: A Search Interface

for Identifying and Self-controlling Web Search Personalization’, in Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020. JCDL ‘20: The ACM/IEEE Joint Conference on Digital Libraries in 2020, Virtual Event China: ACM, pp. 37–46. Available at: https://doi.org/10.1145/3383583.3398519. Yao, Y. et al. (2019) ‘Defending My Castle: A Co-Design Study of Privacy Mechanisms for Smart Homes’, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI ‘19: CHI Conference on Human Factors in Computing Systems, Glasgow Scotland Uk: ACM, pp. 1–12. Available at: https://doi.org/10.1145/3290605.3300428. Yeung, K. (2018) ‘Five Fears About Mass Predictive Personalisation in an Age of

Surveillance Capitalism’, SSRN Electronic Journal [Preprint]. Available at: https: //doi.org/10.2139/ssrn.3266800. Yu, P. S. (1999) ‘Data mining and personalization technologies’, in Proceedings.

6th International Conference on Advanced Systems for Advanced Applications. Proceedings. 6th International Conference on Database Systems for Advanced Applications, Hsinchu, Taiwan: IEEE Comput. Soc, pp. 6–13. Available at: https: //doi.org/10.1109/DASFAA.1999.765731.

Cover of the proceedings volume Human Factors in Software and Systems Engineering

Explore the complete digital volume

Human Factors in Software and Systems Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-16-8

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →