Steven Schmidt, Sandro Hartenstein (2023). Accounting trustworthiness requirements in Service Systems Engineering. In: Human Factors in Software and Systems Engineering. AHFE Open Access, vol. 94, pp. 28–37. https://doi.org/10.54941/ahfe1003767
Trustworthy services are essential for sustainable digitization, especially during ever more demanding times when it comes to service expectations, quality and consumption. Putting the human being behind the service consumer at the centre of service systems engineering also means identifying which requirements are beneficial for the actual usage of the service. Previous research has shown that the degree of trustworthiness of a digital system is directly related to the potential use of the same. Yet, there is a lack of empirical data on the perception of trustworthiness of different systems and approaches on how to integrate these aspects as requirements in disciplines such as Service Engineering in academia. This paper aims to provide insights into a recent empirical study that collected views on the importance of trustworthiness attributes of various digital services and how these results can potentially be integrated into Service Systems Engineering via a Requirements Engineering approach. The base is a multidisciplinary view of trustworthiness in sociotechnical systems. In this expert survey, the perspectives are manifested by trusted Wifi, conventional WebAPIs, AI WebAPIs, and mediation services. Thus, differences in social, technical and socio-technical, as well as with AI-enabled services will also be highlighted. Utilizing the basic approaches of Trust Engineering, Trust determinants are incorporated and empirically evaluated in this study and give new insights into trustworthy Systems Engineering as well as relevant and potentially counter-intuitive features.
The trustworthiness of services is essential for a progressing digitalisation. Only trustworthy services can bring sustainable social benefits through acce- ptance and use and must therefore be improved. For this reason, it makes sense to support the trust. This study uses the following definition for trust: “Trust by definition entails a willingness by the [trustor] to make herself vulnerable to the possibility that another will act to her detriment” (Hill and O’Hara O’Connor 2005, p. 28)
Accounting Trustworthiness Requirements in Service Systems Engineering 29
On the other hand, the services must have trustworthy properties. For trustworthiness, the definition is taken: “Trustworthiness is assurance that a system deserves to be trusted—that it will perform as expected despite environmental disruptions, human and operator error, hostile attacks, and design and implementation errors. Tru- stworthy systems reinforce the belief that they will continue to produce expected behavior and will not be susceptible to subversion.” (Schneider 1999, p. 316) A service is defined as follows: “Abilities by a Service Provider or consumer […] utilized in the context of interactional processes to generate effects on the consumers side and his assets in respect” ((Donabedian 1980); (Donabedian 2005) in: (Bullinger and Scheer 2006), p. 57). Influential conception factors of Service Engineering (SE) can be pictured as in Figure 1.
In the context of increasing technological integrations of software, service- oriented architectures and hardware into service conception, digital services can be defined as a special manifestation. (Pakkala and Spohrer 2019), p. 1886 ff. characterize these by the automated (see (Immonen et al. 2016), p. 151) performance or support using heterogeneous, technical systems that are increasingly distributed and hardware-agnostic. (Williams et al. 2008), p. 506 add to this a digital transaction over IP and define this as a restrictive criterion for the definition of a digital service. It should be noted that this does not necessarily mean that the product must also be digitally designed in the sense of the service definition. This can be the case, for example, with a public WiFi in the form of digital wireless internet access but is different on the example of amazon.com in the form of a physical delivery of goods.
The term requirement is generally defined in the Standard (IEEE 610.12), p. 62, which has been replaced by the current Standard (ISO/IEC/IEEE 24765), p. 380. This definition is supported by the International Requi- rements Engineering Board (Glinz 2020), p. 6, as well as by the Standard (ISO/IEC/IEEE 291488), p. 4 and the considerations of (Sommerville 2016), p. 102; (Wiesner et al. 2015), p. 104; (Macaulay 1996), p. 4 and (Hull 2011), p. 2. The fact that these definitions come more from the field of software
Figure 1: Influential factors and viewpoints of service engineering (cf. (Bullinger and Scheer 2006), p. 102).
30 Schmidt and Hartenstein
engineering is due to the different ages of the disciplines of software engi- neering and service engineering, the latter of which can easily adopt the definition, which is also often mentioned. Through these views, a requirement must contain all necessary information such as need or capability, constraints and conditions to fulfil the criteria and obligations of a technical system, phy- sical product or service and to satisfy an agreement, standard, specification, or other formally imposed documents.
Formally, requirements can be managed in a variety of forms and activi- ties within the scope of requirement engineering (RE). However, the type and scope are mainly dependent on the type of requirement and the associated properties as well as the context field. There are a variety of distinctions of requirements for this purpose (cf. (Sommerville 2016), p. 102; (Glinz 2020), p. 18 ff; (ISO/IEC/IEEE 291488), p. 16 and 64; (Rupp 2021), p. 30 ff.). The variety of requirement types also raises questions regarding the consi- deration of trustworthiness factors. If one follows the thought towards the implementation of trust-creating properties in systems in the sense of trust engineering (cf. (Hoffmann 2014), p. 63, (Hoffmann et al. 2012), p. 4) according to (Söllner et al. 2012), the question quickly arises as how the deri- vation of trust requirements, their implementation, and measurability should be integrated. With regard to sociotechnical systems, (Söllner et al. 2012) accordingly design activities for the method. The core goal is the creation of so-called trust-supporting components (TSC). Trust dimensions and their determinants according to (Muir 1994), enriched by considerations from Lee and See [128], are the target of the TSC (cf. Söllner et al. [127], p. 117). In detail, the assignment of determinants to the dimensions according to (Lee and See 2004) in (Söllner et al. 2012), p. 117 and (Hoffmann 2014), p. 159 ff. is summarized for sociotechnical systems as follows: Through several validating studies, (Hoffmann 2014), p. 189 was finally able to show that these determinants can be applied from the wide selection of possible determinants (cf. (Söllner and Leimeister 2013), p. 139 ff.) to the formation of trust in sociotechnical systems. For the systematic develo- pment of TSC, (Söllner et al. 2012), p. 119 ff. present a framework. Yet this approach misses the methodical definition of a order of importance of these determinants regarding different kinds of sociotechnical systems, respective different digital services, apart from being done by the requirements engineer, which inhabits the potential for bias.
As proposed in (Hartenstein et al. 2020) and (Hartenstein et al. 2021b) an empirical approach on evaluating trustworthiness requirements for soci- otechnical systems should be considered. First steps to do so as part of the EUMoVe Project (cf. (Hartenstein et al. 2021)) have been done by conducting trustworthiness attributes of public wireless networks (cf. (Schmidt 2021)) and web services (cf. (Hartenstein 2021) and (Hartenstein et al. 2021c)). To incorporate these findings into a utilizable framework for the purpose of tru- stworthy digital service engineering, an evaluation of gaps in current models as well as an approach have been presented (cf. (Schmidt 2022)). Combi- ning the first empirical insights with the conceptualized approach, it became necessary to map the relevance of each trust determinant to their empiri- cal relevance regarding different digital services to better understand their
Accounting Trustworthiness Requirements in Service Systems Engineering 31
Table 1. Trust dimensions and corresponding trust determinants.
Table available in the original PDF paper.
Dimension Determinant Performance Eligibility of the system for achieving the target (Muir 1994) (Elig) Reliability of the system (Muir 1994) (Reli) Correctness of information (Muir and Moray 1996) (Corr) Completeness of the required functional scope (Muir 1994) (Comple) Process traceability Authenticity of the system (Muir 1994) (Auth) Consistency of system behaviour (Muir and Moray 1996) (Cons) Comprehensibility of the system’s function (Zuboff 1989) (Compre) Control over the system (Shankar et al. 2002) (Contr) Predictability of system behaviour (Muir 1994) (Pred) Purpose clarity Communication of the system’s purpose (Muir 1994) (Purp) Benevolence of the developers (Muir 1994) (Bene) Faith in the system (Muir 1994)Faith in the system (Muir 1994) (Faith)
distribution and connections in the presence of the proposed task to account trustworthiness requirements into Service Systems Engineering.
METHOD
To gain insights into the empirical relevance of certain trustworthiness deter- minants for digital services in general and selected services as an example to show divergence, a public study has been conducted. In corporation with the respective institutions an online survey has been created and published containing general statistical data as well as sections regarding the adaption of RE elements and the relevance of trustworthiness determinants in general and in detail. Different groups and portals have been addressed with access links.
The first part of the study consisted of questions regarding the degree of adaption of established RE activities within the participants professional environment, as well as their role, amount of experience and related context field. The second part of the survey conducted the perceived relevance of tru- stworthiness requirements for digital services as well as the perception of the current grade of trustworthiness of digital services in general (G). To dee- pen the understanding of formation and variation of the trustworthiness of digital services the state of trustworthiness of eight different digital services has been conducted, with a focus on the current investigation (cf. (Harten- stein et al. 2021)) as is Public WiFis (W), AI WebAPIs (WAI), conventional WebAPIs (WAP) and Online Mediation Services (M) amongst other exam- ples (eGovernment Services (eGov), Online Medical Services (OM), Online Shopping (OS) and Online Banking (OB)).
32 Schmidt and Hartenstein
Further the trust determinants had to be ranked for specific digital services by experts to gain insight into service-related trust building characteristics as well as the distribution across different digital services. The main difficulty of achieving the goal of forming true ranks of trust determinants for individual digital services based on empirical data can be described with Arrow’s Impos- sibility Theorem (Arrow 1963) which states, that with a set of attributes of more than three and a deciding population of more than two, a preference aggregation cannot be invoked while satisfying all conditions. Thus, the auth- ors aim to gain an approximation of ranks of trustworthiness determinants regarding the median and the variance of each determinant.
All options have been randomized to prevent order bias, surrounding statistical data such as gender, age, geographical region of the professio- nal activities and educational background have been conducted too. The questionnaire took about 11 minutes in average to be completed.
RESULTS
Over a period of eight weeks 103 participants finished the study. After evaluating the quality and content of the responses and reducing invalid submissions, 97 datasets remain for investigation. The sample consists of a variety of different roles across different fields, with the majority being Pro- ject Managers (30.93%) and Product Owners (13.40%) and, as expected, users as the least represent group (4.12%). The most prominent professio- nal fields consisted of Software Engineering (46.39%), Service Engineering (19.59%) and Product Engineering (16.49%). The amount of experience is distributed across all options for < 2 years to > 15 years with a greater wei- ght on experiences less than 10 years in the current field of profession. Most participants are based in Europe.
Regarding the RE method formalization of digital services seen from dif- ferent fields of expertise, the general distribution cumulates at an advanced grade of 4 out of 5 with 37.11% and 30.93% at grade 3, with Hardware Engineering (HE) reaching 75.00% at grade 4, followed by Product Engi- neering (PE) (56.25%) and SE (36.84%). The relevance of trustworthiness requirements for the conception of digital services has widely been rated high with grade 4 and 5 accounting for 79.38%. Like before, the evaluation from participants with a HE background ranks the highest with 100.00% rating it a 4 out of 5, followed by SE with 84.21% accounting for rating 4 and 5 and SwE (80.00%).
The trustworthiness itself for digital services in general, on the other hand, is not perceived as high by around 74.23%. Even though there is no signi- ficant correlation between the perception of the trustworthiness of these specific services, all rankings are predominantly on the low side, with eGov and OB being the only exception (cf. Figure 2).
As much as there is no significant correlation between the individual digital service trustworthiness perceptions, there is no significant correlation betw- een the relative trust determinants between digital services and between each other within a specific digital service. It was to be expected that there would be low correlation among them since the attributes are measuring different
Accounting Trustworthiness Requirements in Service Systems Engineering 33
Figure 2: Trustworthiness perception of different digital services.
or distinct aspects of the underlying construct of trustworthiness. By ranking trust determinants for digital services in general and in detail, the results are as follows (cf. Figure 3).
To visualize median and spread of data in accordance to validate the ranks formed for each service, parallel coordinates diagrams such as in Figure 4 were created. This also shows different problematics with the formation of
Figure 3: Median and variance of trust determinants of digital services in general and in detail.
34 Schmidt and Hartenstein
Figure 4: Example parallel coordinates diagram of trust determinants of digital services in general.
these results, as all medians are relatively close, and many variances are too. This again indicates a weaker possibility for distinction between ranks of trust determinants across all digital services.
Regarding the trustworthiness evaluation of all services Cronbach’s alpha is αTW = 0.81 with the selectivity being above .30 in all cases except TW_W (0.24) which would have to be removed according to (Blanz 2015). Low cor- relation among items can lead to a low Cronbach’s alpha value, which does not necessarily indicate low reliability. The reliability of the trust determi- nants seems to follow that conclusion regarding the previously stated low correlation. Only three determinants had an acceptable α (αAuth = 0.67; αBene = 0.68; αFaith = 0.73) with the other determinants remaining at ina- cceptable values, leading again to the interpretation, that each determinant is put into a different perspective by participants in relation to the examined digital service.
CONCLUSION
Based on the examination above, the following can be concluded:
1. The performed study showed strong indicators of a highly individual understanding of the introduced set of trust determinants in relation to the respective service 2. A possible second conclusion can be formed by considering the partici- pants had different criteria to rank trust determinants 3. The fact, that the spread of medians in this study is rather small compa- red to the provided scale can also lead to the conclusion, that differences
Accounting Trustworthiness Requirements in Service Systems Engineering 35
regarding the importance or value of trust determinants are rather insignificant to form ranks 4. A general weighting for trusted properties of WebAPIs could be establi- shed, but the dependencies of the properties need individual considera- tion according to the use cases. 5. This leads to the conclusion, that all of the proposed trust determinants are rather equally relevant for the formation of trustworthiness of digital services in general and of the examples included Limitations for these conclusions may be, that the number of respondents and veritable results after quality checks was not sufficient or representa- tive regarding the field. Another limitation is that almost all participants had their main professional activity in Europe, on which the perceptions provi- ded were based. Results and conclusions for different regions therefore may differ.
At last, further research is needed based on these insights. In following works, TSC shall be implemented and with the help of reflective measurement concepts as provided in (Hoffmann 2014) the potentially higher trustworth- iness of digital services shall be evaluated. Regarding the adaption of the proposed approaches in Service Engineering and the integration into the cor- responding RE as part of the initially proposed work, these studies must be conducted to evaluate the utilizable impact. As for now, a differentiation of differently important trust determinants in Service Engineering is not given based on these results.
REFERENCES
Arrow, Kenneth Joseph (1963). Social choice and individual values. 2nd ed. New Haven, Yale Univ. Press. Blanz, Mathias (2015). Forschungsmethoden und Statistik für die Soziale Arbeit.
Grundlagen und Anwendungen. Stuttgart, Verlag W. Kohlhammer. Bullinger, Hans-Jörg/Scheer, August-Wilhelm (Eds.) (2006). Service Engineering.
Entwicklung und Gestaltung innovativer Dienstleistungen; mit 24 Tabellen. 2nd ed. Berlin, Springer. Donabedian, A. (1980). Explorations in quality assessment and monitoring. Ann Arbor, Mich., Health Administration Press. Donabedian, A. (2005). Evaluating the quality of medical care. 1966. Milbank
Quarterly 83 (4), 691–729. https://doi.org/10.1111/j.1468-0009.2005.00397.x. Glinz, Martin (2020). A Glossary of Requirements Engineering Terminology 2020.
Available online at https://www.ireb.org/content/downloads/2-cpre-glossary-2-0/ ireb_cpre_glossary_de_2.0.1.pdf (accessed 10/15/2021). Hartenstein, Sandro (2021). Vertrauen in Digitale Dienste Technische Aspekte. In: Andreas Schmietendorf (Ed.). Online-Workshop (e) trust – Vertrauen in Digitale Dienste (Werte – Risiken – Prinzipien – Methoden – Techniken). Düren, Shaker Verlag. Hartenstein, Sandro/Schmidt, Steven/Ider, Kadir/Schmietendorf, Andreas (2021).
Forschungsvorhaben EUMovE - Projektsteckbrief. Available online at https:// www.researchgate.net/publication/344477300_Forschungsvorhaben_EUMovE_- _Projektsteckbrief (accessed 3/6/2021).
36 Schmidt and Hartenstein
Hartenstein, Sandro/Schmidt, Steven/Schmietendorf, Andreas (2020). Towards an Empirical Analysis of Trustworthiness Attributes in the Context of Digitaliza- tion. In: The Fourteenth International Conference on Digital Society, ICDS 2020, The Fourteenth International Conference on Digital Society, Valencia. IARIA XPS Press, 112–116. Hartenstein, Sandro/Schmidt, Steven/Schmietendorf, Andreas (2021b). Empirical
Analysis of Trustworthiness Attributes in the Context of Digitization. In: Inter- national Journal on Advances in Security, 48–58. Hartenstein, Sandro/Schmidt, Steven/Schmietendorf, Andreas (2021c). Trust Patterns in Modern Web-API Based Service Architectures - More than Technical Security Aspects. In: Patterns 2021. IARIA, 23–25. Hill, Claire A./O’Hara O’Connor, Erin A. (2005). A Cognitive Theory of Trust. SSRN
Electronic Journal. https://doi.org/10.2139/ssrn.869423. Hoffmann, Axel (2014). Anforderungsmuster zur Spezifikation soziotechnischer Systeme. Standardisierte Anforderungen der Vertrauenwürdigkeit und Rechtsver- träglichkeit. Kassel, Germany, Kassel University Press. Hoffmann, Axel/Söllner, Matthias/Hoffmann, Holger (2012). Twenty Software
Requirement Patterns to Specify Recommender Systems that Users Will Trust. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2484462. Hull, Elizabeth (2011). Requirements Engineering. London, Springer London. IEEE 610.12. 610.12–1990 - IEEE Standard Glossary of Software Engineering Terminology, 31.12.1990. s.l. Available online at https://ieeexplore.ieee.org/stamp/ stamp.jsp?tp=&arnumber=159342. Immonen, Anne/Ovaska, Eila/Kalaoja, Jarmo/Pakkala, Daniel (2016). A service
requirements engineering method for a digital service ecosystem. Service Oriented Computing and Applications 10 (2), 151–172. https://doi.org/10.1007/s11761- 015-0175-0. ISO/IEC/IEEE 24765. 24765–2017 - ISO/IEC/IEEE International Standard - Systems and software engineering--Vocabulary, 28.08.2017. Piscata- way, NJ, USA. Available online at https://ieeexplore.ieee.org/stamp/stamp. jsp?tp=&arnumber=8016712. ISO/IEC/IEEE 291488. 29148–2018 - ISO/IEC/IEEE International Standard - Systems and software engineering -- Life cycle processes -- Requirements engi- neering, 30.11.2018. Piscataway, NJ, USA. Available online at https://ieeexplore. ieee.org/servlet/opac?punumber=8559684 (accessed 04.09.20121). Lee, John D./See, Katrina A. (2004). Trust in automation: designing for appropri-
ate reliance. Human factors 46 (1), 50–80. https://doi.org/10.1518/hfes.46.1.50_ 30392. Macaulay, Linda A. (1996). Requirements Engineering. London, Springer. Muir, B. M. (1994). Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37 (11), 1905–1922. https://doi.org/10.1080/00140139408964957. Muir, B. M./Moray, N. (1996). Trust in automation. Part II. Experimental studies of
trust and human intervention in a process control simulation. Ergonomics 39 (3), 429–460. https://doi.org/10.1080/00140139608964474. Pakkala, Daniel/Spohrer, Jim (2019). Digital Service: Technological Agency in Service Systems. In: Proceedings of the 52nd Hawaii International Conference on System Sciences. Hawaii International Conference on System Sciences. Rupp, Chris (Ed.) (2021). Requirements-Engineering und -Management. Das Hand-
buch für Anforderungen in jeder Situation. 7th ed. München, Hanser. Schmidt, Steven (2021). On the perception and relevance of trustworthiness in public wireless networks. In: Kevin Daimi/Hamid R. Arabnia/Leonidas Deligiannidis
Accounting Trustworthiness Requirements in Service Systems Engineering 37
et al. (Eds.). Advances in Security, Networks, and Internet of Things. Proce- edings from SAM’21, ICWN’21, ICOMP’21, and ESCS’21. Cham, Springer International Publishing. Schmidt, Steven (2022). On the Need for a Generic Requirements Engineering Framework for Trustworthy IT Services. In: Kohei Arai (Ed.). Advances in Information and Communication. Cham, Springer International Publishing, 832–842. Schneider, Fred B. (Ed.) (1999). Trust in cyberspace. Washington, DC, National
Academy Press. Shankar, Venkatesh/Urban, Glen L./Sultan, Fareena (2002). Online trust: a stakeholder perspective, concepts, implications, and future directions. The Journal of Strategic Information Systems 11 (3-4), 325–344. https://doi.org/10.1016/S0963- 8687(02)00022-7. Söllner, Matthias/Hoffmann, Axel/Hoffmann, Holger/Leimeister, Jan Marco (2012).
Vertrauensunterstützung für sozio-technische ubiquitäre Systeme. Zeitschrift für Betriebswirtschaft 82 (S4), 109–140. https://doi.org/10.1007/s11573-012-0584-x. Söllner, Matthias/Leimeister, Jan Marco (2013). What We Really Know About Antecedents of Trust: A Critical Review of the Empirical Information Systems Literature on Trust. In: D. Gefen (Ed.). Psychology of trust. New research. Hauppauge, New York, Nova Science Publishers, 127–155. Sommerville, Ian (2016). Software engineering. Boston/Columbus/Indianapoli-
s/New York/San Francisco/Hoboken/Amsterdam/Cape Town/Dubai/London/Ma- drid/Milan/Munich/Paris/Montreal/Toronto/Delhi/Mexico City/São Paulo/Sydne- y/Hong Kong/Seoul/Singapore/Taipei/Tokyo, Pearson. Wiesner, Stefan/Peruzzini, Margherita/Hauge, Jannicke Baalsrud/Thoben, Klaus- Dieter (2015). Requirements Engineering. In: Josip Stjepandi´c/Nel Wognum/Wim J. C. Verhagen (Eds.). Concurrent Engineering in the 21st Century. Foundati- ons, Developments and Challenges. Cham/s.l., Springer International Publishing, 103–132. Williams, Kevin/Chatterjee, Samir/Rossi, Matti (2008). Design of emerging digital
services: a taxonomy. European Journal of Information Systems 17 (5), 505–517. https://doi.org/10.1057/ejis.2008.38. Zuboff, Shoshana (1989). In the age of the smart machine. The future of work and power. Oxford, Heinemann Professional.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Software and Systems Engineering
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-70-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional