Nuria Brüggemann, Sebastian Preis, Heinrich Claus Arnd Engeln (2024). Empirical analysis of social implications during the development of automated driving. In: Human Interaction and Emerging Technologies (IHIET 2024). AHFE Open Access, vol. 157, pp. 40–46. https://doi.org/10.54941/ahfe1005462
This article presents and discusses studies and their first results on the development of an empirical method for the investigation of social implications caused by automated driving. The basic research project KARLI is funded by the German Federal Ministry for Economic Affairs and Climate Action. One of the objectives of KARLI is to develop a methodological approach for the empirical identification and evaluation of social implications expected in different phases of the user-centred development of automated driving. In KARLI, the empirical analysis of social implications is integrated in different studies throughout an iterative user-centred development process of interaction concepts for automated driving (SAE 0-4). In our definition, social implications address the consequences of a technological development for social structures or processes, as well as the development-related prerequisites necessary for a desired social target state. The empirical identification of possible social implications is conducted in two studies, using a qualitative survey (N = 12) and open-ended questions as part of an online survey (N = 35). Based on the results, a first draft of a questionnaire with closed questions is designed to assess the social implications previously identified. In a forthcoming third study, a VR simulation to test three different concepts to promote level-compliant driver behaviour, the social implications are assessed using the designed questionnaire. In addition, further expected social implications are elicited through open-ended questions. The study runs from the end of March to May 2024 with participants between 35 and 45 years of age (N = 95). According to the results, the statements about probable social implications are well answerable by car users, but the validity of the forecast given by the users remains unclear. That for, for a valid estimation of social implications a combination of users and expert’s perspectives seems to be helpful.
This article bases on the collaborative project KARLI (Artificial Intelligence for Adaptive, Responsive and Level-Compliant Interaction in Future Vehicles), funded by the Federal Ministry for Economic Affairs and Climate Action (BMWK; Diederichs et al., 2022). The main aim of our research is to promote level-compliant driver behaviour across SAE levels 0–4 (SAE International, 2021).
Empirical Analysis of Social Implications During the Development of Automated Driving 41
An additional research aim of the Stuttgart media university within KARLI is to find out how social implications expected by innovations of automated driving can be empirically evaluated. Particularly, methods for identifying and evaluating social implications are implemented and evaluated within the iterative, user-cantered development (DIN EN ISO 9241-210:2020-03, 2020) and evaluation of HMI prototypes to promote level-compliant driver behaviour.
In a literature review of 15 project reports (not published yet), no definition of “social implication” sufficient for our work can be found. So we derive one by ourselves as synthesis from Schubert & Klein, 2021; Lehner, 2011 and suggest it as follows: The term “social” here is understood as the change in society or parts of society with regard to the effects of the innovations to be created on the order, organization and actions of the people acting within it. According to Kemmer (2020), an implication describes a link between a prerequisition A and a consequence B. In formal notation, an implication is represented as follows: A→B and is read as “If A then B”. In relation to KARLI, social implications can mean, for example, the causal or correlative consequences for society or parts of society (B) of an innovation created by KARLI (A). E.g.: If automated vehicles with driver status recognition become established on the market (A), then mobility in our society will change fundamentally (B) (Schweiker et al., 2023).
In our definition, social implications address the consequences of a technological development for social structures or processes, as well as the necessary development-related prerequisites for a desired social target state. In the 15 project reports analysed with the topic of social implications, various methods are used to record social implications. Workshops (Tirschmann & Brukamp, 2021), qualitative studies (Zerth et al., 2017), social media analyses (Niehaves, 2018) and design for all approaches are used (Reinboth & Witczak, 2012).
The aim of this work report is to develop an initial proposal for items that can measure social implications possibly emerging from automated vehicles. Based on the current results, this work report discusses the possible next steps.
METHOD
Method and Results of the Previous First and the Second Study First study (from March to May 2023): The initial data processing stage is involved in a qualitative evaluation of low-fidelity prototypes for automated driving (which are presented in user narratives, a formulation of ideas in form of a story of use) in Germany. The qualitative online interviews are conducted and take approximately 90 minutes, each. 12 participants (three heavy commuters with level 2 experience, three young individuals between the age 18 and 25, three individuals aged 65 or older and three individuals who are responsible for childcare) are included. Within the interviews they are asked to consider the following question: “Imagine that such an automated driving system is established on the market. What do you think the impact will be on society and different social groups?”.
In data analysis, the quotes of the participants are matched to 24 different statements in German language. All statements were formulated in the same way (..., as...; e.g. “Fewer accidents, as the automated system drives purely reasonably”).
Second study (from August to September 2023): In an HMI-evaluation of preference for concepts to motivate adequate driver behaviour in different levels of automation, the statements of social implication from the first study are integrated as closed questions and rated by the participants the first time. 35 participants rate 21 statements (three statements from study 1 are removed as result of the pre-test) according to probability of occurrence from 0 to 100% in a standardized online questionnaire. In addition, to search for further statements, they are asked openly: “Do you have any other ideas about the impact the system could have on society once automated driving systems have established themselves on the market?”.
The standardised results of study 2 show a wide range of the expected probability of occurrence from 0 to 100% for nearly all statements. The mean of the ratings is between m = 23,77% and m = 77,46%, and the standard deviation is between SD = 20.62 and SD = 34.83. Table 1 gives six examples of the results (highest and lowest mean of all statements, four in between).
Table 1. Examples of the results of rating social implication items in evaluation 2.
Table available in the original PDF paper.
Statements of social implication M(SD) 59,00 (33.76) Fewer accidents, as the automated system drives purely reasonably. 77,46 (22.25) The job of cab driver becomes obsolete, as automated vehicles no longer require an active driver.
54,74 (29.84)
Older people can participate more in life as they become more mobile.
47,83 (31.99)
Environment is protected as energy consumption of cars is reduced.
28,86 (25.70)
Some people are burdened by automated driving psychologically, as they are reachable even during the journey and have the opportunity to work.
German automotive industry suffers as driving pleasure gets lost. 23,77 (20.62)
The high variance of the mean values from standardized measurement can be seen as a first hint for a content-related validity of the items.
By the open question, 31 additional statements on social implications are collected. After revision, 19 with regard to content new and well-defined statements are added to the 24 statements from before. These results are published in Brüggemann et al. (2023). The high number of 19 additional statements found in the second study indicate an incompleteness of the first and maybe also of the current collection.
Method of the Third Study In a subsequent third evaluation within a VR simulation study to test against three different concepts for motivating adequate driver behaviour in different levels of automation, the social implication statements are estimated
Empirical Analysis of Social Implications During the Development of Automated Driving 43
according probability of occurrence, again. The evaluation runs from March 2024 to May 2024. At the end of the VR simulation, the participants are now asked to rate a total of 43 statements (the old 24 statements from study one and the additional 19 statements from study two) by a probability scale from 0 to 100%. In addition, they are asked again: “Do you have any other ideas about the impact it could have on society once automated driving systems are established on the market?”.
A total of 95 participants (48 females included) take part in the study. The average age is 39.55 years (SD = 3.13, ranging from 35 to 45 years).
For the evaluation of the items integrated in the third study, the mean values and the standard deviations are looked at. For the statistical verification of the item distributions, the Kolmogorov-Smirnov test for normal distribution is then carried out and the skewness and kurtosis of the individual items are considered (Bortz & Döring, 2006).
RESULTS OF THE THIRD STUDY
Table 2 shows the six examples from Table 1 and in addition the extrema found in the sample according to M, Skewness, Kurtosis and Kolmogorov-Smirnov Test as examples of the results.
Table 2. Examples of the results of rating social implication items in evaluation 3.
Table available in the original PDF paper.
Statements of social implications M(SD) Skewness Kurtosis Kolmogorov- Smirnov 77,11 (23.39) −1.249 1.031 0.000 Automated systems lead to discontent in society, as only financially privileged individuals can afford them.
74,54 (39.00) −1.148 0.024 0.000
Novice drivers never learn manual driving properly, as the automated system takes on so many tasks for them.
68,60 (28.58) −0.849 −0.357 0.000
Fewer accidents, as the automated system drives purely rationally.
65,59 (24.86) −0.684 −0.084 0.006
Many are unsettled by automated vehicles, as they are controlled by machines and not by humans.
56,06 (33.67) −0.237 −1.303 0.005
The job of cab driver becomes obsolete, as automated vehicles no longer require an active driver.
53,86 (33.99) −0.016 −1.496 0.001
Older people can participate more in life as they become more mobile.
51,55 (33.65) −0.034 −1.424 0.003
Environment is protected as energy consumption of cars is reduced.
51,92 (30.28) −0.136 −1.162 0.066
Some people are burdened by automated driving psychologically, as they are reachable even during the journey and have the opportunity to work.
45,07 (30.34) 0.123 −1.085 0.041
German automotive industry suffers as driving pleasure gets lost.
15,45 (23.04) 2.005 3.556 0.000
With fully automated vehicles, one would no longer need a driver’s license, as the vehicle can drive without human intervention.
The mean of the estimations of the 43 statements for probability of occurrence is between m = 15,45% and m =77,11%. The standard deviations of the answers given to the statements is between SD = 23.04 and SD = 39.00. The results of kurtosis are almost all negative, as are the results of skewness. Most data thus show a left skewness and a slightly flattened distribution. Consequently, by a value below 0.05 the Kolmogorov-Smirnov test confirms the non-normally distributed values for 42 of the 43 items, tested in the study.
By the open question, about 100 new statements on social implications are collected. They are not finally edited, yet. Preliminary, we extracted 55 new statements.
DISCUSSION
The variance of the mean values and the scatter in the quantitative scores can be seen as an indicator of a given content responsiveness of the quantitatively tested items. In addition, from a purely descriptive point of view, the mean values and standard deviations of Study 2 and Study 3 are mostly very close together, which can also be seen as hint for a good answerability in terms of content.
A reason for the deviations from the normal distribution could be the specific characteristic of the test sample. The range of answers given by the sample aged 35–45 could be limited. In fact, all people, who move in street traffic, so from almost all age groups are affected by the social implications of automated vehicles. Moreover, we have to note that a non- normal distribution does not mean invalid items, normal distribution is just a prerequisite for some statistical tests.
In sum, the items included as well as the probability rating scale applied here could be a suitable response scale, but are not adequate for some statistical tests.
On current state, the single statements measured in the study cannot be summed up to overarching variables because they refer to different expectations which are independent to each other. To prove this hypothesis in future work, all indicators will be included in an explorative factor analysis to empirically determine the number and characteristics of the factors underlying the measured indicators and the correlations between the indicators (DeVellis, 2003). Unfortunately, the sample size of study three only allows a first approach to a factor analysis. For a well-founded factor analysis with such a high number of items a sample size over N = 300 is needed. At this point, however, it is necessary to see whether meaningful groups can be found or whether, each item should stand on its own.
The forecast validity of the data should also be reflected critically. What does it mean if a future scenario is considered probable or not probable in the vote? In fact, it shows a democratic assessment of the sample of respondents. Currently we do not know, if the forecast by many users is more valid than e.g. the qualitative estimation of a small number of experts for social implications. Perhaps, it is more worth to investigate the range of possible
Empirical Analysis of Social Implications During the Development of Automated Driving 45
social implications with a big number of users than to do the estimation of probability of occurrence with them.
For a multi perspective evaluation of the probability of occurrence of the statements, additional qualitative interviews could be helpful in order to gain more insights into the response behaviour of the test subjects. Or, as in similar projects, a mix of methods could be used, such as additionally rating the statements with experts.
The 55 additional implications from study three are not finally analysed yet. The fact that 55 additional potential social implications are found in addition to the 43 items already developed in the two preliminary studies indicate the impossibility to define a complete set of social implications for empirical evaluation in automated driving. Therefore, in addition to the further development and evaluation of quantitative scoring procedures, the possibility of adding new social implications should be integrated in a method for estimating social implications.
CONCLUSION
The present study focuses on the development of methods for an empirical based prediction of social implications of automated driving. The study uses a combination of qualitative and quantitative methods to identify and evaluate potential social impacts by doing research with users and other people directly or non-directly affected by the automation. While the study provides valuable qualitative insights, a final set of relevant social implications in general cannot be defined. Moreover, the validity of user predictions of probability of implications can be questioned. It is therefore recommended to develop a kind of Delphi method for prediction of social implications: To predict social implications of a specific prototype, it could be useful to firstly generate possible social implications for this specific prototype by qualitative research with users (and experts). In a second step, the implications generated could be evaluated on standardised level according to their probability of occurrence by experts for those social implications. The KARLI project concludes in September 2024. Further work to this topic depends on additional funding.
REFERENCES
Bortz, J. Döring, N. (2006). Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler. Heidelberg: Springer. Brüggemann, N. Preis, S. Pagenkopf, A. Engeln, A. (2023) Empirical analysis of
social implications during the development of automated driving. Hawaii: AHFE (Poster). Deutsches Institut für Normung e. V. (2020). “DIN EN ISO 9241-210:2020- 03, Ergonomie der Mensch-System-Interaktion - Teil 210: Menschzentrierte Gestaltung interaktiver Systeme (ISO 9241-210:2019)”, Deutsche Fassung EN ISO 9241-210:2019: Beuth, Berlin. DeVellis, R. F. (2003). Scale development. Theory and applications. Thousand Oaks:
Sage Publications.
Diederichs, F. Wannemacher, C. Faller, F. Mikolajewski, M. Martin, M. Voit, M.
Widlroither, H. Schmidt, E. Engelhardt, D. Rittger, L. Hashemi, V. Sahakyan, M. Romanelli, M. Kiefer, B. Fäßler, V. Rößler, T. Großerüschkamp, M. Kurbos, A. Bottesch, M. Immoor, P. Engeln, A. Fleischmann, M. Schweiker, M. Pagenkopf, A. Mathis, L-A. and Piechnik, D. (2022). “Artificial intelligence for adaptive, responsive, and level-compliant interaction in the vehicle of the future (KARLI)”, Conference 2022, International Conference on Human-Computer Interaction pp. 164–171. Springer International Publishing, Cham. Kemmer, D. (2020). A →B (≫Wenn A, dann B. ≪): Zur zentralen Rolle von Implikationen für die Konzeption und Praxis einer Integrierten Forschung. Wiesbaden: Springer. Lehner, F. (2011). Sozialwissenschaft. Wiesbaden: VS. Niehaves, B. (2018). INEMAS: Grundlagen interaktions- und emotionssensitiver
Assistenzsysteme. Siegen: Universität Siegen (Report). Reinboth, C. Witczak, U. (2012). Tecla - Autonomie, mobile Beratung zu technischen Assistenzsystemen zur Förderung des selbständigen Wohnens in der zweiten Lebenshälfte. (Report). SAE International (2021). Taxonomy and definitions for terms related to driving
automation systems for on-road motor vehicles. SAE Mobilus. https://www.sae. org/standards/content/j3016_202104/preview/ Schubert, K. Klein, A. (2021). Das Politiklexikon: Begriffe, Fakten, Zusammenhänge.
Bonn: Dietz. Schweiker. M, Pagenkopf. A, Engeln. A. (2023). Integration von ELSI- Fragestellungen in KARLI. Babenhausen: halftime presentation KARLI (Poster). Tirschmann, F. Brukamp, K. (2021). Ethische und soziale Aspekte der Robotikassistenz bei Amyotropher Lateralsklerose (ALS). Ludwigsburg: Evangelische Hochschule Ludwigsburg (Report). Zerth, J. Müller, S. Heidl, C. (2017). SeLe- Sensoren für verbesserte Lebensqualität; Teilvorhaben: Nutzer- und Akzeptanzforschung sowie sozioökonomische Wirkungsanalyse. Fürth: Forschungsinstitut IDC, Wilhelm Löhe Hochschule für angewandte Wissenschaften (Report).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET 2024)
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-33-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional