Kristin Paetzold - Byhain, Franziska Scharold (2024). Classification of uncertainties in agile development of mechatronic systems. In: Human Interaction and Emerging Technologies (IHIET 2024). AHFE Open Access, vol. 157, pp. 450–459. https://doi.org/10.54941/ahfe1005504
Agile methods are increasingly important for developing mechatronic systems in dynamic and volatile environments. These methods help development teams to deal with uncertainties in the development process and utilize them as opportunities. One feature of agile methods is the active involvement of developers in the flow of communication and information, which has a positive effect on the quality of decision making in terms of the development process. It is essential that the developer understands, accepts, and respects the uncertainty in the development task/process to realistically evaluate development scenarios and make well-founded decisions. Tailor-made approaches are required to deal with uncertainties in the development process, as the causes of these are manifold. This paper is based on extensive literature research, analyses of agile development processes at industrial partners and a series of studies on the agile development of physical products that have been carried out regularly for six years. The aim is to differentiate uncertainties to be able to utilize and adapt the artefacts and activities employed in agile development to deal with uncertainties more effectively.The aim is to evaluate the artefacts (e.g. increment, backlog, prototype...) and activities used in agile development in terms of their potential to illustrate and deal with existing uncertainties. It will be of particular importance here which views the individual roles in the development process (product owner, agile master, developer) have on existing uncertainties and how these are interpreted as a consequence. Conversely, it is important to derive recommendations for designing artefacts and activities in agile methods in order to support communication and information flows in the best possible way.
To respond appropriately in the dynamic and volatile development of mechatronic systems, agile methods are becoming increasingly important. The provision of mechanisms serves to facilitate the management of uncertainties inherent to the development process, enabling their acceptance and subsequent exploitation as opportunities (Böhmer et al., 2015). One characteristic of agile development is the active integration of developers into communication and information flows. It has a positive impact on the quality of decision-making in product development. This is precisely why it is necessary to recognise, accept and respect uncertainties in the development process in order to be able to realistically assess development situations and react in a context-specific manner (Thunnissen, 2003). Product development processes are inherently characterized by uncertainties (Freisleben and Vajna, 2002). There are many reasons for this, but they are primarily based on the fact that target systems are initially only theoretical ideas that need to be successively concretised (Bender and Gericke, 2021). Mechatronic systems are developed through the division of labour. Which means that
Classification of Uncertainties in Agile Development of Mechatronic Systems 451
sub-functions must be defined to implement the overall system behaviour, resulting in organizational and functional interfaces. Further interfaces may be added during the specification of the solution due to decisions on solution principles, which are often not sufficiently communicated (Ehrlenspiel and Meerkamm, 2013). Development tasks are often processed asynchronously. This means that the necessary information may not be available at the beginning of the development process and must first be supplemented by assumptions. This leads to uncertainties that are associated with risks, both for the development process itself and for the product or system behaviour (Thunnissen, 2003). From a project perspective, uncertainties arise primarily from the novelty of the development task, but also from development constraints (Rupp, 2014). Product development is aware of these risks due to the inherent uncertainty of the development process and the constraints imposed by the project’s boundary conditions. The utilization of risk analysis methodologies (e.g., as exemplified in Goldberg et al. (1994)) is employed, yet these processes are time-consuming and costly, ultimately impeding the availability of development resources for the identification of solutions (Sutherland, 2014). This domain encompasses the application of agile development methodologies, which are founded upon the values postulated in the Agile Manifesto (Beck et al., 2001) and aim to minimize non-value-adding activities. Agile principles and procedures facilitate the ability of development organizations to remain flexible in dynamic and volatile conditions and to respond quickly to change (Böhmer et al., 2015).
Böhmer et al. (2015) define agility as follows: “Agility is the capability to react, and adopt to expected and unexpected changes within a dynamic environment constantly and quickly; and to use those changes (if possible) as an advantage.”
Uncertainty is an inherent characteristic of complex systems. A key element of agile development is that the iterative and incremental process not only evaluates the results regarding the fulfilment of the requirements, but also the goals underlying the requirements. The simultaneous consideration makes it possible to recognise discrepancies between the original goals and the current reality, thus enabling the goals to be realigned or corrected (Schrof and Paetzold, 2020). This approach provides a foundation for a new way of dealing with uncertainty.
Agile approaches such as Scrum (Schwaber and Sutherland, 2020), which are currently widely used (Weiss et al., 2023), were originally designed to enable teams of eight to ten developers to complete development tasks in a manageable amount of time. Adaptations of agile methods appear necessary, as complex mechatronic systems are always developed based on a division of labour, whereby the individual development tasks are not independent of each other. This aspect is addressed with scaling process models such as LeSS or SAFe (Dingsøyr and Moe, 2014).
The physicality of mechatronic systems requires suitable adaptations (Ovesen, 2012) to employ elements such as increments and prototyping. Ultimately, this is necessary in order to preserve the effects of agile methods in terms of flexibility, self-organisation, transparency and the organisation’s ability to learn (Heimicke et al., 2020).
452 Paetzold-Byhain and Scharold
The research question that forms the basis of this paper is derived from the aforementioned considerations: What are the criteria for identifying and evaluating uncertainties in the development process in order to draw conclusions about necessary and targeted adaptations of agile methods in the specific organizational context?
THEORETICAL FOUNDATIONS To identify and analyse uncertainties in the development of mechatronic systems, a comprehensive understanding of uncertainties is required. To this end, relevant existing approaches for describing and analysing uncertainties as well as agile development methods are presented in the following.
Approaches to the Identification and Classification of Uncertainties Uncertainty plays an important role in all types of decision-making processes, as it is not possible to assume completely deterministic conditions (Walker et al., 2003). Consequently, the management of uncertainties is an issue of concern across various scientific disciplines, including political and social sciences, economics, and engineering. The diverse perspectives on uncertainties arise from a multitude of factors, including the system under consideration, the decisions to be made, and the boundary conditions to be taken into account (Walker et al., 2003). Thunnissen (2003) provides an overview of approaches from various areas of expertise. Despite the disparate approaches employed, it becomes evident that there are similarities and patterns in their interpretation, classification, and root cause analysis. Thunnissen (2003) summarizes these findings in his classification of uncertainties for the design of complex technical systems. He differentiates between ambiguities, epistemic and aleatory uncertainties, as well as interactions, and further subdivides these (Thunnissen, 2003). Ambiguities describe situations in which the decision is not based on a clear idea of its effects. Such ambiguities are based not least on uncertainties regarding cause-effect relationships. As a result, it is not possible to accurately assess the impact of changing boundary conditions (Engelhardt, 2013). Aleatory uncertainties are those that arise due to random variations in influencing variables. In contrast, epistemic uncertainties arise when there is a lack of precise knowledge about the object under consideration or when the available knowledge is incomplete (Bedford and Cooke, 2001). Thunnissen (2003) adds interactions as effects in which system elements interact with each other unexpectedly. In Haberfellner et al. (2019), this phenomenon is also referred to as emergence. Agile methods appear to be particularly suitable for so- called VUCA conditions. VUCA stands for volatility, uncertainty, complexity, and ambiguity. A detailed analysis shows that these characteristics are not independent of each other but are hierarchically dependent (Pendzik et al., 2023). Accordingly, complexity is seen as the primary source of uncertainty. The diversity and multiplicity of elements, which in turn are connected to each other by a diversity and multiplicity of relationships, results in a high degree of variety and connectivity (Patzak, 1982). High connectivity, in turn, can lead to emergence, i.e. to situations in which
Classification of Uncertainties in Agile Development of Mechatronic Systems 453
the system does not react in the expected way. The variability of system elements and their dynamic behaviour over time is understood as a cause of volatility (Patzak, 1982). Uncertainties in the boundary conditions of development lead to ambiguities (Waller et al., 2019) that are either aleatory or epistemic in nature. They also require different strategies for dealing with them (Engelhardt, 2013). While aleatory uncertainties are primarily information problems, i.e., the data required for decision-making exists but is not available in the specific situation, epistemic uncertainties are more of a knowledge problem. Interdependencies are virtually unknown.
McManus and Hastings (2005) focus on the understanding of uncertainties with the aim of deriving implications for the design of complex technical systems. Based on the causal chain, “uncertainties lead to risks/opportunities lead to results” McManus and Hastings (2005, 485) identified causes of uncertainties: Lack of knowledge describes that elements or facts are required but are not or not yet available at the given time to be able to make rational decisions. Lack of specifications or definitions describes that elements or facts are not yet specified but would be necessary for a valid decision. Statistical uncertainties describe elements that are known but whose characteristics fluctuate or are statistically characterized. Known unknowns refer to elements that are known to be considered but are not known. Unknown unknowns are elements that are not known per se and are therefore not considered.
Earl et al. (2005) have further developed this framework by arranging the identified uncertainties on two orthogonal axes. In this way, they create a framework for a further differentiation of uncertainties. The previously defined categories of information problem and knowledge problem can also be assigned here.
Walker et al. (2003) examine conceptual foundations for managing uncertainties for decision support. System models form the basis for classifying and assessing uncertainties. Models are always only representations of reality that make the complexity of the systems manageable by reducing it according to the purpose of the model (Bossel, 2018). Stachowiak (1973) defines three characteristics of models: Reduction Feature, Pragmatic Feature, and Mapping Feature, each of which addresses specific uncertainties and can therefore be taken into account. The basis for the uncertainty analysis is the categorisation of uncertainties into categories similar to those used by Thunnissen (2003), from which specific characteristics for uncertainties can be derived. Three dimensions are defined: Localization of uncertainty: Uncertainties come from the object under consideration itself, the environment of the object under consideration or from the data with which the model of the object is fed.
Degree of uncertainty: Addresses the gradual transition between determinism and complete ignorance. Although subcategories are named here (statistical uncertainty, scenario uncertainty, recognized uncertainty, and complete ignorance), these correspond to the categories identified in McManus and Hastings (2005). However, when applied to different locations or sources of uncertainty, there may well be differences in meaning.
454 Paetzold-Byhain and Scharold
Type of uncertainty: This in turn differentiates according to the effort required to gain knowledge to reduce uncertainties. Analogous to the aforementioned sources, a distinction is also made here between epistemic and aleatory uncertainties.
While the localization of uncertainties helps to identify them, the categories relating to the degree of uncertainty are helpful in analysing and specifying the reason for them. The type of uncertainty describes the effort required and the options for reducing it. In Walker et al. (2003), an uncertainty matrix is derived on the basis of this multidimensional categorization. This overview is designed to enhance awareness of the interdependencies of uncertainties, with the objective of enabling a more detailed analysis and control of their effects.
Agile Approaches in Mechatronic Systems Development Agile approaches are characterized by a high-frequency iterative, incremental approach. Development teams create increments at predefined intervals that can be validated by the customer (Schwaber and Sutherland, 2020). In addition to Scrum as one of the most popular methods (Digital.Ai, 2021), there are many other methods that essentially follow the same logic. Comprehensive descriptions of agile methods can be found in Dingsøyr et al. (2012).
Figure 1: Scrum process; according to Schwaber and Sutherland (2020).
The process model behind Scrum is summarized in Figure 1. The indicated cycle comprises both technical-physical development activities and technical management activities. The cycle begins with sprint planning based on user stories and the product backlog (Plan). During the sprint, the defined task is completed using product development methods (Do). Daily meetings serve as a control element, where the current work status is recorded, and problems and difficulties are discussed. At the end of a sprint, the functionalities implemented in prototypes (Increment; Control) are discussed with customers, product owners and other stakeholders in sprint review meetings (Act). This creates further potential to sharpen requirements, recognize problems and initiate changes. It enables the development team to remain agile, minimize waste in resources and ultimately deliver a
Classification of Uncertainties in Agile Development of Mechatronic Systems 455
product tailored to the customer’s needs. Sprint retrospectives, which take place after a sprint review but before the next sprint planning, give the team the opportunity to reflect on their own work as part of a learning process. An analysis of agile development methods and their elements reveals fundamental strategies for dealing with uncertainties:
Elements that support team communication create transparency and help to solve information problems.
Elements of customer integration help to clarify and concretize requirements and boundary conditions regarding customer needs and product use.
Prototyping serves two primary purposes: it validates the results with the customer and it facilitates exploration by developing technical interrela- tionships. This approach helps to reduce the knowledge problem. The incremental and iterative approach in short cycles allows decisions to be postponed, thereby creating opportunities to complete the knowledge and information base, which in turn helps to reduce uncertainty.
However, in the development of complex mechatronic systems, there are also uncertainties that are determined by the product characteristics but are not explicitly addressed by elements of agile development:
Development is based on a division of labour; the separate view can lead to a loss of integration into the overall context. Conflicts of objectives are difficult to identify, functional and organisational interfaces are not considered.
The required regular prototyping after each increment is difficult to realize. This is not only due to the division of the development process into phases (planning, conceptualizing, designing, integrating (Bender and Gericke, 2021)), but also due to the resulting product itself. As a result, vertical prototypes are increasingly being used (Weiss et al., 2023), which only represent parts of the product. This means that the properties of the prototype do not correspond to those of the entire product, which leads to distortions in the interpretation of the system behaviour.
The previous points demonstrate that agile methods for the development of mechatronic systems require adaptations, additions, and novel elements to realize improved uncertainty management.
APPROACH FOR IDENTIFYING AND CLASSIFYING UNCERTAINTIES IN AGILE DEVELOPMENT PROCESSES The aforementioned classification schemes must be integrated to analyze uncertainties in the development process and adapted problem specific. The starting point is the differentiation and definition of types as proposed by McManus and Hastings (2005). Following Walker et al. (2003) the types should be differentiated according to the place of origin too. The systemic approach is useful here, as it also considers uncertainties in relation to the system description and the data and information used. The distinction between epistemic and aleatory uncertainties serves above all as an indication
456 Paetzold-Byhain and Scharold
IPWP IP
about user behaviour WP
strategies IP
relationships WP
requirements
Unknown requirements
requirements WP
Sources of information are
not known
WP Inadmissible assumptions
WPIP Unknown cause-effect
IP Uncertainties about communication structures
User behaviour
Diversity of variants
Freedom of choice in decision
Scope for decision making
unclear about secondary functions
additional interfaces
overall system IP
Responsibility for requirements IP Conflicting objectives in
Access to data sources IP Existence of the data IP
channels
Inhibitions to use information carriers
IP Unclear responsibilities IP Misunderstanding the
IP Data storage location
IP Uncertainties about communication
IP Importance of roles for own task
Variation in degree of novelty IP Additional unknown effects WP
organisation Uncertainties about specific contact persons
IT-structure Current versions, releases IP Authorisation for data access IP
Communic. structure Accessibility of information carriers IP Which persons are information carriers in context IP
behavioural architecture Variations in the function parameters WPIP Uncertainties about interfaces
Physical architecture Variations in the structural characteristics WPIP Uncertainties about interfaces IPWP
Development task Variations in development tasks
Costumer access Degree of individuality IP Usage situation
Company Strategy Current order situation IP Priorities in development
Task distribution Organisa-tional constraints Importance of one’s own task in the
Requirements management Uncertainty about the timeliness of the
Task description Personal interpretation of the task
Information sources (communication) Temporal unavailability of information
Table 1. Structuring matrix for uncertainties in agile product development.
Table available in the original PDF paper.
Location Level
Context Market/ environment Variations in relevant values IP Requirements uncertainties IP Novelties in technology IPWP
model Product architecture Functional/
System
Input Data availability Actuality of the data
IP = Information problem; WP = Knowledge Problem
Classification of Uncertainties in Agile Development of Mechatronic Systems 457
of which strategies can provide support in dealing with them, as these are indicative of an information or knowledge problem. The identification of uncertainties is a consequence of the analysis of the development process, which must be considered in conjunction with the management processes that are employed to organise the development work. The findings were summarised in a matrix for identifying and describing uncertainties based on Walker et al. (2003) (see Table 1). The next step is to complete and refine the listed uncertainties. To this end, interviews and workshops will be conducted together with an industry partner, to check the approach for accuracy and plausibility, and to discuss and complete the contents of the matrix itself. This then serves as the basis for determining specific elements for the development process, with the objective of enabling the management of uncertainties.
CONCLUSION
The paper presents a matrix for identifying uncertainties in the development process. Several uncertainties were also found to be linked through causal chains as the matrix was built. This is in line with the findings from the studies on VUCA, as described above. For taking this into account, it is necessary to extend the approach to consider the dependencies between uncertainties in the development process. Based on the matrix, elements for dealing with these uncertainties were developed and integrated into the development process with the intention of supporting agile product development.
REFERENCES
Beck, K./Beedle, M./van Bennekum, A./Cockburn, A./Cunningham, W./Fowler, M./Grenning, J./Highsmith, J./Hunt, A./Jeffries, R./Kern, J./Marick, B./Martin, R./Mellor, S./Schwaber, K./Sutherland, J./Thomas, D. (2001). Manifesto for Agile Software Development. Available online at agilemanifesto.org (accessed 4/16/2024). Bedford, Tim/Cooke, Roger (2001). What is uncertainty? In: Probabilistic Risk
Analysis: Foundations and Methods, 17–38. Bender, Beate/Gericke, Kilian (2021). Pahl/Beitz Konstruktionslehre. Berlin, Heidelberg, Springer Berlin Heidelberg. Böhmer, Annette Isabel/Beckmann, Andreas/Lindemann, Udo (2015). Open Innovation Ecosystem - Makerspaces within an Agile Innovation Process. In: ISPIM Innovation Summit. Bossel, Hartmut (2018). Modeling and Simulation. A K Peters/CRC Press. Digital. Ai (2021). 15th Annual State of Agile Report. Available online at https://digi tal.ai/resource-center/analyst-reports/state-of-agile-report/ (accessed 11/5/2023). Dingsøyr, Torgeir/Moe, Nils Brede (2014). Towards Principles of Large-Scale Agile
Development. In: Torgeir Dingsøyr/Nils Brede Moe/Roberto Tonelli et al. (Eds.). Agile Methods. Large-Scale Development, Refactoring, Testing, and Estimation. Cham, Springer International Publishing, 1–8. Dingsøyr, Torgeir/Nerur, Sridhar/Balijepally, VenuGopal/Moe, Nils Brede (2012). A decade of agile methodologies: Towards explaining agile software development. Journal of Systems and Software 85 (6), 1213–1221. https://doi.org/10.1016/j.js s.2012.02.033 Earl, Chris/Johnson, Jeffrey/Eckert, Claudia (2005). Complexity. In: John Clarkson/Claudia Eckert (Eds.). Design process improvement. London, Springer London, 174–197.
458 Paetzold-Byhain and Scharold
Ehrlenspiel, Klaus/Meerkamm, Harald (2013). Integrierte Produktentwicklung.
Denkabläufe, Methodeneinsatz, Zusammenarbeit. 5th ed. München, Hanser. Engelhardt, Roland Alexander (2013). Uncertainty mode and effects analysis - heuristische Methodik zur Analyse und Beurteilung von Unsicherheiten in technischen Systemen des Maschinenbaus. Düsseldorf, VDI-Verlag. Freisleben, Do¨rte/Vajna, Sa’ndor (2002). Dynamic Project Navigation: Modelling,
Improving, and Review of Engineering Processes. In: Volume 2: 28th Design Automation Conference, ASME 2002 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Montreal, Quebec, Canada, 29.09.2002 - 02.10.2002. ASMEDC, 919–925. Goldberg, Benjamin E./Everhart, K./Stevens, R./Babbitt III, N./Clemens, P./Stout, L.
(1994). System engineering toolbox for design-oriented engineers. Haberfellner, Reinhard/Weck, Olivier de/Fricke, Ernst/Vössner, Siegfried (2019).
Systems Engineering. Cham, Springer International Publishing. Heimicke, J./Chen, R./Albers, A. (2020). Agile Meets Plan-Driven – Hybrid Approaches in Product Development: A Systematic Literature Review. Proceedings of the Design Society: Design Conference 1, 577–586. https://doi.org/10.1017/dsd.2020.259 McManus, Hugh/Hastings, Daniel (2005). A Framework for Understanding Uncertainty and its Mitigation and Exploitation in Complex Systems. INCOSE International Symposium 15 (1), 484–503. https://doi.org/10.1002/j.2334-5837. 2005.tb00685.x Ovesen, Nis (2012). The Challenges of Becoming Agile. PhD Thesis. Aalborg
University. Patzak, G. (1982). Systemtechnik, Planung komplexer innovativer Systeme. Grundlagen, Methoden, Techniken. Berlin/New York, Springer-Verlag. Pendzik, Martin/Sembdner, Philipp/Paetzold, Kristin (2023). Identification and Classification of Uncertainties as the Foundation of Agile Methods. Proceedings of the Design Society 3, 2165–2174. https://doi.org/10.1017/pds.2023.217 Rupp, Chris (2014). Requirements-Engineering und -Management. Aus der Praxis
von klassisch bis agil. 6th ed. München, Hanser, Carl. Schrof, J./Paetzold, K. (2020). Agile Produktentwicklung in einer zunehmend dynamischen Automobilwirtschaft: Potentiale und Grenzen/Agile Product Development in an Increasingly Dynamic Automotive Industry: Potentials and Limits. In: Bernhard Geringer/Hans-Peter Lenz (Eds.). 41. Internationales Wiener Motorensymposium 22–24. April 2020. VDI Verlag, II-362-II-376. Schwaber, Ken/Sutherland, Jeff (2020). The Scrum Guide. Available online at https:
//scrumguides.org/scrum-guide.html (accessed 10/2/2023). Stachowiak, Herbert (1973). Allgemeine Modelltheorie. Wien [etc.], Springer. Sutherland, Jeff (2014). Scrum: The Art of Doing Twice the Work in Half the Time.
New York, Crown. Thunnissen, Daniel P. (2003). Uncertainty classification for the design and development of complex systems. In: 3rd annual predictive methods conference. Walker, W. E./Harremoës, P./Rotmans, J./van der Sluijs, J. P./van Asselt, M. B. A./Janssen, P./Krayer von Krauss, M. P. (2003). Defining Uncertainty: A Conceptual Basis for Uncertainty Management in Model-Based Decision Support. Integrated Assessment 4 (1), 5–17. https://doi.org/10.1076/iaij.4.1.5.16466
Classification of Uncertainties in Agile Development of Mechatronic Systems 459
Waller, Robert E./Lemoine, Pamela A./Mense, Evan G./Garretson, Christopher J./Richardson, Michael D. (2019). Global Higher Education in a VUCA World: Concerns and Projections. Journal of Education and Development 3 (2), 73. https://doi.org/10.20849/jed.v3i2.613 Weiss, Stefan/Paetzold-Byhain, Kristin/Michalides, Marvin/Pendzik, Martin/Scharold, Franziska/Stoiber, Lino (2023). Agile Entwicklung physischer Produkte 2023. Technische Universität Dresden.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET 2024)
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-33-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional