Iris Graessler, Deniz Oezcan (2024). Quality Key Figures for Developing Future Scenarios. In: Human Systems Engineering and Design (IHSED2024): Future Trends and Applications. AHFE Open Access, vol. 158, pp. 298–307. https://doi.org/10.54941/ahfe1005553
The usefulness of the consistency-based Scenario-Technique depends on expertise in identifying future influence factors, assessing their relationships as well as aggregating raw scenarios into alternative visions of the future. Typically, three alternative scenarios are developed, including extrema like positive and negative futures. In this paper, a new set of quality key figures is developed for the selection of these scenarios to reduce room for interpretation and to increase their quality. The three quality key figures include normal distribution, consistency and differentiability. These quality key figures are used iteratively to achieve an optimal balance. For validation purpose, Scenario-Technique was implemented in an internationally active mechanical engineering company for strategy alignments. The resulting future scenarios were developed and integrated into the strategy in eight workshops and three iterations with managing directors. The new metrics proposed in this paper help to create reliable future scenarios for the strategy.
Today’s business is becoming increasingly fast-moving and fragile. Due to the volatility of the sales markets, a convenational sales forecast and technology roadmap are not sufficient for the development of a strategy (Gräßler et al., 2023). In a strategy, decisions are made as to which product ideas should be pursued further, considering limited development budgets and available expertise within a company. In this process, the chances of success of alternative product ideas must be carefully weighed up against each other. The consistency-based Scenario-Technique provides a profound basis for this. It is characterised by anticipating contradiction-free developments of key factors in the entrepreneurial business environment, so called consistent scenarios (Linneman and Klein, 1983). In the absence of knowledge regarding future developments and disruptive events, probabilities are not considered in the consistency-based Scenario-Technique. Instead, the aim is to anticipate all conceivable future developments so that opportunities can be seized early and challenges are overcome more easily (Reymann, 2013).
Quality Key Figures for Developing Future Scenarios 299 developing company strategies. In order to ensure that decisions based on future scenarios are comprehensible and transparent at all times, it is necessary to determine quality key figures (QKF) that allow a statement to be made about quality and thus enable scenarios to be compared and evaluated at regular intervals (Graessler et al., 2024).
The following research questions were derived from this problem statement:
1. Which QKF can be used to improve strategy development of companies and institutions using Scenario-Technique? 2. What are the benefits of using QKF in decision-making processes for strategy development in companies and institutions?
The research questions are answered in the following five sections. After the introduction, a state of the art of Scenario-Technique is given (section 1), followed by an explanation of the systematic approach (section 2). Identified QKF for the Scenario-Technique are developed on the basis of the previous chapters (section 3). Finally, the applicability of the QKF is evaluated in a validation (section 4) and the paper is summarised in a conclusion (section 5).
STATE OF THE ART This section provides an overview of the state of the art of strategic planning using Scenario-Technique. One part of this is the identification of QKF for assessing the quality of scenarios. The results in this section are based on a structured literature analysis that follows the method of Machi and McEvoy (Machi and McEvoy, 2012).
Scenario-Technique The Scenario-Technique is a methodical approach to strategic planning and decision-making in which various possible future developments (scenarios) are systematically developed and analysed as shown in Figure 1. This technique is often used in business, politics and research to overcome uncertainties and develop long-term strategies. Due to the lack of formalisation in the process, the quality of the results depends on the individual expertise in applying of the methodology. After identifying influence factors, the relationships are evaluated and documented in an impact matrix. Possible future developments, so-called projections, are anticipated for each individual influence factor (Gordon and Hayward, 1968). Conditional possibilities for the realisation of a projection are evaluated if another projection occurs (Gordon and Hayward, 1968). Based on these estimates, overall estimates for scenarios with a projection of each influence factor can be calculated algorithmically (Kolmogorov, 1977). The selection criteria include the active and passive sum as well as the scope and time frame of the scenario project to support decision- making in strategy process. After deriving projections, their consistency is evaluated in pairs in the consistency matrix. According to Kosow and Reibnitz, consistency is understood as the logic of the simultaneous occurrence of two projections in a scenario (Kosow, 2015; Reibnitz, 1992). Raw scenarios are generated mathematically based on the consistency matrix. With the help of algorithms such as clustering or branch-and-bound, the number of raw scenarios can be
300 Graessler and Oezcan
reduced according to their consistency, difference and stability (Mißler-Behr, 1993). A number of three to five scenarios are selected from this set. In the mathematical definition, a scenario is a set of projections with one projection for each influence factor. The scenarios are then transferred to prose text in a more intuitive format and presented to the user (Gausemeier, 1995). The intuitively interpretable scenarios can be used to derive measures for various procedures. It is possible to derive measures for a company’s future strategy or for product ideas to be focused on in the future (Gräßler et al., 2017b). Figure 1 shows a simplified procedure of the Scenario-Technique:
Figure 1: Simplified representation of Scenario-Technique.Figure 2: Data of an influence factors in the Scenario-Technique.
The Integrated Scenario Data Model (ISDM) according to Gräßler and Pottebaum is used to store the respective data. In addition to supporting the formalisation of the procedure, the data shown in Figure 2 is also stored in a traceable manner. The data includes influence factors consisting of the attributes: Area of influence, description, descriptors, projections, and assigned to the projections the data. The area of influence describes the field of influence, e.g. politics, the market or other relevant areas. The description of the influence factor and the descriptor are the textual description and the measure of the influence factor. The projections of the influence factor are based on the future development of the influence factor based on the recorded descriptor. This data is all collected and utilised in an influence factor. This data is stored in the ISDM (Figure 2) (Gräßler et al., 2017a).
Quality Key Figures for Developing Future Scenarios 301 Systematic Approach This work is based on an methodology-driven five-step approach, as shown in Figure 3. In the first step, a systematic literature review and practical workshops are conducted to identify practical problems and relevant scientific approaches in the field of QKF in Scenario-Technique. Based on the identified approaches and practical problems, existing QKF for quality assurance of future scenarios are identified in a second step. In the third step, the identified QKF are analysed for their applicability in expert workshops. In step four, a new enhanced set of QKF for evaluating future scenarios is developed. In the fifth step, the developed QKF are applied to the strategy development of an internationally operating machine and plant manufacturer. The results are validated in eight expert workshops.
Figure 3: Systematic approach to define QKF for Scenario-Technique.
QUALITY KEC FIGURES FOR SCENARIO-TECHNIQUE In the following, the results of the five steps of the systematic approach are described sequentially.
Identify Practical Problems The derivation of the influence factors and the creation of the scenarios require not only statistical key figures and specialist knowledge, but also transparent and comprehensible communication. The results of the literature review is presented in table one.
302 Graessler and Oezcan
Table 1. Search strings.
Table available in the original PDF paper.
Vektor String Result
3 relevant results of 8 total results
9 relevant results of 514 total results
3 (“quality key figures“ OR “quality metrics” OR “quality measurement” OR “quality characteristics”) AND (“product strategy” OR “product ideas”) AND (“future scenarios” OR “future development” OR “alternative visions” OR “resulting scenarios” OR “future direction” OR “creating scenarios” OR “reproduce scenarios”) AND (“reproducibility” OR “distribution” OR “heterogeneity” OR “consistency” OR “differentiability” OR “recreating”) 8 ( “key quality figures” OR “quality metrics”) AND (“products” OR “product strategies”) AND (“future scenarios” OR “future results” OR “development” OR “future developments”) 9 (“quality key figures“ OR “quality metrics” OR “quality measurement”) AND (“products” OR “product”) AND (“future”)
4 relevant results of 186 total results
Based on the application-oriented workshops and the literature analysis, it was possible to extract the problem that interruptions in the argumentation arise when relying on untested scenarios, especially in the control after a certain period of time. The problems identified in the workshops were analysed in various approaches of literature. One problem is the necessary measurability of quality in order to be able to derive measures on an objective data basis. Another problem is the quality of the results, which can only be measured subjectively. The comprehensibility of the scenarios is particularly important in strategy development process, as their effectiveness must also be checked after time periods. Another problem is that easy- to-extract key figures are currently not used to assess the quality of the scenarios. Statements are made in the literature that the traceability of individual decisions in the procedure is not given. There are also statements about the unmeasurable quality and usability of scenarios generated by Scenario-Technique in workshops. The experts state that they need a basis for decision-making for the comprehensible derivation of measures and a traceable quality check.
Analysis of Identified QKF for Consistency-Based Scenario-Technique The respective key figures found, such as transparency, quality, plausibility, consistency, comprehensibility, selectivity, integration, effort and participants, were then analysed. The basis was used to record steps for obtaining valid QKF. The basis for discussion is thus objectified. The QKF collected from the literature were categorised on the basis of requirements. The requirements were derived from the literature and expert opinions through interviews. Exemplary requirements are, for example, an efficient
Quality Key Figures for Developing Future Scenarios 303 generation of results in the scenario project, an intuitive and comprehensible measurement parameter and the necessity of existing data for analysis. All requirements were then clustered. The 13 requirements were then evaluated and the three most relevant QKFs were selected on the basis of these.
Development of New QKF for Consistency-Based Scenario-Technique The set of QKF comprises three QKF, which are defined in Table two. Quality key figure one (QKF1) measures the Normal Distribution of the raw scenarios calculated by the algorithms. The data is recorded based on the steps of the Scenario-Technique and forms the consistency sum based on the consistency analysis. The raw scenarios are shown on the abscissa (as shown in Figure 4) based on their consistency sum. If these are distributed normally, a high quality can be derived from this. QKF2 Consistency, requires the fulfilment of a high consistency sum in order to generate consistent scenarios. The consistency sum is calculated from the sum of all consistencies wthin the scenario. QKF3, Heterogeneity, ensures sufficient differentiability of the scenarios. QKF 3 can be used to determine the degree of similarity between the respective scenarios. The key figure can be used to increase diversification within the derivation of measures. The QKF are used iteratively to achieve an overall optimum of focusing, normal distribution, heterogeneity, consistency. By iteratively applying the individual steps of the Scenario-Technique, a highly valid and effective picture of the future scenarios can be generated and used for strategy development. Figure four shows these elements, with the consistency sum plotted on the abscissa and the number of scenarios on the ordinate, these are relevant for QKF 1 and QFK 2.
Figure 4: QKF for development of future scenarios.
Set of QKF for Evaluating Future Scenarios The different QKF and the intended iterative procedure are shown in Table two.
304 Graessler and Oezcan
Table 2. The aim and purpose of the QKF.
Table available in the original PDF paper.
No. Quality Key Figure Aim and Purpose QKF1 Distribution of the raw scenarios
QKF2 Consistency of scenarios
The consistency sums in the Scenario-Technique are usually normally distributed because they consist of a sum of variables. The differences between the values of the same variable in different scenarios are random variables that are independent from each other. This means that the probability of a difference assuming a certain value does not depend on the probability of another difference assuming the same value. The normal distribution is a probability distribution that is described by two parameters: the expected value and the standard deviation. The expected value of the consistency sum is zero, as the differences between the values of the same variable in different scenarios are zero on average, because the scale ranges from -x to +x. The standard deviation of the consistency sum depends on the number of variables and the number of scenarios. The normal distribution of the consistency sums has some important consequences. Firstly, the probability that a consistency sum exceeds a certain value can be determined. Secondly, the consistency sum can be calculated for a specific confidence interval. In practice, the normal distribution of consistency sums is often used to assess the quality of scenario data. If the consistency sums of a sample scenario data are not normally distributed, this may indicate that the scenario data is not suitable for assessing future development. A consistency sum is a measure of the internal consistency of a set of data. It is calculated by dividing the sum of the deviations of the individual data points from the mean value of the set by the number of data points. The consistency sum can be used to assess the reliability of measurements or to test the agreement of data with a model. The consistency sum is a simple and easy to calculate measure and is an intuitive measure of the internal consistency of a set of data. The higher the consistency sum, the better it is for scenarios. It is therefore optimal to utilise scenarios with the highest consensus sum. The heterogeneity of scenarios describes the variety and diversity of scenarios. It can refer to various aspects of the scenarios, such as
QKF3 Heterogeneity of scenarios
Topics: The scenarios can deal with different topics, e.g. climate change, energy transition, digitalisation or demographic change.
Perspectives: The scenarios can be developed from different perspectives, e.g. from the perspective of politics, business, science or civil society.
Target groups: The scenarios can be developed for different target groups, e.g. for decision-makers in politics or business, for the general public or for the scientific community.
The heterogeneity of the scenarios can help to ensure that a broad spectrum of possible developments is taken into account and the scenarios can contribute to making the scenarios more robust and crisis- proven. The heterogeneity of the scenarios can help to ensure that the scenarios are relevant for different target groups.
A new proposed set of QKF supports the selection of suitable future scenarios as a starting point for the selection of promising product ideas.
Quality Key Figures for Developing Future Scenarios 305 This reduces the scope of interpretation when creating future scenarios and increases the quality of the resulting scenarios.
Application and Validation of QKF in Industrial Context The results were validated using a strategy development process project from mechanical and plant engineering in a medium-sized German company. Eight workshops were held with managing directors to assess the applicability of the QKF and to arrive at reliable future scenarios for the product strategy in the example application. Decisions on new products, innovations and corporate strategy are made by a group of managers. Due to the company structure, there is not much space for errors when making decisions about strategy. The application example for validation was carried out with a group of eight decision- makers in the group who were also involved in a strategy development process. All of the decision makers are managing directors of different daughter companies for which the strategy is being developed. The application of these QKF and adaptive steps of the Scenario-Technique shows that they support successful strategy development based on future scenarios. The observational study showed that a decision can be made very objectively and comprehensibly using the QKF provided in combination with the iterative procedure of the Scenario-Technique. The users stated that the utilisation of the results using the QKF is more comprehensible and therefore easier to trace and argue for decisions. Both the QKF and the iterative approach help to structure the informal process of strategy development in the company. This reduces the time required to collect the relevant information for the decision on the influences on the company. For another expert, in addition to the results of the Scenario-Technique, the quality of the scenarios created was very important in order to be able to derive the right measures from them, in the sense that if the scenarios are error-free, only the right measures can be created. The measures can be derived directly from the quality-checked and scenario analyses. Furthermore, observations showed that the experts felt encouraged and more confident when using the QKF than without it, especially when deriving measures.
CONCLUSION
The Scenario-Technique helps organisations to be better prepared for uncertain and complex future situations by anticipating possible developments and systematically thinking through options for action in advance. This development of alternative visions of the future can be characterized by QKF. In this contribution, a new set of key figures is proposed that supports the selection of appropriate future scenarios as a starting point for the selection of promising product ideas. This reduces the scope of interpretation used in generating future scenarios and increases the quality of the resulting scenarios. The basis for discussion is thus objectified. The RQ1 is answered by a set of three QKF: QKF1 incorporates the normal distribution of the raw scenarios calculated by the algorithms. QKF2 requires the fulfilment of a high consistency sum in order to generate contradiction- free scenarios. QKF3, heterogeneity, ensures sufficient differentiability of
306 Graessler and Oezcan
the scenarios. The QKF could be used iteratively to achieve an overarching optimum. With the use of the QKF it was possible to answer RQ2 with the help of expert statements from the validation. In a survey, the managing directors stated that the QKF provide a transparent basis for decision- making in the comprehensible development of a corporate strategy from future scenarios. It can therefore be concluded that the QKF promote precise added value in strategy development. In the accompanying feedback discussion, the experts attached importance to ensuring that the steps and decisions during the strategy development process are comprehensible, transparent, understandable and self- explanatory, so that a new selection of a corporate strategy is very efficient in the event of topic-specific or time-related modifications in the weighting. The results are aimed at decision-makers in SMEs. The validation in a strategy development project proved that the QKF support the strategy development process. The QKF can also be determined on the basis of the available data. The QKF are particularly suitable for selecting the designed narratives, as they support a comprehensible decision, especially when selecting the right scenarios to embed them in the development process. Both decision-makers and decision preparers are the target group of the results presented here. The paper enables these decision makers to select the most appropriate strategy for their company based on QKF tested future scenarios with a methodological approach. The QKF can be generalised and transferred to other areas of application. By researching and applying strategy development, the Scenario- Technique can be extended to capability assessment. Here, competences of individual roles can be used to identify and select the most appropriate strategy for the company based on highly qualitative future scenarios.
Acknowledgment The research leading to these results has received funding from the European Union`s Horizon Europe Programme under the CREXDATA Project, grant agreement n° 101092749. We extend our gratitude to all collaborators, partners, and contributors involved in CREXDATA for their dedication. REFERENCES Gausemeier, Jürgen (1995). Die Szenario-Technik. Werkzeug für den Umgang mit einer multiplen Zukunft. Paderborn, Heinz Nixdorf Institut, Universität-GH Paderborn. Gordon, T. J./Hayward, H. (1968). Initial experiments with the cross impact matrix
method of forecasting. Futures 1 (2), 100–116. https://doi.org/10.1016/S0016- 3287(68)80003-5. Graessler, Iris/Wiechel, Dominik/Oezcan, Deniz/Taplick, Patrick (2024). Tailored metrics for assessing the quality of MBSE models. Proceedings of the Design Society 4, 2545–2554. https://doi.org/10.1017/pds.2024.257. Gräßler, I./Tušek, A./Özcan, D. (2023). Entwicklung zukunftsfähiger Geschäftsmodelle
in der Automatisierungstechnik anhand von Megatrends. In: Automation 2023. Transformation by Automation. Düsseldorf, VDI Verlag, 863–874. Gräßler, Iris/Pottebaum, Jens/Scholle, Philipp (2017a). Integrated Process and Data Model for Agile Strategic Planning. In: Sandor Vajna (Ed.). 11th International Workshop on Integrated Design Engineering, IDE Workshop, Magdeburg, 05.04. - 07.04. Gräßler, Iris/Scholle, Philipp (2016). Identification of indirect influences under time-
variant impacts in Scenario-Technique for foresight. In: Proceedings of R&D Management Conference 2016, From Science to Society, Innovation and Value Creation, R&D Management Conference, Cambridge, UK, 03.-06.07.2016, 1–12.
Quality Key Figures for Developing Future Scenarios 307 Gräßler, Iris/Scholle, Philipp/Pottebaum, Jens (2017b). Integrated process and data
model for applying scenario-technique in requirements engineering. In: Anja Maier/Stanko Škec/Harrison Kim et al. (Eds.). DS 87-Abstracts. Proceedings of the 21st International Conference on Engineering Design, ICED 17, Vancouver, Canada, 21.-25.08.2017. Design Society, 261–270. Kolmogorov, Andrej N. (1977). Grundbegriffe der Wahrscheinlichkeitsrechnung.
Berlin, Springer. Kosow, Hannah (2015). New outlooks in traceability and consistency of integrated scenarios. European Journal of Futures Research 3 (1), 16. https://doi.org/10. 1007/s40309-015-0077-6. Linneman, Robert E./Klein, Harold E. (1983). The use of multiple scenarios by U. S.
industrial companies. A comparison study, 1977–1981. Long Range Planning 16 (6), 94–101. https://doi.org/10.1016/0024-6301(83)90013-4. Machi, Lawrence A./McEvoy, Brenda T. (2012). The literature review. Six steps to success. 2nd ed. Thousand Oaks, Calif., Corwin. Mißler-Behr, Magdalena (1993). Methoden der Szenarioanalyse. Wiesbaden, Dt.
Univ.-Verl. Reibnitz, Ute von (1992). Szenario-Technik. Instrumente für die unternehmerische und persönliche Erfolgsplanung. 2nd ed. Wiesbaden, Gabler. Reymann, Felix (2013). Verfahren zur Strategieentwicklung und -umsetzung auf
Basis einer Retropolation von Zukunftsszenarien. Paderborn, Heinz-Nixdorf- Inst., Univ. Paderborn. Ulrich, Hans (1981). Die Betriebswirtschaftslehre als anwendungsorientierte Sozialwissenschaft. In: Manfred Geist/Richard Köhler (Eds.). Die Führung des Betriebes. Curt Sandig zu seinem 80. Geburtstag gewidmet. Stuttgart, Poeschel, 1– 27.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Systems Engineering and Design (IHSED2024): Future Trends and Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-34-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional