Can Cagincan, Juliane Balder, Roland Jochem, Rainer Stark (2026). METIS: A quality-oriented multi-stage decision framework for IT tool-stack optimization. In: Industrial and Systems Engineering. AHFE Open Access, vol. 219, pp. 128–137. https://doi.org/10.54941/ahfe1007783
Small and medium-sized enterprises (SMEs) frequently operate heterogeneous IT tool landscapes that evolve incrementally, resulting in fragmented digital work environments and limited support for integrated digital transformation. Existing tool selection approaches mainly evaluate isolated software solutions and rarely address system-level configuration or digital work quality. This paper proposes a human-centered Situation-Aware Digitalization Framework for systematic IT tool stack reconfiguration in collaborative product development. The framework combines CEAM-derived Engineering Collaboration Building Blocks (ECBBs) to standardize digital work activities, quality science–based prioritization to identify improvement needs, and a multi-stage multi-criteria decision-making (MCDM) mechanism to generate situation-aware tool stack recommendations. Functional capability, user-perceived quality derived from crowdsourced ratings, and SME-specific digitalization targets are integrated into a unified scoring model. Two iterative strategies support either compact reconfiguration or incremental transformation. Implemented in the METIS application and validated in industrial workshops, the framework demonstrates feasibility and practical decision-support value for aligning IT tool environments with SME-specific digitalization objectives.
Small and medium-sized enterprises (SMEs) increasingly rely on a broad range of information technology (IT) tools, particularly in collaborative product development activities such as requirements management, design coordination, project planning, and cross-functional communication. In recent years, the rapid availability of cloud-based has enabled SMEs to digitalize selected activities without large upfront investments. As a result, many SMEs operate heterogeneous IT tool landscapes composed of applications adopted incrementally to address local or short-term needs rather than following a coordinated and strategic digitalization approach (Brink & Packmohr, 2023).
However, such IT tool landscapes often evolve organically rather than through systematic planning. New tools are frequently added without reassessing existing solutions, leading to redundant functionalities; currently, Received February 13, 2026; Revised April 7, 2026; Accepted April 22, 2026; Available online July 20, 2026
METIS: A Quality-Oriented Multi-Stage Decision Framework for IT Tool-Stack Optimization 129 ~50% of installed software goes unused, representing direct financial waste (Nexthink, 2024). This lack of coordination leads to weak system integration and heterogeneous user interfaces. From a systems engineering perspective, this fragmentation without strategic planning negatively affects digital work quality, resulting in increased coordination effort and manual data transfers. Employees frequently face ~12 hours of wasted time per week searching for information across disconnected systems, while fragmented data increases resolution times by 43% (Salesforce, 2024).
Furthermore, fragmented tool environments hinder data centralization and limit the effective adoption of artificial intelligence (AI) (Kintone, 2024). This is exacerbated by the fact that 70% of AI investments fail to produce business impact due to tool misalignment and low digital adoption (McKinsey, 2024). These inefficiencies are costly, as large enterprises report losses of $104 million annually due to digital friction, and productivity losses in complex IT landscapes can equal 7% of annual revenue (Freshworks, 2024). These limitations constrain digital collaboration, reduce the ability of SMEs to adapt to highly competitive markets, and represent a growing obstacle to sustainable digitalization, particularly in resource-constrained SME environments (Mittal et al., 2018).
Research Gap and Contribution Existing approaches to IT tool selection and digitalization support address parts of the challenges faced by SMEs yet remain limited in scope. Many methods tend to treat tools in isolation and do not consider the IT environment as an integrated system, evaluating software independently rather than considering interactions and dependencies at the IT tool stack level (Jadhav, 2009).
Digital maturity and readiness models provide high-level assessments of organizational digitalization status but offer little guidance on concrete systematic reconfiguration of existing IT tool environments (Jeanneret, 2024). While such models are valuable for strategic orientation, they do not support activity-level analysis or decision-making for heterogeneous and evolving IT tool landscapes.
Multi-criteria decision-making (MCDM) techniques have been applied to IT tool selection to structure complex decision problems. However, most existing applications rely on static, single-stage evaluations of predefined alternatives (Madanchian, 2025). These approaches struggle to address the combinatorial nature of IT tool stack configuration and rarely incorporate user-perceived quality of digital activities as an explicit decision criterion.
From a quality science perspective, this reveals a methodological gap. While digital work quality can be conceptualized using established principles such as cause-and-effect analysis, critical-to-quality characteristics, drivers, and perceived performance, these concepts are seldom integrated into decision-support methods for IT tool ecosystem design. In addition, existing approaches provide limited support for explicitly defining SME-specific digitalization quality goals or tailoring evaluations to concrete use cases based on the collective experience of multiple users.
This paper addresses this gap by introducing a situation-aware framework that enables SMEs to define digitalization quality goals at the level of digital activities and evaluate heterogeneous IT tool environments in a use-case- specific manner. By leveraging quality science concepts, standardized activity representations, aggregated multi-user ratings, and multi-stage decision logic, the framework supports proactive, context-sensitive reconfiguration of IT tool stacks to optimize digital work quality in collaborative engineering environments.
IT Tool Knowledge Database for Situation-Aware IT Stack Analysis A structured IT tool database was developed and aligned with the Collaborative Engineering Activity Model (CEAM) introduced in our previous work (Balder, 2025). The database provides the foundational data structure for the Situation-Aware Digitalization Framework and enables the METIS application to analyze and reconfigure SME IT tool environments in a situation-aware manner. In doing so, it considers digital work capabilities, digitalization goals, and organization-specific priorities, and provides the empirical basis for applying and validating the proposed framework.
Figure 1: (a) ECBB model, IT tool stack decomposition. (b) Structure of the METIS IT tool database, where tool- and usage-related information is systematically stored and the CEAM method is applied to standardize and compare tool functionalities.
For each IT tool, the database stores structured tool-related and usage- related information, including functional capabilities expressed through CEAM-derived Engineering Collaboration Building Blocks (ECBBs) (Çağıncan, 2025), as well as user-based tool ratings, see Figure 1. These ratings are collected from large-scale online software evaluation surveys, where tools are assessed across multiple evaluation criteria. In addition, the digitalization level of each tool is captured in terms of the model and stored in the database.
Situation-Aware Digitalization Framework Situation-Aware Digitalization Framework was developed that supports systematic, proactive, human centered, data-driven, and goal-oriented
METIS: A Quality-Oriented Multi-Stage Decision Framework for IT Tool-Stack Optimization 131 digital transformation. The framework enables SMEs to assess, evaluate, and reconfigure their IT tool environments holistically, considering not only technical capabilities but also human-perceived quality, usage experience, and organizational priorities. Rather than optimizing individual tools in isolation, the framework explicitly treats the IT tool landscape as an integrated digital work environment.
The framework builds on the concept of ECBB to operationalize complex digital work systems. IT tools are decomposed into standardized digital work activities (ECBBs). The ECBBs are evaluated individually by SME users. User evaluations are complemented by structured prioritization techniques, including cause–effect Ishikawa analysis and numerical weighting, to identify ECBBs with the highest need for improvement.
Figure 2: The figure illustrates situation-aware digital transformation cycle that structures digital work into standardized ECBB functions, integrates quality analysis and digitalization requirements assessment, and enables continuous IT tool stack optimization.
The framework also supports the definition of digitalization objectives for individual ECBBs. Organizational priorities and digitalization goals are incorporated as weighting factors, ensuring that recommendations align with the specific strategic context of each SME. A multi-stage IT tool stack recommendation mechanism is then applied to generate alternative IT tool configurations, emphasizing tool combinations that have demonstrated high user acceptance and perceived effectiveness in comparable situations. The METIS application operationalizes this process and enables its automated execution. The framework is structured into three consecutive steps: Digital Perception, Digital Comprehension, and Digital Projection.
1. Digital Perception: Digital System Decomposition
The first step of the Situation-Aware Digitalization Framework is Digital Perception, which aims to establish a transparent and comparable
representation of the existing digital system. This step focuses on identifying what digital work is currently performed across the IT environment.
ECBBs describe standardized digital work activities that occur in collaborative product development and allow heterogeneous tools to be described in a uniform environment. ECBBs are organized under four overarching domains- product design, data and knowledge management, project management, and communication-providing a structured yet flexible representation of an IT Tool. IT tools are described as fluid, compositions of multiple ECBBs and may therefore span several domains simultaneously. This decomposition reflects the complexity of modern digital work environments and enables heterogeneous IT tool landscapes to be expressed in a comparable manner.
2. Digital Comprehension: Human-Centered Evaluation of ECBBs
Following Digital Perception, the second step of the Situation-Aware Digitalization Framework is Digital Comprehension, which aims to understand how well the identified digital works are currently supported and where improvement is most critical. While Digital Perception makes digital work explicit through ECBB decomposition, Digital Comprehension focuses on evaluating these digital work activities from a human-centered and organizational perspective.
In this step, SMEs evaluate the identified ECBBs directly, independent of specific IT tools. Users who actively perform the digital work rate individual ECBBs under multiple criteria, capturing human-perceived quality and practical experience. The applied criteria reflect aspects such as frequency of use, time efficiency, output quality, ease of use, integration with other tools, reliability, and user satisfaction. This evaluation captures how digital work is experienced in practice rather than how tools are specified technically.
The resulting ECBB-level evaluations are subsequently analyzed using a structured cause–effect approach based on an Ishikawa diagram. This analysis supports SMEs in identifying underlying pain points and systemic deficiencies within their digital work environment. This approach highlights problematic digital work activities and their root causes, enabling a functional rather than tool-driven prioritization.
Based on this analysis, SMEs prioritize ECBBs according to their need for change (not, can, should, must change), operationalized as numerical values (0–3). This prioritization expresses the urgency and relevance of improving individual digital work activities. In addition to prioritizing the need for change, SMEs define target digitalization levels for each prioritized ECBB. These targets specify the desired degree of digital support, including data centralization, system integration, and AI-enabled automation. This ensures that the digitalization level of supported activities is aligned with SME’s strategic objectives. In parallel, SMEs numerically rate organizational criteria such as integration capability, usability, licensing models, methodological support, and cost. These priorities, identified as core factors in prior work, serve as boundary conditions for feasible transformation (Balder, 2025). The same criteria are represented in the IT tool database through aggregated online
METIS: A Quality-Oriented Multi-Stage Decision Framework for IT Tool-Stack Optimization 133 user ratings. By weighing these criteria according to SME-defined priorities, personalized evaluation scores are derived from crowdsourced rating data.
Together, the prioritized ECBB change requirements, personalized user- weighted tool ratings, and ECBB-specific digitalization targets form a consolidated requirement profile. This profile constitutes the structured input for the subsequent Digital Projection step.
3. Digital Projection: Multi-Stage IT Tool Stack Recommendation
The third step is Digital Projection, which focuses on deriving future-oriented IT tool stack configurations based on the consolidated requirements identified in the preceding steps. While previous steps establish an understanding of existing digital work and its prioritized improvement needs, Digital Projection addresses how the IT tool environment should be systematically reconfigured from a quality science perspective to meet these requirements.
This step is based on a holistic decision mechanism that combines MCDM principles with iterative recommendation logic. MCDM is applied to structure and analyze pairwise comparisons of IT tools with respect to the specific use case, while iterative models determine the sequence in which these comparisons are conducted to identify suitable combinations of IT tools for the SME context.
MCDM Scoring Model In the proposed model, the overall suitability of an IT tool for a given use case is determined by its functional capabilities, user-perceived quality, and alignment with the SME’s digitalization objectives. These aspects are aggregated into an integrated score, where higher values indicate a better fit for the specific application context:
Equation reproduced from the original PDF.
Capability (C) represents how well an IT tool supports the digital activities required by the SME. The functionality of each tool within the IT tool stack is decomposed into ECBBs, where each ECBB is conditionally evaluated as either supported (1) or not supported (0) by the tool. The ECBBs are weighted according to their relevance for the SME, resulting in a normalized measure of functional coverage:
x x i
Equation reproduced from the original PDF.
where wx denotes the SME-defined priority of ECBB x, and ex∈{0,1} indicates whether the tool supports this ECBB.
Preference (P) represents the user-perceived quality of a tool. It is calculated by combining crowdsourced ratings from large online platforms with SME- specific weighting of key evaluation criteria, including usability (b), support
quality (s), integration capability (in), cost (k), and functionality (f). These criteria capture both the practical acceptance of the tool and its alignment with the strategic priorities of the SME. The preference score for tool is computed as: b b s s i in k k f i g p g p g p g p g P p p p p ⋅ + ⋅ + ⋅ + ⋅ + ⋅ = + + + + ⋅
5 ( 5) 5
b s in k
where gx denotes the crowdsourced rating of criterion x (scale 0-5), px represents the SME-defined priority weight assigned to criterion x (scale 0-5, except px = 5), and x∈{b, s, i, k, f}. The resulting preference score is normalized between 0 and 1 and reflects both general market perception and the SME-specific importance assigned to the evaluated criteria.
Digitalization (D) represents the alignment between a tool and the SME’s targeted digitalization level. It is calculated by comparing the SME’s target levels for the ECBBs in terms of automation (a), AI maturity (AI), and system synchronization (synch), expressed on ordinal scales. The digitalization score for tool i is computed as:
1 1 2 2 3 3
a AI synch i
w e d w e d w e d D w e w e w e ⋅ ⋅ + ⋅ ⋅ + ⋅ ⋅ = ⋅ + ⋅ + ⋅
( ) ( ) ( ) ( ) ( ) ( )
1 1 2 2 3 3
where da, dAI, and dsynch denote the deviation between the SME’s target level and the tool’s capability for automation, AI maturity, and synchronization, respectively; wx represents the importance weight assigned to each dimension; and ex∈{0, 1} indicates whether the respective capability is supported by the tool. The resulting score reflects how strongly the tool contributes to the SME’s intended digital transformation rather than merely supporting current operations.
Iterative IT Tool Stack Construction Models Due to the combinatorial complexity of IT tool stack configuration, a single- step MCDM evaluation is insufficient to derive suitable recommendations. Instead, the framework applies iterative decision models that construct IT tool stacks stepwise by repeatedly applying the integrated MCDM score. This approach enables systematic exploration of alternative configurations while accounting for interdependencies between tools and ECBBs. Two complementary iterative models are implemented to address different transformation strategies and organizational constraints:
The Stack-Exchange Approach considers all prioritized ECBBs as a single combined requirement. The process starts with an empty tool stack. In each iteration, all candidate tools in the database are compared using the MCDM score. The tool with the highest score is selected and added to the stack. After selection, the ECBBs covered by this tool are removed from the list of remaining requirements. This iterative procedure continues until all prioritized ECBBs are adequately covered.
METIS: A Quality-Oriented Multi-Stage Decision Framework for IT Tool-Stack Optimization 135 The One-by-One Exchange Approach evaluates the existing IT tools of an SME sequentially in a prioritized order. Tools are first ordered based on the relevance of the prioritized ECBBs they support. The order of priority for tool is calculated as:
x i
Equation reproduced from the original PDF.
where wx represents the SME-defined priority weight of ECBB x, and l is the number of all ECBBs that tool has. The summation includes all ECBBs covered by tool. This ensures that tools supporting highly prioritized digital activities are evaluated first. Tools from the stack are evaluated in order of prioritization and compared with all alternatives in the database, the highest- scoring option is selected. Two variants are implemented:
1-by-1 Exchange: A tool is replaced once a better-rated alternative is identified. If the replacement does not cover all ECBBs of the original tool, the uncovered ECBBs are carried forward to subsequent iterations.
1-by-1 Forced Exchange: A replacement is only accepted if the alternative fully covers all ECBBs of the original tool. This results in a more conservative and structurally stable reconfiguration.
The iterative comparison continues until all SME-required ECBBs are addressed.
RESULT
The METIS demonstrator that demonstrates the Situation-Aware Digitalization Framework was validated in practical workshops with industrial partners and external SMEs. The validation focused on demonstrating feasibility, usability, and perceived applicability of the Situation-Aware Digitalization Framework in real use-case settings.
A total of 36 participants from heterogeneous organizational backgrounds (including management, IT/consulting, and engineering roles, and companies of varying sizes) took part in the evaluation. Usability was assessed using the standardized System Usability Scale (SUS), a validated 10-item questionnaire widely used to measure perceived usability and learnability (Brooke, 1996). SUS responses show low perceived complexity and good learnability, with approximately 80% of participants indicating that they would recommend the system. User experience was further evaluated using AttrakDiff, a validated instrument for measuring pragmatic and hedonic quality dimensions of interactive systems (Hassenzahl et al., 2003). The results confirm positive perceptions regarding clarity, simplicity, professionalism, and innovativeness.
Participants further reported enhanced transparency in IT tool evaluation and more structured decision-making. More than two-thirds expect measurable efficiency improvement through the systematic configuration of IT tool stacks. Overall, the validation demonstrates the feasibility, usability,
and perceived decision-support value of the framework for situation- aware IT tool stack reconfiguration in SME collaborative engineering environments.
Figure 3: User interface of the METIS application displaying the Digital Projection step and generated IT tool stack recommendation results.
CONCLUSION
This paper presented a Situation-Aware Digitalization Framework for systematic IT tool stack reconfiguration in SME collaborative product development. By combining CEAM-derived ECBB decomposition, quality science–based prioritization, and multi-stage MCDM recommendation logic, the framework enables proactive, structured, and context-sensitive digital transformation beyond isolated tool selection.
The two recommendation models provide complementary transformation strategies. The Stack-Exchange approach supports compact and functionally comprehensive tool stacks but may introduce disruptive changes. In contrast, the One-by-One variants preserve structural stability and enable incremental transformation, though potentially retaining legacy constraints. This allows SMEs to balance efficiency gains with organizational readiness.
Building on the validated feasibility and usability of the METIS application, future work will integrate advanced AI-based recommendation methods into the framework. AI can enable more flexible ECBB structures, extend CEAM to additional industrial processes, and uncover complex correlations between digital work activities, use cases, and organizational characteristics to generate highly SME- and use-case-specific configurations. A key challenge remains the availability of structured, fine-grained IT tool usage data. Ongoing research therefore focuses on computationally extending the quality science methodology and strengthening AI support, particularly in the perception and projection stages.
ACKNOWLEDGMENT
The topic of this paper stems from the “Metis” project. The authors would like to sincerely thank the IGF (Grant No. 22534N) for their financial support of the METIS project, which enabled this research. We also wish to express our appreciation to all those who contributed to the success of this project, including our collaborators and participants. Their invaluable insights and feedback have played a crucial role in shaping the outcomes of this research.
REFERENCES
Balder, J., Cagincan, C. (2025). Analyzing the deployment of IT tool stacks in SME.
Int. J. Eng. Bus. Manag., 17. Balder, J., Cagincan, C. (2025). Systematic classification of collaborative engineering activities. Procedia CIRP, 136, 740–745. Brink, H., Packmohr, S. (2023). Barriers to Digital Transformation in SMEs. ICDEc
2023, LNBIP 483, Springer. Brooke, J. (1996). SUS: A quick and dirty usability scale. In: Usability Evaluation in Industry, 189–194. Cagincan, C., et al. (2025). IT tool stack optimization in collaborative projects.
AHFE Open Access, 181. Freshworks. (2025). The Cost of Complexity: Global Report on Software Waste and Revenue Drain. Freshworks Inc. / EY Research. Hassenzahl, M., et al. (2003). AttrakDiff: Measuring perceived hedonic and
pragmatic quality. Human-Computer Interaction, 187–196. Jadhav, A.S., Sonar, R.M. (2009). Evaluating and selecting software packages: A review. Inf. Softw. Technol., 51(3), 555-563. Jeanneret Medina, M., et al. (2024). Digital Business Transformation for SMEs.
WorldCIST 2023, LNNS 801, Springer. Kintone. (2024). SME Digital Productivity Survey 2024. Kintone Southeast Asia. Madanchian, M., Taherdoost, H. (2025). MCDM in IS for Strategic and Operational Decisions. Computers, 14(6), 208. McKinsey & Company. (2021). Why Digital Transformations Fail: 70% of Large-
Scale Programs Don’t Reach Their Stated Goals. McKinsey Digital. Mittal, S., et al. (2018). Critical review of Industry 4.0 maturity models. IFAC- PapersOnLine, 51(11), 1010–1014. Nexthink. (2024). Soft-WASTE: How Much Does IT Waste on Unused Software
Licenses? Research based on 6 million devices. Nexthink Insights. Salesforce. (2024). Connectivity Benchmark Report 2024: 85% of IT Leaders See AI Boosting Productivity, but Data Integration and Overwhelmed Teams Hinder Success.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Industrial and Systems Engineering
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-95-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional