Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The Human Factor in Systems Engineering: Engineering Systems Thinking

Faculty of Technology Management, HIT-Holon Institute of Technology, Holon 58102, Israel ↗
Publication year
Digital Object Identifier10.54941/ahfe100137

Abstract

To successfully perform systems engineering tasks, systems engineers need a systems view or, in other words, a high capacity for engineering systems thinking (CEST). A tool for assessing systems thinking of engineers, once validated, may be used for systems engineering workforce selection and development, developing systems engineering curriculum, and education and training programs and a standard tool for assessing systems engineers' competencies. Since there is no known way for directly 'measuring' systems thinking in general and CEST in particular, an indirect way is needed. This paper proposes an idea for developing an indirect means, i.e. a questionnaire for assessing the CEST of systems engineers. The idea is composed of four logic layers.

INTRODUCTION

Systems engineering is an interdisciplinary field of engineering that focuses on how to design and manage complex engineering projects over their life cycles. Systems thinking is what makes systems engineering different from other kinds of engineering and is the underpinning skill required to do systems engineering (Beasley & Partridge, 2011). Engineering Systems Thinking is a major high-order thinking skill that enables individuals to successfully perform systems engineering tasks (Frank, 2000; 2002).

To successfully perform systems engineering tasks, systems engineers need a systems view or, in other words, a high capacity for engineering systems thinking (CEST). The main characteristic of systems engineers possessing high CEST is the ability to see the whole picture and identify the system emergent properties, capabilities, behaviors, and functions without looking inside the details of the system. It is assumed that CEST is a measurable and consistent quality of personality, and it can be used to distinguish among individual engineers.

Systems thinking seeks to address and solve complex problems by understanding the system parts and their interactions within the context of the whole system, rather than in isolation (Hitchins, 2003). Systems thinking is a discipline for seeing the whole. It is a framework for seeing interrelationships and repeated events rather than things, for seeing patterns of change rather that static ‘snapshots’. Systems thinking is the perception of the “constructs” underlying complex problems (Senge, 1994).

Systems thinking is considered as high order thinking skill. The simplest thinking skills are learning facts and recall. Higher order skills include critical thinking, creative thinking, analysis, problem solving and systems thinking.

The battery for assessing CEST of individuals will comprise a set of paper-and-pencil, field and lab tests. This paper discusses one of the paper-and-pencil tests – an interest inventory. The paper discusses rationale, principles, uses, stages of development, and presents some examples and results of studies aimed at checking its reliability and validity.

HOW TO ASSESS CAPACITY FOR ENGINEERING SYSTEMS THINKING (CEST) BY A PAPER-AND-PENCIL QUESTIONNAIRE: THE IDEA Why to Assess?

A tool for assessing systems thinking of engineers, once validated, may be used for systems engineering workforce selection and development, developing systems engineering curriculum, and education and training programs and a standard tool for assessing systems engineers' competencies. Such a tool may be used for selection, filtering, and screening of candidates for systems engineering job positions, and for placing the 'right person to the right job'. The tool may be used also as an instrument for evaluating the effectiveness of systems engineering curriculum and training programs.

Since there is no known way for directly 'measuring' systems thinking in general and CEST in particular, an indirect way is needed. For example, IQ tests are paper-and-pencil indirect tests for 'measuring' the intelligence of individuals. This paper proposes an idea for developing an indirect means, i.e. a questionnaire for assessing the CEST of systems engineers. The idea is composed of four logic layers.

The First Layer As stated above, engineering systems thinking is a major high-order thinking skill that enables individuals to successfully perform systems engineering tasks (Frank, 2000; 2002). Actually we deal here with a thinking skill. So, let's learn what are the common methods of "measuring" other thinking skills such as intelligence and creativity.

An intelligence quotient, or IQ, is a score derived from one of several standardized tests designed to assess intelligence. One of the common ways for assessing intelligence is by separately assessing abilities such as arithmetic, spatial imagery, reading, vocabulary, memory and general knowledge and then weighting the individual scores to one general index (the IQ). For example, The WAIS-III version of the Wechsler Adult Intelligence Scale was released in 1997. It provided scores for Verbal IQ, Performance IQ, and Full Scale IQ, along with four secondary indices - Verbal Comprehension, Working Memory, Perceptual Organization, and Processing Speed. The Verbal Comprehension Index (VCI) included the following tests: Information, Similarities and Vocabulary. The Working Memory Index (WMI) included: Arithmetic, Digit Span and Letter-Number Sequencing. The Performance IQ (PIQ) included six tests and it also provided two sub-indexes: perceptual organization and processing speed. The Perceptual Organization Index (POI) included: Block Design, Matrix Reasoning and Picture Completion. The Processing Speed Index (PSI) included: Digit Symbol-Coding and Symbol Search (Kaufman and Lichtenberger, 2006).

The current version of the test, the WAIS-IV, which was released in 2008, is composed of 10 core subtests and five supplemental subtests, with the 10 core subtests comprising the Full Scale IQ. With the new WAIS-IV, the verbal/performance subscales from previous versions were removed and replaced by the index scores. The General Ability Index (GAI) was included, which consists of the Similarities, Vocabulary and Information subtests from the Verbal Comprehension Index and the Block Design, Matrix Reasoning and Visual Puzzles subtests from the Perceptual Reasoning Index. The GAI is clinically useful because it can be used as a measure of cognitive abilities that are less vulnerable to impairments of processing and working memory (Pearson, 2008).

There are four index scores representing major components of intelligence: Verbal Comprehension Index (VCI), Perceptual Reasoning Index (PRI), Working Memory Index (WMI), Processing Speed Index (PSI). Two broad scores are also generated, which can be used to summarize general intellectual abilities: Full Scale IQ (FSIQ), based on the total combined performance of the VCI, PRI, WMI, and PSI General Ability Index (GAI), based only on the six subtests that the VCI and PRI comprise.

A similar approach can be found in creativity test. For example, Torrance (1984), Building on Guilford's work and created by Torrance, the Torrance Tests of Creative Thinking (TTCT), is based on four scales: Fluency: The total number of interpretable, meaningful, and relevant ideas generated in response to the stimulus. Flexibility: The number of different categories of relevant responses. Originality: The statistical rarity of the responses. Elaboration: The amount of detail in the responses. The third edition of the TTCT in 1984 eliminated the Flexibility scale from the figural test, but added Resistance to Premature Closure and Abstractness of Titles as two new criterion- referenced scores on the figural. With the five norm-referenced measures that he now had (fluency, originality, abstractness of titles, elaboration and resistance to premature closure), he added 13 criterion-referenced measures which include: emotional expressiveness, story-telling articulateness, movement or actions, expressiveness of titles, syntheses of incomplete figures, synthesis of lines and circles, unusual visualization, extending or breaking boundaries, humor, richness of imagery, colorfulness of imagery, and fantasy (Cramond et al., 2005).

The Second Layer From IQ and creativity assessing approaches we have then learned that the first stage of developing a tool for assessing systems thinking in engineers is to decompose the 'engineering systems thinking' to factors that can be separately assessed. But how to do so? In order to answer this question, let's learn from one of the approaches taken in Leadership research.

There is vast amount of books, articles, and studies available which attempt to identify the competencies and qualities of effective leaders. For example, Morrison (2000) presents a global leadership model. Jokinen (2005), reviews and discusses the main findings of previous leadership studies aimed at identifying characteristics and competencies of effective and successful leaders.

Thus we can see that a basic approach for understanding the 'leadership phenomena' is to identify the competencies of successful leaders. It is suggested here that, in the same manner, in order to understand the 'engineering systems thinking phenomena' let's try to identify the competencies and characteristics of engineers who are systems thinkers. But how can we know who are those engineers that can be considered as systems thinkers as the tool for assessing CEST has not been developed yet?

The Third Layer As mentioned above, to successfully perform systems engineering tasks, systems engineers need a systems view or, in other words, a high capacity for engineering systems thinking (CEST). Thus, for the sake of this study we may hypothesize that successful systems engineers are characterized by high Capacity for Engineering Systems Thinking (CEST). In other words, we may hypothesize that successful systems engineers are systems thinkers. This hypothesis should be checked later when the development of the systems thinking assessment tool will be completed. Thus in order to understand the 'engineering systems thinking phenomena' let's try to identify the competencies and characteristics of successful systems engineers.

Actually, this has already been done. Frank (2006), aimed at identifying the characteristics of successful systems engineers. The study included observations of systems engineers who were evaluated—by at least three peers and two supervisors—as being “successful.” Eighty-three which later aggregated to thirty-four competencies of successful systems engineers were identified in this study. These 34 competencies were classified into:

  • Ten cognitive characteristics – understanding the big pictures, interconnections, systems synergy, multiple perspectives, systems without getting stuck on details, implications of proposed change, a new system or concept immediately upon presentation, analogies and parallelism between systems, limits to growth and thinking creatively. All these ten characteristics are related to systems thinking.
  • Eleven abilities – analyzing the need, analyzing/developing the concept of operations, requirements analysis, conceptualizing the solution, generating the logical solution (functional analysis), generating the physical solution (architecture synthesis), “seeing” the future, using simulations and systems engineering tools, optimizing, using systems design considerations, leading trade studies
  • Ten individual traits – management skills, team leader, building and controlling the work plan, defining boundaries, taking into consideration non-engineering factors, good human relations, team player, good communication skills, good interpersonal skills, autonomous and independent learner, strong learning skills, willing to deal with systems, curious, innovator, initiator, promoter, originator, ask good questions. Computing, Software, and Systems Engineering (2018)
  • Three dealing with multidisciplinary knowledge and experience.

Thus, the content validity of the proposed tool can be achieved by basing its items on the finding found in the latter study (Frank, 2006). Each competency of successful systems engineers, found in that study, may be separately assessed by a single item or several items of the proposed tool and then the individual scores will be weighted to one general index.

The Fourth Layer One of the findings found by Frank (2006) is that in order to be a successful systems engineer, one must have both a will and an interest to be a systems engineer. In addition, as mentioned, successful systems engineers possess a high capacity for engineering systems thinking (CEST). Thus, the three components discussed here - success in a systems engineering position, an interest in systems engineering positions and CEST - are all interconnected and interrelated. The will and interest to be a systems engineer basically means the desire and interest to be involved with job positions that require CEST. In other words, we may hypothesize that there is a high positive correlation between the engineering systems thinking extent (CEST) of an individual and his/her interest in what is required from successful systems engineers. Figure 1 is a simple concept map that depicts the relationships between these three components:

characterized possesses Will/Desire

High level of

Successful

CEST

systems engineer

to be a systems engineer

Figure 1. The relationships between the desire, successful SE and CEST

Figure available in the original PDF paper.

If this hypothesis is supported, then it enables developing a method for assessing the extent of CEST of individuals. This is because interests may be assessed by an interest inventory which is a very common and frequently used to help people choose a profession, and as a selection tool (to determine whether a certain individual is suitable for a certain role) in the recruiting process (Anastazi, 1988).

ASSESSING THE INTERST FOR ENGINEERING POSITIONS' REQUIRED CAPACITY FOR ENGINEERING SYSTEMS THINKING (CEST) Frank (2010) introduces a tool for assessing interest for systems engineering positions and other engineering positions' required capacity for engineering systems thinking (CEST). Usually, the items in interest inventories deal with preferences, specifically likes and dislikes regarding a diverse group of activities, jobs, professions or personality types. Likewise, the items included in the tool discussed in this chapter refer to ranges of likes and dislikes regarding systems engineering activities, various disciplines and knowledge required from systems engineers, systems engineering activities and types of people involved in projects.

In its present version the tool consists of 40 pairs of statements. For each pair, the examinee has to choose between the two statements according to his/her preference. The examinee checks answer “A” if he/she prefers the first statement or answer “B” if he/she prefers the second statement. In order to improve the questionnaire’s reliability, questionnaire items were reorganized, so in some cases “A” represented the systems thinking answer and in other cases “B” represented the systems thinking answer. Each "A" answer receives 2.5 points, while each "B" answer receives no point. Thus, the range of the scores is 0-100.

As of the tool validity – four types of validity have been checked in a series of studies - content validity, contrasted groups validity, concurrent validity and construct validity (Frank, 2010).

CONCLUSIONS

Every enterprise strives to fill positions in the organization with employees who have the best chance to succeed. Employees are also interested in entering positions that fulfil their aspirations. Selection and screening processes can help match the interests of both parties, thus contributing both to the organization and the individual. The selection process for systems engineering positions should reliably predict those employees who can succeed and reject those who are likely to fail. Out of the employees who can succeed as systems engineers, it is necessary to choose those who have the highest chance of succeeding.

From the organization’s point of view, rejection of candidates who might have succeeded in systems engineering positions can be critical, especially under conditions of an ever-increasing shortage of systems engineers. Likewise, placing engineers who later fail in systems engineering positions is also an expensive error, taking into consideration the necessary training which will be invested and the subsequent damage which might be caused to the projects in which they are involved. The tool presented in this paper may be used for selection, filtering, screening of candidates for systems engineering job positions and for placing the 'right person to the right job'.

REFERENCES

Anastasi, A. (1998), "Psychological testing", 6th edition, New York, NY: 1988. Beasley, R., Partridge, R. (2011), "The three T's of systems engineering - trading, tailoring and thinking", Paper presented at the 21st Annual Symposium of the International Council on Systems Engineering (INCOSE). Denver, CO, USA. June 20-23, 2011. Cramond, B., Matthews, M.J., Bandalos, D., & Zuo, L. (2005). "A report on the 40 year follow-up of the Torrance

Tests of Creative Thinking: Alive and Well in the New Millennium". Gifted Child Quarterly, 49, 283-291. Frank, M. (2000). 'Engineering systems thinking and systems thinking". INCOSE Journal of Systems Engineering, vol. 3, no. 3, pp. 163-168. Frank, M. (2002). "Characteristics of engineering systems thinking - A 3-D approach for curriculum content". IEEE

Transaction on System, Man, and Cybernetics, vol. 32, no. 3, Part C, pp. 203-214. Frank, M. (2006). "Knowledge, abilities, cognitive characteristics and behavioral competences of engineers with high Capacity for Engineering Systems Thinking (CEST)". INCOSE Journal of Systems Engineering, vol. 9, no. 2, pp. 91-103. Frank, M. (2010). "Assessing the interest for systems engineering positions and other engineering positions’

required capacity for engineering systems thinking (CEST)". INCOSE Journal of Systems Engineering, vol. 13, no. 2, pp. 161-174. Hitchins. D.K. (2003), "Advanced systems thinking, engineering and management". Norwood, MA: Artech House, INC Jokinen. T. (2005), "Global leadership competencies: a review and discussion", Journal of European Industrial Training. Vol. 29 No. 3, pp. 199-216. Kaufman, A.S, Lichtenberger, E. (2006), "Assessing Adolescent and Adult Intelligence" Hoboken, NJ: Wiley. Morrison, A.J. (2000). "Developing a global leadership model". Human Resource Management, Vol. 39, Nos. 2 &

3, Pp. 117–131. PEARSON (2008). Wechsler Adult Intelligence Scale, Fourth Edition Now Available from Pearson. http://www.pearsonclinical.com/psychology/news/2008/wechsler-adult-intelligence-scale-fourth-edition-nowavailable-from-pearson.html Senge, P.M. (1994), "The fifth discipline: The art and practice of the learning organization", New York, NY:

Doubleday.

Cover of the proceedings volume Advances in Human Factors, Software, and Systems Engineering

Explore the complete digital volume

Advances in Human Factors, Software, and Systems Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2096-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →