Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Multiple-view Visualization Awareness Model and Measurement Method Construction: Based on Quantitative Analysis of Situation Awareness

School of Design, Hunan University, Changsha, Hunan, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1003441

Abstract

Multiple-view Visualization (MV) is the current trend in information visualization, presenting massive and complex data from multiple perspectives and through different aggregations. MV designs are often used in industrial inspection scenarios, and their interaction design affects the operator's Situational Awareness (SA) and thus the operator's decision-making. However, SA assessments of MV exist mostly after the design is completed and lack effective measurement methods. In this paper, we present an in-depth analysis of existing SA models, combine the design process and elements of MV, summarize the MV perception model based on the Quantitative Analysis of Situational Awareness (QASA), and propose a method to measure the SA of MV, so as to help designers take the SA into consideration before completing the design. A formal user study was conducted to compare the SA of different layouts, starting with the layout as a design element. Quantitative results showed significant differences in participants' SA when using different layouts. Participants’ feedback confirmed that this measurement method can help designers to conduct user studies on different MV elements to improve operators' SA when using the system.

INTRODUCTION

With the process of Industry 4.0, the digitization of the entire product life- cycle and the entire manufacturing process has deepened (Damayanti, 2013). Multiple-view visualization is widely used in the manufacturing industry for converting massive amounts of information into knowledge and reasoning and presenting it to operators. The processing and programming of large amounts of information can help operators make timely decisions at diffe- rent stages throughout the manufacturing industry (Marconi et al., 2021; Yao et al., 2019). For the operators of the system, Multiple-view visualiza- tion must be able to help them correctly perceive useful information while ignoring or neglecting other stimuli (Oury et al., 2021). And situational awa- reness theory fulfills this need by providing a framework for understanding performance and decision making (Endsley, 2017). Situational awareness is an evaluation metric that can improve operator operational efficiency and

© 2023. Published by AHFE Open Access. All rights reserved. 807

808 He and Ji

reduce human-system friction. However, current research on Multiple-view visualization has focused on areas such as intelligent design of Multiple-view visualization systems (Al-Maneea & Roberts, 2020), aesthetic studies, and the need for applications. In the current research, there is a lack of consi- deration of the metric of situational awareness in the design process. The inclusion of situational awareness requires the construction of a Multiple- view visualization model of situational awareness and its evaluation based on this model.

Therefore, this paper extensively studies the situational awareness model, analyzes the correlation between the elements of Multiple-view visualiza- tion and the situational awareness model, extracts the situational awareness model that best fits the scenario of Multiple-view visualization, and proposes the situational awareness model and evaluation method for Multiple-view visualization by combining the study of Multiple-view visualization design process and design elements. Then, through user research, we propose an evaluation method based on the model for layout, one of the elements of Multiple-view visualization, and validate the model and evaluation method through experiments.

Related Work

Endsley’s three-stage model of situational awareness is the classical model. the three stages of SA are commonly referred to as perception, comprehen- sion, and projection, i.e., the operator needs to perceive valid information from the task environment, subsequently comprehend the information, form a self-perception, and subsequently make decisions based on the perception (Endsley, 2017). American cognitive psychologists Joe Gregory Bedny and David Meister (Bedny & Meister, 1999) described situational awareness through the activity approach theory, and Smith and Hancoke proposed the perceptual cycle model, which views situational awareness as a process of knowledge-generating generation and situation-dependent action execution (Smith & Hancock, 1995).

It was found that Edgar et al. proposed a cyclic model of SA, as shown in Figure 1, as an excellent basis for situational awareness model building for Multiple-view visualization. This cyclic model of SA describes the genotype and environment acting on the participant’s phenotype to drive the parti- cipant to make different decisions and behaviours. At the same time, the genotypes of participants receive changes as a result of the influence of the phenotypes and decisions. And participants differ by a tendency to receive less or more information, known as bias.

Current approaches for assessing situational awareness include freeze online probe techniques (Salmon et al., 2009), real-time probe techniques (Cha & Yu, 2022), observer measurements (Chegini et al., 2019), subje- ctive measurements (Salmon et al., 2009), and physiological test measure- ments(Moore & Gugerty, 2010; Van De Merwe et al., 2012). Yet, these measurements are subjective to the individuality of subjects/observers and are difficult to be carried out. The research found that Edgar, Graham K et al. proposed QASA based on signal detection theory (Edgar et al., 2017), which

Multiple-View Visualization Awareness Model and Measurement Method Construction 809

Figure 1: A cyclical model of SA based on schema theory. Figure redrawn and modified by authors. Based on a figure from Edgar et al., 2018.
Original PDF Figure 1:

can be applied to small sample sizes while avoiding the influence of subje- ctive factors on the measurement results. So far, QASA has been applied and validated in search and rescue (Epling et al., 2018), fire control (Sallis et al., 2022), driving (Wijayanto, Marcillia, et al., 2021; Wijayanto, Ramadhan, et al., 2021), and other fields. Therefore, this model facilitates us to find the Multiple-view visualization design that brings the least bias to the partici- pants through user studies. Also, Edgar et al. proposed measures based on this model, which are Actual situation awareness (ASA), Bias and Perceived situation awareness (PSA). Compared with other measurements, this method provides a combination of subjective and objective indicators to help rese- archers discuss the assessment of situational awareness in a comprehensive manner, while the presence of bias can help designers to analyze the results of the study in order to optimize the design.

Multiple-view is a popular strategy in information visualization that ena- bles users to explore a dataset perspective from different angles or to explore different datasets from the same perspective. The design of Multiple-view visualizations mostly relies on the designer’s experience and preference. According to Shao et al. (Shao et al., 2021) the design process of Multiple- view visualization is summarized as generating various patterns of MVs based on a set of Multiple-view visualizations in terms of view types, number of views, and their interrelationships, based on the presented viewport. In this paper, statistics on the elements of Multiple-view visualization mentioned in different articles are summarized key elements that can be used to support situational awareness research, as shown in Table 1.

Method

Combining the common process of situational awareness evaluation, based on the study of Multiple-view visualization process by SHAO et al. this paper describes the traditional process of design and evaluation for Multiple-view visualization through Figure 2 (Shao et al., 2021). In the traditional process,

810 He and Ji

Table 1. MVs element statistics - based on literature study.

Table available in the original PDF paper.

Articles MV Elements visual encoding, layout, and interaction design Effects of View Layout on Situated Analytics for Multiple-View Representations in Immersive Visualization (Wen et al., 2022)

Modelling layout design for multiple-view visualization via Bayesian inference (Shao et al., 2021)

view coordination, view type, and viewport A tool to help lay out Multiple View Visualisations guided by view analysis (Al-Maneea & Roberts, 2020)

view spacing, background colour Exploring the Design Space of Composite Visualization (Javed & Elmqvist, 2012)

visual composition, visual structure, view Composition and Configuration Patterns in Multiple-View Visualizations (Chen et al., 2020)

Configuration, Composition, view type

Figure 2: Traditional design process and new design process of MVs.
Original PDF Figure 2:

situational awareness evaluation is often performed after the design is com- pleted or after an error has occurred in use. The designer would redesign the Multiple-view visualization based on the results of the situational awareness evaluation. This leads to an increase in the designer’s workload and can also lead to serious losses in real life during the formal use.

Therefore, in this paper, the design elements of Multiple-view visualiza- tion are incorporated into the cyclic model of situational awareness to form a Multiple-view visualization perception model (shown in Figure 3), and the QASA measurement method derived from the cyclic model is used to achieve the acquisition of situational awareness results. The Multiple-view visualized situational awareness model allows the new design approach to be established. That is, by prioritizing the operator (user) situational awareness in different situations through the measurement of different Multiple-view visualization design elements, the designer can prioritize the metric of situa- tional awareness and synthesize the measurement results of different parts to design a Multiple-view visualization interface that can improve the operator’s situational awareness.

The steps of the measurement method based on the Multiple-view visuali- zation situational awareness model and the QASA measurement method are as follows.

Multiple-View Visualization Awareness Model and Measurement Method Construction 811

Figure 3: Multiple-view visualization situational awareness model.
Original PDF Figure 3:

1. The experimenter generates probe statements in response to the experi- ment. The participant is informed of the probe statements, accepts the test, and then begins to perform the experiment formally. For example, in a test for a layout, the probe statement could be In this layout, there are two hexagons. Yes or No? 2. Participants observed N experimentally relevant stimuli (videos, pictures, etc.) and answered probing questions with subjective ratings (for stati- stical PSA) on their perception of posture. The researchers collected the participants’ responses and classified them according to signal detection theory. 3. The investigators used signal detection theory indicators to analyse responses to T/F statements, counting hit rate and miss rate, to give a measure of ASA versus Bias.

True/false responses were analysed using the QASA method, and A′ and B′′ were calculated using the formula described as followed: A′ = 0.5 + (sign(H −F) (H −F)2 + |H −F| 4 max (H, F) −4HF) (1)

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

H (1 −H) + F(1 −F)) (2)

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

Experiment and Discussion Layout is an important and fundamental aspect of Multiple-view visualiza- tion design, which serves to adjust the user’s attention flow (Lewis et al., 2006). The focus of layout design is to find feasible spatial configurations for a set of interrelated objects. Therefore, the researcher chose the element of layout as a practical validation of a situational awareness model and measure- ment method for Multiple-view visualization. Based on Chen et al.’s summary

812 He and Ji

of Multiple-view visualization documents (shown in Figure 4) (Chen et al., 2020), combined with the researcher’s case collection, the layout when the view is 7 is selected for the experiment in this paper, and the five selected layouts are shown in Figure 4 (Layout A-E). Thirty users were recruited for testing in this study, and the experimental scenario simulates the operator viewing the Multiple-view visualization interface, as shown in Figure 4.

Figure 5 shows the ASA, PSA and Bias in 5 different layouts. We can see that both ASA and PSA are optimal for Layout E layout. That is, the user’s situational awareness is at the optimal value under the layout of LAYOUT E. Combined with the metric Bias, we can understand that Layout E is the one that will bring less cognitive bias to the participants and easier to perceive the information. At the same time, we found that there is a gap between the ASA and PSA of the participants, which can bring different observation perspectives to the designers.

Figure 4: The layout of the summary (Chen et al., 2020), the layout chosen for the experiment and the experimental scenario are shown.
Original PDF Figure 4:
Figure 5: ASA, Bias and PSA.
Original PDF Figure 5:

Multiple-View Visualization Awareness Model and Measurement Method Construction 813

CONCLUSION

Introducing the evaluation of situational awareness in the design process of Multiple-view visualization can reduce the friction between the system and human and improve the sales. Therefore, this paper extensively studies the situational awareness model and proposes a situational awareness model and evaluation method for Multiple-view visualization. Then, we propose an evaluation method based on the model for layout, one of the elements of Multiple-view visualization, and validate the model and the evaluation method through experiments. The experimental results show that the evalu- ation method can help designers to analyse the results and lead to feasible design directions.

LIMITATION AND FUTURE WORK Although we only studied five view layouts, simple and practical layouts can cover a large number of arrangements in multi-literate visualizations (Roberts & Al-Maneea, 2018). Therefore, five of them were selected for this experi- ment as deemed appropriate by the researchers. In subsequent research, we will explore other elements of the multiitem visualization design to improve the overall situational awareness assessment system.

ACKNOWLEDGEMENT

The authors would like to acknowledge the financial support from the Natu- ral Science Foundation of Hunan Province (Grant number 2022JJ40064).

REFERENCES

Al-Maneea, H. M., & Roberts, J. C. (2020). A Tool to Help lay out Multiple View Visualisations Guided by View Analysis. EuroVis, Bedny, G., & Meister, D. (1999). Theory of activity and situation awareness.

International Journal of cognitive ergonomics, 3(1), 63–72. Cha, J. S., & Yu, D. (2022). Objective Measures of Surgeon Non-Technical Skills in Surgery: A Scoping Review. Human Factors, 64(1), 42–73. Chegini, M., Andrews, K., Schreck, T., & Sourin, A. (2019, Jan 06-09). Multiple

Linked-View Exploration on Large Displays Facilitated by a Secondary Handheld Device. Proceedings of SPIE [International workshop on advanced image tech- nology (iwait) 2019]. International Workshop on Advanced Image Technology (IWAIT), Singapore, SINGAPORE. Chen, X., Zeng, W., Lin, Y., Al-maneea, H., Roberts, J., & Chang, R. (2020).

Composition and Configuration Patterns in Multiple-View Visualizations. IEEE Transactions on Visualization and Computer Graphics, pp, 1–1. https://doi.org/ 10.1109/TVCG.2020.3030338 Damayanti, N. (2013). Internet of Things: a vision, architectural elements, and future directons. Internet of Things: a vision, architectural elements, and future directons. Edgar, G., Catherwood, D., Baker, S., Sallis, G., Bertels, M., Edgar, H., Nikolla, D.,

Buckle, S., Goodwin, C., & Whelan, A. (2017). Quantitative Analysis of Situation Awareness (QASA): modelling and measuring situation awareness using signal detection theory. Ergonomics, 61, 1–16. https://doi.org/10.1080/00140139.2017. 1420238 Endsley, M. R. (2017). Toward a theory of situation awareness in dynamic systems.

In Situational awareness (pp. 9–42). Routledge.

814 He and Ji

Epling, S. L., Blakely, M. J., Edgar, G. K., Russell, P. N., & Helton, W. S. (2018).

Memory impairment during a climbing traverse: implications for search and rescue climbing. Experimental Brain Research, 236(11), 3043–3052. Javed, W., & Elmqvist, N. (2012). Exploring the design space of composite visualization. Proceedings of the IEEE Pacific Symposium on Visualization, 1–8. https://doi.org/10.1109/PacificVis.2012.6183556 Lewis, D., Haroz, S., & Ma, K.-L. (2006). Layout of multiple views for volume

visualization: A user study. Advances in Visual Computing: Second International Symposium, ISVC 2006 Lake Tahoe, NV, USA, November 6-8, 2006. Proceedings, Part II 2. Marconi, M., Menghi, R., Papetti, A., Pietroni, G., & Germani, M. (2021). An interactive resource value mapping tool to support the reduction of inefficiencies in smart manufacturing processes. International Journal on Interactive Design and Manufacturing (IJIDeM), 15, 211–224. Moore, K., & Gugerty, L. (2010). Development of a novel measure of situation awa-

reness: The case for eye movement analysis. Proceedings of the human factors and ergonomics society annual meeting. Oury, J. D., Ritter, F. E., Oury, J. D., & Ritter, F. E. (2021). How user-centered design supports situation awareness for complex interfaces. Building Better Interfaces for Remote Autonomous Systems: An Introduction for Systems Engineers, 21–35. Roberts, J. C., & Al-Maneea, H. M. (2018). Study of Multiple View Layout Strategies

in Visualisation. Sallis, G., Catherwood, D., Edgar, G. K., Baker, S., & Brookes, D. (2022). Situation awareness and habitual or resting bias in high-pressure fire-incident training command decisions. Fire Safety Journal, 128, 103539. Salmon, P. M., Stanton, N. A., Walker, G. H., Jenkins, D., Ladva, D., Rafferty, L.,

& Young, M. (2009). Measuring Situation Awareness in complex systems: Com- parison of measures study. International Journal of Industrial Ergonomics, 39(3), 490–500. https://doi.org/https://doi.org/10.1016/j.ergon.2008.10.010 Shao, L., Chu, Z., Chen, X., Lin, Y., & Zeng, W. (2021). Modeling layout design for multiple-view visualization via Bayesian inference. Journal of Visualization, 24(6), 1237–1252. https://doi.org/10.1007/s12650-021-00781-z Smith, K., & Hancock, P. A. (1995). Situation awareness is adaptive, externally

directed consciousness. Human Factors, 37(1), 137–148. Van De Merwe, K., Van Dijk, H., & Zon, R. (2012). Eye movements as an indicator of situation awareness in a flight simulator experiment. The International Journal of Aviation Psychology, 22(1), 78–95. Wen, Z., Zeng, W., Weng, L., Liu, Y., Xu, M., & Chen, W. (2022). Effects of View

Layout on Situated Analytics for Multiple-View Representations in Immersive Visualization. IEEE Transactions on Visualization and Computer Graphics, 29(1), 440–450. Wijayanto, T., Marcillia, S. R., Lufityanto, G., Wisnugraha, B. B., Alma, T. G., & Abdianto, R. U. (2021). The effect of situation awareness on driving performance in young sleep-deprived drivers. IATSS Research, 45(2), 218–225. Wijayanto, T., Ramadhan, R. M., Falikha, T., Cahyani, S. D., & Mahenra, R.

Y. (2021). Effects of In-Vehicle Navigational Display Modality on Situational Awareness and Driving Performance. Joint Conference of the Asian Council on Ergonomics and Design and the Southeast Asian Network of Ergonomics Societies. Yao, X., Zhou, J., Lin, Y., Li, Y., Yu, H., & Liu, Y. (2019). Smart manufacturing based on cyber-physical systems and beyond. Journal of Intelligent Manufacturing, 30, 2805–2817.

Cover of the proceedings volume Ergonomics In Design

Explore the complete digital volume

Ergonomics In Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-53-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →