Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The Effects of Visual or Auditory Interruption on Task Performance - Combination of Behavior Data and Eye Movement Analysis

School of Machanical Engineering, Southeast University, Nanjing 211189, China ↗
Publication year
Digital Object Identifier10.54941/ahfe100948

Abstract

Interruptions are prevalent phenomena in complex human-computer interaction. And various studies have proved that interruptions have negative effects on user performance. Therefore, study on the law of interruption in the field of human-computer interaction has certain guiding significance of enhancing working efficiency. Our study examined the effects of interruption modality (visual or auditory) on performance of primary task (visual) and interruption task itself to determine which modality was less disruptive. An experiment was carried out to explore the difference between two interruption modalities. The results demonstrated that different interruption modalities caused similar disruptive effects on performance of primary task, but auditory modality was more disruptive than visual modality on performance of interruption task. Our findings indicated that intra-modal interruption was a better way than cross-modal interruption in visual primary task.

INTRODUCTION

Human Interruption is becoming an increasingly common and frequent occurrence in human–computer interaction. A growing body of studies has been discussed about the negative effects of interruptions. Generally, interruptions increase the task completion time, worsen decision making, and lead to more errors, frustration, annoyance, and anxiety (Cutrell et al. 2000, Bailey & Konstan, 2006, Carayon et al. 2007).

The ubiquity and inevitability of interruptions has led researchers to explore ways of reducing the costs of interruptions. The adjustment of interruption modality has been examined as a possible method of interruption management. There are several theoretical explanations of the effects of interruption modality on task performance. The first theory is preemption theory which suggests that information presented via the auditory channel has greater attention-capturing properties compared with the visual channel (Spence, 2001). The second is multiple resource theory which demonstrates that an interruption presented in a different modality from the primary task should be less disruptive than an interruption presented in the same modality as the primary task (Ho et al. 2001). The third is memory for goals which indicates it is not modality but activation level that takes effect, and the higher the activation level of the suspended goal, the more easily that goal can be retrieved (Ratwani & Trafton, 2001). These three theories are controversial and worthy of further study. Although there have been some studies on interruption modality, most of their attention was primary task. Few studies considered the effects of interruption on interruption task itself.

Therefore, our goal in this study was to examine how visual or auditory interruption influence primary task and interruption task itself. We designed an experiment to determine which modality was less destructive. The primary task in the experiment was a counting task and the interruption task was an addition task. The tasks both needed mathematical operations, so mutual interference was inferred. Completion time and accuracy of task were the indexes of user performance. Meanwhile, eye tracker was used to provide scientific support for the results. In the next sections, the experiment will be introduced and the results will be discussed.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

METHOD

PARTICIPANTS According to the needs of the experiment, 20 undergraduate and graduate students (10 males and 10 females) were recruited, aged 18-29 years (Mean=23.5, SD=2.0). All participants had normal intelligence, normal hearing, normal vision (or corrected vision), and no color blindness. Participants were randomly divided evenly into visual group and auditory group. Visual group experienced visual interruptions while auditory group experienced auditory ones.

MATERIALS

Counting task, counting the number of target graphics, was utilized as the primary task. A matrix of 55 graphics was presented to participants (see Figure 1). Participants need to calculate the number of graphics in the graphic matrix that are consistent with the target graphics including color and shape in the upper left corner.

Figure 1. An example of primary tasks. (counting the number of target graphics) Addition task was utilized as the interruption task, which had two modes of vision and hearing. When visual interruption occurred, the original interface was replaced by an interface with addition question (see Figure 2). When auditory interruption occurred, a voice broadcast the addition task and the interface remained unchanged.
Original PDF Figure 1.
Figure 2. An example of visual interruption tasks. (Addition question was expressed in
Original PDF Figure 2.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Chinese to ensure consistency with auditory interruption tasks.)

EQUIPMENT A desktop, a laptop and a set of Dikablis Eye Tracker were used in our experiment. The desktop was utilized to run the D-Lab experiment platform and connect with the Dikablis Eye Tracker. The laptop was utilized to present the experimental tasks, and record completion time and answers of participants. The screen dimension of the laptop was 16 inches, and the resolution was 1920*1080. The sampling frequency of the eye tracker was 50 Hz. The experiment program was run in the E-Prime software platform. Participants were seated 50cm from the screen and completed interaction with the laptop by keyboard.

PROCEDURE When participants arrived at the lab, we went through an informed consent process. After wearing the Dikablis Eye Tracker, participants were given a brief overview of the experiment and did the practice experiment to get familiar with the experiment process. Then the formal experiment began.

The experimental process is shown in Figure 3. Firstly, participants were asked to answer four addition questions to test their addition ability. Visual group answered visual addition questions while auditory group answered auditory ones. Then, eight primary tasks were presented at random in order. Four of them were interrupted by addition tasks. When interrupted, participants were asked to dictate the addition answer and press the space bar to return to the primary task. Once completing one primary task, participants pressed the space bar to jump to the answer page and input their answers. Participants were instructed to complete tasks as quickly as possible while maintaining accuracy. The program automatically recorded the completion time of each task and answers to the primary tasks. Answers to the interruption tasks were recorded manually. Meanwhile, the eye movement data were recorded.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Figure 3. Schematic diagram of experimental process.
Original PDF Figure 3.

RESULTS

COMPLETION TIME AND ACCURACY The following sections analyze completion time and accuracy of primary task and interruption task.

Primary tasks. First, Independent-Samples T-test and Chi Square Test were used to compare visual group with auditory group on completion time and accuracy of four non-interrupted primary tasks. Results indicated that there was no significant difference in completion time (p=0.460>0.01) and accuracy (p=0.500>0.01) between two groups. The abilities of counting between the two groups were similar.

The effect of interruption on primary task was examined by comparing completion time and accuracy among conditions (non interrupted, visual interrupted and aural interrupted). The ANOVA comparing completion time was significant (p<.001). Tukey HSD post hoc comparisons revealed that completion time in the non- interrupted condition was significantly shorter than in the visual interrupted (p<.001) and aural interrupted (p<.001) conditions, but the interruption modality had no significant effect (p=0.484). As shown in Figure 4, When there was an interruption, the average time to complete primary task increased by 36.9% in the visual group and 46.3% in the auditory group. Chi Square Test indicated that interruption had no significant effect on the accuracy of primary task (p=0.186).

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Figure 4. Completion time on the non interrupted vs. visual interrupted vs. aural interrupted primary tasks.
Original PDF Figure 4.

Interruption tasks. First, Independent-Samples T-test was performed to compare completion time of visual addition questions with aural addition questions in the addition ability test. Results indicated that completion time of visual addition questions was significantly shorter than aural addition questions (Mvisual=4113.400ms, Maural=6522.375ms, p<.001). And Chi Square Test demonstrated significantly higher accuracy in visual addition questions than aural ones (p=.001).

The effect of interruption on interruption task itself was examined by comparing addition ability test with interruption tasks. Single T-test indicated that interruption had a significant effect on visual addition time (p=.010) and auditory addition time (p<.001). According to Figure 5, The visual operation time increased by 12.5% while the auditory operation time increased by 21.5%. Chi Square Test showed that interruption had no significant effect on accuracy of visual addition (p=0.247) and auditory addition (p=0.126).

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Figure 5. Comparison of completion time on addition questions in the addition ability test and the interruption tasks.
Original PDF Figure 5.

EYE MOVEMENT DATA The following sections analyze eye movement of participants during the experiment.

Resumption strategy. The eye movement of participants in the resumption period after interruption was analyzed. Participants had two strategies (see Figure 6): one was to recount and the other was to continue counting. For any strategy, participants needed to relocate the target position(starting position or pre-interruption position).

Figure 6. The strategies of participants to resume counting tasks.
Original PDF Figure 6.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Saccade Data. Number of saccades was a measure of workload. The right eye saccade data during primary task were selected, and several abnormal data were eliminated. The ANOVA indicated that the effect of interruption on saccades (p=0.559>0.01) was not statistically significant, but the number of saccades still showed an increasing trend when interrupted in the primary tasks (see Figure 7).

Figure 7. Number of saccades during the non interrupted vs. visual Interrupted vs. aural interrupted primary tasks.
Original PDF Figure 7.

DISCUSSION

The results demonstrated that the effects of interruption were manifested in the completion time but not accuracy. The completion time of primary task and interruption task significantly increased in the interruption situations, while there was no significant difference in accuracy. The increasing time in the primary task can be explained by eye movement data. The results showed that participants needed to relocate the targets after interrupted, which caused more saccades. The increasing time in the interruption task can be explained by interruption lag, which is a period of reaction time to switch from the primary task to the interruption task.

Different interruption modalities caused similar disruptive effects on performance of primary task. There was no significant difference in the increase of completion time of primary task between visual group and auditory group. The results were similar to the study suggesting that interruption modality was not an important factor for reducing the disruptiveness of interruptions on primary tasks (Ratwani & Trafton, 2001). But It was contrary to some other studies, which held that visual interruptions led to the largest number of errors on a visual primary task, whereas auditory

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 interruptions led to the least amount of interference (Ho et al. 2001).

We also included the performance of interruption task in the study. The result demonstrated that in the interruption situation, the increase in completion time of interruption task was greater in auditory group than in visual group. Auditory modality was more disruptive than visual modality on performance of interruption task itself. So, comprehensively considering the user performance of primary task and interruption task, auditory interruption led to more interference than visual interruption in the visual primary task.

CONCLUSIONS

The study investigated the effects of interruption modality on task performance. The results showed that the performance of primary task and interruption task were both affected by interruption. The completion time of task was longer but accuracy was not significantly changed in the interruption situation. For primary task, there was no significant difference in the effects on task performance between visual and auditory interruption. While for interruption task, auditory modality led to worse performance than visual modality. Thus, we estimate that for visual primary task, intra-modal interruption is a better way than cross-modal one comprehensively considering primary task and interruption task. We suggest to present interruption task visually in visual human-computer interaction. For further study, the effect of interruption on user emotion can be considered.

ACKNOWLEDGMENTS

This work was supported jointly by National Natural Science Foundation of China (No. 71901061).

REFERENCES

Bailey, B.P., and Konstan, J.A. (2006). On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state. Computers in human behavior, 22(4), pp.685-708. Carayon, P., Wetterneck, T.B., Hundt, A.S., Ozkaynak, M., DeSilvey, J., Ludwig, B.,

Ram, P. and Rough, S.S. (2007). Evaluation of nurse interaction with bar code medication administration technology in the work environment. Journal of Patient Safety, 3(1), pp.34-42. Cutrell, E.B., Czerwinski, M., and Horvitz, E. (2000). Effects of instant messaging interruptions on computing tasks. In CHI '00 Extended Abstracts on Human Factors in Computing Systems, pp.99-100.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Ho, C.Y., Nikolic, M.I., and Sarter, N.B. (2001). Multimodal information presentation in support of timesharing and effective interruption management. In 20th Digital Avionics Systems Conference, 1, pp.5D2/1-5D2/8. Ratwani R., Trafton J.G. (2010). An eye movement analysis of the effect of

interruption modality on primary task resumption. Human factors, 52(3), pp.370-380. Spence, C. (2001). Crossmodal attentional capture: a controversy resolved? Advances in Psychology, 133, pp.231-262.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-7923-8988-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →