Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Effects of speaker’s voice pitch variation on listener’s intellectual concentration during online lecture

Graduate School of Energy Science, Kyoto University, Yoshida-Honmachi, Sakyo, Kyoto, 606-8501, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1002413

Abstract

Background & PurposeUnder the Covid-19 pandemic, most schools all over the world had to suspend their face-to-face lectures and adopted online courses instead. Some surveys have revealed that while this new educational format has advantages of convenience and flexibility, there also exist some problems, such as students' increased fatigue and lower intellectual concentration. However, it's difficult to find any effective solutions that can directly improve the intellectual concentration from the students' side currently.To solve this problem, we have focused on the online course system. From the viewpoint of cognitive psychology, sound is one of the essential factors that can influence people's intellectual concentration, and the speakers' voice is one of the most important parts of the sound source in online courses.The purpose of this study is, therefore, to explore the influence of speakers' vocal parameters, specifically pitch variation, on listeners' intellectual concentration during online lectures.Outline of this study(Step1) Selection of the optimal vocal pitch level This step is aiming to select the optimal vocal pitch level that does no harm in keeping intellectual concentration nor increasing the cognitive fatigue among the listeners during the online course system. (Step 2) Exploration of the optimal pitch variation on listeners' intellectual concentrationIn this step, the optimal pitch variation of different patterns on listeners' intellectual concentration will be explored by experiments. For example, Normal Pitch→High Pitch→Normal Pitch (NHN pattern), or Normal Pitch→Low Pitch→Normal Pitch (NLN pattern) etc.(Step 3) Development of feedback control to recover listeners’ intellectual concentrationAfter finding optimal pitch variation for refreshing listeners’ intellectual concentration in step 2, then in this step, feedback-control will be developed to recover listeners’ intellectual concentration during the online course system.In this presentation, we will report the method and result of Step 1.MethodAn experiment was conducted with 6 students at Kyoto University (3 males and 3 females), and all of them are native Chinese speakers.The stimulus given to the participants were lecture videos in which audios in 3 pitch levels (Low, High and Normal) and 2 types of timbre (Male and Female) were prepared.Since this study is focusing on the auditory factors, to simulate the online course system, the audio stimuli using the content of passages of the Reading part in the TOEFL iBT exam were used because they are similar to university-level online course content. Furthermore, to alleviate the influence of different English levels between each participant, all the contents have been translated into Chinese.After listening to each audio, each participant was asked to answer a problem related to the content, and the correct answer rate would be used as a scale of their intellectual concentration level.In addition, their brain waves were measured by a single-channel device, NeuroSky MindWave Mobile 2 headset while they were listening to the audio. The index ‘Attention’ calculated by NeuroSky software has been also used as a reference of listeners’ intellectual concentration level.As a result of the experiment, female voice under normal pitch has been verified as the optimal pitch level to proceed to the next step of this study.

INTRODUCTION

With the development of media technology, e-learning has been getting popular around the world. Especially under the Covid-19 pandemic, most universities all over the world had to suspend their face-to-face lectures and adopted online courses instead. Some surveys have revealed that while this new educational format has advantages of convenience and flexibility, there also exist some problems, such as students’ increased fatigue and lower intellectual concentration.

According to this problem, some studies committed to proposing effective methods to measure and monitor students’ intellectual concentration levels during online lectures (Shah et al. 2021), (Revadekar et al. 2020). However, currently it’s difficult to find any effective solutions that can directly improve their intellectual concentration.

To achieve this goal, this study has been focusing on the online course system and trying to explore the possibility to improve students’ intelligent concentration through the sound interface. From the viewpoint of cognitive psychology, sound is one of the essential factors that can influence people’s intellectual concentration (McAdams et al. 1993), and the speakers’ voice is

© 2022. Published by AHFE Open Access. All rights reserved. 295

one of the most important parts of the sound source in online courses. Besi- des, pitch variation has been verified effective to improve people’s cognitive processing of a complex communicative message (e.g., radio ads) (Rodero et al. 2017). Therefore, the purpose of this study is to explore the influence of speakers’ pitch variation on listeners’ intellectual concentration during online lectures.

EXPERIMENTAL METHOD Sound Stimulus for Improving Intellectual Concentration

Since this study is mainly focusing on the online lecture system and especially its auditory settings, a similar condition should be simulated in the experi- ment. In this study, audios recorded by Siri voice assistant (Apple, 2022) were used as the sound stimulus. Furthermore, to make it closer to the university course-level, we used the passages of the Reading part in TOEFL iBT exami- nation as the content, because they are similar to the textbook of introductory courses in universities.

Meanwhile, to alleviate the influence of different English levels between each participant, all the contents have been translated into Chinese because all the participants were native Chinese speakers.

Evaluation Method Objective evaluation Post-test

After listening to each audio, participants were asked to answer the post- test including one question based on the audio content. The question requires them to select all the correct answers from 5 options, which can express the important ideas of the passage. The score would be scaled from 0 to 5, which can reflect participants’ overall grasp of each audio content.

Attention level EEG (electroencephalogram) data measured by a single channel device, NeuroSky Mind Wave Mobile 2 headset (NeuroSky, 2015), was used as an auxiliary reference to quantify and analyze students’ intellectual concentra- tion level during online lectures. The reason choosing this device was that it was verified usable in many conventional studies (Crowley et al. 2010) but also low-price and convenient to use.

In specific, the software named NeuroView (NeuroSky, 2020) developed by NeuroSky company can output an index called ‘Attention’, calculated by the NeuroSky built-in algorithm, which reflects intellectual concentration, and the value ranges from 0 to 100.

Subjective evaluation Post-questionnaire As shown in Table 1, two questions were asked after the experiment to investigate participants’ subjective feelings towards the pitch variation condition.

Effects of Speaker’s Voice Pitch Variation on Listener’s Intellectual Concentration 297

Table 1. Questions of post-questionnaire Questions

Table available in the original PDF paper.

1. Which kind of audio do you think makes you more focused? A. Without pitch variation B. With pitch variation 2. Did you feel awakened at the moment of the pitch variation? A. Yes B. No

Figure 1: Pitch variation sequence.

Figure available in the original PDF paper.

EVALUATION EXPERIMENT More details of the evaluation experiments will be introduced in this part, including the experimental conditions and the procedure.

Experimental Condition Before this experiment, two preliminary experiments have been conducted to find the optimal pitch level and suitable contents. Specifically, in prelimi- nary experiment 1, male’s and female’s voice with 3 pitch levels (low, normal, high) has been evaluated with 6 students of Kyoto University (3 males and 3 females), and the results showed that female voice with normal pitch had the best performance which gave no harm on keeping listeners’ intellectual concentration nor increasing their cognitive fatigue under the online course system. Besides, to alleviate the influence of different contents, in preliminary experiment 2, we selected 6 contents with similar difficulties among 12 con- tents, in order that the difference of the contents do not affect the score of the post-tests.

The formal experiment aimed to explore the influence of speaker’s voice pitch variation on listener’s intellectual concentration during online lectures. This experiment was conducted twice, each time with 4 students of Kyoto University (5 males and 3 females in total), and all of them are native Chi- nese speakers. As the pitch variation pattern, females’ voice with high pitch level have been used as the variation target, because high pitch tends to be more effective on awaking and alarming. Figure. 1 shows the pitch variation strategy in this experiment. Audios with pitch variation were edited by shif- ting the pitch of the 20 seconds in the middle of the audio to +4 semi-tones, via the Pitch Shifter effect of Adobe Audition (Adobe, 2021).

Table 2 shows the outline of the experimental procedure, and Table 3 shows the details of each session. First, the participants were asked to take a practice session including 3 trials to get familiar with the post-tests, after that they were asked to take session 1 and 2, each session including 3 trials. Within each trial, they need to listen to an audio of 4-5 minutes, and then answer a post-test.

Table 2. Outline of experimental procedure.

Table available in the original PDF paper.

Process (min) Practice Session 20 Rest 15 Session 1 20 Rest 15 Session 2 20 Questionnaire 2

Table 3. Procedure of each session.

Table available in the original PDF paper.

Practice Session Session 1 Session 2 Practice 1 Trial 1 Trial 4 Practice Test 1 Test 1 Test 4 Practice 2 Trial 2 Trial 5 Practice Test 2 Test 2 Test 5 Practice 3 Trial 3 Trial 6 Practice Test 3 Test 3 Test 6

Table 4. Order of audio contents and pitch variation conditions.

Table available in the original PDF paper.

ID Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6 P1 PitchV (A) Normal (B) PitchV (C) Normal (D) PitchV (E) Normal (F) P2 Normal (A) PitchV (C) Normal (B) Normal (D) PitchV (F) Normal (E) P3 PitchV (B) Normal (C) PitchV (A) PitchV (E) Normal (F) PitchV (D) P4 Normal (C) PitchV (B) Normal (A) PitchV (F) Normal (E) PitchV (D) P5 PitchV (A) Normal (B) PitchV (C) Normal (D) PitchV (E) Normal (F) P6 Normal (A) PitchV (C) Normal (B) Normal (D) PitchV (F) Normal (E) P7 PitchV (B) Normal (C) PitchV (A) PitchV (E) Normal (F) PitchV (D) P8 Normal (C) PitchV (B) Normal (A) PitchV (F) Normal (E) PitchV (D)

Additionally, while they were listening to the audios, their attention levels were measured by NeuroSky Mind Wave Mobile 2 headset and recorded by NeuroView software mentioned in 2.2 section. Finally, after finishing all the sessions, they need to answer a short questionnaire as shown in Table 1.

To further avoid the influence of contents and the order of experimen- tal situation, conditions for each participant have been carefully designed. Table 4 shows more details of the experimental condition for each partici- pant. “PitchV”in the table means audios with pitch variation, and “Normal” means audios without pitch variation. Besides, selected through the prelimi- nary experiment 2, six different contents were used in this experiment and the alphabet in the brackets shows the content ID. Table 4 shows the experiment conditions for participant 1 to 8.

Effects of Speaker’s Voice Pitch Variation on Listener’s Intellectual Concentration 299

Figure 2: Average scores of post-tests under conditions with and without pitch variation.
Original PDF Figure 2:
Table 5. Results of the questionnaires.

Table available in the original PDF paper.

Questions (Person)

1. Which kind of audio do you think makes you more concentrated? A. Without pitch variation 1 (P8) B. With pitch variation 7 2. Did you feel awakened at the moment of the pitch variation? A. Yes 7 B. No 1 (P8)

RESULTS AND DISCUSSION

As mentioned in section 2.2, objective and subjective evaluation indices have been used in this study and the results are introduced and discussed in this part.

Figure 2 shows the results of post-tests. The participants got an average score of 3.42 under the condition with pitch variation (PV), which is 0.25 point higher than that of 3.17 under normal condition with an improvement of 7.9%.

Since the refreshing effect of pitch variation tends to be short-time, we analyzed the data focusing on the average values of attention level in the 20 seconds during, before and after the pitch variation (PV). As shown in Figure 3, the attention level measured by NeuroSky device had the highest average value of 46.24 in the 20 seconds during the pitch variation, with the following of that of 44.65 in the 20 seconds after the pitch variation, and both of them are higher than that of 42.36 in the 20 seconds before the pitch variation.

Table 5 shows the results of the questionnaire. Seven out of eight partici- pants thought the condition with pitch variation made them more concen- trated on the audio content, and they also felt awakened at the instant of the pitch variation. Only 1 participant (Participant 8) had the opposite attitude.

Based on the experiment results, it can be found that the participants have better performance in the post-tests under the condition with pitch vari- ation, and the attention level measured by NeuroSky device also showed

Figure 3: Average values of attention level in the 20 seconds during, before and after the pitch variation.
Original PDF Figure 3:

higher values in this situation. In addition, the results of subjective eva- luation showed that most participants (7 out of 8) held positive attitudes towards the pitch variation condition and thought that it can help them more concentrated and keep sober.

SUMMARY To solve the problem of lower intellectual concentration during online lecture, this study has been trying to verify the possibility of pitch variation of lecturer’s voice. Evaluation results showed that compared with the normal condition, the participants got a 7.9% improvement in their post-tests under the condition with pitch variation. Moreover, their attention level also sho- wed an increasing trend since the moment of the pitch variation. In addition, the subjective results of most participants felt there was positive effect helping them concentrate and keep awake under pitch variation condition. Therefore, as exploratory research, our study showed that the pitch variation tended to improve listeners’ intellectual concentration.

Undoubtably, there were still some limitations in this study. The number of participants should be increased in the further experiments. The post- test consisted of only one question, and more questions should be used to improve the accuracy of objective evaluation. Besides, only one pattern of pitch variation has been used in the experiment, and more patterns should be evaluated.

REFERENCES

Adobe. (2021). Automatic Pitch Correction effect. https://helpx.adobe.com/audition /using/timt-pitch-manipulation-effects.html [accessed 31 January 2022]. Apple. (2022). Have your Mac speak text that’s on the screen. https://support.apple.

com/en-jo/guide/mac-help/mh27448/mac [accessed 31 January 2022].

Effects of Speaker’s Voice Pitch Variation on Listener’s Intellectual Concentration 301

Crowley, K., Sliney, A., Pitt, I., & Murphy, D. (2010). “Evaluating a brain-computer interface to categorise human emotional response.” International Conference on Advanced Learning Technologies, pp. 276–278. McAdams, S. E., & Bigand, E. E. (1993). “Thinking in sound: The cognitive psych-

ology of human audition.” In Based on the fourth workshop in the Tutorial Workshop series organized by the Hearing Group of the French Acoustical Society. Clarendon Press/Oxford University Press. NeuroSky. (2015). MindWave Mobile 2. Available from: https://store.neurosky.com /pages/mindwave [accessed 19 January 2022]. NeuroSky. (2020). NeuroView. https://www.neurosky.jp/about-neuroview/ [accessed

19 January 2022]. Revadekar, A., Oak, S., Gadekar, A., & Bide, P. (2020). “Gauging attention of students in an e-learning environment.” Conference on Information & Commu- nication Technology, pp. 1–6. Rodero, E., Potter, R. F., & Prieto, P. (2017). “Pitch range variations improve cogni-

tive processing of audio messages.” Human Communication Research, 43(3), pp. 397–413. Shah, N. A., Meenakshi, K., Agarwal, A., & Sivasubramanian, S. (2021). “Assessment of Student Attentiveness to E-Learning by Monitoring Behavioural Ele- ments.” International Conference on Computer Communication and Informatics, pp. 1–7.

Cover of the proceedings volume Training, Education, and Learning Sciences

Explore the complete digital volume

Training, Education, and Learning Sciences

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-35-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →