Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Remote Cheering System with Voice in Live Streaming

College of Engineering and Design, Shibaura Institute of Technology, 3-9-14 Shibaura, Minato-ku, Tokyo 108-8548, Japan ↗ ↗
Publication year
Digital Object Identifier10.54941/ahfe1001753

Abstract

In recent years, live streaming has become the mainstream. Because the music live has been canceled or postponed by the influence of the new coronavirus infection. Even now that the number of infected people is decreasing, hybrid live performances with both concert and live streaming are being held. Live streaming can reduce travel costs and time, so it has the merit of being able to watch it easily. However, it is difficult to feel a sense of unity and presence, and it does not create more excitement than concert. It has become a problem in the music industry. In order to solve this problem, we think it is necessary to pay attention to the presence or absence of audience sharing the same place and voice, which is a big difference between concert and live streaming. This study examines how to make it satisfactory live where we can feel a sense of unity and presence even if we are alone at home in a live streaming. To this end, we clarified the behavior of the audience watching concert and analyzed how to cheer.Therefore, we conducted a survey of the excitement of it based on the pyramid of Freytag.We investigated the behavior of the audience from concert videos of idols, singers and rock bands. As a result, audience’s cheering method has three types of cheering: those using voice, those using hands and those using entire body. Cheering using voice plays an important role in deciding the excitement.Live streaming has comments, social tipping, and posting on SNS as a service. However, none of them share the voice of the audience. This analysis clarified the reason why live streaming is not more exciting than concert. Thereby we considered that sharing emotions aloud between the audience create a sense of unity in live streaming. From the above, we produce a live streaming cheering system using voice. This system uses the call program to communicate with other audiences, visualize the voice of the audience and project it on the screen. It’s mechanism that increases the number of effects that express excitement as the audience’s voice gets louder. We produce it to use TouchDesigner. Moreover, subjects watched the concert video with this system. we experimented with whether the subject felt a sense of unity and presence compared to conventional live streaming. Subjects were able to shout even more by sharing voices with other audiences and visualizing their voices. In addition, conventional live streaming shared emotions by discussing their impressions with other audiences using SNS. By contrast, this system can share emotions directly through the call program, which makes it more exciting. On the other hand, subjects have an opinion that it would be better to project effects tailored to the concept of songs and concerts on the screen so that the audience would not get bored. Therefore, this system is room for the development. From this experiment, the remote cheering system using voice improve the concert experience at home.

INTRODUCTION

In recent years, live streaming has become the mainstream (Piatex Co., Ltd, 2020), (Fujii, 2021). Because the music concert has canceled or postponed by the influence of the new coronavirus infection. The new coronavirus infection is causing a lot of damage to the music and entertainment industry (Pia Co., Ltd, 2020). However, the intention to use flat-rate music distribution service and livestreaming is increasing with the advancement of entertainment online (Recording Association of Japan, 2020). We can easily watch live streaming because it can reduce travel costs and time. Furthermore, live streaming tries various measures not to let audiences get bored such as comment function, social tipping and virtual live production (Zhenhui et al, 2017), (Mitsubishi UFJ Research and Consulting Co., Ltd, 2018). Even when that the number of infected people is decreasing, hybrid live performances that perform both concert and live streaming are being held. Fans who cannot go to the venue or who did not win a ticket to the concert are able to purchase a ticket for live streaming and participate in the concert wherever they like. However, the fans

© 2022. Published by AHFE Open Access. All rights reserved. 1

2 Takatama and Yang

who participate in the live streaming are more likely to feel alienated than the fans who go to the venue. While entertainment has rapidly become online, live streaming is difficult to feel a sense of unity and presence (Teramoto et al,2010), (Horie, 1989). Therefore, it has not created the same or higher excitement as concert. SKIYAKI Co., Ltd conducted an awareness survey of 2,822 music live fans comparing concert and live streaming (SKIYAKI Co., Ltd, 2020). In terms of overall satisfaction, the questionnaire exceeded 75% of the response that real concert is better. In addition, more than 90% of the respondents answered that real concert is better in terms of sense of unity and presence of the live performance. In other words, very few people evaluate live streaming. Live streaming has less information that the audience can get from the screen, and there is no sharing of hot emotions between the audiences that naturally occur in real concert. Consequently, it has become a problem in the music and entertainment industry. The purpose of this research is to support a live experience where audiences can feel the excitement even if they are alone at home, and to increase the value of the live streaming.

CURRENT SITUATION SURVEY OF AUDIENCE DURING CONCERT AND LIVE STREAMING Survey of Audience Behavior at Concert

We observed and analyzed behavior and the excitement transition of the audience during the concert. Furthermore, we conducted a questionnaire on the cheering using the voice of the audience during live streaming. First, we watched concert video of idols, singers and rock bands. Then we observed and analyzed behavior of the audience. As a result, we analyzed that there are three types of cheering for the audience: those using voice, those using hands, and those using the entire body.

Next, we analyzed the behavior of the audience in the song sung during the concert, based on Freytag’s Pyramid. Freytag’s Pyramid is the theory that music and theater starts with a rising action and ends with a calming action after reaching the climax (Cole, 2020). We measured the audience’s excite- ment from 0 to 5. Specifically, we defined 0 for the audience to be quiet and relaxed and 5 for the audience to move their entire body while making a loud voice. Figure 1 is a graph of the behavior of the audience during the song. As a result of the analysis, the audience often cheers using voice toward the chorus, which is an important part of the song, before the chorus such as melody B. The cheering using voice creates the most exciting climax scene. It is an important role that determines the excitement of the live streaming. There is a strong connection between the excitement of live performance and voice.

Survey of Audience Behavior at Live Streaming We conducted a questionnaire survey for 67 male and female in their teens and twenties who had watched live streaming about cheering using voice during the live streaming. First, we investigated whether they enjoyed watch- ing the live streaming. Then, 60 subjects answered “No”. In addition, we

Remote Cheering System with voice in Live Streaming 3

Figure 1: Behavior of audience in song sung concert.
Original PDF Figure 1:
Figure 2: Reasons for not cheering with voice during live streaming.
Original PDF Figure 2:

investigated the situation in which the 60 subjects would watch the live stre- aming and how to cheer them on. 41 of 60 subjects watch the live streaming alone, 13 subjects watch it with their family, and 10 subjects watch it with their friends. 32 of 41 subjects who watch it alone answered that they would not cheer using voice. Then, Figure 2 summarizes the reasons why the 32 sub- jects do not cheer using voice. Many of them said, “Lonely alone at home” and “Not related to the excitement of the venue”. From the results of this que- stionnaire, we believe that the live streaming is less exciting than the concert because it is not possible to share emotions with other audiences on the spot and there is no sense of unity or presence. We need a system that could share the voice with others and visualize them so that we can easily see the excite- ment of the live performances. Based on the above, in this research, we will create a cheering system using voice in live streaming.

CREATING A PROTOTYPE SYSTEM System Concept

We decided on two system concepts: a design that shows the changes in the excitement of the live streaming immediately, and a design that does not interfere with its video.

4 Takatama and Yang

Figure 3: System configuration diagram.
Original PDF Figure 3:

System Configuration We built a remote cheering system that audience use a computer and a smart- phone. Figure 3 shows the configuration of this system. In this system, we acquire the voice information of each audience with the build-in microphone of the personal computer and collect the voices of the audience with the WEB call program. We used LINE Phone for the WEB call program. We created a program with TouchDesigner that changes the graphics in real time accor- ding to the loudness of the collected audience’s voice and built a system to display it in combination with livestreaming video. We also need to deliver video screen created in TouchDesigner. Therefore, we use TouchDesigner’s NDI Out TOP to stream video captures in OBS Studio. We used YouTube as our live distribution site to allow audiences to watch the live streaming.

RESEARCH METHOD This research aimed to investigate the usefulness of this system and the vali- dity of the concept of using voice. We experimented twice with the prototype system that we created. The content of the two evaluation experiments is to watch live performance video using this system for 30 minutes. Furthermore, we asked the subjects to complete some questionnaires after experiment. In the questionnaires, we investigate the satisfaction obtained when using this system, the change in the sense of unity and presence of the live performance, the change in the shout during the live streaming.

The contents of the questionnaire are as follows:

1) Do you feel satisfied this system more than the conventional live streaming? 2) Do you feel satisfied this system more than the conventional concert? 3) Do you feel that this system has a sense of unity in the live performance compared to the conventional live streaming? 4) Do you feel that this system has a sense of presence in the live performa- nce compared to the conventional live streaming? 5) Did you cheer louder when using this system than when you watched the live streaming before? 6) Did you feel like cheering more with voice when using this system?

Remote Cheering System with voice in Live Streaming 5

Figure 4: Using remote cheering system.
Original PDF Figure 4:
Figure 5: Interface design used in the primary evaluation experiment.
Original PDF Figure 5:

RESEARCH CONTENT AND RESULTS Research Content

We experimented twice and asked the subjects to complete 6 questionnai- res after using this system. After that, we asked the subjects in detail what they liked about system and what they were dissatisfied with. Figure 4 is a photograph of the actual experiment.

First, we experimented with 6 male and female subjects in their twenties who had watched live streaming. This time, we displayed sound level maters graphic on the sides of the screen so as not to interfere with the live streaming video and designed it to show how loud the audience’s voice is. Figure 5 shows the interface design used in the primary evaluation experiment.

The second time, we experimented with 26 male and female subjects in their twenties who had watched live steaming in groups of 4 to 5 people. After the primary evaluation experiment, we improved the design so that the graphics of particle like fireworks appearing from the sides and center of the screen. Figure 6 shows the interface design used in the secondary evaluation experiment.

6 Takatama and Yang

Figure 6: Interface design used in the secondary evaluation experiment.
Original PDF Figure 6:
Figure 7: Satisfaction obtained when using this system.
Original PDF Figure 7:

Results

In the primary evaluation experiment, all subjects highly evaluated the use- fulness of this system. In contrast, they commented, “I feel like I’m making a voice by myself” and “The graphics are monotonous and boring.”

In response to subject’s opinions, we make improvements the two problems in the primary evaluation experiment. Specifically, we improved the point that “I feel like I’m making a voice by myself.” to graphic design that makes it look like each audience make a voice. And we improved the point that “The graphics are monotonous and boring.” to a design that the graphics appear not only from both sides of the screen but also from the center with a certain amount of excitement.

As a result, more than 90% of the subjects answered that there is a sense of unity and presence in the live performance by comparing this system with the conventional live streaming. In addition, more than 90% of the subje- cts answered that they are more likely to cheer with a louder voice than the conventional live streaming, and all subjects responded that they were motivated to speak aloud more. And 96% of the subjects answered that they were satisfied with the live streaming when using this system compared to the conventional live streaming. Compared to the concert, many subjects answ- ered that both concert and live streaming were good, and about 60% of the subjects answered positively. Figure 7 shows the satisfaction obtained when using this system.

Remote Cheering System with voice in Live Streaming 7

CONSIDERATION In this research, the attention of the audience’s voice is appropriate. Voice and live performance are closely related. We also believe that this system is useful not only for live streaming, but also for watching sports, watching movies and distributing games.

This research has two issues. The first is a functional problem. This system requires graphics that change in real time according to the concept and the world view of the live performance. Live performance is the world view expresses by the concept and set list that artists imagine. Therefore, when the live streaming has quiet scenes or video, we design inconspicuous graph- ics so that the audience can concentrate on what they are watching. On the other hand, we design flashy graphics to match when the artist has a flashy production with flames and fireworks. The immersion in the live streaming is enhanced by changing the graphics depending on the scene. And the value of live streaming will increase. The second is a technical problem. Namely, the graphics appearing in the center interfere with the subtitles when live strea- ming broadcast video with subtitles. In this system, particle graphics appear from the center of the screen when the volume of the audience’s voice exceeds a certain level. However, we need to fine-tune the appearance of the particle graphics by setting the graphic’s appearance time and range of external forces in detail in order not to interfere with the video with subtitles.

CONCLUSION

In this research, we constructed a remote cheering system using the audie- nce’s voice for the purpose of making live streaming more satisfying than conventional it. We showed how important the voice is through the analy- sis of the behavior of the audience during the concert and the questionnaire survey on the cheering of the audience during the live streaming. This system visualizes the voice of the audience and changes the graphic in real time. As a result, the audience’s shouts increased and the live satisfaction was enha- nced. Therefore, we concluded that this system is effective for cheering on live streaming. Through this research, audibly and visual shared the audie- nce’s shouts in real time leads to a fulfilling live experience. Furthermore, it is room for the development and needs to be improve d to make the live strea- ming more satisfying. From this experiment, the cheering system using voice improve the concert experience at home.

REFERENCES

Cole, S. (2020) Freytag’s Pyramid and the Importance of a Dramatic Structure: https: //www.tckpublishing.com/freytags-pyramid/ Fujii, D. (2021). “Music live streaming and future stories”, in: Textbook for the first

year of live streaming. pp. 36–41. Horie K., (1989). On the Principle of the Feeling of Togetherness, Journal of Japan Industrial Management Association 40(4), pp. 266–273.

8 Takatama and Yang

Mitsubishi UFJ Research and Consulting Co., Ltd (2018). Organize the trends of live streaming services: https://www.caa.go.jp/policies/policy/consumer_policy/policy _coordination/internet_committee/pdf/internet_committee_190117_0002.pdf Pia Co., Ltd. (2020). Damage to the music and entertainment industry due to the new

coronavirus: https://corporate.pia.jp/news/detail_covid-19_damage200529.html Piatex Co., Ltd. (2020) Survey on online events: https://documents.peatix.com/Peati x_Online_Event_Survey_2020.pdf Recording Association of Japan. (2020). 2020 Music Media User Survey: https://ww

w.riaj.or.jp/f/pdf/report/mediauser/softuser2020.pdf SKIYAKI Co., Ltd. (September 9, 2020). Fan Platform Service, SKIYAKI Co., Ltd Website: https://skiyaki.com/contents/339428 Teramoto, W., Yoshida, K., Asai, N., Gyoba, J. and Suzuki, Y. (2010). What is

“Sense of Presence”? A Non-researcher’s Understanding of Sense of Presence, Transactions of the Virtual Reality Society of Japan 15(1), pp. 7–16. Zhenhui, Z., Zhi, Y. and Yafei, D. (2017). Understanding the Gift-Sending Interaction on Live Streaming Video Websites, Social Computing and Social Media. Human Behavior, pp. 274–285.

Cover of the proceedings volume Kansei Engineering

Explore the complete digital volume

Kansei Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-16-2

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →