Shogo Yanaze, Keiichi Watanuki, Yusuke Osawa (2025). Construction of a VR music live performance system to support improved sense of presence. In: Affective and Pleasurable Design. AHFE Open Access, vol. 164, pp. 136–141. https://doi.org/10.54941/ahfe1006068
Entertainment such as live music performances (live performances) and events are often held in urban centers for reasons such as transportation accessibility and venue size, resulting in regional disparities in entertainment. On the other hand, recent years have seen the emergence of virtual reality (VR)-based live performances, which could solve regional disparities in entertainment. VR live performances can provide many opportunities for people who are unable to attend live performances due to location and time constraints.However, the quality of the experience is expected to deteriorate due to the sense of realism and other sensations that are difficult to reproduce in Reality live performances. However, few studies have compared real-life live performances with VR live performances and examined the effects of factors such as the sense of presence on the audience. In this study, an experiment was conducted with the aim of constructing a system to support the improvement of the sense of presence in VR live music performances. In the experiment, we experienced an on-demand VR live performance and investigated the degree of realism and the differences from a real live performance. From the results, we analyzed the factors that are important for improving the realism of VR live performances. The system is then constructed and evaluated.Ten male participants (23.6±1.8 years old) who had participated in a live concert in the past and remembered the sensation were asked to experience a VR music concert in a standing position for 5 minutes, and a post-experience questionnaire was administered to compare the realism of the actual concert and the VR concert.The post-experience questionnaire consisted of a subjective questionnaire and open-ended questions regarding the sense of presence and the five senses. The subjective questionnaire used a 5-point Likert scale. The items related to the sense of presence consisted of 13 items, which were added by the researcher based on previous studies that have analyzed the sense of presence. 3 was used as the standard for the 5-point scale, based on the comparison with previous real-live experiences. The higher the value, the higher the evaluation of each element in the VR experience.According to the results of the post-experience questionnaire, the vibration and communication elements were rated low. The VR live experience and its components in this experiment did not reproduce the vibrations from the large speakers and communication with other people, such as the performers and the surrounding audience, that are felt in a real live performance. Therefore, the experience tended to feel like a one-way live performance, and the quality of the experience was considered to be compromised. In addition, factor analysis of the questionnaire results showed that among the body perception factors with the highest contribution rate, the values of dynamism, vibration, and sense of self-presence were the most significant, and thus had a large impact on the sense of presence in the live performance.
Live music concerts and events are often held in urban centers because of the convenience of transportation and size of the venues, resulting in regional disparities in entertainment. In contrast, live streaming platforms using virtual reality (VR) have become popular in recent years, because live shows can be attended regardless of location and time, providing an opportunity for those who cannot attend live shows because of these restrictions. However, the quality of the experience deteriorates, because it is difficult to reproduce realistic sensations that can be experienced in a real live performance. In previous studies, the sense of presence was classified into three categories: spatial elements, such as three-dimensionality; temporal elements, such as simultaneity; and physical elements, such as interactivity. The methods
Construction of a VR Music Live Performance System 137
used to evaluate the sense of presence include subjective evaluation and biometric signal measurements. In a previous study, we focused on the presence of the audience and the viewer’s behavior in response to actions, and examined the factors that contribute to realizing a sense of presence in VR. To enhance the live performances, the experimenter considered the movements of the audience, and by referring to the way the audience behaved, the experimenter’s movements were more active in accordance with the audience. This finding suggests that information about the surrounding audience is a factor in the sense of presence. This study focused on audience presence and nonverbal communication in live VR performances, and aimed to determine the effect of hand gestures on presence in live VR performances. In this experiment, we evaluated the effect of hand gestures on the sense of presence using subjective evaluation and biometric measurements.
EVALUATION OF THE EFFECT OF HAND GESTURES ON THE SENSE OF PRESENCE VR Live Overview
Ten male subjects (24.0 ± 1.1 years old) were included in this experiment. An experimental video was captured using a 360-degree camera (THETA Z1, Ricoh) during an indoor live performance at a university festival. The camera was set up at a distance of approximately 2 m from the stage, and the height from the floor to the lens was 165 cm, considering the average height of Japanese males and height of their shoes. The video images and hand gestures in the experiment were presented using Unity (2022.3.43f1), a game development platform, using VR equipment comprising an HMD and a controller (VIVE Pro 2, HTC). Figures 1 and 2 show the experimental and live VR environments, respectively.
Figure 1: Experimental environment.Figure 2: VR live environment.
Experimental Conditions Experiments were conducted under two conditions. Under the first condition, the experimenter was instructed to perform hand gestures in the same manner as the spectator during the live experience. Under the second condition, the experimenter was instructed not to perform hand gestures during the live experience. The hand gestures were always presented during the VR live experience, and by waving the controller in time with the surrounding audience, hand gestures were presented in real time in the VR space. The same videos were presented under both conditions. The experiment was conducted on different days under each condition to prevent a sense of deja vu.
Experimental Procedure A pre-experimental explanation and questionnaire regarding VR sickness was provided to the participants. Subsequently, an electrocardiogram sensor and VR equipment were attached and the volume was adjusted for the experiment. After 5-min of pre-rest in a seated position, the participants experienced approximately 8-min of live VR in the standing position. After the experience, the participants were seated again, rested for 5 min, and answered two post-experiment questionnaires regarding VR sickness and sense of presence. The experiments were conducted using similar procedures under both conditions.
Valuation Index The igoup presence questionnaire (IPQ) was used as an index to subjectively evaluate the sense of presence experienced during a live VR performance after the experiment. The IPQ is a 7-point scale (1-7) comprising 14 questions that evaluate four indices: general presence (gp), spatial presence (sp), involvement (inv), and experienced realism (real).
The simulator sickness questionnaire (SSQ) was used as a subjective evaluation index to assess the physiological changes and VR sickness before and after the experiment. The SSQ is a 4-point scale (0 to 3), and the indices of nausea, oculomotor function, disorientation, and total severity can be calculated from the 16 questions.
Biometric signals were measured to objectively evaluate the sense of presence during the experiment. An electrocardiograph (WEB-1000, Nihon Kohden, Tokyo, Japan) was used to measure biometric data. The LF/HF ratio
Construction of a VR Music Live Performance System 139
was measured and calculated throughout the experiments. High LF/HF ratios indicate a state of tension, stress, or excitement, whereas low values indicate a state of relaxation.
RESULTS
IPQ The IPQ results are shown in Figure 3. The sense of presence was higher in the with-gesture condition for all three indicators, except for general presence (gp). In both conditions, the value of the involvement (inv) index was the highest, but the experienced realism (real) index was low, with an average value of less than 4, suggesting that realism may have been lacking. Wilcoxon signed-rank tests were conducted on the results of both conditions for each indicator and significant differences were found only for the spatial presence (sp) indicator at the 5% significance level.
Figure 3: Comparison of presence (IPQ).Figure 4: Comparison of SSQ.
SSQ The SSQ results are shown in Figure 4. This was performed by standardizing the response values for each participant, calculating the difference between the pre- and post-experiment responses, and then comparing them. The mean value was less than zero for all indices under both conditions, suggesting that live VR content may suppress VR sickness. The mean values in the with-gesture condition were lower than the mean values in the no-gesture condition for all three indicators, except nausea. In the indicators of the “oculomotor” and “disorientation,” the variability in the with-gesture condition was higher than that in the without gesture condition. Wilcoxon signed-rank tests were conducted for each index, and significant differences were found in the nausea index at the 5% significance level.
LF/HF
Figure 5 shows the LF/HF values before and after resting for each condition. There was more variability in the no-gesture condition than in the gesture condition. A t-test was conducted between the pre- and post-rest periods for each condition. No significant differences were observed at the 5% level under either condition.
Figure 6 shows the amount of change in the front and rear rests for each condition, based on the results shown in Figure 5. A t-test was performed between the two conditions; however, there was no significant difference at the 5% significance level.
Figure 5: Comparison of LF/HF.Figure 6: Comparison of variable quantity.
Construction of a VR Music Live Performance System 141
DISCUSSION
The IPQ results showed that the value of the involvement (inv) index was the highest in both conditions compared to the other indices, suggesting that the participants’ awareness of and attention to reality may have been relatively lowered during the live VR experience because of their interest in and attention to the content. In addition, the index of spatial presence (sp) was significantly higher in the condition with gestures, suggesting that gestures make it easier to recognize one’s own presence in virtual space, making it easier to immerse oneself in a live performance.
The SSQ results showed significant differences in the nausea index, suggesting that participants were less likely to feel sick in the with-gesture condition because of fewer physical actions. In the no-gesture condition, the subjects were more likely to feel uncomfortable owing to the increased visual information and intentional gestures.
The t-test showed no significant differences between the LF and HF results. This suggests that the presence or absence of gestures had no clear effect on the LF/HF ratio. It is also possible that the presence or absence of gestures had no significant effect on the biometric information, even if the participants felt a sense of presence.
CONCLUSION
In this study, we focused on the presence of audiences and non-verbal communication in live VR performances. We conducted an experiment under two conditions to clarify the effect of hand gestures on the sense of presence during live VR performances. The experimental results suggest that gesturing with the audience contributes to a feeling of physical synchronization and enhances the sense of presence. However, it has been suggested that this may be one of the factors that cause nausea. In the no-gesture condition, it was difficult to feel uncomfortable owing to the lack of physical movement, whereas it was difficult to feel a sense of presence owing to passive experience and gesture limitations. In a future study, we will investigate the impact of a prolonged live VR experience. In addition, to clarify whether the presence or absence of a gesture display affects the sense of presence, we aim to add a condition in which gestures are not displayed in VR even when the user is instructed to perform them, for comparative evaluation.
REFERENCES
Ando, Hiroshi. (2009). Cognitive mechanisms of human perception of presence and evaluation techniques, Journal of the Institute of Image Information and Television Engineers, Volume 63, No. 12. Bannai, Musashijima, Y. Y. (2020). Interactive Virtual Live Performance Viewer,
WakWak, Tube 3D, IPSJ SIG Technical Report Volume 2020-GN-1, No. 2. Kennedy, Robert, S. (1993). Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness, International Journal of Aviation Psychology Volume 3, No 3. Schubert, T. Friedmann, T. Regenbrecht, H. Igroup presence questionnaire (IPQ)
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-40-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional