Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

User Requirement Matching Model for Distributed Conferencing Systems in the Post-Epidemic Era

1 School of Industrial Design, China Academy of Art, China ↗
2 Design Innovation Center, China Academy of Art, China ↗
3 System Planning Lab, Chiba University, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1003366

Abstract

The epidemic (COVID-19) has incredibly released the demand for teleconferencing, creating conditions for developing distributed conferencing systems. However, due to the complexity of distributed conferencing systems, there needs to be more clarity regarding matching product supply with user demand. In addition, the currently adopted product evaluation methods are mostly single-view assessments, which do not fully represent the degree of matching between them and user needs. Therefore, this study will take academic conferences as an example and explore and construct a user-demand matching model for distributed conference systems from the perspective of conference organizers. This model's construction can help decision-makers get feedback on user requirements matching degrees quickly and provide design strategies for product system development and iteration.

INTRODUCTION

The development of information technology has broadened the boundaries of the meeting model, and meetings are no longer limited to face-to-face communication in the same space and time but also include distributed communication, including audio conferencing, video conferencing, and tele- presence (Standaert et al. 2021; Yankelovich et al. 2004). The outbreak of the epidemic (COVID-19) in 2020 revolutionized people’s lives and also drove the growth of the distributed meeting industry. Today, this trend continues and expands. According to Gartner, by 2024, 75% of the world’s meetings will be distributed, up from 40% before the outbreak (Standaert et al. 2021).

From traditional offline meetings to distributed meetings, the mode tran- sformation is behind the continuous updating of meeting needs and gradually reveals users’ pain points in the meeting process. For example, although video conferencing can meet the synchronous communication needs of partici- pants in different locations, long-time video conferencing makes participants feel physically and mentally exhausted and increases the communication

© 2023. Published by AHFE Open Access. All rights reserved. 93

burden of both parties (Xiao, 2021). The technical performance variability of conferencing hardware makes the audio-visual experience of distributed meetings uneven, and people prefer to have multiple people present in the meeting (Berndtsson et al. 2012). These issues have shown a partial dissona- nce between conferencing systems and user needs. In conclusion, in product innovation, the matching of user needs of distributed meeting systems needs to be considered and evaluated to achieve a good meeting experience. (Zhang and Wang, 2021) This paper will be divided into seven parts. In the second part, we review the theories related to distributed conferencing and product and user require- ments matching. In the third part, we present the basic overview of the study and the research steps. In the fourth section, we detail the source and defini- tion of the matching degree model. Immediately after that, in Part V, we show the data collection, calculation, and analysis process. In Part VI, we discuss this paper’s experimental results and research implications and highlight the research conclusions, limitations, and future research directions in Part VII.

LITERATURE REVIEW

Overview of Distributed Conferencing Systems Distributed conferencing can be defined as the communication of two or more people in different locations by any means of communication. Distribu- ted conferencing systems provide many features and services for this purpose (Yankelovich et al. 2004) and rapidly iterate under the influence of epidemic normalization. However, there is still a gap between its product supply and user needs. For complex products like this (including intelligent products, apps, web pages, etc.), the interaction experience and the definition of the product itself do not fully match due to the diversity of their practical expe- riences and the complexity of their usage processes. Moreover, not all users can successfully perceive the value of the services provided by the product due to the differential influence of the participants’ comprehension ability, the nature of the meeting, and other factors (Chen et al. 2021). Therefore, unilateral product evaluation or performance indicators do not fully repre- sent the degree of matching, and such evaluation mechanisms should also be based on user perception and demand evaluation (Wang et al. 2022; Zhang, 2018).

Product-User Demand Matching Model The existing literature on product-user needs matching is divided into two main theoretical perspectives: unilateral and bilateral (Gale and Shapley, 1962). The unilateral perspective refers to researchers who focus on a unilate- ral perspective from the product side or user side to evaluate existing products or service systems comprehensively and propose corresponding evaluation models and strategic approaches (Du, 2018; Berndtsson et al. 2012; Zhou et al. 2013), or to conduct user-needs-centered (Fu and Li, 2021; Zhang and Wang, 2021; Chi, 2020) to derive the corresponding requirement sets from guiding design practice. The bilateral perspective is to match product

User Requirement Matching Model for Distributed Conferencing Systems 95

evaluation with user requirements effectively. Due to the complexity of distri- buted conferencing systems, the bilateral matching perspective can effectively evaluate and examine users’ subjective feedback on objective attributes of exi- sting conferencing systems (Berndtsson et al. 2012). Therefore, it is vital to explore the user requirement matching model with a complete perspective for the future development of distributed conferencing systems.

METHODOLOGY

Research Subjects and Data Sources In this paper, we will take academic conferences as an example and explore the user requirements matching mainstream conference systems at home and abroad from the perspective of conference organizers. The data for this study comes from two primary sources: first, the user needs to be collected and summarized through online voice interviews; second, quantifiable data colle- cted through an online questionnaire based on preliminary interview findings. Based on the principle of information saturation, a total of seven users par- ticipated in the interviews, their ages ranged from 25–40 years old, and they had different professional identities (including associate professor, doctoral student, researcher, etc.). All interviewees had experience in using distributed meeting systems to organize or participate in academic conferences. A total of 266 questionnaires were collected, and the actual number of valid que- stionnaires was 163 (including 105 females and 58 males) based on time to fill out and consistency and regularity of response scores. The vast majority (about 93%) of the participants were between 18 and 40, covering different professional identities (about 66% were students or teachers, while the rest included technical developers, managers, professionals, etc.).

Research Steps Given the advantages of the Grey Relation Analysis(GRA) in evaluating obje- ctivity and the strengths of the Coupling Coordination Analysis(CCA) in analyzing the matching degree of things, this paper will combine the above two methods (Zhang et al. 2020; Hao, 2016) to quantify the user demand matching degree of distributed meeting systems. The specific research steps are shown in Table 1.

EVALUATION INDEX SYSTEM CONSTRUCTION Distributed Meeting System Product Evaluation Index

A combination of literature research, general generalization, and screening construct the product evaluation index system of the distributed conference system. Furthermore, considering the quantitative balance among the indica- tors on the product side and the difficulty of setting up the questionnaire at the later stage, it was finally summarized into three primary indicators (A) and 16 secondary indicators (a), as shown in Table 2.

Table 1. Study steps and methods.

Table available in the original PDF paper.

Stage Research methodology Specific content Determine metrics

Form product evaluation index system and user requirements index system.

Literature Research General Induction Qualitative Interview Data acquisition Quantitative Questionnaire Obtain product evaluation satisfaction score and user demand importance score. Data calculation

Entropy Weighting Method Determine the weights of each index for product evaluation and user needs. Grey Relation Analysis Derive the grey rational degree of each conferencing system and analyze it. Coupling Coordination Analysis

Derive the coupling coordination degree of each conferencing system and analyze it.

Table 2. Distributed meeting system product evaluation index system.

Table available in the original PDF paper.

References

Primary Indicators (A)

No. Secondary indicators (a) a1 Dynamic display effect Presenting Effect A1 Berndtsson et al. 2012; Zhou et al. 2013; LeRouge et al. 2002; Vucic and Skorin-Kapov, 2015 a2 Static display effect

Kong et al. 2019; LeRouge et al. 2002

Berndtsson et al. 2012; Zhou et al. 2013; LeRouge et al. 2002 a3 Sound effects Berndtsson et al. 2012; Zhou et al. 2013; LeRouge et al. 2002 a4 Interface aesthetics a5 Operational reliability Technology Performance A2 Du, 2018; Li, 2014; Zhou et al. 2013; LeRouge et al. 2002; Vucic and Skorin-Kapov, 2015 a6 Interoperability LeRouge et al. 2002; Vucic and Skorin-Kapov, 2015 a7 Security Kong et al. 2019; LeRouge et al. 2002 a8 Load force Du, 2018; Li, 2014; Kong et al. 2019 a9 Synchronization Berndtsson et al. 2012; Li, 2014; LeRouge et al. 2002 Function Using A3

a10 Ease of use LeRouge et al. 2002; Townsend et al. 2001; Khalid and Hossan, 2016 a11 Convenience Kong et al. 2019; LeRouge et al. 2002; Vucic and Skorin-Kapov, 2015 a12 Practicality Kong et al. 2019; LeRouge et al. 2002; Townsend et al. 2001; Khalid and Hossan, 2016 a13 Integrity CSUQ, 1995 a14 Comprehensibility CSUQ, 1995 a15 Reasonableness Vucic and Skorin-Kapov, 2015 a16 Controllability Li, 2014; Khalid and Hossan, 2016

Distributed Meeting System User Requirements Evaluation Index The distributed conference system user requirement evaluation index system was constructed from literature research and qualitative interviews. Also,

User Requirement Matching Model for Distributed Conferencing Systems 97

Table 3. Distributed meeting system user requirements index system.

Table available in the original PDF paper.

References

Primary indicators (B)

No. Secondary indicators (b)

Berndtsson et al. 2012; Interviews

Experiential Demand B1

b1 Multimodal participation experience

Berndtsson et al. 2012; Interviews

b2 Visual quality Li, 2014; Zhou et al. 2013; Interviews b3 Sound quality Li, 2014; Zhou et al. 2013; Interviews b4 Sense of reality Interviews b5 Communication atmosphere

Interviews

b6 Communication effectiveness

Li, 2014; Interviews

b7 System reliability

Technical Demand B2

Interviews

b10 Immediate feedback visibility

b8 Network Signal Li, 2014; Zhou et al. 2013; Interviews b9 Confidentiality Interviews Functionality Demand B3 b11 Fun Interviews b12 Adaptability Interviews b13 Fault Tolerance Interviews b14 Controllability Interviews; Li, 2014; Khalid and Hossan, 2016 b15 Ease of use Interviews; LeRouge et al. 2002; Townsend et al. 2001; Khalid and Hossan, 2016 b16 Programmatic Interviews

considering the balance of quantity among the indicators on the demand side and facilitating data statistics and calculations at a later stage, it was finally determined as three primary indicators (B) and 16 secondary indica- tors (b), matching the number of indicators on the product side, as shown in Table 3. After the indicator system’s construction, the indicators’ names and definitions were adjusted and improved by professionals’ evaluation and suggestions.

DATA ANALYSIS AND DISCOVERY Data Calculation and Analysis

Conference system evaluation feedback is collected utilizing subjective sati- sfaction scoring (extremely dissatisfied - extremely satisfied), and user demand feedback is collected utilizing subjective importance scoring (extre- mely unimportant - essential). Taking into account the influence of the level setting on the calculation results, the final questionnaire is issued in the form of a ten-point Likert scale and combined with personal information, parti- cipation, and other fundamental questions. (The amount of data in some

meeting systems is too small for statistical purposes, so the data analysis and conclusions later take Tencent Meeting, Ding Talk, and ZOOM meeting systems as examples) According to the scores of each index, the collected data were analyzed using SPSS (Statistical Package for Social Sciences). Firstly, the entropy wei- ght method was used to determine the weight of each indicator. Secondly, the total grey rational degree of the three meeting systems and the grey rational degree of each level indicator was measured by the GRA, and then the final grey rational degree was obtained by weighting and summing, as shown in Table 4. The larger the value is, the stronger the correlation with the parent series (composite score), which means the higher the rating. As we can see, the total grey rational degree of product evaluation after weighted summation is from highest to lowest: Tencent Meeting (0.82887), ZOOM (0.77984), and Ding Talk (0.77164), and the total grey rational degree of user demand is also ranked in the same order. In the grey rational degree of product evaluation level 1 index (A1-A3), Tencent Meeting still maintains the leading position. Compared with Ding Talk, ZOOM scores lower in presentation effect. Regar- ding the grey rational degree of user requirements (B1-B3), Tencent Meeting, ZOOM, and Ding Talk continue to lead from the highest to the lowest.

Matching Results Display The above calculation can only highlight the comprehensive evaluation and score from a unilateral perspective. In order to better understand whether the product evaluations of distributed conferencing systems and user needs match, a final step is required in this study. The coupling coordination degree (D-value) is calculated using the standardized grey rational degree between product evaluation and user needs, and the above three meeting systems and their first-level indicators are ranked separately. The degree of matching is

Table 4. GRA results for distributed meeting systems.

Table available in the original PDF paper.

Conference System

Total grey rational degree of product evaluation

Total grey rational degree of user needs

Grey rational degree of each level of user demand indicators (B1-B3)

Grey rational degree of each level of product evaluation indicators (A1-A3)

0.82887 0.83065 0.814 0.823

Tencent Meeting 0.812 0.845 0.845 0.830 ZOOM 0.77984 0.77975 0.735 0.771 0.778 0.793 0.798 0.777 Ding Talk 0.77164 0.76592 0.789 0.763 0.768 0.761 0.767 0.769

Note: The total grey rational degree is retained to five decimal places due to the slight difference in values

User Requirement Matching Model for Distributed Conferencing Systems 99

compared according to the results, as shown in Tables 5 and 6. In this paper, the D-values are divided by referring to the decile interval method (Liao, 1999), and 0–1 is divided into ten intervals on average. From the total matching results, Tencent Meeting is significantly higher than ZOOM and Ding Talk, and its product evaluation and user needs have the highest matching degree, which is at the level of quality matching. In terms of the matching degree of the first-level indicators, Tencent Mee- ting still maintains a high level of matching. At the same time, ZOOM, as a similar competitor to Tencent Meeting, has the lowest matching degree in terms of presentation effect and experiential demand (A1-B1), and the remaining two dimensions maintain a high level of coordination. Due to its product positioning and nature, Ding Talk is at a lower matching level in all indicators.

Table 5. Results of user requirements matching for distributed conferencing systems.

Table available in the original PDF paper.

Conference System

Coupling degree (C- value)

Coordination (T-value)

Matching level

Matching degree

Coupling coordination (D-value)

Tencent Meeting

1.000 0.990 0.995 10 Quality Match ZOOM 0.982 0.185 0.426 5 Close match Ding Talk 1.000 0.010 0.100 2 Serious mismatch

Table 6. Results of user requirement matching for primary indicators.

Table available in the original PDF paper.

Conference System

Coordination (T-value)

Matching level

Matching degree

No. Coupling degree (C- value)

Coupling coordination (D-value)

Tencent Meeting

A1-B1 1.000 0.725 0.851 9 Good Match A2-B2 0.985 0.846 0.913 10 Quality Match A3-B3 0.995 0.903 0.948 10 Quality Match ZOOM A1-B1 0.528 0.066 0.187 2 Serious mismatch A2-B2 1.000 0.389 0.624 7 Primary Matching A3-B3 0.873 0.385 0.580 6 barely match Ding Talk A1-B1 0.455 0.260 0.344 4 Mild mismatch A2-B2 0.350 0.158 0.235 3 Moderate mismatch A3-B3 0.872 0.196 0.413 5 Close match

DISCUSSION

Focusing on product evaluation or data calculation from a unilateral perspe- ctive, especially when analyzing a complex product like a distributed meeting system, may make the results biased. In this paper, we explore the relationship between product evaluation and user requirement matching from a bilate- ral perspective and aim to find a more comprehensive evaluation model for distributed meeting systems through qualitative and quantitative research. In the process of data processing, we have the following findings.

First, the meeting system with the highest satisfaction level is not necessa- rily the highest in user requirement matching. According to the preliminary scores after data collection, the highest to lowest order is Ding Talk (7.419), ZOOM (7.220), and Tencent Meeting (6.965). After the GRA and the CCA, the ranking of the three meeting systems was switched, and the highest match- ing degree was Tencent Meeting. Although the functions of Ding Talk are powerful, the overly elaborate design in academic meetings will instead make users feel confused or bewildered. Moreover, from the perspective of product positioning, Ding Talk is more suitable for internal corporate communica- tion. As a local meeting system, Tencent Meeting is similar to ZOOM in terms of presentation effect, technical performance, or functional experience, but it is better than the other in terms of user scale, promotion and dissemination, and economic burden.

Second, in academic conferences, users are still more concerned about indi- cators such as sound quality, controllability, immediate visibility of feedback, especially network signal and system reliability. Although the interview- ees have mentioned several times in the interviews about the multimodal participation experience or the need for authenticity and presence in the con- ference, we were able to find low scores in the follow-up questionnaire for the indicators of fun (6.773), multimodal participation experience (7.110), and authenticity (7.221).

Finally, from the interviews, we uncovered some neglected details. Some respondents would like a more interactive sense of scholarly communication, e.g., politely interrupting each other and joining in heated discussions more comfortably. Other respondents mentioned that long listening to the lectures at the academic conference might lead to inattentiveness and the possibility of having real-time content captioning. Although the matching metric model is a more objective and intuitive way to evaluate, talking directly with users can help us to obtain more in-depth value.

CONCLUSION

In this paper, a quantitative model of user demand matching is constru- cted by collecting the product satisfaction and demand importance of each distributed meeting system and combining the GRA and the CCA. Moreo- ver, taking academic conferences as an example, three mainstream meeting systems at home and abroad are calculated and evaluated, and a primary rea- son analysis is conducted. The study’s conclusion shows that in the evaluation method of distributed conference systems, the degree of matching between product supply capability and user demand needs to be considered, and the

User Requirement Matching Model for Distributed Conferencing Systems 101

evaluation results from a unilateral perspective are not entirely accurate. The construction of the model helps decision-makers clarify the matching relati- onship between the product supply side and the demand side and provides development directions for the iteration of distributed meeting systems.

Although most of the research in this study adopts the method of targe- ted questioning and targeted questionnaire distribution, there are still many shortcomings and areas for improvement due to the realistic conditions. For example, increasing the sample size and diversity including combined hardw- are and software conferencing systems). At the same time, the categories and classification of evaluation indicators are still open to discussion if hardw- are and software combined conferencing systems are involved. In addition, the follow-up study should also consider the bias effects of factors such as geographic scope, user size, and the number of measured samples on the results.

ACKNOWLEDGEMENTS

Thanks to everyone who participated in this study.

REFERENCES

Berndtsson, G. Folkesson, M. and Kulyk, V. (2012). Subjective quality assessment of video conferences and telemeetings. in 2012 19th International Packet Video Workshop (PV) (pp.25–30). IEEE. Chen, S. L. Guo, K. X. Gong, K. and Ouyang, T. H. (2021). Research on the

Matching Mechanism between Chinese Traditional Cultural and Creative Pro- ducts and Users--Taking Yuxian Ceramics as an Example. foreign Economics and Management (07), 141–152. Chi, D. X. (2020). Research on Matching Spiritual Service Needs of the Elderly in Urban Community. https://kns.cnki.net/KCMS/detail/detail.aspx?dbname= CMFD202101&filename=1020312052.nh Du, B. S. (2018). Research and Application of Comprehensive Evaluation Method for

Video Conferencing System Operation Quality. https://kns.cnki.net/KCMS/detail/ detail.aspx?dbname=CMFD201802&filename=1018116909.nh Fu, M. L. and Li, G. (2021). Research on Matching Job Seekers’ Needs and Enterprises’ Demands Based on Big Data Analysis of Recruiting Websites. statistics and Management (08), 91–97. Gale, D. and Shapley, L. S. (1962). College admissions and the stability of marriage.

The American Mathematical Monthly, 69(1), 9–15. Hao, H. Y. (2016). Based on gray relational analysis of my country’s conference market demand characteristics. Journal of Urumqi Vocational University (02), 51– 54. Khalid, M. S. and Hossan, M. I. (2016). Usability evaluation of a video confere-

ncing system in a university’s classroom. In 2016 19th International Conference on Computer and Information Technology (ICCIT) (pp.184–190). IEEE. Kong, Q. Y. Hou, J. Zhang, H. Z. Yang, Y. and Wang, Y. S. (2019). Evaluation of desktop paperless meeting system effectiveness based on fuzzy theory. Information Communication (06), 283-284. LeBlanc, L. A. and Nosik, M. R. (2019). Planning and leading effective meetings.

Behavior Analysis in Practice, 12(3), 696–708.

Li, W. (2014). Research on Performance Management of Video Conference System.

telecommunication Engineering Technology and Standardization (07), 56–58. Liao, C. B. (1999). Quantitative evaluation and classification system of the coordinated development of environment and economy--Take the Tropical Geography (02). Liu, S. F. Dang, Y. G. Fang, Z. G. Xie, N. M. Zhang, K. Zeng, B. and Lin, Y. (2012). Gray System Theory and Its Application (Fifth Edition). 2023-01-30, https://kns.cnki.net/KCMS/detail/detail.aspx?dbname= SNAD&filename=SNAD000001462376. Standaert, W. Muylle, S. and Basu, A. (2021). Understanding the importance

of meeting mode capabilities for different meeting objectives. information & management, 58(1), 103393. Townsend, A. M. Demarie, S. M. and Hendrickson, A. R. (2001). Desktop video conferencing in virtual workgroups: anticipation, system evaluation and performance. Information Systems Journal, 11(3), 213–227. Vucic, D. and Skorin-Kapov, L. (2015). The impact of mobile device factors on

QoE for multi-party video conferencing via WebRTC. In 2015 13th international conference on telecommunications (Contel) (pp. 1–8). IEEE. Wang, Z. Deng, R. and Jiang, Q. L. (2022). Research on Factors of Continuous Use of Fitness APP Based on User Perception. packaging Engineering (20), 202– 211+255. Xiao, X. (2021). Beware of “Zoom Fatigue”. Procuratorial Situation (10), 86–87. Yankelovich, N. Walker, W. Roberts, P. Wessler, M. Kaplan, J. and Provino, J. (2004).

Meeting central: making distributed meetings more effective. in Proceedings of the 2004 ACM conference on Computer supported cooperative work (pp. 419–428). Zhang, J. T. Dong, Y. C. and Fang, B. (2020). Coupling Analysis of Supply and Demand of Cultivated Land Reserve Resources Based on Gray Relational Method. regional Research and Development (01), Regional Research and Development (01), 163–167+174. Zhang, N. (2018). Research and Construction of Ability Matching Model for the

Elderly in Product Innovation Design. https://kns.cnki.net/KCMS/detail/detail. aspx?dbname=CDFDLAST2020&filename=1020022230.nh Zhang, N. and Wang, Q. Y. (2022). Elderly Icon Design Based on Ability-Needs Matching Model. Packaging Engineering (06), 103–109+133. Zhou, C. Feng, W. D. Zhou, Z. Tian, X. and Tian, J. W. (2013). Research on Audio

and Video Quality Evaluation System of Video Conference System. television Technology (08), 80–85.

Cover of the proceedings volume Ergonomics In Design

Explore the complete digital volume

Ergonomics In Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-53-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →