Explore a scientifically compound talent evaluation system which is suitable for the needs of Industry-University-Research Collaborative innovation, to better serve the collaborative innovation activities of various industries and fields. This research upgrading and innovating a method for constructing talent evaluation system based on Industry-University-Research collaborative innovation needs, design and reconstruction the university laboratory talent evaluation system under the background of "New Engineering Disciplines" with this method. This paper demonstrates how to construct a new user group for talent evaluation system, export and transfer their requirements. This paper design and reconstruction of evaluation system, mining of evaluation indicators, visualization of evaluation results, and design of how to use evaluation results. This research case outputs a modularized combination of University Laboratory talent evaluation system, which consists of 5 major modules, 22 secondary indicators and 95 tertiary indicators. The feasibility, validity and innovation of the results of this study were verified by expert review.
With the continuous development and expansion (Wang Yun, 2021) of the participants in the IUR collaborative innovation institutes, the needs for interdisciplinary talents in IUR collaborative activities continues to increase. However, at the current stage of China IUR collaborative activities, there are certain knowledge transfer obstacles in the collaborative activities of talents in different fields of the team. On the one hand, it is because the methods and paths of design collaborative activities still need to be optimized, and on the other hand, the talent cultivation, selection, and evaluation system based on the needs of collaborative innovation activities in the IUR collaborative external cycle has yet to be perfected, and the “use-oriented” system has yet to be improved. This research takes compound talents as the evaluation sub- ject, explores the design method of building a Compound Talent Evaluation System (CTES) suitable for IUR collaborative units, and guides IUR collabo- rative units to build a customized CTES according to their actual needs, so that the talents are suitable for the job.
Figure 1: The linkage mechanism of the talent evaluation three system.
DESIGN METHOD OF IUR COLLABORATIVE INNOVATION CTES Definition of Talent Evaluation system
The Talent Evaluation System (TES) generally refers to a dynamic system of talent development and management consisting of three systems: target system, organization system, and method technology system (Hou Zifang, 2006). The TES consists more than two elements, including evaluation objectives, evaluation subjects and objects, evaluation methods, evaluation contents, evaluation norms, and evaluation results ect. The elements are organically interconnected and interact with each other, based on a systematic, scientific and standardized evaluation process (see Figure 1).
Design Method of IUR Construction Collaborative innovation CTES This study proposes a composite CTES for the IUR collaborative innovation needs which should be an evaluation system with modular characteristics and can be used in multiple and flexible combinations. To optimize the design of multi-user group framework with the organizational system, the method technology system should be combined with new technologies, new designs, and new services in the industry 4.0 design era for design deepening.
Research on the Target System of the Multi-User Group Optimized x Definition of multiple user group: The user groups can be divided into three categories according to macro groups, meso groups and micro groups. The macro-level user is the state. The state uses talents but does not directly use the CTES, so the state is an indirect user group. The users at the meso-level are the industries and companies where talents are located, which belong to the direct users. At the micro level, the user of CTES is an individual, who belongs to the direct user group. According to the above research and analysis, this research proposes a user group framework of the CTES’ SICP-User Group, namely State-Industry-Company-Personal. y Export of usage needs of multiple user groups: User needs can be divided into explicit needs and implicit needs. In the multi-user group, the explicit needs of the state and the industry clearly communicated through policy
“Industry-University-Research” Collaborative 125
Table 1. User demand derivation table.
Table available in the original PDF paper.
User Group State Industry Company Personal User Chara- cteristics Macro & Indirect Users
Meso & Direct Users
Meso & Direct Users Talent evaluation and employment
Micro & Direct Users Pain Points & Demand Acquisition Path
National Relevant Policy & Literature Review
Industry standard & Industry Policy & Literature Review
policy & evaluation standard study &evaluation expert Talent evaluation micro-use function
Talent evaluation overall framework design
behavior Design goals of demand transforma- tion
design Design of talent evaluation
Talent evaluation macro-use function design Talent evaluation design principles
method Talent evaluation factor design
documents; the companies have explicit needs such as job title evaluation and performance appraisal, and there are also implicit needs to understand the composition and training status of the unit’s talent team; individuals have explicit needs such as job title evaluation and performance appraisal, as well as implicit needs such as evaluation of individual career development progress. Combined with the characteristics of users’ explicit and impli- cit needs, this study proposes the derivation process of different types of users’ explicit and implicit needs according to the research mechanism of “policy analysis &status quo investigation-problem formulation &pain point extraction-demand derivation-design transformation”(Table 1). z Evaluation goal design in the goal system: The state hopes the CTES has a guiding role in the development of talent cultivation; The industry hopes that the CTES can build a new format of the industry; the company hopes to recruit counterpart talents through the CTES and build the talent team needed by the unit; individuals hope that the CTES can reasonably evaluate individual work results and realize the development of talent career planning.
Research on the Organization System of the Multi-User Group Optimized CTES: Since 2016, the General Office of the State Council, the Ministry of Educa- tion and other departments of my country have proposed new policies such as implementing the peer review system and focusing on the actual social contribution of results in response to the dilemma of the “Five-Dimensions Only” evaluation indicators of TES in China, which has brought the revo- lution to evaluation factors design. The inverse framework of “collaborative innovation work content-result output-talents in related fields-result stan- dardization selection of appraisers” is proposed to deduce the candidates for evaluation agencies, and design the organizational structure of appraisers.
Research on the Method and Technology System of the CTES Under the New Technology, New Design and New Service in the Industrial 4.0 Design Era New technologies such as big data and cloud computing have brought earth- shaking changes to the production and operation modes of all different fields. Under the support of new technologies, new designs and new services, the CTES should provide users with a more concise way of visualizing evalua- tion results, and bring users a new service experience of the CTES. Talent evaluation theory, evaluation factor design method, evaluation factor weight assignment method and other design systems are relatively mature. According to the design requirements of the CTES based on the needs of IUR collabora- tive innovation, research and select career theory, competency-based theory and competency model (Hao Zhaojie, 2017) as talent evaluation basic the- ory of system design. The evaluation factor design adopts the combination of Behavioral Event Interview (BEI), Literature Research Method, Analytic Hierarchy Process (AHP) and Delphi expert evaluation method (Wang Ling, 2021).
METHOD APPLICATION The New Engineering Construction (NEC) is one of China’s national-level IUR collaborative innovation strategies. University laboratories play an important window platform for IUR collaborative innovation activities. The universities laboratory talents (ULTs) group is compound talents. Based on that, this paper conducts research on the design of the evaluation index system in the ULTs.
Sample Selection and Study Conduct Based on the SICP-UG user group framework, the research focuses on the user group of ULTs, and proposes a user group of “state-teaching industry- university laboratory-university laboratory talents”. The user group research is using the literature research method combined with the BEI method (Table 2).
Definition of University Laboratory Talents This study proposes that the ULTs under the needs of IUR collaborative inno- vation refer to: working in university laboratory institutions, engaged in daily laboratory management, experimental teaching, development, management and maintenance of experimental instruments and equipment, technical sup- port for scientific research projects, and collaboration in IUR collaborative innovation projects. The ULTs under the needs of IUR collaborative innova- tion need to have the following connotative characteristics: x Good teacher morality; y Good political quality; z Have a solid technical practice experi- ence; { Have strong business ability; | Have an open spirit and innovation spirit and collaboration.
“Industry-University-Research” Collaborative 127
Table 2. Sample selection list.
Table available in the original PDF paper.
Research keywords Final document
User Research method The State Literature Research
NEC, Collaborative Innovation, TES, IUR
5 Authoritative Chinese Policies (The Central Committee of the CPC and the State Council,2020) Teacher Industry
Literature Research
Teacher evaluation, education evaluation, university laboratory
3 industry authoritative policies & 60+ research papers related to TES University Laboratory
Literature Research & BEI
University laboratory, talent evaluation, teacher evaluation, competency model
10 high-quality ULTs evaluation and employment policy docu- ments & A university’s 2016-2020 ULTs file records Talent BEI Problems in the evaluation process
5 ULTs interview records & recordings
Research on the Current Situation of ULTs The research team used a University in Hunan Province as a research plat- form to collect ULTs information in the school from 2016 to 2020. 982 pieces of available information were obtained from the department to sort out the data, each of the information including age/ professional and technical qua- lification level/ length of service etc. Combined with the pain points of talent evaluation obtained by the research group’s formal interviews with talents, this paper summarizes the four main dilemmas of the UTLs: x The evalu- ation content list is not really matching most of their daily work. y Most of those talents are facing more than 2 different types of work z The are less long-term evaluation to track talents career growth. { Talent evaluation results are displayed in a single way, and there are not enough results analysis.
Evaluate the Derivation of Usage Requirements x Make the talent appropriates the positions. Carry out talent evaluation around the job content and responsibilities of UTLs, use suitable talents in suitable positions; y Use talents at the right time. Through reasonable evalu- ation, managers can be assisted in arranging appropriate jobs at appropriate stages, and efficiently and with high quality to give full play to the value of talent capabilities of UTLs; zMake the best use of the talents. Hierarch- ical and classified talent evaluation perspective, multi-dimensional view on the possibility and growth potential of ULTs, and optimize the allocation of human resources; { The evaluation could lead or suggest the talent “full life cycle” career planning. Periodically and continuously evaluate talents, so that the evaluation data in different time periods are connected into a line, the evaluation results are “sustainable development”, and the growth trend of talent ability is visualized.
Figure 2: Framework for the ULTs system based on 5 ability.
DESIGN PRACTICE Framework Design of the ULTs System for University
The development direction of colleges and universities under the needs of IUR collaborative innovation has put forward compound ability requirements for ULTs. Based on the investigation of the ULTs current situation and the sorting out of the work content, the research proposes a framework for the ULTs system based on the type of ability (Figure 2). The framework includes five types of abilities, namely personal accomplishment, teaching ability and research ability, management and service capabilities. Based on different evaluation purposes, different construction needs of colleges and universities, and diffe- rent job types of talent team responsibilities, the five major ability evaluation tables can be modularized and combined. Analysis the research on the com- petency factor index and the work content factor index of the ULTs’ and the teacher’s post, then count the frequency of the index, conduct APH analysis on the high-frequency factors, and further analysis the factor index through the Delphi Method, finally build five kinds of ULTs Capability index factor.
Factor Design of ULTs Personal Accomplishment: In the personal accomplishment factor design, the professional quality, moral quality, self-characteristics, interpersonal chara- cteristics and knowledge literacy of the ULTs (Figure 3). This competency system includes the evaluation of teachers’ morality and style, and the cor- responding national user requirements should be emphasized in the weight design. This competency system also includes the “Five-Dimensions Only” evaluation factors, which are also included in the comprehensive evalua- tion of personal capabilities; At the same time, this part of the proficiency assessment involves a large number of psychological fields, and a third-party assessment should be conducted in conjunction with professional institutions. Teaching Ability Factor Design: The Teaching ability factor design focuses on returning to the work of experimental talents in colleges and universities (Figure 4). Teaching ability mainly evaluates the main work of teachers to
“Industry-University-Research” Collaborative 129
Figure 3: Personal accomplishment.Figure 4: Teaching ability.Figure 5: Evaluation of scientific research ability.
students, such as the ability of lesson preparation, teaching ability, practical ability, sense of responsibility, answering ability, and guidance ability. Evaluation of Scientific Research Ability Factorial Design: The factor design of scientific research ability mainly focuses on the collaborative innovation ability of talents in industry (Figure 5). In this ability module, we jump out of the “Five-Dimentions Only” evaluation factors, and mainly evaluate the level of scientific research ability and the importance of achievements from the aspects of creating economic value, solving major social problems, and improving the quality of social production and life. Management Ability Factorial Design: At present, laboratory personnel management, asset management and other functions are scattered in various institutions, and it is difficult to coordinate laboratory construction planning. Therefore, in the scope of the ability of the experimental series of talents, we have added the management ability, to prepare for the detailed classification of the future experimental series of talents (Figure 6). Service Capability Factorial Design The service capability module mainly reflects the work generated during the collaborative innovation activities of industry-university-research institutes (Figure7).
Hierarchy A B C D E correspond percentile intermediate Beginner primary Evaluation meaning advanced Intermediate and advanced
Evaluation Results Calculation Method Design Because the evaluation rights are delegated to colleges and universities, colle- ges and universities can adjust the evaluation factors according to the actual situation, and calculate the evaluation weight according to the APH analysis method. This research has a five-level system (Table 3), and there is no descri- ption of negative ability in the evaluation results of the level system. The description of the ability level should reflect the step-by-step improvement process from the primary level to the advanced level of ability. Visualization Scheme Design of Evaluation Results Combined with the data visualization method of design, this research displays the results of talent evaluation graphically through the visualization of radar charts and percentage histograms, and conveys the hidden information of the talent evaluation results.The visual design can adjust the display method of evaluation results according to different evaluation objectives: Single-Person Evaluation Results Display →meet the distribution of personal short-term ability values (Figure 8); Single-Person Multiple Evaluation Results Display →meet the reference role of personal medium and long-term career planning ability growth (Figure 9);
Multi-Person Single Evaluation Results Display →Visually analyze the ove- rall planning of the talent team. The layout provides a reference for the construction of the university laboratory talent team (Figure 8); Multi-Person Multi-Evaluation Results Display →Visual evaluation of the comprehensive growth direction of the talent team (Figure 9);
CONCLUSION
The design plan of the talent evaluation system for the university laboratory talents was approved by the expert review meeting of the China Talent Deve- lopment Committee in Beijing, China in September 2021, which theoretically demonstrated the design method of the compound talent evaluation system under the needs of collaborative innovation activities of IUR institutes. In this study, the talent factors of university laboratory talents are mostly obtained from high-frequency words in the frontier research theory of collaborative innovation between industry, university and research in the laboratory, and are still based on theoretically established models. The research conducted a preliminary exploration of the visualization scheme for the evaluation of the whole life cycle of talents’ careers, and achieved certain results. The future research direction will be to consider and study the construction of a visualization platform for the cross-industry industry-university-research collaborative innovation talent evaluation system.
ACKNOWLEDGMENT
The authors would like to acknowledge all the helps and support from the research group of “Research on China’s Design and Manufacturing Collabo- rative Innovation Model”, the research group of “Study on Laboratory and Engineering Talent Evaluation Standards and Norms”. This research was funded by the Talent Development Professional Committee of China Soci- ety of Educational Development Strategy and the National Social Science Foundation of China’s Major Art Project “ Research on China’s Design and Manufacturing Collaborative Innovation Model”. Fund NO: 20ZD09.
REFERENCES
Hou Zifang. (2006) Research on Talent Evaluation System of my country’s Vocational Qualification System [Z]: National University of Defense Technology. Hao Zhaojie, Pan Lin. (2017) Research on the Construction of Teaching Com-
petency Model for College Teachers’ Flipped Classroom Teaching——Also and New Reflections on Teaching in the Background of “Artificial Intelligence +”[Z]: Journal of Distance Education, pp: 68-77. The Central Committee of the CPC and the State Council (October 13, 2020) “Overall Plan for Deepening the Reform of Education Evaluation in the New Era”. Wang Ling, Yin Yujia, et al. (2021) An empirical study on the evaluation system of
high-level medical talents based on Delphi method and AHP[Z]: Modern Hospital, pp: 72–75. Wang Yun, Zhu Jihong, ed. (2021).The Construction of Chinese Design Intellectual Value System from an International Perspective [Z]: Packaging Engineering, pp. 36–42. Website of the Central People’s Government of the People’s Republic of China: http:
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-35-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional