Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Industry and Academia Collaborative Learning: The CSULB and ISSIP AI Collab Pilot Program

California State University Long Beach, Long Beach, CA 90804, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1005101

Abstract

This paper is an analysis of the case study of an industry-academia collaboration between the International Society of Service Innovation Professionals Academic/Industry Collaboration (ISSIP AI COLLAB) and a senior UX design course at California State University, Long Beach. The ISSIP AI Collab Pilot Program was launched in Fall 2023 with four participating academic partners; San Jose State University, California State University, Long Beach, Pennsylvania State University, and the University of Washington. The pilot program included both graduate and undergraduate participants from programs ranging from service systems engineering to design. The ISSIP AI COLLAB program is designed to engage ISSIP institutional members with faculty and students from leading educational institutions in value co-creation and exploration of the capabilities of generative AI in an analysis of historical service systems. This case study discusses the design of an academia and industry collaboration course including teaching strategies, course management, project scope, and deliverables from the CSULB and ISSIP AI COLLAB project, “Harms and Benefits Analysis of Historical Service Systems Innovations Using Generative AI.” (https://issip.org/ai-collab-offering-participating-universities/). This case study discusses the course description, project definitions, scope, and evaluation strategy of the ISSIP AI COLLAB Pilot Program.

INTRODUCTION

ISSIP is a professional organization that promotes and recognizes service innovations, with a special emphasis on the development of T-shaped adap- tive innovations with a breadth communications knowledge and a depth for problem solving that improving learning as it contributes to the area of service innovations (Sphorer et al., 2022; Demirkan et al., 2015). As stated on their website, the goal of the ISSIP AI COLLAB program is to work with academic institutions to engage undergraduate and graduate stu- dents and their faculty with innovative research and content in industry relevant priority focus areas using a documented process, findings, how- to ‘playbooks’ for replication, building brand eminence via ISSIP platform (https://issip.org/ai-collab-offering/). ISSIP offers digital credentials to record key events and outcomes in the areas of knowledge sharing eminence and

© 2024. Published by AHFE Open Access. All rights reserved. 221

222 Satterfield and Chang

upskilling to students and professionals in the form of badges that can be added to online professional bios. ISSIP promotes “Systems thinkers who understand both the benefits and harms of innovations (e.g., benefits may include improved productivity, quality, compliance, sustainability for some, and harms may include growing inequity, exclusion, decrease in diversity, stakeholder losers-and-winnners so not win-win, etc)”.

High Impact Practices Education-related data has been used to predict a variety of critical student outcomes including performance, success, satisfaction, and dropout rates. The bulk of research in this area has focused on grade point average (GPA) as one narrow measure of academic success (Alyahyan et al., 2020). Post- admission factors that influence student success have been found to include external factors, institutional factors, and individual factors including affinity with their studies and their peers (Fischer et al., 2022). However, high- impact practices (HIPs) are another metric that has been linked to academic success in university students. HIPs are often incorporated into the fresh- man experience at universities as a way to lower first-year student dropout rates. HIP participation is well documented in its ability to enhance student engagement, however, freshman participation in HIPs does not predict reten- tion in subsequent years. According to Provencher and Kassel, embedding HIPs in the curriculum with an emphasis on freshman and sophomore stu- dents can ensure greater participation. A well-designed HIP incorporates a learning environment that allows for career exploration, social integration, and identity exploration. Therefore, HIPs may offer students a focus on social integration and identity exploration beyond what is found in tradi- tional internships. (Provencher & Kassel, 2019). In addition, upper division and transfer students also experience more positive outcomes in terms of student retention and academic achievement as a result of participating in HIPs that incorporate strategies such as collaborative learning. This includes project-based student collaboration and peer support (Thomas et al., 2021).

Industry Collaborations and Student Success Industry collaborations have been identified as contributing to student suc- cess. Higher education is concerned with both student retention and student engagement. The latter of which is the ability of the university to ensure that students experience a deep and relevant educational experience. Stu- dent retention and student engagement together are closely related and can be thought of as a success framework (Tight, 2019). Industry collaborations have been shown to encourage student engagement. Students that are actively engaged in problem-solving, teamwork, and learning clusters have shown greater integration in their learning. This has also been linked to a more effective strategy for students from diverse backgrounds by allowing them to bring their varying educational, cultural, and language backgrounds into the learning experience (Crosling, 2017). In addition, industry experiences

Industry and Academia Collaborative Learning: The CSULB and ISSIP AI COLLAB 223

that are embedded into a classroom and engage students directly with indus- try professionals can effectively give a virtual internship type of collaboration (Hurley et al., 2022).

DISCUSSION

The AI COLLAB pilot course was introduced at a large urban public uni- versity with a highly diverse student body including a high percentage of first generation college students and students from underrepresented minor- ity groups. The course was an undergraduate upper division design elective in User Experience Design (UXD) research. The AI COLLAB project incor- porated a series of HIPs as part of the student engagement. The project incorporated a virtual internship aspect where students worked directly with ISSIP professionals to give guidance during the projects and conduct mid and final reviews. In addition, members of the ISSIP Executive Leadership Team gave professional talks and engaged in student-conducted interviews. Three industry-focused projects were given over approximately 10 weeks during the 15 week semester as part of the AI COLLAB pilot project.

Project One: ISSIP Member Survey Analysis The three projects for the class were: 1. Data analysis of the ISSIP member sur- vey, 2. Content analysis, benchmarking, and recommendations for the ISSIP website redesign, and 3. Content creation and analysis of historic service systems and product innovations using generative AI tools to research and document these service systems and identify the harms and benefits of those systems regarding social, emotional, physical, and environmental impacts. Each of the projects was designed to include interactions with the ISSIP indus- try partners, data collection and analysis, and a series of professional reviews. The first project was conducted over 3 weeks and included the following industry-aligned HIPs:

  • Obtaining CITI human subject research certification;
  • Outlining the book, “Service in the AI Era: Science, Logic, and Architec- ture Perspectives;”
  • Conducting Interviews with two ISSIP industry professionals; and
  • Analyzing, scrubbing, and coding the ISSIP Member Survey and preparing executive summaries.

Five student teams of 3–4 members each coded the ISSIP member survey raw data. The survey question included 51 respondents to the open-ended question, “Do you have a "Tech for Good" position statement to share?” Of the responses, 21 were coded as null sets. Of the remaining open-ended responses, the students were asked to code them according to positive, neg- ative, or neutral and group them according to codes developed from a grounded theory approach. The scrubbed and coded data was then presented according to the codes developed by each team in a final format that was pre- sented to the ISSIP industry professionals. As part of this final presentation, the students were required to write the following in the form of an executive summary:

224 Satterfield and Chang

Data Summary - List the codes (Categories) and a brief statement of the category and what it indicates or the common themes in the categories. For each category list the number of respondents n = 51 Overall Summary Statement of Tech for Good Position Statements - Give an overall summary that contextualizes the categories in terms of most to least represented and any linkages or relationships between categories. You can quote a specific answer where needed for each part of your discussion to add specific data to the general discussion. You can also link this to other parts of the survey and show how it relates to those answers, You can also relate this to any of the interviews that we have had or to the book we read.

Example Data Executive Summaries Team 1 - The survey analysis, encompassing 41 responses, offers a nuanced perspective on the impact of AI and innovation on human society. It reveals a balanced split in respondents’ sentiments, with 18 respondents highlight- ing the benefits of AI and 13 expressing concerns about the harms. Beyond this binary division, the analysis unveiled multifaceted themes which encom- passed the technological, social, medical, and environmental dimensions. This ultimately aided in underscoring the intricate relationship between AI, innovation, and our lives. These findings reflect a diverse range of feelings and perspectives among the respondents, mirroring the complex and evolving nature of society’s relationship with AI and innovation.

Team 2 - Overall, many respondents had differing ideas of what they were wanting to propose as their position statement centering around Tech for Good. These responses varied from tech promoting unity, cautionary steps that will be needed to be taken for the good of humanity, how tech can benefit humanity, and more questions about tech for good that have come up within the survey. In conclusion, although all of these responses did have differing opinions, they are all important to look into and understand for the future of tech and humanity.

Project Two: ISSIP Website Content Analysis and Recommendations The goal for Project 2 was to do a UX analysis of the content, messages, and usability of the ISSIP organization website (issip.org.) During Project 2, industry professionals in UX came into the class as guest speakers. They pro- vided a framework of three sprints for this part of the course. The sprints were aligned with UX industry standards, and the students were provided with examples of how to conduct the research and how to prepare their find- ings into client presentations based on the expectations in the UX field. The industry-aligned HIPs for this project included client interviews, development of a timeline for the project, identifying organizations for benchmarking, creating user stories and user flow diagrams, and presenting 1–3 recommen- dations for improving the ISSIP user web experience. The following is the list of activities and deliverables and examples of student-produced client presentations:

Industry and Academia Collaborative Learning: The CSULB and ISSIP AI COLLAB 225

UX Portfolio Case Study Client Presentation Template

1. Project Title & Subtitle (A headline and subtitle that indicates the name and goal of the project) 2. Client Project Summary (An overview that summarizes the ISSIP website and a series of your scavenger hunt data slides) 3. Project Timeline/Gantt Chart of the Sprints (A timeline that shows each phase of our project and what was developed during that phase) 4. Interview with Jim Spohrer and Deb Stokes (An overview that summarizes the interviews) 5. Benchmarking Data (What specific problem, user needs, business require- ments and/or pain points were identified.) 6. Survey Data Charts and Summaries (An overview that summarizes the ISSIP surveys) 7. Solutions and Supportive Data (What method/process were used to solve specific problem, user needs, business requirements and/or pain points? How did features address the objectives?) 8. User Stories (Align the Personas with the "why" for each of your proposed solutions) 9. Flow Diagram of ISSIP Website (An overview shows the structure of the website) 10. Conclusion Slide (A slide with final questions for client)

Project Three: Service Systems Harms and Benefits Analysis The goal Project 3 was to analyze the use of generative AI tools to create informative content for the ISSIP website. Projects 2 and 3 were conducted in conjunction with ISSIP and UX industry professionals and were intended to give students the experience of working in teams on research and presen- tations as a virtual internship experience. The HIPs in Project 3 included creating a Gantt chart for their schedule, writing content, creating and editing a business presentation, and making a presentation to our industry professionals. The following class timeline was used to simulate industry “sprints.”

ISSIP AI COLLAB Harms and Benefits Project Timeline

1. Timeline - Create schedules and deadlines and track Your progress using a Gantt chart format. 2. Write - write and create content for the report including an introduction of the problem, an explanation of the research, and 2–3 conclusions or recommendations. SPRINT 1 -Weeks 1–3 3. Edit - check the report for consistency in the text and proof the text for errors. Ensure all sources are reliable and fact checked. Make sure all links work and that the quotes are accurate and sourced properly. SPRINT 2 Weeks 3–4 4. Design - Create the final layout of the report and follow a style guide. Select a font, color palette, and set up the page layout. Proof the book and deck for visual consistency and adherence to type styles, point sizes,

226 Satterfield and Chang

and margin consistency, and create a compelling title page and a layout for the slides and pages. SPRINT 3 Weeks 4–6 5. Present - Each person will create a final report, use one quote to support their work, and create a visual or infographic in the form of a pie chart, an explanatory infographic, a timeline, a visual comparison, or an illustra- tion of a process or a diagram. The page layout will be done in InDesign and all infographics will be created in Adobe Illustrator. The final book should be prepared as an 8.5x11 inch book with photos at 150 dpi and use Pantone-coated colors for the color system. The final pdf of the book and your deck will be submitted on Canvas and on Miro. Week 6

Project Three: Final Deliverables The students were asked to assess the AI tools regarding their ability to pro- duce reliable content and to evaluate the relative strengths and weaknesses of each tool regarding potential bias or misinformation. The project’s three deliverables included:

  • Develop the content for a service system and technology using GenAI to write, document, and illustrate the service system both historically and with future harms and benefits outlined.
  • Develop 3–4 cases of historic examples of service innovations in a specific category (i.e., transportation, medicine, communications, etc.) Each case will consist of content created by a variety of generative AI tools.
  • Format content and strategies to the needs of the ISSIP audiences in terms of members, corporations, students, and volunteers.
  • Reflect on the AI tools used regarding bias and quality of responses.

Course Evaluation This pilot course used a university evaluation program called the Faculty For- mative Feedback Project (FFFP) to allow the instructor to receive immediate and constructive feedback for improvement during the first pilot semester. FFFP was developed in 2020 at CSULB to help instructors improve student engagement by providing trained observations of instruction, gathering of student engagement survey data, and providing a mechanism for consultative discussions about instruction.

FFFP Process A subset of literature focuses specifically on the use of consultation to aug- ment student evaluations of teaching (SET) or interpret formative feedback data. We distinguish formative feedback as data collected during instruction from students or colleagues for the primary purpose of making instructional change; this is not a summative evaluation of perceived faculty effectiveness. Past meta-analytic work has suggested collecting formative feedback data mid-semester (or mid-instruction) could be useful if instructional change is possible and that consultation might support how faculty interpret feedback data to improve teaching (Cohen, 1981).

Industry and Academia Collaborative Learning: The CSULB and ISSIP AI COLLAB 227

The Student Engagement Survey was developed based on previous work that considered engagement as multifaceted and including cognitive, affec- tive, and behavioral components (Handelsman, et al, 2005). This conceptu- alization of engagement goes beyond merely examining whether students are turning in accurate assignments or posting on discussion boards, it aims to also include how students feel about a class or area of study. This emotional component is associated with connecting material to one’s own experience and the ability to apply knowledge gained in other contexts. The Online Student Engagement Scale (Dixson, 2015) is a tool with Likert-style ques- tions that considers students’ self-report of their own: skills, participation, performance, and emotions about a specific class. FFFP leadership adapted the OpenScienceEd (OSE) and created questions to address student percep- tions of course design and teaching, questions linked to culturally responsive and sustaining practices, and two open-ended, narrative questions to address what is and is not helpful about the class. The Student Engagement Survey is anonymous, and it is administered digitally. It takes an average of 4 minutes for students to complete.

A protocol was created to provide one mutually arranged observation, done virtually or in-person, of classroom teaching to support those who want further feedback. The Instructor Observation Tool (IOT) centers around six qualitative domains that were adapted from the COPUS observation protocol (Smith et al., 2013) and features of classroom culture that support equitable sensemaking and culturally responsive and sustaining practices (OpenSciEd, 2019.) In the FFFP evaluation, each participating faculty is paired with a trained faculty Partner from outside of their discipline and asked to critically exam- ine their teaching philosophies and learning objectives. The process of which a faculty receives feedback is guided by FFFP’s own Student Engagement Survey and Instructor Observation Tool. The first meeting begins with a rapport-building and a goal-setting activity and is followed by a debriefing of the collected data within the context of the course and objectives. FFFP provides the instructor of this course a mechanism to iteratively revise the curriculum based on formative student feedback. The FFFP FACILITATOR asks the following reflection questions of the course faculty before and survey data debriefing questions after the survey results are in: Getting to Know You and Your Class

  • Tell me about yourself/How are your classes going so far?
  • Tell me a little about your teaching philosophy.
  • What are the courses you are examining and what are the student demographics in those courses?
  • What are you most interested in learning about student engagement?
  • What is one goal you really hope to accomplish from participating?

Survey Data Debrief

  • What patterns do you notice in the data?
  • What are the strengths and areas of need?

228 Satterfield and Chang

  • Do high/low levels of engagement match how you’ve set up the course?
  • What do you glean from open-ended questions?
  • What changes can/will you make based on the results?

Evaluation of Teaching and Learning The FFFP assessment process was conducted during the semester to give feedback to the instructor regarding student response to the course. Typi- cal assessments are conducted at the end of the course and data from the assessment is provided to the instructor weeks or months after the course has been completed. By using an external assessment conducted by a faculty men- tor, the course could be improved in a very dynamic way giving the professor more control over the outcomes and the opportunity to enhance the student learning experience. For this course, student feedback (N = 12) DATA was collected on 11/7-14/2023 during weeks 12 and 13 of the semester after the completion of the ISSIP projects but before the end of the course. See Table 1.

Table 1. Student FFFP survey responses.

Table available in the original PDF paper.

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

1 The assignments are material for personal portfolios, real-world scenarios, and research, treating the class like a mini internship! 2 The ability to learn from professionals in the field. 3 The attitude of my professor is the most helpful in this class. She is able to keep a positive attitude when she lectures, and lets her students be comfortable with her. It is really easy to ask for help as she is very encouraging. 4 Being able to use Miro and collaborate with others in breakout rooms. And all of the projects we have been doing have been very interesting as well as applicable to developing stronger skills and more experience. 5 The format of the weekly assignments. It makes organizing work very easy. Also the use of Miro which makes collaboration easy as well. 6 The overall attitude Professor Debra has towards the course, she is very approachable and helpful. 7 Instructor’s patience and understanding. 8 The professor’s engagement and her desire to teach us the material so we can learn it no matter how long it takes us and apply it to our lives. 9 Learning to speak with professionals and to talk about my work clearly.

Industry and Academia Collaborative Learning: The CSULB and ISSIP AI COLLAB 229

CONCLUSION

The ISSIP AI COLLAB course successfully engaged all students in a virtual internship experience by engaging with professionals from ISSIP as project leaders and mentors. Students were exposed to industry professionals in UX who were able to guide them on how to conduct research and present find- ings according to professional guidelines. The students were exposed to HIPs in the form of conducting interviews with industry professionals as stake- holders, conducting primary research, cleaning and coding data from an ISSIP membership survey, and presenting data-driven recommendations to ISSIP professionals in the role of clients. In the FFFP qualitative evaluation students mentioned “learning from professionals in the field, real-world sce- narios, mini-internship experience, and collaboration” as important to their learning in this class.

The case study of the ISSIP AI COLLAB pilot project allowed for the devel- opment of three unique real-world projects for the ISSIP organization. It allowed both ISSIP and the faculty in this course to better understand how to work together and develop best practices for future collaborations. Students were exposed to a variety of professionals and gained confidence in working with them as mentors. Students were given badges from ISSIP to display on their LinkedIn profiles. The collaboration successfully transformed the class into a virtual internship for all of the students and motivated them to be highly engaged with the content and to learn important professional network skills.

ACKNOWLEDGMENT

The authors would like to acknowledge Jim Sphorer, ISSIP Co-Founder and Executive Leadership Team Member; Deb Stokes, ISSIP President and Execu- tive Leadership Team 2024 Member; and the Michele Carrol, ISSIP Executive Director and President of Carrollco Marketing Services.

REFERENCES

Alyahyan, Eyman, and Dilek Dü¸stegör. (2020). “Predicting academic success in higher education: literature review and best practices.” International Journal of Educational Technology in Higher Education 17.3. Cohen, P. A. (1981). Student Ratings of Instruction and Student Achievement: A

Meta- analysis of Multisection Validity Studies. Review of Educational Research, 51 (3), 281–309. doi: 10.3102/00346543051003281. Demirkan, H., & Spohrer, J. C. (2015). T-Shaped Innovators: Identifying the Right Talent to Support Service Innovation. Research-Technology Management, 58, 12–15. Dixson, M. D. (2015). Measuring Student Engagement in the Online Course: The

Online Student Engagement Scale (OSE). Journal of Asynchronous Learning Networks, 19. doi: 10.24059/olj.v19i4.561. Fischer, Christian, et al. (2020). “Increasing Success in Higher Education: The Relationships of Online Course Taking With College Completion and Time-to- Degree.” Educational Evaluation and Policy Analysis 44.3.

230 Satterfield and Chang

Handelsman, M. M., Briggs, W. L., Sullivan, N., & Towler, A. (2005). A Measure of College Student Course Engagement. The Journal of Educational Research, 98 (3), 184–191. doi: 10.3200/JOER.98.3.184–192. Hurley, J., Raddatz, N., Satterfield, D. (2022). Bridging the Gap Between Industry

and Education: Engaging Design Professionals in the Education of Student Design- ers. In: Christine Leitner, Walter Ganz, Clara Bassano and Debra Satterfield (eds) The Human Side of Service Engineering. AHFE (2022) International Conference. AHFE Open Access, vol. 62. AHFE International, USA. http://doi.org/10.54941/ ahfe1002537. OpenSciEd (2019). Features of a classroom culture that support equitable sensemaking. Retrieved from: https://www.openscied.org/wp- content/uploads/2019/07/Copy-of-Handout-Features-of- Classroom-Culture- OpenSciEd-2.pdf. Provencher, Ashley, and Ruth Kassel. (2019). “High-Impact Practices and Sophomore Retention: Examining the Effects of Selection Bias.” Journal of College Student Retention: Research, Theory & Practice 21.2. Rossoni, André Luis, Eduardo Pinheiro Gondim de Vasconcellos, and Renata Luiza

de Castilho Rossoni. (2023). “Barriers and facilitators of university-industry collaboration for research, development and innovation: A systematic review.” Management Review Quarterly. Smith, M. K., Jones, F. H. M., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize University STEM Classroom Practices. CBE—Life Sciences Education, 12 (4), 618–627. doi: 10.1187/cbe.13–08-0154. Spohrer, J., Maglio, P. P., Vargo, S. L., & Warg, M. (2022). Service in the AI Era:

Science, Logic, and Architecture Perspectives. Business Expert Press. Tight, Malcolm. (2020). “Student Retention and Engagement in Higher Education.”

Journal of Further and Higher Education 44.5 (2020): 689–704. Thomas, D. T., Walsh, E. T., Torr, B. M., Alvarez, A. S., & Malagon, M. C.

(2021). Incorporating High-Impact Practices for Retention: A Learning Commu- nity Model for Transfer Students. Journal of College Student Retention: Research, Theory & Practice, 23 (2). doi: 10.1177/1521025118813618.

Cover of the proceedings volume The Human Side of Service Engineering

Explore the complete digital volume

The Human Side of Service Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-19-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →