Elena Cavallin (2024). Exploring the Use of GenAI in the Design Process: A Workshop with Design Students. In: Artificial Intelligence and Social Computing. AHFE Open Access, vol. 122, pp. 89–95. https://doi.org/10.54941/ahfe1004645
The introduction of artificial intelligence into the design process is bringing fundamental changes. AI can be used to improve or even radically change the process of designing digital solutions (Agner et al., 2020). In particular, AI has been hailed for providing many important possibilities, including the possibility of greater customization at scale, more precise analysis of the use of digital solutions, and aiding the creative process of designers (Oh et al., 2018).For this reason, it is important to continue to explore how these technologies can be developed and integrated into the design process in ways that facilitate and improve the work of designers.It is argued that the introduction of new digital tools such as AI into the design process brings about major changes in the nature of designers' work (O'Donovan et al., 2015) and that these changes bring about radical changes in ways of working, as well as some potentially negative and unintended consequences (Gaffney, 2017). The main problem encountered is that the use of such new and complex technologies to date means that designers and end users may have limited understanding of the ways in which these technologies can influence the design and thus also its outcome, i.e., the product/service.This is not necessarily a skills issue, but also relates to the fact that artificial intelligence is still a black box that still does not provide comprehensive explanations of its decision-making process (Asatiani et al., 2020).Most of the studies analyzed believe that design should not be merely automated by AI, with complete removal of the designer. It is believed that it should be an AI-supported process, a co-design, and that this is more likely to be used to automate only the most repetitive aspects of the process. Although AI-designer collaboration requires designers to enrich their skills, and many implementations of the artificial partner, the research collected shows that AI-designer interaction achieves better results than design processes in which AI is absent or the process is fully automated. Given this background, this paper investigates some aspects of designers' behavior during co-design. It was noted that designers establish ways of relating to the artificial partner that vary accordingly to how they can understand (or rather, believe they understand) the black box, i.e., the information processing procedure of AI. For example, the design process is made smooth when the designer can find ways to communicate its intentions to the machine in a way that facilitates its response, even if the content is complex. Through a review of the literature and direct observation of workshops where young designers test and use AI, some valuable observations and insights have been clustered.
Throughout 2023, with the proliferation of generative AI platforms, design- ers have begun to experiment with these new tools and incorporate them into their design practices. The most utilized platforms during the design process are text-to-image ones, with Dall-e, Midjourney, Stablediffusion, and Lex- ica.art being the most popular for image creation (Banh & Strobel, 2023). The behaviour of designers towards these new technologies is a broad subject. This article aims to analyse the behaviour of design students to understand their approach, initial challenges, learning processes, and particularly how these platforms, especially MJ, are used by an audience that has a basic understanding of ChatGPT but is not yet considered AI experts, though they already possess foundational design skills. This study will examine MJ’s inter- face and the interaction established by designers through analyses conducted
during workshops and platform use. The MJ platform was chosen because it is one of the most widespread in its category and used by designers in the past three years. It has seen significant developments both in AI models and user interface, evidenced by the upgrades in platform releases and documen- tation. Furthermore, it has catalysed the creation of practitioner communities dedicated to exchanging information and strategies for refining prompts. Prompt engineering resembles a dialog with the text-to-image system. A practitioner typically runs a prompt, observes the outcome, and adapts the prompt to improve it. Thus, prompt engineering is iterative, and practition- ers formulate prompts as probes into the generative models’ latent space. The online community quickly discovered that the aesthetic qualities and subjec- tive attractiveness of images could be enhanced by adding specific keywords and phrases to the textual input prompts (Oppenlaender, 2023). It is impor- tant to note that generating images from textual descriptions is exceedingly complex because the image is, in its entirety, instantaneous and not didactic (Esposito, 2022). An image captures numerous details that to mention all would require disproportionately extensive writing. The disproportionality comes from the fact that when one draws, the result is immediately appar- ent to the observer. Engaging in dialogue with AI for co-designing can be an excellent tool as Elena Esposito explicates in Artificial Communication (2022), AI tools are excellent communicative partners. Communication dur- ing the design process is significant, as described by studies on the think-aloud method (Shealy et al., 2023; Wright & Monk, 1991), precisely because it is a mechanism that allows designers to articulate their thoughts and thereby cognitive processes (Cavallin, 2024).
Aim The objective is to highlight the aspects that can influence designers’ behav- iors when they engage with generative AI of the x-to-image type, which may either facilitate, hinder, or even obstruct their activities. This is significant as it could foster progress in solving user interface problems and in under- standing which elements are essential when working with generative AI. It will be demonstrated that even when graphical user interfaces (GUIs) are implemented, AI still mediates the design process.
STUDY The study is divided into two components. The first involves the analysis of the interface of Midjourney, one of the most used platforms by designers. This is followed by a report of analyses conducted from the direct observation of design students using this AI during a workshop.
Interface Analysis of Midjourney The Midjourney interface presents itself as a simple chat, indeed it is found on the Discord messaging platform. Here, the user has a field to type text. To activate the image generation command, one must type /image, which opens an additional box for typing the prompt. The user writes in natural language and receives four images that correlate and correspond to the entered text.
Exploring the Use of GenAI in the Design Process: A Workshop With Design Students 91
It is important to note that although the interaction is command-action based, the action is always mediated by AI, which will use generative data processing techniques to execute the command. Up to this point, the level of complexity is not particularly high. In fact, anyone can generate images of very high quality and complexity. The problem lies in managing the content of that image and, consequently, both the cognitive load of the designers and their relative frustration increase significantly. Indeed, if one wants to adjust the colors, shapes, positions - especially - (starting from the most basic adjustment of the image frame size), problems begin. Currently, there is also a feature for modifying a portion of an image that can be erased with an eraser - but this is reserved for pro versions only. Without the pro version, the designer must describe the content in detail, and to obtain satisfactory results, some basic knowledge of prompt engineering is required. In MJ, there are textual commands, Parameters (encoded expressions), that are used to achieve more reliable results, i.e., those that respond to if-then logic rather than to GenAI algorithms.
However, many iterations are needed to achieve an image that comes close to the desired outcome. This is because even the designers who are designing do not immediately know the precise result they want to achieve, but they approach it through iterations of problems and solutions (Cross, 2023).
Workshop Analysis and Observation The workshop took place from November 30 to December 1, primar- ily focusing on the use of MidJourney for design ideation and product communication.
Approximately 50 second-year bachelor students in industrial and multi- media design from the University of San Marino (UNIRSM Design) partici- pated in the workshop. During the early months of the course - October to November - they experimented with the use of Chat GPT 3.5 as a support for website design, thus acquiring a basic understanding of prompting.
The workshop was divided into an introductory segment, where various commonly used Generative AIs were presented along with a brief overview of their distinctive and shared features. This was followed by various exer- cises, introduced with a short brief and notes for better completion of the specific task. Subsequently, one of these tasks was chosen for analysis due to its targeting of a specific objective, which was not highly complex.
Task The requirements were: develop all components (logo, hero image, slogan); in this analysis, the focus will be solely on the logo. The title of the task was: “Making a logo for a tourism office website for the region/country of your colleague”. Ensure these match the ‘clients’ vision and are enticing enough to encourage visitation. Students were asked to describe the place before proceeding to creation via MJ.
Data and General Observations The analyses took into account the following factors:
Number of iterations
Timing
Text variation
Use of parameters
Use of commands
Language used The number of iterations by the students ranged from a maximum of 25 to a minimum of 7, with the majority completing the task within 10 iterations. The time taken for processing and selecting the final image for the logo was about 40 minutes, with some students returning to the task after completing
Exploring the Use of GenAI in the Design Process: A Workshop With Design Students 93
the other elements, but the total time spent on this specific task remained the same.
It was observed that those who exceeded 10 iterations, at a certain point, radically changed the prompt, seeking vastly different solutions. The prompts used were very simple, averaging about 20 words each.
In this task, students made little use of MJ’s specific command references [https://promptfolder.com/midjourney-prompt-helper/], some, even if finding less effectiveness, wrote prompts in Italian, or used a translator. Language thus becomes one of the crucial points, which can determine the failure to achieve the students’ intentions. The iterations are very quick, as seen from MJ’s timing, often even before seeing the achieved result, students copy and paste the previous prompt and edit it. A point of frustration was that for every task, a common chat was opened; therefore, to find their works (iter- ative steps), one had to scroll a lot because there were 50 students writing simultaneously.
Specific Example Below is the example of a student, showing that they significantly altered the logo in two ways: in the first case, by trying to introduce text (a function not yet properly supported by MJ), thereby changing the content, and in the second case, by significantly editing the style, invoking the artist Jackson Pollock. These two changes are noticeable because they remained isolated compared to the flow that echoes the same content and graphic style. No improvement in prompt capabilities is observed; the initial idea of surfing remains in all prompts. It was better articulated in the penultimate prompt, but not achieving the result that we humans can deduce from the prompt, the student decides to choose the logo with a single surfboard, settling for the outcome. It can therefore be hypothesized that while the student probably formulated an increasingly detailed image in their mind, they failed to express it in such a way that MJ could create an image matching their intentions.
Analysis From the workshop, it was observed that students often attempt multiple times to present the same prompt to the AI, hoping it will generate the desired result, changing only one or two terms at most. It is seen that in a simple task, to which no evaluation is attributed, students approach it as when they need to develop a concept. Here, the interaction based on objectives and intents works very well (source) because in the development of concepts, objectives are to be defined, and iteration with the AI helps to reason and bring them out (the objectives). Consequently, students made little use of control commands; however, probably more in-depth introductions to the GUI were needed to allow them to preview the consequences of the commands.
The new paradigm offered by Generative AI allows people to express what they want using natural language, but to achieve a result close to their intentions, they must be able to express themselves accurately (Jagadeesh, 2024), and most designers in the initial phase do not know how to express - especially in words - what they are imagining (mental images).
94 Cavallin
Figure 4: Designer-AI iterations, from Discord platform.
Figure available in the original PDF paper.
Designers can refine their thinking and formulation of ideas precisely through dialogue with AI, but these dialogues, which are cycles of itera- tions, are a form of interaction currently poorly supported, even though they are continuously being implemented (see the documentation related to Midjourney releases).
The frustration and adaptation to the result generated by the AI alternate with enthusiasm for unexpected but satisfying generations. These aspects highlight that students do not have full control over what is happening as they design. Only a few, driven by the will to succeed, attempt more itera- tions than average, with different prompts, using communicative strategies or inserting “command and control” functions (Nielsen, 2023).
CONCLUSION
The workshop was instrumental in understanding the behaviors and approaches adopted by young designers in generating images through gener- ative AI. The students tackled the tasks with an exploratory spirit; objective achievement was required, but no evaluations had been announced. This approach avoided burdening them with concerns but likely stimulated less determination to complete the tasks rigorously.
The analyzed parameters provide a clear framework to understand how interactions with AI are structured. It reveals that one of the fundamental changes we are witnessing in GenAI interfaces is a shift from traditional
Exploring the Use of GenAI in the Design Process: A Workshop With Design Students 95
“command and control” interactions to “objectives and intents” based inter- actions. With new artificial intelligence systems, the user no longer tells the computer what to do. Instead, the user tells the computer what outcome they desire. Thus, the third user interface paradigm, represented by current genera- tive artificial intelligence, is outcome specification based on intents (Nielsen, 2023). This new paradigm allows people to express what they want using natural language, but even though there are manuals and communities that articulate strategies on how to do so, it is a skill that is acquired with difficulty and much practice.
Many GenAI platforms are evolving their user interfaces. Krea.ai is cur- rently one of the platforms that allows for the most control over image generation.
As Jakob Nielsen emphasizes, GUIs have superior usability because they show people what can be done instead of asking them to articulate what they want. For this reason, in perspective, UX and UI designers have a lot of work to do to support the current type of interaction, starting with conducting user research to discover better ways for humans to control these systems.
ACKNOWLEDGMENT
Gratitude is extended to the students for their active collaboration, and to the professors and tutors who facilitated the realization of the workshop.
REFERENCES
Cavallin, E. (2024). Interaction between designers and AI: Literature review of research to highlight the characteristics of co-creation and methods for analyz- ing it. Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems, 119(119). https://doi.org/10.54941/ahfe1004509 Cross, N. (2023). Design Thinking: Understanding How Designers Think and Work
(2
edizione). Ava Pub Sa. Experience, W. L. in R.-B. U. (n.d.). AI: First New UI Paradigm in 60 Years. Nielsen Norman Group. https://www.nngroup.com/articles/ai-paradigm/ Explore Midjourney Prompting. (n.d.). https://docs.midjourney.com/docs/prompts Jagadeesh, D. (2024, March 3). 5/6: UI Patterns to power your AI products.
Medium. https://bootcamp.uxdesign.cc/5-6-ui-patterns-to-power-your-ai-produ cts-ae1591981a78 Nielsen, J. (2023, May 20). AI Is First New UI Paradigm in 60 Years. UX Tigers.
https://www.uxtigers.com/post/ai-new-ui-paradigm Oppenlaender, J. (2023). A Taxonomy of Prompt Modifiers for Text-To-Image Generation. Behaviour & Information Technology, 1–14. https://doi.org/10.1080/ 0144929X.2023.2286532 Shealy, T., Gero, J., Ignacio, P., & Song, I. (2023). Changes in Cognition and
Neurocognition when Thinking Aloud During Design. Proceedings of the Design Society, 3, 867–876. https://doi.org/10.1017/pds.2023.87 Wright, P. C., & Monk, A. F. (1991). The use of think-aloud evaluation methods in design. ACM SIGCHI Bulletin, 23(1), 55–57. https://doi.org/10.1145/122672. 122685
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Artificial Intelligence and Social Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-98-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional