Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evaluating Human – AI Design Interaction From the Perspective of Cognitive Abstraction

Department of Architecture, Xi’an Jiaotong-Liverpool University, Suzhou, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1004563

Abstract

Together abstraction and iteration are two foundational cognitive processes used by designers to expand intellectual experimentation, realize innovation, and construct theoretical positions and formulate generic models. Nersessian’s work on conceptual change highlights abstraction and the role of non-specificity in the formation of generic models, understood to be mental abstractions of a system or relationships seen in objects and dynamic processes. (Nersessian, 2005) we analyze the way small groups of designer structure data inputs for AI tools vizcom and Midjourney to rapidly materialize futuristic visualizations. The construct of consistency is used to evaluate the designers creative process utilizing AI tools and examine future applications, scenarios, design strategies, solutions, and futuristic product feasibility. The aim of this study is to identify degrees of interface abstraction in prompt inputs and AI visualization outputs that may indicate the formation of generic models characterized by non-specificity, understood here as abstraction of a design intent. The ubiquity of iteration in design projects also highlights its importance in the design process. A second aspect of this study is the way designers decompose, integrate or revise semantic structures of prompts for Human-AI integrated workflows to improve design visualization outcomes. Human-artificial intelligence (AI) visualization outputs are aligned to prompts and sketches, and examined to identify levels of non-specificity (expression of abstraction) using the domain constructs of spatial structure, social quality, and spatial quality, to indicate the possible present of a generic representational model. Nersessian, Nancy. 2005. Abstraction via generic modeling in concept formation in science, in Martin R Jones, Nancy Cartwright, Idealization XII Correcting the model: Idealization and abstraction in the sciences. Pozanan’ Studies in the Philosophy of the Sciences and Humanities, Vol. 86. Amsterdam Brill.

INTRODUCTION

Together abstraction and iteration are two foundational cognitive processes used to expand intellectual experimentation, realize innovation, and con- struct theoretical positions and test scientific models. In this paper, I examine how the semantic structure of prompts apply abstraction as a method and control consistency of output quality in the Human-Artificial Intelligence (AI) speculative design conceptual production. The research questions exam- ined are i) how do prompts and AI visualization transform through iteration? and ii) how do Human-AI speculative design workflows engage the construct of abstraction in the development and change of a conceptualization? Find- ings highlight how iterative testing of semantic constraints (non-specificities) by the designer to create inputs, is further advanced by the AI tool through

© 2024. Published by AHFE Open Access. All rights reserved. 133

134 Middleton

the semantic reference of shared meaning. The methodology tested in this study may be useful to further explore structuring properties of H-AI iterative conceptual development and change.

LITERATURE REVIEW

On the Ontology of Abstraction in Conceptualizations Ontology present an interesting interactive dynamic by which to examine the semantic structure of Human-AI (H-AI) interaction. Ontological specifica- tions act as the approximate of an intended model of a logical language that reflects a conceptualization (Oltamari, 2019). The definition of an ontol- ogy by Gruber is ‘an explicit specification of a conceptualization of the world by means of a given language, a formal specification that may also be a shared conceptualization.’ Conceptualizations are language indepen- dent abstractions containing different degrees of specification that may hold shared or negotiated meaning (Oltamari, 2019). Domains are inherently part of conceptual spaces that may be characterized by a number of qualitative dimensions related to its [morphology] perception (temperature, brightness, weight and spatial dimensions of height, width depth) (Gärdenfors, 2017). Grasping the understanding of a domain requires referent to its characteris- tic distinctions and specificities, communicated through words, gestures and icons that refer to different regions of the domain. Concepts may be under- stood by their collection of properties contained with intention, expressing how their properties are corelated, typically expressed by adjectives (Gär- denfors, 2017). These properties may identify a single domain or a complex of properties may form a number of domains are expressed typically as nouns (Gärdenfors, 2017). Iterative semantic structure may indicate change or transformation in the noun phrase reflecting a loss of domain and rela- tional specificity, which may indicate abstraction or the way the noun phrase uses determinacy of reference, indicated by degree of constraint in specificity (Farkas, 2002).

Abstraction and Specificity in Generic (Conceptual) Models Research on conceptual change highlights abstraction and the role of non- specificity in the formation of generic models, understood to be mental abstractions of a system or relationships seen in objects and dynamic pro- cesses (Nersessian, 2005). Generative AI systems are useful to rapidly perform cognitive abstraction tasks and construct a synthesis of new patterns that are presented as the output materializations of a concept (visualization arti- facts). Designers engaging in reflective analysis of AI artifact outputs may further extract specificity features that achieve greater levels of abstractions in the input sematic language in order to clarify the expression of design ideas. Nersessian (2005) notes a generic model should satisfy the abstracted domain constraints that constitute the domain field. The generic model will ideally represent the class of phenomena in each domain that are capable of producing specific configurations. The aim of this study is to evaluate how designers integrate or revise semantic structures to change design concepts

Evaluating Human-AI Design Interaction From the Perspective of Cognitive Abstraction 135

in prompts for Human-Artificial Intelligence (AI) integrated workflows that aim to improve design visualization outcomes as a type of generic model. The goal of this study is to establish an evaluative tool that is able to assess con- sistency and change in conceptualizations between Human-AI systems. Two case studies included in a recent exhibition showcasing the speculative design workshop “The Quality That Lights Up” are examined.

METHODOLOGY

In this study prompt semantic structure is categorized through an ontological schema that distributes the phrase elements according to ontological aspects. The goal is to align the semantic structure of inputs (prompts and sketches) to AI materialized visualizations (outputs) in order to establish the continuity of transformation of the image through iterative development. Table 1 outlines the aspects examined in this study. Text prompts (inputs) are deconstructed to identify their semantic structure aligning to ontological categories of domain, condition relationship, constraint, technical-functional. Interpretations of the AI materializations of conceptualization, as individual images are aligned to patterns identified in the semantic structure of prompts. The visual inter- pretation of AI conceptual materialization distinguishes three aspects; i) the presence of specificity and non-specificity (abstraction) in the visualization, ii) the presence of a shared semantic reference (interpreted meaning), iii) the presence and distinctiveness of semantic continuity of spatial and social ref- erences seen in the composition of object/subject and in the phenomenal aspects of individual and iterative AI output visualizations. The presence of distinctions in the prompt and AI materialized image are identified refer- ring to how variables are introduced by the phrase or assigned values. In the case examples presented, iteration is used by designers to rephrase prompts with new semantics. The maintenance of non-specificity [in the prompt input during iterative testing with AI tools] may establish the development and continuity of structuring logic for a generic model. Two case studies shown in an exhibition of AI-Designer interface as examined. As the speculative design workshop aimed to develop futuristic conceptualizations of lighting, I assume an architectural categorization in phrase structure based on its domain morphology and scopal condition characteristics.

ANALYSIS AND FINDINGS This section presents the analysis of two Human-AI project cases in order to illustrate the presence of abstraction and non-specificity in conceptual artifacts of prompts and sketches (inputs) and AI conceptual materialization (outputs).

Case 1 Overview In case 1 the designer uses the generative AI software Vizcom to iteratively develop the conceptual materialization as a visualization output. The designer input consists of text prompts and hand drawn sketches. Light and space are the main focal elements abstracted in the structure of human input. In concept

136 Middleton

A the AI visualization presentation is highly abstract. Through the iteration the aspect of light is seen to materialize a spatial form rather than provide source of illumination. The light sources are not defined. The spatial form is presented as planes and interior volume (cube).

Table 1. Aspects of semantic structure in H-AI conceptualization.

Table available in the original PDF paper.

Aspect Sub-category Definition Domain Morphology Attributes of shape, line, point, dimensions, scale etc. (Farkas, 2002). Scopal Configuration Structure and configurational properties, that are in dependency on scope, temporal aspects or reflect relationships to typology specificity such as context (Farkas, 2002). Quality Distinction Spatial, atmosphere, and social qualities, that illicit distinctions and specificity (Gärdenfors, 2017). Condition Relationship

Assigned Value Precision succinct linking value to objects, people, space (Farkas, 2002). Quantification Assignment of value, quantity (Farkas, 2002). Connection Relations in the introduction of variables and their assigned values (Farkas, 2002). Constraint Restriction Rules for operation or assignment of value or quality (Farkas, 2002). Ordering Evidence of formalization, classification in topological relations (Farkas, 2002). Technical- Functional

Operational Performance oriented attribute or brand technological indicator (Farkas, 2002). Tasks Indicators of activity and action, doing (Farkas, 2002).

Figure 1: Case 1 Human input (prompt – sketch) iteration and AI output (visualization).
Original PDF Figure 1:

Evaluating Human-AI Design Interaction From the Perspective of Cognitive Abstraction 137

Case 2 Overview In case 2 the designer uses the AI tool Midjourney. The semantic structure of the domain reflects a spatial room in the early visualizations with ambi- ent colour mask to indicate differentiations of light with an exception of the rear wall plane. This has a specific element of window that is assigned the role of scopal configuration referencing light source in the AI interpretation of meaning of human input. Window is understood to be a sources of light even as this is not incorporated in the semantics of the human prompt. In the second series of iterations the frame of the window maintains continuity in the output visualizations but is interpreted in two different ways. The distinc- tive specificity in the visualization image references the mood as experssions of colour, curving lines and which modify the shape and form of the win- dow frames. The aspect of colour is not specified in the prompt sequence.

Figure 2: Case 2 Human input (prompt – sketch) iteration and AI output (visualization).
Original PDF Figure 2:

138 Middleton

Table 2. Analysis semantic structure (prompts) Case 1.
Original PDF Table 2.

The colour palette in AI output images relate to mood through the different colour assignment to spatial elements of furniture and partitions (Wexner, 1954). In the third iteration, a frame of coloured light encapsulates human activity, with layering and shape configurational differences aligned to mood. To reflect a change of mood, the AI system tranfers blue and square shape to warmth and soft curving partition in the foreground. This is then scaled up

Evaluating Human-AI Design Interaction From the Perspective of Cognitive Abstraction 139

with AI distributing human activity throughout a further abstracted structure of frames and dematerialized spaces. The final visualization is a conceptu- alization and generic model of space within a space. The AI visualization constructs a structural grid scaffold system with possibilities for multiple spa- tial domain definition but it is maintaining an abstract visualization. The AI tool applies non-specificity to compose the materialization, scales up the con- cept to indicate the formation of a generic model. The image is abstract in many aspects, from the level of domain, scopal conditions, and relational structure. Aspects of technology, task and function are not distinct in the visualization.

Table 3. Analysis semantic structure (prompts) Case 2.
Original PDF Table 3.

The analysis of semantic structure aligned to ontological categories (aspects) reveals a clear non-specificity at the domain structure, with qual- ity distinctiveness and quantification highlighted as specificities operating in the prompt in case 2. Patterns are noticeable in the composite for both case 1

140 Middleton

and case 2 and highlight the designers strategy to vary deployment of words to express variety of to clarify the different concepts to be visualized.

CONCLUSION

This study examines how generative AI tools may be used to develop generic modelling processes through representations of the general dynam- ical properties of prompt and visualization semantic structure. As seen in the transformation of prompt semantics structure and AI materialization visual- izations in Case 2, the the AI system process involves the extrapolation of a shared meaning from the prompts for specific words and phrases ex. mood, light. Together these become expressed in the window frame, and are car- ried forward as a spatial distinctive quality materialized through application of opacity and color to the required partition that gives light. This was a construct that are not intended by the designer in their formation of a con- cept through the prompt composition aiming to elucidate a design direction of vision. The extrapolations of physical features in the materialization by generative AI achieves an idealization of the human conceptualization input. Idealization is a cognitive process need to reduce and simplify phenomena to form a model to which mathematics can be applied as an ideal model. Cartwright notes that the point of constructing an ideal model is to establish an abstract law, where abstraction is extrapolation beyond the ideal model.’ (Cartwright, 1989, P 187 in Nersessian, 2005) This study of lighting atmo- sphere through manipulations of AI materialization and prompts, present us with different degrees of abstraction across iterations to provide a general description that is in some way common to all.

ACKNOWLEDGEMENT

The exhibition on the speculative design workshop The Quality That Lights Up was supported by the industrial patner IGuzzini and delivered by faculty from the Industrial and Architecture Departments at Xián Jiaotong-Liverpool University in 2023.

REFERENCES

Farkas, D. (2002). Specificity Distrinctions, Journal of Semantics. 19:213–243 Oxford University Press. Gärdenfors, P. (2017) Semantic Knowledge, Domains of Meaning and Conceptual

Spaces in Meusburger, P. Werlen, B. Saursana, L. (eds) Knowledge and Action. Knowledge and Space. Vol. 9. Springer. www.doi.org/10.1007/978-3-319-44588- 5_12 Nersessian, Nancy. (1999) Model-based Reasoning in Scientific Discovery. Ed. L.

Magnani, N J. Nersessian, P. Thegard. Klower Academic Plenum Publishers. Nersessian, Nancy. (2005). Abstraction via generic modeling in concept formation in science, in Martin R Jones, Nancy Cartwright, Idealization XII Correcting the model: Idealization and abstraction in the sciences. Pozanan’ Studies in the Philosophy of the Sciences and Humanities, Vol. 86. Amsterdam Brill.

Evaluating Human-AI Design Interaction From the Perspective of Cognitive Abstraction 141

Oltamari, A. (2019). Artificial Intelligence within the Bounds of Ontological Reason, in Borgo, S., Ferrario R., Masolo, C, Vieu L., Ontology Makes Sense. Essays in Honor of Nicola Guarino. Vol. 316, Frontiers in Artificial Intelligence and Applications. IOT Press B. V. Amsterdam Netherlands. Sukwoojang Ki-Young Nam, (2022). Utilization of speculative design for designing

Human AI interactions. Archives of Design Research 35. (2) 57–71. Vrolijk, Ademir-Paolo, Deng, Yuanzhe, Olechowski Alison. (2023). Connecting Design Iterations to Performance in Engineering Design. International Conference on Engineering Design ICED 23 24–28 July 2023. Bordeaux France. Wexner, L. B. (1954) The degree to which color (hues) are associated with mood-

tones. Journal of Applied Psychology 36 (6), 432–435.

Cover of the proceedings volume Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications

Explore the complete digital volume

Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-96-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →