Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Generative Adversarial Network Algorithms in Art: Data Video

School of Arts and Social Sciences, Hong Kong Metropolitan University, Ho Man Tin, Kowloon, Hong Kong ↗
Publication year
Digital Object Identifier10.54941/ahfe100957

Abstract

The recent development of machine learning to synthesize the dataset and manipulate images into new works of art, bringing essential changes in visual art and the method of art creation. The paper aims at applying the Generative Adversarial Network (GAN) to the new media art in particular the image generation and video synthesis through latent space interpolation, through the indirect training in GAN to process a series of still images as the dataset, the generated work presents the ability of machine algorithms in learning and processing the image creation, as well as the next stage of machine-made art. The generated images through latent space interpolation are the artificial imitation among the images by the machine, indicating a new form of image interpretation and representation where human’s intervention in art creation is restricted in the pre-data selection and post-data appreciation.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

INTRODUCTION

With the development and improvement of computational and imaging technology, manipulation and visualization of datasets become one of the important areas to test the ability of artificial intelligence in image creation. Machine learning has been widely used as one of the most effective tools and technologies in image generation, and to a certain extent become an important research area in art and technology, for example, Mark Coeckelbergh in his research concerning the artistic creation ability of machine and human/non-human forms of creativity (Coeckelbergh, 2016). Image, in general, can be classified into three types, namely still images, moving images and expanded images. Although there are still a lot of things that have to be addressed regarding still image generation, how to tackle the difficulty and improve the synthetic ability of moving images is already a focusing point in the development and research of imaging and machine learning technologies. This could be regarded as the visual mechanism of humans that we are living in a condition based on time and space. Time is constructing our concept towards the beings and lives, to understand the constitution of moving things is to understand the very notion of time as it shows the characteristics of dynamical and unstable condition which can better reflect reality. One of the main attempts of this paper is to explore the possibilities of transforming the still images as a dataset into moving images through a machine learning neural network, and to observe the results generated by the neural network, trying to understand the model of machine-generated images and the nature of machine-made art based on datasets manipulation.

GENERATIVE AND DISCRIMINATIVE MODELS It was presented by Alce Radford et al. in their research Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks that the deep convolutional generative adversarial networks (DCGANs) demonstrate the ability of unsupervised learning through generator and discriminator (Alec et al., 2016). Generative and discriminative networks of the GAN architecture were used to generate new images in this paper. A generative network imitates the training dataset and generates new data, while a discriminator network discriminates the generated data from the real data of the training set (Figure 1). This mechanism enables both networks to perform unsupervised learning dynamically in order to synthesize new models based on the training of the dataset.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Figure 1: Mechanism of GAN based on the Generative and Discriminative Models In this paper, two sets of pictures were used for GAN training in order to test the ability of GAN in image generation on a previous set of generated images, and explore the possibility of creating new images by adapting the aesthetic framework of “art is an imitation of another imitation” as addressed in Plato’s Republic. In the first stage, a set of 135 pictures of Plato were collected from the internet and used to train GAN for the generation of a new set of images. The second set of 15 pictures of David produced from digital illustrations, 3D face reconstruction, photography and 3D scanning were used to train the GAN for synthesizing the previous models generated based on the 135 pictures of Plato. It is anticipated that the final outcome after the training of GAN on the 2 sets of data (Plato and then David on Plato dataset) should have a certain level of changes in terms of the visual expression and representation from the original datasets. The training on the generator and discriminator of GAN on the 2 sets of data enables the model to generate new data on the new data under unsupervised learning condition. In this case, to a certain extent, can implement the training of the machine in image manipulation of an image manipulation. The generated moving images were examined in order to have a better understanding of the visual representation and the relationship between the art creation and the data generation under the application of GAN.
Original PDF Figure 1:

IMAGE GENERATION The AI and algorithmic technologies provide new approaches to creativity and artefact creation. Digital objects become an essential topic to understand the artefacts being produced artificially and digitally. Understanding the ability of GAN in digital image generation can help us examine some natures of digital objects. Given that there are researches and examples proven that GAN is able to generate still portraits of highly realistic, such as Karras Tero et al. in their research about generating artificial photos of fake people (Tero et al., 2018), this paper aims at applying the GAN to further examine the ability of the adversarial network to utilize

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 the data in generating moving images (short video synthesized via latent space interpolation) and other possible visual expressions.

A set of 135 pictures of Plato was used to train the GAN by using the AI tool RunwayML and its corresponding GPU. The generated model and the gallery of the video was presented in Figure 2. The result shows that GAN is able to synthesize the dataset, generate new images and video (moving images). It is noted that latent interpolation image appears among the original images, representing the ability of artificial imitation and generation of artefacts by machine learning in the training. The latent interpolation images can shed light on the various visual expressions through GAN.

Figure 2: The result from GAN in synthesizing 135 pictures of Plato which were collected from the internet. To examine the ability and advantage of the GAN to imitate the training dataset and generate artefacts, and examine the representation of the images, the generated “Plato” video (Figure 2) was used as the dataset to train another set of images. 15 pictures of David (Figure 3) were used in the second training in RunwayML. The results show that the latent interpolation is more obvious compared with the results of the “Plato” video (Figure 2), superimposition of the two sets of images was obtained, which led to obscured hybrid appearances of the two sets of models - Plato and David.
Original PDF Figure 2:

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Figure 3: A set of 15 pictures of David, produced through digital illustrations, 3D face reconstruction, photography and 3D scanning

Figure available in the original PDF paper.

Figure 4: The result from GAN in synthesizing 15 pictures of David in the mapping from the images generated in the first stage (Figure 2) According to previous researches about data on discriminator training in GAN such as training GAN with limited data done by Karras Tero et al. (Tero et al., 2020), constraints were also set in the training of GAN in this research to test its ability in latent space interpolation. 135 pictures of Plato ran for 2000 training steps and 15 pictures of David ran for 500 training steps were used to create limited data and training conditions, good results were obtained in terms of the latent interpolation
Original PDF Figure 4:

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 effects and generated new forms of images. The data on GAN generated images could be the next research approach to examine the diversification of the latent interpolation images and the length of the moving images by increasing the number of pictures and training steps.

DATA VIDEO Data training and image generation play an important role in this research as the image synthesized through latent space interpolation indicating not only the architecture of the generative and discriminative networks, but also what has the network learnt in unsupervised learning. The video, to a certain extent, is a data-driven and neural network synthesized artefact. The unsupervised GAN learning system and its automation in images generation allow the GAN to maximize the operation without the intervention of humans or artists, and therefore somehow echo and address the concerns by Lev Manovich in his book AI Aesthetics about the high ability of AI automation and generation of contents (Manovich, 2018). The nature of the video artwork under the use of GAN in this research has been expanded to machine and data-based, the video is therefore an unconventional form of work that allow the observation and appreciation of the artificial artefacts further expand to the perception of the data manipulation of GAN.

CONCLUSION

The ability of GAN in data training and manipulation to generate images and synthesize moving images through generative and discriminative networks was tested. The results of the trained dataset and images obtained are obscured but that could be very interesting in terms of the new forms generated and the latent space interpolation by the unsupervised machine learning. The dynamical representation of two sets of pictures and the formation of images synthesis from latent interpolation are new approaches to artistic creation, indicating the possibility of machine-made art and data video generation. Given the nature of unsupervised learning of the GAN, the intervention and participation of artists in video creation is therefore limited and transformed to machine-oriented synthesis.

REFERENCES

Alec, R., Luke, M. and Soumith, C (2016) “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks”, in: 4th International Conference on Learning Representations. ICLR 2016. pp.1-16. Coeckelbergh, M. (2016) Can Machines Create Art?. Philosophy & Technology,

30(3). pp.285-303.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Images of Plato. (n.d.) Available at: https://www.google.com/search?q=plato&source=lnms&tbm=isch&sa=X&sqi =2&ved=2ahUKEwj89t- w1dX1AhUZ4jgGHYDvA04Q_AUoAXoECAIQAw&biw=1651&bih=888& dpr=2. (Accessed 4 January 2021). Manovich, L. (2018) AI Aesthetics. Strelka Press. Plato. (n.d.) The Republic. Bookwise.io.: https://www.bookwise.io/plato/the-

republic. (Accessed 13 November 2021). Tero, K., Miika, A., Janne, H., Samuli, L., Jaakko, L. and Timo, A (2020) "Training Generative Adversarial Networks with Limited Data", in: Thirty-fourth Conference on Neural Information Processing Systems, Advances in neural information processing systems, Vol. 33. Tero, K., Timo, A., Samuli, L. and Jaakko, L (2018) "Progressive Growing of

GANs for Improved Quality, Stability, and Variation", proceedings of the International Conference on Learning Representations, Vancouver, Canada.

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-7923-8988-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →