Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Computational Aesthetics of Visual Artworks: Review and Outlook

1 School of Design, Shanghai Jiao Tong University, 800 Dong Chuan Road, Shanghai 200240, China ↗
2 China National Gold Group Gold Jewelry Co., Ltd., 1 Liuyin Park South Street, Beijing 100011, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1001833

Abstract

Beauty has always been the goal of human activity. The meaning of beauty is defined during its creation. The discussion of beauty by mathematicians has created the computational aesthetics. This paper reviews the related works in computational aesthetics of visual arts and summarizes the features used in aesthetic measurement. The features extracted from the visual artworks are divided into five types including visual, compositional, statistical, perceptual and artistic features. Based on those features, the visual images are quantified and analyzed on their aesthetic and artistic properties. The previous studies are mostly focused on the aesthetics of art production, the continuous process for artistic creation cannot be represented. Moreover, new forms of visual arts such as generative art emerged, which provide new opportunities and challenges to the field of computational aesthetics. A prospective on the algorithms for the evaluation of this new forms of visual arts will be given.

INTRODUCTION

Aesthetic is a discipline studying mankind’s aesthetic judgements and experi- ences especially of art (Zangwill, 1998). Art is the direct expression of human creativity which not only conveys individual’s emotion but also contains col- lective intelligence. Because of its mysterious but charming nature, scholars in various fields are attracted to define beauty from their own profession. Birkhoff (1933) proposed a formula measuring the aesthetic merit of artw- orks by complexity, balance symmetry and balance. This idea of quantifying aesthetic judgements through mathematical methods leads to the birth of computational aesthetics.

The computational aesthetics refers to the discipline of computational methods that can imitate human aesthetic decision (Hoenig, 2005). As a subfield of computer vision, computational aesthetics aims to develop com- puter’s capability of imitating the ability of human’s visual system to process aesthetic images. This task corresponds to a classification based on the inter- pretation of image content and the pre-set evaluation criterion, also called the classification criterion. Generally, the classification criterion can be divi- ded into two types, aesthetic quality and artistic attribution. For the aesthetic

© 2022. Published by AHFE Open Access. All rights reserved. 8

Computational Aesthetics of Visual Artworks: Review and Outlook 9

quality, the methods to build an automated model in evaluating the aesthetics of visual artworks such as photographs and paintings are explored. The com- putational method for aesthetic features extraction is a key part. For the artistic attribution, the methods to classify images based on art style, artist and creation period are developed. The feature sets representing artistic attri- butions, particularly for paintings, are discussed. The characteristics of art image content are also used in assisting the analysis of art. Through the quan- titative study of large-scale visual artworks, the development of art, which are not discovered by historians and critics, are revealed. The potential capacity of these computational methods in art research is demonstrated (Brachmann and Redies, 2017).

The computational aesthetics research of visual artworks can be categori- zed into two types. One is building a classifier based on aesthetic properties which is determined by the research purpose of computer vision. The other is implementing quantitative analysis to discover the evolution of art from the perspective of computational tools developed. Both types require quan- tifying image which includes establishing a computational method to extract features representing image pattern and content. Therefore, the selection of image features reflects the insights of the researchers. In this paper, the related works in computational aesthetics of visual arts are reviewed and the diffe- rent types of features used in aesthetic measurement are summarized. New computational aesthetic method for future artworks will be explored.

METHODS

The computational aesthetics of the visual artworks studies the image featu- res of paintings and photographs, including visual, compositional, statistical, perceptual, and artistic features, to conduct a quantitative analysis of their aesthetic and artistic properties. The visual features include the low-level fea- tures, high-level features, and deep-level features. The low-level features of an image describe the basic elements of its content, such as edges, colors, lumi- nance, textures, shapes, and saliency. The high-level features can describe the semantic information of the art images, including their regions and contents. The features extracted by neural networks are defined as the deep-level fea- tures which combined various features and are hard to describe its explicit meaning. The compositional features reveal the composition of the image which are generally used in photography to assess its appealing. The statisti- cal features are extracted by statistical approaches based on their statistical properties. The perceptual features are the perception of human for the entire image. The artistic features are associated with the characteristics of artw- orks, such as the outcome and the process of the art creation. Details of the methods for image feature extraction based on different applications will be discussed below.

Image Classification Based on the Aesthetic Properties In the field of computer vision, the visual, compositional, perceptual, and statistical features of the images are extracted to investigate their aesthetic properties.

For photo quality assessment, the visual features measure the appearances of photographs. Tong et al. (2004) use a set of low-level features depicting blur, contrast, colorfulness, saliency, energy, texture and shape of the image to classify the photographs took by photographers and home users. Datta et al. (2006) propose a classifier using visual features of exposure, colorfulness, saturation, hue, texture and shape to discriminate pictures of high and low aesthetic value. Luo and Tang (2008) focus on the subject regions of the pho- tographs and formulate a set of features measuring clarity contrast, lightness, simplicity and color harmony to assess the aesthetic quality. Obrador et al. (2012) measure low-level features both on the entire image and its contrasting regions. The global visual features include luminance, contrast, colorfulness and color harmony. The contrasting region features include sharpness, expo- sure, chroma and saliency. With the development of deep learning, the neural network techniques are introduced for visual features extraction. Dong et al. (2015) use the well-trained AlexNet, a convolutional neural network propo- sed for ImageNet classification, to get the features for photograph quality assessment.

The compositional features used in photo quality evaluation indicate the object arrangements. The photographic rules of composition such as Rule of Thirds and golden ratio contribute to the aesthetic measurement of the photographs. Datta et al. (2006) propose a method to detect the low depth of field and macro images. As the size of an image is considered as a factor affecting the photo ratings, the size feature is computed by the sum of width and height. Both the low depth of field indicator feature and the size feature are added to the feature set for judging the aesthetic qualities of photographs. Obrador et al. (2012) use templates to extract image composition features such as the Rule of Thirds, the golden mean and the golden triangles in order to attain the global compositional features.

Perceptual features represent the direct feelings of the images by observers. Datta et al. (2006) deem the familiarity that observers feel for the image content affects their appreciation. The measure of the familiarity is computed by the integrated region matching (IRM) image distance.

From the classification of photographs based on their appealing, the clas- sification of paintings according to the artistic properties, such as author, style and period, are explored. The visual features of a painting can express the color palette and technique of the artist. Gunsel et al. (2005) propose a novel method to calculate low-level features from the luminance and gradient information of the image, along with the statistical information of grey level histogram. Based on the feature set, the classifier can differentiate their pain- ters as well as art movements. Li and Chen (2009) extract the color features representing hue, saturation and lightness (HSL) space from the paintings and build a model to classify high-quality and low-quality paintings. Siddi- quie et al. (2009) propose a feature set containing texture, Histograms of Oriented Gradients (HOG), color and saliency and use Multiple Kernel Lear- ning technique in classification of painting genre. Cetinic and Grgic (2013) consider the color, brushstroke and composition as the main characteristics of the paintings and extract visual features of color and texture to recognize paintings by artist.

Computational Aesthetics of Visual Artworks: Review and Outlook 11

The compositional features extracted from paintings describe the arrange- ment of the visual elements designed by artists. Li and Chen (2009) separate the focus region in paintings based on Golden Section and Rule of Thirds to organize objects. Computed on the segmented regions, the composition fea- tures can indicate shapes and spatial relationship of different segments inside the image.

The statistical features describe the general characteristics inside the pain- tings. Redies et al. (2007) compute Fourier power spectra in images to discover the statistical differences between photographs of faces and por- traits. Cetinic and Grgic (2013) extract the statistical features of image intensity and combine with other visual features to build a system for auto- mated painter recognition. Carballal et al. (2018) extract complexity features based on image compression, Zipf’s Law and fractal dimension to distinguish paintings from photographs.

The perceptual features in fine art include balance, symmetry and comple- xity. Aleem et al. (2017) measure the vertical symmetry indices in portrait paintings by subtracting the intensities of each pixel and its pair across the midline of the face. A new method is proposed to measure the balance of portraits. al-Rifaie et al. (2017) introduce swarm intelligence technique to quantify the symmetrical complexity of patterns.

Quantitative Analysis of Visual Artworks The digitization of the painting arts allows the large-scale quantitative stu- dies on artworks. With digital image processing techniques, the features in paintings from different artists and periods in the human history are investigated.

The visual features correspond to the unique style of each artist, which can be used in the authentication of the artworks and the study of art history. Berezhnoy et al. (2005) analyze the color and brushstroke texture in Van Gogh’s paintings to develop a technique for the authenticity detecting. Abe et al. (2013) extract the Classeme feature vector as the visual feature to compute the artist similarity. Kim et al. (2014) quantify the usage of indivi- dual colors, the variety of colors and the roughness of the brightness which showed great differences in art period.

Lee et al. (2020) analyze the compositional features of western landscape paintings. An information-theoretic framework is proposed to capture com- positional proportion for dissecting landscape paintings. Based on the mutual information of the color palettes and the partitions in the landscape paintings, the dissection method can capture the characteristics which distinguishing their style and creation date.

The statistical features are gained from statistical analysis of fine-arts. Taylor et al. (1999) use the box-counting method to calculate the fractal dimension of Pollock’s drip paintings for authentication. Lyu et al. (2004) apply the wavelet filters to decompose the image in spatial position, orienta- tion and scale. The wavelet statistics can be used for authenticating artworks. Hughes et al. (2010) propose a novel method for quantitative analysis of artistic style based on sparse coding, which is capable of detecting authentic

Bruegel drawings. Sigaki et al. (2018) propose a method to calculate the per- mutation entropy and the statistical complexity of paintings, and quantify a large number of paintings spanning a long art history to investigate the performance of the proposed indices in distinguishing artworks in style and period.

Differing from visual features, the artistic features reveal the characteri- stics of artworks in artistic creation. Montagner et al. (2016) combine Gabor filter and Scale Invariant Feature Transform to extract the brushstroke featu- res. Pigment features are obtained from hyperspectral imaging and elemental analysis. Based on the proposed features, the brushstroke and materials analysis is capable of authenticating paintings by Amadeo de Souza-Cardoso. Ellis and Johnson (2019) demonstrate that image processing techniques can assist art historians and conservators in analyzing materials and techniques applied in artworks.

NEW FORMS OF VISUAL ARTS By studying the aesthetic merit of artworks, the computational aesthetics may contribute to the art creation by guiding the improvement of visual expression. Therefore, the application of aesthetic measurement in art cre- ation is worth to be discussed. The previous works on the computational aesthetics of visual artworks normally extract the features of the final produ- ction from visual art creation and analyze its aesthetic properties. However, from the perspective of art creation, the artistic analysis based solely on the final production has certain limitations. For traditional visual artworks, there are many obstacles in proposing an artistic measurement based on artistic creation because it is difficult to obtain the image data for the creative process.

The emergence of the visual artworks produced by the computer systems, such as the generative arts, provides a possibility in realizing the aesthetic measurement of visual art creation process. The generative art refers to any art practice where the artist uses a system, such as a set of natural language rules, a computer program, a machine, or other procedural invention, which is set into motion with some degree of autonomy contributing to or resulting in a completed work of art (Galanter, 2003). A generative art system consi- sts of four components, namely entities, processes, environmental interaction and sensory outcomes (Dorin et al. 2012). The entities are the basic consti- tuents in the generative system which will be operated by the processes to change their states to create art. The entities define the fundamental elements appeared in the generative artworks. The spatial, temporal and formal attri- butes of the entities determine the diversity of the visual expression for the art creation. The processes are the mechanisms managing the behaviors of entities which allows the diverse creation modes in generative art. The envi- ronmental interaction describes the information flows between the process and the execution environment of the generative system. It indicates that the generative system is multi-agent involved. The entities, processes and envi- ronmental interaction of generative art will determine its sensory outcomes which refer to the results perceived by the observers. The characterization of

Computational Aesthetics of Visual Artworks: Review and Outlook 13

generative system makes the generative art differ from the traditional visual art because of its creative process and outcome forms.

With the development of technology, the creator of arts will no longer be limited to human, but also the others. Created by computers, the generative art, as a new form of visual arts, inspires the exploration in exploiting compu- tational aesthetics to support the future art production. The multi-dimension attributes of the generative art in artistic expression, urge its aesthetic evalua- tion to involve spatial and time-based features in order to depict its aesthetic properties based on its unique creation rules. Considering that the intera- ctions between the generative system and the operation environment may affect the aesthetic experience, it is necessary to add features depicting the information flow to the prevailing feature sets for aesthetic assessment. Con- sequently, the aesthetic measurement for new forms of visual art should examine the information of a process which involved the creators and the observers, starting from art creation to communication. By modeling the complete process of human perception for visual artworks, it is helpful in improving the capacity of the machines in imitating the aesthetic decision of the human.

SUMMARY This paper reviews the state-of-art of the computational aesthetics for the visual artworks. There are various computational methods to classifying art images including photographs and paintings by their aesthetic properties, as well as quantifying paintings to assist the study of art. The image features used in quantifying artworks are categorized into five types which are visual, compositional, statistical, perceptual, and artistic features. As the aesthetic value of artworks may contribute to art creation, the aesthetic evaluation method for future artworks, take the generative art as example, is discussed. The aesthetic evaluation of the new forms of visual arts should focus not only on their visual expressions, but also other perceptual factors in aesthetic experience.

REFERENCES

Abe, K., Saleh, B., and Elgammal, A. (2013). An early framework for determining artistic influence. In International Conference on Image Analysis and Processing (pp. 198–207). Springer, Berlin, Heidelberg. Aleem, H., Correa-Herran, I., and Grzywacz, N. (2017). Inferring master painters’

esthetic biases from the statistics of portraits. Frontiers in human neuroscience, 11, 94. al-Rifaie, M., Ursyn, A., Zimmer, R., and Javid, M. (2017). On symmetry, aesthetics and quantifying symmetrical complexity. In International Conference on Evolutionary and Biologically Inspired Music and Art (pp. 17–32). Springer, Cham. Berezhnoy, I., Postma, E., and van den Herik, J. (2005). Computerized visual analysis

of paintings. In Int. Conf. Association for History and Computing (pp. 28–32). Birkhoff, G. D. (1933). Aesthetic Measure. Cambridge: Harvard University Press. Brachmann, A., and Redies, C. (2017). Computational and experimental approaches to visual aesthetics. Frontiers in computational neuroscience, 11, 102.

Carballal, A., Santos, A., Romero, J., Machado, P., Correia, J., and Castro, L. (2018). Distinguishing paintings from photographs by complexity estimates. Neural Computing and Applications, 30(6), 1957–1969. Cetinic, E., and Grgic, S. (2013). Automated painter recognition based on image

feature extraction. In Proceedings ELMAR-2013 (pp. 19–22). IEEE. Datta, R., Joshi, D., Li, J., and Wang, J. (2006). Studying aesthetics in photographic images using a computational approach. In European conference on computer vision (pp. 288–301). Springer, Berlin, Heidelberg. Dong, Z., Shen, X., Li, H., and Tian, X. (2015). Photo quality assessment with

DCNN that understands image well. In International Conference on Multimedia Modeling (pp. 524–535). Springer, Cham. Dorin, A., McCabe, J., McCormack, J., Monro, G., and Whitelaw, M. (2012). A framework for understanding generative art. Digital Creativity, 23(3-4), 239–259. Ellis, M., and Johnson Jr, C. (2019). Computational Connoisseurship: Enha-

nced examination using automated image analysis. Visual Resources, 35(1-2), 125–140. Galanter, P. (2003). What is generative art? Complexity theory as a context for art theory. In In GA2003–6th Generative Art Conference. Gunsel, B., Sariel, S., and Icoglu, O. (2005). Content-based access to art paintings.

In IEEE International Conference on Image Processing 2005 (Vol. 2, pp. II-558). IEEE. Hoenig, F. (2005). Defining computational aesthetics. in Proceedings of the First Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging (Girona: Eurographics Association), 13–18. Hughes, J., Graham, D., and Rockmore, D. (2010). Quantification of artistic

style through sparse coding analysis in the drawings of Pieter Bruegel the Elder. Proceedings of the National Academy of Sciences, 107(4), 1279–1283. Kim, D., Son, S., and Jeong, H. (2014). Large-scale quantitative analysis of painting arts. Scientific reports, 4(1), 1–7. Lee, B., Seo, M., Kim, D., Shin, I., Schich, M., Jeong, H., and Han, S. (2020). Disse-

cting landscape art history with information theory. Proceedings of the National Academy of Sciences, 117(43), 26580–26590. Li, C., and Chen, T. (2009). Aesthetic visual quality assessment of paintings. IEEE Journal of selected topics in Signal Processing, 3(2), 236–252. Luo, Y., and Tang, X. (2008). Photo and video quality evaluation: Focusing on the

subject. In European Conference on Computer Vision (pp. 386-399). Springer, Berlin, Heidelberg. Lyu, S., Rockmore, D., and Farid, H. (2004). A digital technique for art authentication. Proceedings of the National Academy of Sciences, 101(49), 17006–17010. Montagner, C., Jesus, R., Correia, N., Vilarigues, M., Macedo, R., and Melo,

M. (2016). Features combination for art authentication studies: brushstroke and materials analysis of Amadeo de Souza-Cardoso. Multimedia Tools and Applications, 75(7), 4039–4063. Obrador, P., Saad, M. A., Suryanarayan, P., and Oliver, N. (2012). Towards category-based aesthetic models of photographs. In International Conference on Multimedia Modeling (pp. 63–76). Springer, Berlin, Heidelberg. Redies, C., Hänisch, J., Blickhan, M., and Denzler, J. (2007). Artists portray human

faces with the Fourier statistics of complex natural scenes. Network: Computation in Neural Systems, 18(3), 235–248.

Computational Aesthetics of Visual Artworks: Review and Outlook 15

Siddiquie, B., Vitaladevuni, S., and Davis, L. (2009). Combining multiple kernels for efficient image classification. In 2009 Workshop on Applications of Computer Vision (WACV) (pp. 1–8). IEEE. Sigaki, H., Perc, M., and Ribeiro, H. (2018). History of art paintings through the

lens of entropy and complexity. Proceedings of the National Academy of Sciences, 115(37), E8585-E8594. Taylor, R., Micolich, A., and Jonas, D. (1999). Fractal analysis of Pollock’s drip paintings. Nature, 399(6735), 422–422. Tong, H., Li, M., Zhang, H., He, J., and Zhang, C. (2004). Classification of digi-

tal photos taken by photographers or home users. In Pacific-Rim Conference on Multimedia (pp. 198–205). Springer, Berlin, Heidelberg. Zangwill, N.. (1998). The concept of the aesthetic. European Journal of Philosophy, 6(1), 78–93.

Cover of the proceedings volume Cognitive Computing and Internet of Things

Explore the complete digital volume

Cognitive Computing and Internet of Things

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-19-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →