Repeating And Remembering: The Associations Of GANs In An Art Context

Author: Anna Ridler

Translator: Wang Mengyao

Editor: Zheng Zhuyun

Repeating and remembering: the associations of GANs in an art context

Keywords: GAN Generative Adversarial Network, training set, pix2pix, image generation

Note:GAN (Generative Adversarial Network): A machine learning framework designed by Ian Goodfellow and his colleagues in 2014, it is a method of unsupervised learning that learns through a game between two artificial neural networks.

“Abstract”

Given the lack of discussion about GAN-generated art in artistic environments, this article explores how we should think about GANs and training sets in artworks, just as we treat other materials. I will focus on the potential connections between training sets and GANs, particularly pix2pix (Note: a general image-to-image translation implemented using conditional adversarial networks, invented by Phillip Isola et al.), and how I have tried to embed these connections into my own work.

“Introduction”

Some research has looked into whether artificial intelligence, especially machine learning, can create art. However, the focus of these projects has been on assessing and judging whether the results are “art” by studying the visual parameters’ impact on viewers (i.e., “Does this look like art?”). This neglects the important consideration for artists regarding the impact of the materials used in creating the work. Using GANs to generate images prompts viewers to think about different experiences, expectations, histories, trajectories, and contexts compared to any other method. What are these associations? How are they used in a work? Materiality is “one of the most controversial concepts in contemporary art, often marginalized in academic writing” [5], and until recently, digital art has only begun to be explored; to my knowledge, it has not been applied to works generated by neural networks. This is an important gap: art should be able to comment on advanced contemporary scientific theories as these theories are used in creative practices, while also ensuring “a critical distance to prevent the status of these fields from ascending to unquestionable authority” [5]. Images created by GANs are becoming increasingly common in the international art scene (for instance, at Ars Electronica in 2017 and the Serpentine Gallery Miracle Marathon), but there is almost no language to discuss them outside of science. Can GANs or training sets become “intentional agents and drivers in the artistic process” [5] like other materials? I will explore some of these questions by studying the potential connections between training sets and GANs (especially pix2pix), and by attempting to embed these connections into my own work.

“The Association of Training Sets”

Training sets are rarely discussed, but they are central to the final image output. Most ready-made (and linked from GitHub pages) training sets are compiled by researchers using various methods, but due to the inherent subjectivity of humans involved in the original content or process, they inevitably contain certain cultural or social attitudes. These datasets are usually very large, meaning that artists cannot control which biases and prejudices are replicated and repeated through selection. Moreover, the invisible labor and related power relations of the mechanical turks typically used to create scenes are seldom discussed in the context of GAN-generated images as art and what that might mean. However, small-batch machine learning programs (such as pix2pix) require much smaller training sets, which can be manually curated. I will explore how artists can begin to use these tools to create carefully selected small training sets to counter these issues.

“The Association of GAN-Generated Images”

Although creating training sets can give artists some control (what images, what labels, etc.), GANs offer a quality where “materials develop their own incredible life and demonstrate their ability to metamorphose” [5], which, in the context of art, is more closely related to biological or natural art than digital art. The styles of these images are unique (especially those models that are “under-trained”), perhaps echoing Georges Bataille’s concept of “bas materialite” — broken, decayed, and decomposed, in contrast to the smooth surfaces of capitalist commodities [7]. I believe that a significant part of contemporary digital art aesthetics is thus constituted. Art historians and cultural theorists should unpack these associations and their implications.

Quoting Petra Lange-Berndt’s definition of materials in “electronic technologies” (such as digital technologies), the outcome is not imitation, but “as mixed, mutated materials, a new material emerges that also critically comments on the initial components of the experiment” [5]. I explore how GAN-generated images can critically manifest, and in the context of some recent AI artists (such as Memo Akten and Mario Klingemann), begin to explore some assumptions about creativity, authorship, and control considering the material background.

“The Fall of the House of Usher”

I reflect on how I attempted to consider these two factors while creating my own work, “The Fall of the House of Usher.” This is a 12-minute animation made using pix2pix [8]. It could have been created by hand, but by choosing machine learning, I was able to emphasize and highlight themes surrounding the role of the creator, the interaction between art and technology, and various aspects of memory that I could not achieve in any other way. For example, by choosing to create my training set using hand-painted ink wash paintings from the 1929 film (Fig. 1, Fig. 2), I was able to emphasize the historical interaction between digital and physical painting, question the roles of humans and AI, and begin to think about labor issues.

By limiting the training set to the first four minutes of the film, I could control the level of “correctness” to some extent: as the animation progressed, it increasingly drew less from the reference, leading to the incredible moments where I could not predict where the information would begin to collapse, especially towards the end. I deliberately adopted the “decay” offered by images made in this way and turned it into the central part of the work to echo the destruction and “replication” occurring in the narrative (the work is a copy of a copy of the film, which was originally a book) (Fig. 3).

Finally, I will summarize how GAN images have allowed me to discover new possibilities [3]. It has provided a mirror for my process, revealing elements I was unaware of (how I attract the eye, what I choose to emphasize). I am now creating a new training set in which I will draw GAN-generated images (see Fig. 4), replicating artifacts and errors (some of which I made in the original training set, like a chair that appears and disappears depending on whether I remember to draw it). The copy of the copy of the copy, decay and decomposition become more volatile and uncontrollable. The results of the model will be merged and absorbed into the animation, creating a series of repeated works that, in Borges’s words, are “no less real, but more akin to expectation,” with “pure lines not found in the original” [1] emerging over time and cycles.

Fig. 1: Hand-painted training set material set for “The Fall of the House of Usher.”

Fig. 2: Training set for “The Fall of the House of Usher.”

Fig. 3: Still from “The Fall of the House of Usher.”

Fig. 4: Redrawn GAN image samples in “The Fall of the House of Usher.”

References

[1] Jorge Luis Borges, Andrew Hurley, and Andrew Hurley. Collected fictions. Viking New York, 1998.

[2] Simon Colton. The painting fool: Stories from building an automated painter. In Computers and creativity, pages 3–38. Springer, 2012.

[3] Mark d’Inverno, Jon McCormack, et al. Heroic versus collaborative AI for the arts. 2015.

[4] Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. CAN: Creative Adversarial Networks, Generating “Art