The Art and Science of Alexander Mordvintsev

Alexander Mordvintsev is a researcher and artist based in Zurich, Switzerland. He is best known for creating the DeepDream algorithm, which was introduced in 2015 and marks a major milestone in AI image generation. DeepDream demonstrates how machines can assist in creating images that abstract reality in ways humans might not conceive on their own, offering a new way of seeing. It has since inspired a new wave of artists experimenting with AI. Apart from his significant contributions to this technology, Mordvintsev is also an active artist. In our article, we delve into both his invention and the beginning of his artistic journey. As we mark the 5th anniversary of our iconic “Automat und Mensch” exhibition, where we first showcased his creations, we are pleased to present a selection of his early pieces created before DeepDream was open-sourced (2015 July). These works, initially not offered for sale, are now available, providing collectors a rare chance to engage with an important moment in art and technology.

When Alexander was about six years old, father brought home an amazing thing: a computer and a bunch of books full of tiny programs that one could type in. Many of these programs displayed interesting geometric patterns on an old black-and-white TV that served as a monitor. That’s how Alex fell in love with computers and generative art. This was the beginning of an exciting journey full of meeting and learning about great people, discovering great ideas from the past, and observing recent developments. Simulations of emergent phenomena, when simple rules lead to complex and unpredictable behavior, were particularly exciting. Demoscene was another source of inspiration for Alexander and this interest culminated in the display of own work “The Flow” at an Assembly demo party in Helsinki in 2009. Soon after graduation from Saint Petersburg State University of Information Technologies, Mechanics, and Optics Mordvintsev got heavily interested in Computer Vision. The challenge of teaching computers a skill that is so easy for us and so difficult for them was very deeply fascinating. After joining Google in 2014 Alexander got introduced to a modern generation of neural networks, and they certainly exceeded his expectations on what computers are capable of. This opened a new chapter for Alexander, in which he spent several years trying to figure out what’s going on inside. DeepDream happened to be one of the experiments trying to answer that question that got world attention.

Alexander Mordvintsev, Custard Apple, 2015/05

Alexander Mordvintsev Just before DeepDream: fractalization #3, 2015/04

At the time, Google led the field in neural network research with figures like Geoffrey Hinton and Jeff Dean, who headed the teams behind the Google Brain in Mountain View. In Mordvintsev’s initial years at Google, he studied key research papers and experimented with pre-trained recognition systems. He took advantage of Google's policy, which encourages employees to dedicate twenty percent of their work time to personally inspired projects, to delve into reverse-engineering image-trained neural networks. His goal was to unravel and visualize the inner workings of these technologies.

By early 2015, he had achieved some visually interesting results, but the true breakthrough came unexpectedly late on the night of May 18. Awakened from a nightmare at 2:00 am, he was inspired to start an experiment. When the experiment finally succeeded, he immediately shared the results on a company social network and then returned to sleep. The following morning, he received numerous comments from his colleagues at Google who quickly recognized the significance of his work. This initial interest in DeepDream was just the beginning of what would soon become a viral phenomenon.

Alexander Mordvintsev, DeepDream is like looking at the clouds, 2015/05 (Not for sale)

Mordvintsev's innovation allowed the computer some autonomy to "dream", thus revealing the complex and often surreal workings of its data processing. His experiment reversed the standard image recognition process in convolutional neural networks (ConvNets), primarily used for vision. These networks usually process images layer by layer, each interpreting and refining the data to construct a final image from basic shapes to detailed recognitions. Mordvintsev interrupted this forward processing to manipulate the mid-layers, coaxing the network to enhance and generate features from partial data, resulting in bizarre, hybrid creations. This exploration of neural networks' "hidden layers" provided valuable insights into their operation. Mordvintsev's work suggested a parallel between the function of these neural networks and human perception, both capable of identifying shapes and patterns where none explicitly exist. 

Alexander Mordvintsev, Father Cat, 2015/05

Alexander Mordvintsev DeepDream is looking for patterns everywhere #2, 2015/05

The final images that emerge from DeepDream are often characterized by intricate, hallucinogenic patterns and a dreamy, surreal aesthetic. Common visuals include enhanced textures and repetitive patterns, such as eyes or architectural forms, which emerge organically within the image. At the beginning, the system often created images that combined elements of dogs and cats into a fantastical creature. Mordvintsev utilized a pretrained neural network known as ImageNet, which has been a benchmark for image classification since its establishment around 2010. ImageNet is particularly noted for including 120 categories of dog breeds to demonstrate its capability for "fine-grained classification”. Due to this focus, there's a strong bias towards dog breeds in the dataset, which influences the results significantly. For one of his first experiments, he fed ImageNet a digital wallpaper image featuring a beagle and a kitten, each perched on tree stumps in a meadow. Interestingly, the output image was predominantly influenced by the dog category—despite the presence of the kitten in the image. This led to fragments of dog faces appearing in unexpected places throughout the image. The outcome was surreal, resembling something out of a hallucination, though its origins were not psychiatric or psychotropic but purely algorithmic.

Alexander Mordvintsev Just before DeepDream: 1000 classes #3, 2015/01

Alexander Mordvintsev Just before DeepDream: 1000 classes #2, 2015/01

Later, Mordvintsev published his work on DeepDream on Google’s public research blog, alongside his colleagues Christopher Olah, a software engineering intern, and Mike Tyka, a software engineer. The publication quickly captured the internet's attention. Within just a few days, the images were featured in over 100 articles, and spread across countless tweets and Facebook posts. The technology was made accessible to the public through a variety of apps and APIs, allowing anyone interested to create their own DeepDream images. This accessibility helped foster a thriving DeepDream community, significantly influencing public engagement with machine learning and computer vision. The impact of DeepDream extended beyond casual interest, inspiring some individuals to pursue PhDs in related fields and leading notable artists like Mario Klingemann to explore this technology and neural networks more broadly.

Alexander Mordvintsev, Sunset, 2015/05

At that time, Mordvintsev didn’t view himself as an artist and considered these images to be merely byproducts of his research. However, in 2017, encouraged by his wife, he began to produce more art using DeepDream and eventually embraced the title of “artist” alongside his role as a research scientist. Since then, his art has been showcased worldwide in prestigious venues such as the Barbican Centre in London, Art Fair Zurich, and Gray Area in San Francisco, CA.

We are proud to have been among the first to exhibit his artworks in 2019 at our "Automat und Mensch" show, which highlighted the evolution of generative art from the 1960s to the present, featuring both pioneering and contemporary artists. At this exhibition, we displayed “Cats”, one of the first pieces he created using DeepDream in 2015. Initially, his works were not for sale. Now, as we celebrate the five-year anniversary of our landmark show, we are excited to present exclusive early works, giving collectors the opportunity to enrich their collections with these historically significant pieces.