Warning: this post contains images that may be disturbing, particularly to the trypophobic and those averse to badly drawn cats

Last week in Fun With Neural Networks we covered the delightful neural network projects of Janelle Shane, who wants to "Char-rnn ALL THE THINGS" with often hilarious results.

The things she tends towards, though, are largely text-based data. This week, we're going deeper. Deep Dream deep.

Dive, dive, dive!

Released in July of 2015, Google's Deep Dream computer vision program delighted and horrified humanity with its image manipulation capabilities. Using what has been dubbed "algorithmic pareidolia," Deep Dream uses its neural network programming (that's the algorithmic part) to classify and manipulate images in a way that it recognizes; the pareidolia part comes from its similarity to the human psychological phenomenon of seeing human faces on non-human objects (/r/pareidolia always has a fresh supply of examples, but you can probably find one in the objects around you right now!)

But Deep Dream doesn't see faces... at least not the way humans do. While some of the resulting images are charmingly abstract and would make a lovely psychedelic print:

Deep Dream image from Google via The Telegraph

Others, like this GIF that made it to the front page of Reddit, are pure bad trip:

The network's propensity for finding animal faces and eyes is a result of its training data, but some have theorized that the hallucinogenic resemblance of its output may be evidence of neural networks' impending ability to replicate human brain complexity (or at least some aspects of our visual cortex). If you're interested in even wilder theories about what Deep Dream could evolve into, MONTAG published a short fiction entitled Deep Awakening on this very subject.

Even Google admits on the Deep Dream site that use of the algorithm has primarily become a new medium for creating psychedelic and abstract art. A recent example is the music video for Foster the People's "Doing It For The Money." Hold on to your pumped up kicks as you watch what is a fairly standard pop video (the band in a room, the silhouette of a woman, shots of driving, someone running somewhere, for some reason) get a little extra flavor from sticking a square of circuit board-printed blotter paper under its tongue:

Steal Her Style!

Less disturbing and surreal, the Deep Dream network's cousin, Deep Style, can perform style transfer, where it uses the network's knowledge of painting styles to combine two images, overlaying the textures, shapes, and colors of one, onto the composition of another (check out the Deep Dream feed for the source images as well as the results):

Candy hearts and citrus fruits combined by Conrad Roberts

A house and clouds combined by Don Eidson

Style transfer was originally developed by a research team from the University of Tubingen in Germany, who released A Neural Algorithm of Artistic Style, but you can now see style transfer algorithms everywhere: Google, Adobe, Facebook, and tons of standalone apps like Prisma use style transfer to make your photos look they were painted by Van Gogh or Picasso.

One neural network-driven style transfer art project recently went viral on Twitter: dinosaurs and flowers by Chris Rodley (@chrisrodley) used the web app DeepArt.io to combine books of prehistoric beasts with flowers and 19th-century engravings of fruit:

Due to the crazy amount of interest the internet expressed in these floral monstrosities, Chris Rodley hosted the answers to some frequently asked questions on his blog about how to do it yourself.

If you want to get really deep into the code and set up your own neural networks, there are open source projects available on Github that you can use to start, or try a web app like DeepArt.io or Ostagram.

Unlike a simple photo filter, web apps like these let you enter both images: the one you want to apply style to and the one to take the style from. It's a very different process when the user chooses both parts of the equation (as opposed to when we let the neural network impose its own idea of a style) because, as we'll see in later examples of neural network processed images, the source material really matters.

Neural networks have a very vague sense of what cats are made of

Let's take cats as a wild example. Imagine a neural network exists that wants to turn anything you feed into it into what it thinks a cat is.

First, of course, the network has to have some definition of what a cat is, which is easier said than done.

Here are several projects illustrating the fact that sometimes neural networks are very bad at finding out exactly what things are.

This image set, which circulated on the internet as "Adversarial images for machine learning," was attributed to @teenybiscuit. These are problems that even humans might not figure out at first glance: is that a chihuahua? Or a blueberry muffin?

Chihuahua or muffin via Imgur

But it turns out that neural networks can be tricked even more easily than human eyes, as explained in this article by Julie Evans: "How to trick a neural network into thinking a panda is a vulture." Without getting deep into the math, these mistakes are incredibly easy for neural networks to make.

Sam Winiger, who also created neural networks that wrote in the style of a TED Talk, shared his attempts at "computational comedy" by having a neural network create image captions that are occasionally hilariously wrong:

via Medium

The pix2pix image demo Edges 2 Cats uses a Generative Adversarial Network (or GAN) which is another type of neural network. In this particular demo, it has been trained on 2000 stock photos of cats, and will try to transform any line drawing into a cat-like object.

Pix2pix example of edges2cats

Flagrant abuse of edges2cats

As you can see, misuse of this tool creates fuzz-covered balls of weird that resemble animals just enough to be disturbing.

And it's even worse when pix2pix tries to make people.

As we saw in Deep Dream's manipulations, we've blown through the uncanny valley with neural network image processing and into the Uncanny Mariana Trench.

Motherboard covered the use of the same pix2pix algorithm in a Dutch photo-generation tool that was trained on many images of a single person and has since been shut down after its viral popularity incurred crippling server costs:

(To see more examples of the now-defunct fotogenerator, check out coverage by Digg and The Verge)

For those who are afraid that artificial intelligence-powered voice and video manipulation will mean that in the future no digital artifact can by trusted, these experiments should give you hope. For now, you can tell when a neural network is generating images, because it still doesn't look quite right in the best scenarios, and in the worst cases, it's the stuff of nightmares.

Permalink