What Is Pix2Pix, and How Do You Use It?

Artificial intelligence has made significant strides in recent years. And now, with help from machine learning, it's becoming more intelligent by the day.

It's against that backdrop that a new solution called Pix2Pix, from Dutch public broadcasting network NPO, is now making its rounds on the web.

Pix2Pix creations can have uneven results, even if you supply a lot of detail. Original Image
If you don't know much about Pix2Pix, there's a good reason for that: It's in its fledgling stages and relies on technology that still needs some polishing. However, by relying on a neural network and using machine learning, it can generate some rather interesting creations.

Using a simple line drawing as its reference point, Pix2Pix converts it into an oil painting based on its understanding of shapes, human drawings and the real world. And if you're an accomplished artist, it might actually spit out something interesting.

Read on to learn more about Pix2Pix, how it works and whether it's something you should try out.

What is Pix2Pix?

Pix2Pix is the brainchild of the Dutch public broadcasting network NPO. It was developed as part of the NPO's desire to create its own artificial intelligence system and see how well machines would do at analyzing human creations and turning them into lifelike paintings.

Pix2Pix uses a range of features, including an algorithm to interpret information, a discriminator that can analyze both the original and the output when it's created, and a neural network that it relies on to guess what the image should look like.

In other words, your creations are tossed around inside a computer's brain to spit out an image that hopefully looks realistic.

MORE: Best Camera Phone - Top-Rated Smartphone Cameras

What can Pix2Pix do, exactly?

Pix2Pix is a service that can instantly convert your drawings and illustrations  into paintings. It uses artificial intelligence, machine learning and conditional adversarial networks (more on these below), among other features, to create the picture.

You can try out Pix2Pix and either submit a drawing or create your own quick drawing in the small white box on the dedicated site. From there, you can click to convert the image into a painting and spit out something that hopefully looks like your original.

What is a conditional adversarial network, and why should I care?

At the heart of the Pix2Pix technology is something called a conditional adversarial network (CAN).

Despite its nebulous name, CAN can be boiled down rather simply. The technology takes an image that you want to create and delivers what it believes is the best result, based on its understanding of what the image should look like. It then uses artificial intelligence to determine how accurate the depiction is and uses that information to create more images.


The hope is that as time goes on, and the system learns more about the user's desires and accuracy requirements, CAN will create better-looking pictures.

However, it's important to note that, for CAN to work, users around the world need to submit images to the system. Generally speaking, the more opportunities Pix2Pix has to analyze and create pictures, the more opportunities it has to learn what works and what doesn't. And that, of course, should translate to better pictures.

MORE: 6 Pro Tips for Taking Better Pictures With Your iPhone 7 Plus

Where can I try it out?

Pix2Pix itself can be accessed here. As you can see, it features two boxes: one for the input, or what you would draw, and another for the output, or what the system creates.

If you want to try it out, clear the box on the left, and use your mouse to draw the image of your choice. Make a face, draw an object or create whatever else you want. When you're ready, click rocess. Your output box will be populated automatically with the Pix2Pix creation.

If you like what you see, you can keep your creation by clicking the Save button. It will immediately download a PNG file both of your input and your output.

When you use the service, you might be surprised by how fast it is. That's an accomplishment of artificial intelligence and machine learning that shouldn't be overlooked.

Credit: GitHub Original Image

Can I submit my own professional drawings?

Yes, you can! In fact, it might get you better results.

The more detail there is in your drawing, the more likely it is that Pix2Pix will identify different features within your creation and output what you hope might be an accurate painting.

MORE: The Best Photo Editing Software and Apps - Beginner to Pro

How much does it cost?

Here's the best part: Pix2Pix is free to use. And you can submit as many pictures as you want without fear of incurring a charge. 

This thing doesn't look right. What gives?

Well, that's part of the problem with artificial intelligence today. And it also goes back to what we said about conditional adversarial networks. The more inputs, or drawings, Pix2Pix gets and can process through its servers, the more accurate it'll become. But right now, its ability to deliver accurate results is rather minimal.

Although the examples provided by the creators show exceedingly sophisticated images, or outputs, I spent considerable time trying to create something (anything!) that would resemble a real person, and I didn't have much luck.

But if you keep trying (and your friends keep trying), there's a possibility that Pix2Pix will improve and deliver better results. 

Can I do more than portraits?

If you were to ask most industry experts, they'd say the future of artificial intelligence and machine learning is bright. They'd say that as computers become more sophisticated — and therefore, more intelligent — services like Pix2Pix will only get better.

But it's not just portraits that you can expect to look better in the future.


Credit: Christopher Hesse/Image-to-Image Demo Original Image
The same Pix2Pix technology model has been used in other tools to allow computers to create drawings. For instance, a developer named Christopher Hesse created a service that lets you create a wide array of paintings.

In one of his tools, called Facades, you can create walls, doors, windows and more. Then, based on your creations, the service spits out what looks like a real building.

MORE: 12 Tips for Taking Amazing Landscape Photos

And if you're a cat lover, you can use the same site to draw a cat sketch and have the service create your real-world version.

Hesse's tool even lets you create shoes and handbags.

According to Hesse, its tools were developed by using reference images of cats, shoes, handbags and buildings. The service evaluates what a person draws, tries to compare that to the reference images and then creates its own.

But much like the human portraits, the cat, shoe and handbag models I created left much to be desired.

Loading ...