Click the “AI Insight” above and select “Star” to follow the public account.
Heavyweight content delivered first-hand.
From: Open Source Frontline (ID: OpenSourceTop)
Comprehensive from:
https://github.com/yuanxiaosc/DeepNude-an-Image-to-Image-technology, programmers, etc.
Some time ago, a programmer developed an application called DeepNude. “Is Technology Innocent?” The AI stripping app was taken offline just hours after its launch.
The app is easy to use: open the software → convert → generate photos. From the results provided online, the effect can indeed be quite realistic, as shown below:
Currently, this program has been taken offline, but a user found a project related to image generation and image restoration technology concerning DeepNude on GitHub, created by yuanxiaosc.
This repository contains the pix2pixHD algorithm (proposed by NVIDIA) used in DeepNude, and more importantly, the general Image-to-Image theory and practical research behind DeepNude.
Image-to-Image Demo
This section provides a playable Image-to-Image demo: turning black-and-white sketches into colorful representations of cats, shoes, and handbags. The DeepNude software primarily uses Image-to-Image technology, which theoretically can convert any input image into any desired image.
In the left box, draw a simple sketch of a cat as you imagine it, then click the process button to output a model-generated cat.
Experience address: https://affinelayer.com/pixsrv/
DeepNude’s Technology Stack
-
Python + PyQt
-
PyTorch
-
Deep Computer Vision
Image-to-Image Theoretical Research
This section outlines the AI/deep learning theories (especially in computer vision) related to DeepNude.
1. Pix2Pix
Pix2Pix is a general solution for the image-to-image translation problem using conditional adversarial networks proposed by Berkeley University. (GitHub address: https://github.com/phillipi/pix2pix)
2. Pix2PixHD
Obtaining high-resolution images from semantic maps. A semantic map is a colored image where different color blocks represent different types of objects, such as pedestrians, cars, traffic signs, buildings, etc. Pix2PixHD takes a semantic map as input and generates a high-resolution realistic image from it. Previous techniques mostly generated rough low-resolution images that did not look realistic. This research generates 2k by 1k resolution images, which are very close to full HD photos. (GitHub address: https://github.com/NVIDIA/pix2pixHD)
3. CycleGAN
CycleGAN uses a cycle-consistency loss function to achieve training without paired data. In other words, it can convert from one domain to another without a one-to-one mapping between the source and target domains. This opens up possibilities for many interesting tasks, such as photo enhancement, image colorization, style transfer, etc. You only need source and target datasets.
Using the CycleGAN neural network model, four main functions are achieved: photo style transfer, photo effect enhancement, seasonal changes in scenery in photos, and object conversion.
4. Image Inpainting
In the demonstration video, you simply need to use a tool to brush away unwanted content in the image, even if the shape is irregular, NVIDIA’s model can “restore” the image, filling in the erased area with a very realistic picture. It’s like one-click photo editing, and there are “no traces of editing”. This research comes from the team of Guilin Liu and others at NVIDIA, who released a deep learning method that can edit images or reconstruct damaged images, even if the image has holes or missing pixels. This is currently the state-of-the-art method as of 2018.
In fact, it may not require Image-to-Image. We can use GANs to generate images directly from random values or from text:
1. Obj-GAN
A new AI technology developed by Microsoft Research AI, Obj-GAN can understand natural language descriptions, sketch, synthesize images, and refine details based on the sketch framework and individual words provided by the text. In other words, this network can generate images of the same scene based on textual descriptions of everyday scenes.
Effects
Model
2. StoryGAN
Microsoft’s new research proposes a new type of GAN—ObjGAN, which can generate complex scenes based on textual descriptions. They also proposed another GAN that can draw stories—StoryGAN, which outputs comic strips based on the input text of a story.
The current best text-to-image generation model can generate realistic bird images based on a single sentence description. However, text-to-image generators can do much more than just generate a single image from one sentence. Given a multi-sentence paragraph, it can generate a series of images, each corresponding to a sentence, fully visualizing the entire story.
Effects
The most commonly used Image-to-Image technology today is beauty apps, so why not develop a more intelligent beauty camera?
Technology is innocent, but let’s not entertain ourselves to death. It’s important to understand what can and cannot be done with these technologies. I hope everyone can use these technologies for good.
Recommended Reading
(Click the title to jump to read)
A complete AI learning path, the most detailed resource compilation!
Essentials | Selected historical articles from the public account
My deep learning entry route
My machine learning entry roadmap
Found this article helpful? Please share it with more people.
Follow AI Insight and star it, to get the latest AI insights.
The latest AI insights, I am reading❤️