Source:OpenSourceTop
ID:OpenSourceTop
A while ago, a programmer developed an application called DeepNude. “Is Technology Innocent? AI Nudity App Taken Down Hours After Launch”
The app is very simple to use: open the software → convert → generate a photo. Based on the results provided online, the effect can indeed be quite realistic, as shown below:
Currently, this program has been taken offline, but I found a GitHub project related to the image generation and image restoration technologies used in DeepNude, created by yuanxiaosc.
This repository contains the pix2pixHD algorithm (proposed by NVIDIA) used in DeepNude, and more importantly, the general theory and practical research behind Image-to-Image.
Image-to-Image Demo
This section provides a demo of Image-to-Image: converting black and white sketches into colorful cats, shoes, and handbags. The DeepNude software primarily uses Image-to-Image technology, which theoretically can convert any image you input into any image you desire.
In the left box, draw a simple sketch of a cat as you imagine it, then click the process button to output a model-generated cat.
Experience it here: https://affinelayer.com/pixsrv/
DeepNude’s Technology Stack
-
Python + PyQt
-
pytorch
-
Deep Computer Vision
Image-to-Image Theoretical Research
This section elaborates on the AI/deep learning theories related to DeepNude (especially in computer vision).
1. Pix2Pix
Pix2Pix is a general solution for image-to-image translation problems using conditional adversarial networks proposed by researchers at Berkeley University. (GitHub link: https://github.com/phillipi/pix2pix)
2. Pix2PixHD
Obtaining high-resolution images from semantic maps. A semantic map is a color image where different colored regions represent different types of objects, such as pedestrians, cars, traffic signs, buildings, etc. Pix2PixHD takes a semantic map as input and generates a high-resolution realistic image from it. Previous technologies mostly generated rough low-resolution images that did not appear realistic. This research, however, generates images with a resolution of 2k by 1k, which is very close to full HD photos. (GitHub link: https://github.com/NVIDIA/pix2pixHD)
3. CycleGAN
CycleGAN uses a cycle-consistency loss function for training without requiring paired data. In other words, it can convert from one domain to another without a one-to-one mapping between the source and target domains. This opens up the possibility of performing many interesting tasks, such as photo enhancement, image coloring, style transfer, etc. You only need source and target datasets.
Using the CycleGAN neural network model to achieve four major functionalities: photo style transfer, photo effect enhancement, seasonal changes in landscape photos, and object conversion.
4. Image Inpainting
In the demonstration video, simply use the tool to brush away unwanted content in the image, even if the shapes are irregular, NVIDIA’s model can “restore” the image, filling in the erased gaps with very realistic visuals. This is truly one-click photo editing, and there are “no traces of Photoshop”. This research comes from a team led by Guilin Liu at NVIDIA, who released a deep learning method that can edit images or reconstruct damaged images, even if the image has holes or lost pixels. This is currently the state-of-the-art method from 2018.
In fact, Image-to-Image may not be necessary. We can use GANs to generate images directly from random values or from text:
1. Obj-GAN
A new AI technology developed by Microsoft Research AI, Obj-GAN can understand natural language descriptions, draw sketches, synthesize images, and then refine details based on the sketch framework and individual words provided by the text. In other words, this network can generate images of the same scene based on textual descriptions of everyday scenarios.
Effects
Models
2. StoryGAN
Microsoft’s new research proposes a new type of GAN—ObjGAN, which can generate complex scenes based on textual descriptions. They also proposed another GAN—StoryGAN, which can output comic strips based on the text of a story.
The current best text-to-image generation model can generate realistic bird images based on single-sentence descriptions. However, text-to-image generators can do much more than just generate a single image from one sentence. Given a multi-sentence paragraph, it can generate a series of images, each corresponding to a sentence, fully visualizing the entire story.
Effects
Currently, the most widely used Image-to-Image technology should be beauty apps, so why not develop a smarter beauty camera?
Technology is innocent, but we shouldn’t entertain ourselves to death. I hope everyone can use these technologies for good.
Summary:
https://github.com/yuanxiaosc/DeepNude-an-Image-to-Image-technology, programmers, etc.