Deepfake vs Anti-Deepfake: A Long-Term Cat and Mouse Game.
The following report from Zhiyuan on June 18 asks, can you spot any issues with the photos of these people?
These smiling individuals are all fake, generated by a well-known AI model called StyleGAN, and do not exist in real life.
Deepfake technology is becoming increasingly powerful, making it difficult for people to distinguish the authenticity of the images it creates. If this technology is used maliciously on a large scale, it could lead to endless problems.
In response, Facebook has collaborated with Michigan State University (MSU) to propose a new research method that can not only detect fake images but also reverse-engineer how the AI model that generated the fake image was designed.
Notably, some generative models have never been seen before; through a series of hyperparameter analyses, this new research method can still identify common sources of some fake images.
This will help effectively trace misleading images circulating on various social networks and uncover coordinated misinformation or other malicious attacks initiated by deepfakes.
Facebook research scientist Tal Hassner stated, “We achieved state-of-the-art results on standard benchmarks.”
Identifying Image Sources with ‘Fingerprints’
How does Facebook’s new AI method work?
▲ Model analysis process
Researchers first ran a set of deepfake images through a Fingerprint Estimation Network (FEN) to estimate the fingerprint details left by the AI generative model.
For humans, fingerprints are like personal identifiers, characterized by invariance, uniqueness, and classifiability.
Similar identifying features exist in devices. For example, in digital photography, due to imperfections in the manufacturing process, specific devices leave unique patterns on every image they produce, which can be used to identify the digital camera that generated the image. This pattern is referred to as a device fingerprint.
Similarly, image fingerprints are unique patterns left by generative models in the images they create, which can be used to identify the generative model from which the image originates.
Before the era of deep learning, researchers often used a small set of hand-crafted, well-known tools to generate images. The fingerprints of these generative models were estimated based on their handcrafted features. However, deep learning allows tools to generate images infinitely, making it impossible for researchers to identify fingerprint attributes through handcrafted features.
Given the endless possibilities, researchers decided to estimate fingerprints based on general attributes, using different constraints including fingerprint size, repetitive nature, frequency range, and symmetric frequency response.
These constraints are then fed back into the FEN through different loss functions to enforce that the generated fingerprints possess these desired properties. Once the fingerprint generation is complete, it can be used as input for model analysis.
By identifying the unique fingerprints in these images, Facebook’s AI can discern which fake images were created by the same generative model.
▲ Image attribution: Identifying which images were produced by the same generative model
Simulating Hyperparameters to Infer Deepfake Model Structures
Each generative model has its own unique hyperparameters.
Hyperparameters are variables used to guide the model’s self-learning process. For example, the hyperparameter settings for the model’s network structure and training loss function type will affect how the generated images are created and their results.
If the various hyperparameters can be understood, it is possible to identify the generative model that created a specific image.
To better understand hyperparameters, the Facebook team likens generative models to cars, with hyperparameters representing various specific engine components. Different cars may look similar, but under the hood, they can have very different engines and components.
Researchers describe their reverse engineering technique as somewhat akin to identifying car parts by sound, even if they have never heard of that car before.
▲ Reverse engineering technology can identify features of unknown models
Once the system can consistently separate true fingerprints from deepfake fingerprints, it will dump all the fake fingerprints into an analysis model to simulate their various hyperparameters.
Through its model analysis method, researchers can estimate the network structure used to create deepfakes, such as how many layers there are or what loss functions were trained.
For training convenience, they normalized some continuous parameters in the network structure and performed hierarchical learning on the types of loss functions.
Given the significant differences in network architecture and training loss functions among generative models, mapping from deepfakes or generated images to the hyperparameter space allows them to critically understand the characteristics of the models used to create them.
▲ Through model analysis, it is possible to infer how unknown models are designed
Synthesizing 100,000 Fake Images from 100 Generative Models
To test this method, the research team at Michigan State University generated a dataset of 100,000 synthetic images from 100 publicly available generative models.
Each of these 100 generative models corresponds to an open-source project developed and shared by researchers from the scientific community. Some open-source projects have already released fake images.
In this case, the Michigan State University research team randomly selected 1,000 images. In instances where there were no available fake images from the open-source projects, the research team ran their published code to generate 1,000 synthetic images.
Considering that the test images may come from generative models not visible in the real world, the research team simulated real-world applications through cross-validation to train and evaluate their model on different splits of the dataset.
▲ Each image generated from the 100 generative models produces an estimated fingerprint on the left and a corresponding spectrum on the right. Many spectra show different high-frequency signals, while some spectra appear similar to each other.
In addition to model analysis, its FEN can be used for deepfake detection and image attribution. For these two tasks, researchers added a shallow network that inputs the estimated fingerprints and performs binary (deepfake detection) or multi-class classification (image attribution).
Although Facebook’s fingerprint estimation was not tailored for these tasks, researchers claim that they still achieved competitively technical-level results, indicating that their fingerprint estimation exhibits excellent generalization capabilities.
The diverse collection of deepfake images from 100 generative models means their model was built through representative selection, with better generalization across human and non-human representations.
Despite some original images used to generate deepfakes being real personal images from publicly available facial datasets, the Michigan State University research team initiated forensic-style analysis using deepfake images rather than the original images used to create them.
Since this method involves deconstructing deepfake images into their fingerprints, the research team analyzed whether the model could map the fingerprints back to the original image content.
The results indicated that this was not the case, confirming that the fingerprints mostly contained traces left by the generative model rather than the original deepfake content.
All fake face images used in this study, as well as all experiments in the reverse engineering process, came from Michigan State University.
Michigan State University will open the dataset, code, and training models to the broader research community to facilitate research across various fields, including deepfake detection, image attribution, and reverse engineering of generative models.
Conclusion: Deepfake vs Anti-Deepfake, A Long-Term Cat and Mouse Game
This research by Facebook and Michigan State University pushes the boundaries of understanding deepfake detection, introducing model analysis concepts more suitable for real-world deployment.
This work will provide tools for researchers and practitioners to better investigate coordinated misinformation events involving deepfakes and open new directions for future research.
However, it is worth noting that even the most advanced results are not entirely reliable. Last year, Facebook held a deepfake detection competition, and the winning algorithm could only detect 65.18% of AI-manipulated videos.
Researchers believe that using algorithms to discover deepfakes remains an “unsolved problem.” Part of the reason is that the generative AI field is very active, with new technologies released every day, making it nearly impossible for any detector to keep up completely.
When asked whether there would be generative models that this new method would fail to detect, Hassner agreed: “I expect that will happen.” He believes that the development of deepfakes and the development of deepfake detection will “continue to be a cat-and-mouse game.”
Source: Facebook AI, The Verge
（This article is original content from Zhiyuan, part of the NetEase News • Netease Account Feature Content Incentive Program. Unauthorized reproduction is prohibited.）
