In early December 2024, a promotional video appeared online, featuring Dr. Zhang Wenhong, the director of the National Infectious Disease Medical Center and head of the Infectious Disease Department at Huashan Hospital affiliated with Fudan University, passionately endorsing a certain food product.
Many netizens, trusting Dr. Zhang, not only purchased the product themselves but also actively shared the video in family groups.However,Dr. Zhang quickly clarified that the video was not recorded by him and was likely AI-generated.
Network Screenshot
In recent years, with the rapid development of artificial intelligence technology, such AI forgery video incidents have been frequently exposed. Criminals exploit AI technology to impersonate celebrities for fraud, publish false advertisements, and even create forged videos to defraud large sums of money, severely threatening the social trust system and information security.
Today, we will analyze such incidents from a technical perspective.
AI Forged Video Technology:
From Generative Adversarial Networks to Deepfake
The technology used for these AI forged videos did not just appear recently; research on face replacement and image generation began in academia in the 1990s. In 2014, Ian Goodfellow proposed Generative Adversarial Networks (GAN), enabling computers to generate more realistic and high-quality images.
Generative Adversarial Networks consist of two parts: the generator and the discriminator. The generator creates content, while the discriminator checks whether the created content is real and provides feedback to the generator. During the training process, the two engage in a back-and-forth, improving their capabilities through this adversarial process, ultimately allowing the generator to produce very realistic images.
In 2017, a user created an account named “deepfakes” on the online community Reddit and released several celebrity face-swap videos made using GAN technology. Since then, the term “Deepfake” has been widely used, typically referring to the technology used for these face-swap videos.
In January 2018, a desktop application called FakeApp was released. Subsequently, open-source tools with similar functionalities, such as Faceswap and DeepFaceLab, emerged. These tools significantly lowered the barrier for creating Deepfake videos, allowing ordinary users without professional knowledge to easily generate face-swap videos.
With ongoing advancements in related technologies, today’s Deepfake videos are not only higher in resolution and more naturally synchronized in facial expressions compared to earlier versions, but they also require less data and shorter training times.Earlier Deepfake training often needed hundreds to thousands of images of the target person or several minutes to hours of video to capture facial features from different angles and expressions. However, with the development of GAN technology itself and the emergence of techniques like Transfer Learning and Few-Shot Learning, now only a few dozen or even a single photo is needed to generate a Deepfake video.
Copyrighted images, unauthorized reproduction may lead to copyright disputes
Additionally, early Deepfakes could only generate visuals, but now, combined with Voice Cloning and Voice Style Transfer technologies, they can also generate highly realistic target voices and synchronize mouth movements with the audio in the video.
In simple terms, now it only takes a single photo and a few seconds of audio to generate a Deepfake video.Of course, if more photos and longer audio are available, the generated video will be even more realistic.
Positive Applications and Negative Impacts of Deepfake Technology
Although Deepfake is a form of “forgery,” it can have many positive applications when done with the consent of the person being “forged.” For example, in 2019, British football star David Beckham released a video calling for the eradication of malaria, using Deepfake technology to generate his voice in nine languages, including Swahili and Yoruba, aside from English.
Moreover, the various technologies used in Deepfake have broad applications in fields like digital humans, video hosting, film production, education and training, and psychological rehabilitation.
However, any technology can be misused. Before the previously mentioned fake Dr. Zhang, there were already instances of fake Jin Dong, fake Jack Ma, fake Lei Jun, fake Yu Donglai, and others. Impersonating celebrities to sell products is not the worst misuse; Deepfake technology has also been used in many more malicious contexts, such as fraud.
On December 20, 2024, the BBC reported a story titled “Love Scammers Use Deepfake to Steal £17,000 from Me.” The victim was 77-year-old Nikki MacLeod. The scammer claimed to work on an oil drilling platform and asked Nikki to purchase Steam gift cards and transfer money via Paypal to obtain internet access on the platform and cover travel expenses to meet her in Scotland. Nikki was initially skeptical but believed the scammer after seeing a video from the drilling platform.
In January 2024, an employee at a Hong Kong company transferred $25 million from the company account to a scammer. The scammer conducted a video call impersonating the Chief Financial Officer, demanding the transfer. During the video call, the employee saw not only the “CFO” but also other “colleagues.”
According to a report released by the renowned accounting firm Deloitte in May 2024, Deepfake fraud in the US increased by 700% in 2023, resulting in $12.3 billion in losses, and this figure could reach $40 billion by 2027.
Copyrighted images, unauthorized reproduction may lead to copyright disputes
Besides fraud, Deepfake technology can also be used to deceive facial recognition systems. Many mobile applications use facial recognition to verify user identity. To prevent the use of photos to impersonate faces, current facial recognition technologies often employ liveness detection, which requires special hardware for techniques like 3D structured light, but only some phones support this.
Currently, many applications still use 2D facial recognition based on the front camera of the phone. 2D facial recognition liveness detection typically requires users to perform actions such as blinking or nodding, along with the screen flashing specific colored lights. As a result, some people have used Deepfake technology combined with other methods to deceive facial recognition systems and steal online accounts.
How to Identify and Detect Deepfake Videos?
Deepfake videos that are not well-made can often be identified by the naked eye. For instance, unnatural facial expressions or eye movements, too few blinks, blurred facial edges, or unnatural transitions with the background, and mismatched lighting effects on the face compared to the surrounding environment are common signs. However, as Deepfake technology improves, these abnormal features become less frequent.
Currently known Deepfake technology can mimic facial expressions, but it cannot replicate the deformations caused by external pressure on the face. Therefore, during video calls, if there is suspicion of Deepfake fraud, one can request the other party to press one side of their nose or one side of their cheek with their index finger.
Besides visual detection, using AI to identify AI-generated content is also a popular research direction. For example, synthetic videos may exhibit discontinuities between frames; temporal consistency analysis could reveal anomalies. Additionally, slight color changes in the skin due to heartbeat rhythms can provide pulse information, while Deepfake videos may lack this feature.
However, we must also recognize that the GAN technology used in Deepfake inherently consists of both a generator and a discriminator; any detection technology can also be incorporated into the Deepfake discriminator, enabling the generation of content that is difficult to detect.
Copyrighted images, unauthorized reproduction may lead to copyright disputes
In recent years, our country has implemented regulations such as the “Regulations on the Management of Deep Synthesis of Internet Information Services” and the “Interim Measures for the Management of Generative Artificial Intelligence Services,” but these laws mainly constrain the platforms providing relevant services. With the development of software and hardware technology, Deepfake can now be completed using models running on personal computers.
Therefore, to address the various issues brought by Deepfake, comprehensive governance mechanisms are needed; in the future, a multi-faceted collaboration of technology, platforms, and laws will be required.Using Deepfake to impersonate celebrities and deceive others into purchasing products not only potentially violates Articles 1019 and 1024 of the Civil Code, infringing on others’ portrait rights and reputation rights, but may also involve criminal offenses such as fraud under Article 266 of the Criminal Law and false advertising under Article 222 of the Criminal Law. Legal action is necessary against illegal and criminal acts involving Deepfake.
Planning and Production
Author: Yu Yang, Head of Tencent Xuanwu Laboratory
Reviewed by: Yu Naigong, Head of the Robotics Engineering Program at Beijing University of Technology, Director of the Robotics Research Center at the Beijing Academy of Artificial Intelligence, Ph.D. Supervisor
Zhao Jingwu, Associate Professor at the School of Law, Beihang University, Deputy Director of the Key Laboratory of Legal Strategy and Management of the Ministry of Industry and Information Technology, and Deputy Director of the International Governance Research Base for Cyberspace
Editor: Lin Lin
Proofreader: Xu Lai
Related Articles
1. A Stone Actually Grew Hair?! But the Truth Is Scarier Than You Think…
2. “Drinking Water” Really Works! Latest Research Shows: Drinking More Water Helps with Weight Loss and Treating Kidney Stones…
3. Why Do Women Always Complain of Pain? It’s Not Just Whining, But…
4. The Must-Eat Divine Vegetable in Winter! Nutritious, Delicious, and Cancer-Fighting; Promise Me You’ll Try It
5. Why I Don’t Recommend You to Be “Open and Honest” with Your Partner About Everything?
The cover image and images in this article come from copyright-protected stock images.
Unauthorized reproduction may lead to copyright disputes. For original text and images, please reply “Reprint” in the background.
