In early December 2024, a promotional video surfaced online, featuring Dr. Zhang Wenhong, the director of the National Center for Infectious Disease Medicine and the head of the Infectious Disease Department at Huashan Hospital affiliated with Fudan University, passionately promoting a certain food product.
Many netizens, trusting Dr. Zhang, not only made purchases themselves but also actively shared the video in family groups.However,Dr. Zhang quickly clarified: the video was not recorded by him and was likely fabricated using AI.
Network Screenshot
In recent years, with the rapid development of artificial intelligence technology, such AI forgery video incidents have been frequently exposed. Criminals utilize AI technology to impersonate celebrities for fraud, publish false advertisements, and even create forged videos to swindle large sums of money, posing a serious threat to the social trust system and information security.
Today, we will analyze such incidents from a technical perspective.
AI Forged Video Technology:
From Generative Adversarial Networks to Deepfake
The technology used for these AI forged videos is not new; research on face swapping and image generation began in academic circles in the 1990s. In 2014, Ian Goodfellow proposed Generative Adversarial Networks (GANs), enabling computers to generate more realistic and high-quality images.
Generative Adversarial Networks consist of a generator and a discriminator.The generator creates content, while the discriminator detects whether the created content is real and provides feedback to the generator. During the training process, the two engage in a game of offense and defense, enhancing their abilities through confrontation, ultimately enabling the generator to produce very realistic images.
In 2017, someone created an account named “deepfakes” on the online community Reddit and published several celebrity face swap videos made using GAN technology. From then on, the term “Deepfake” began to be widely used, usually referring to the technology used for such face swap videos.
In January 2018, a desktop application called FakeApp was released. Subsequently, open-source tools with similar functionalities, such as Faceswap and DeepFaceLab, emerged. These tools significantly lowered the barrier for creating Deepfakes, allowing ordinary users without professional knowledge to easily generate face swap videos.
With continuous advancements in related technology, today’s Deepfakes not only have higher resolution and more natural facial expression synchronization compared to earlier versions, but they also require less data and shorter training times.Early Deepfake training often required hundreds to thousands of images of the target person or several minutes to hours of video to capture facial features from different angles and expressions. However, with the development of GAN technology itself and the emergence of techniques such as Transfer Learning and Few-Shot Learning, it is now possible to generate Deepfake videos with just a few dozen or even a single photo.
Stock Image, copyright issues may arise from unauthorized use
Additionally, early Deepfakes could only generate visuals; now, combined with Voice Cloning and Voice Style Transfer technologies, they can also generate convincingly realistic target individuals’ voices and synchronize mouth movements with the audio in the video.
In simple terms, now, with just a photo and a few seconds of audio, a Deepfake video can be generated. Of course, if more photos and longer audio are available, the generated video will also be more realistic.
Positive Applications and Negative Impacts of Deepfake Technology
Although Deepfake is a form of “forgery,” it can have many positive applications when the consent of the person being “forged” is obtained. For example, in 2019, British football star David Beckham released a video calling for the eradication of malaria. In the video, he used nine languages, including Swahili and Yoruba, while all languages except English were generated using Deepfake technology.
Furthermore, the various technologies used in Deepfake have broad applications in fields such as digital humans, video streaming, film production, education and training, and psychological rehabilitation.
However, any technology can be misused. Before the previously mentioned fake Zhang Wenhong incident, there were already cases of fake Jin Dong, fake Jack Ma, fake Lei Jun, and fake Yu Donglai. Impersonating celebrities for selling products is not the worst; Deepfake technology has also been used in many more malicious ways, such as fraud.
On December 20, 2024, the BBC reported a story titled “Love Scammers Use Deepfake to Steal £17,000 from Me.” The victim was 77-year-old Nikki MacLeod. The scammer told her he worked on an oil drilling platform and asked Nikki to purchase Steam gift cards and send money via Paypal to obtain internet access on the platform and cover travel expenses to meet her in Scotland. Nikki was initially skeptical but believed him after seeing the video he sent from the drilling platform.
In January 2024, an employee from a Hong Kong company transferred $25 million from the company account to scammers. The scammer conducted a video call with him, impersonating the Chief Financial Officer and requesting the transfer. During the video call, the employee not only saw the “CFO” but also other “colleagues.”
According to a report released by the renowned accounting firm Deloitte in May 2024, Deepfake fraud in the United States increased by 700% in 2023, resulting in losses of $12.3 billion, and this figure could reach $40 billion by 2027.
Stock Image, copyright issues may arise from unauthorized use
Besides fraud, Deepfake technology can also be used to deceive facial recognition systems. Many mobile applications use facial recognition to verify user identity. To prevent photos from impersonating faces, current facial recognition technology often requires liveness detection, which may involve special hardware like 3D structured light, but only some phones support it.
Currently, many applications still rely on 2D facial recognition using front-facing cameras. The liveness detection for 2D facial recognition primarily requires users to perform actions like blinking or nodding, and the screen flashes specific colored lights. Therefore, some people use Deepfake technology combined with other methods to deceive facial recognition systems, thereby stealing online accounts.
How to Identify and Detect Deepfake Videos?
Poorly made Deepfake videos can often be detected by the naked eye. For example, unnatural facial expressions or eye movements, too few blinks, blurry facial edges, or unnatural transitions with the background, and inconsistent lighting effects on the face compared to the surrounding environment, etc. However, as Deepfake technology advances, these abnormal features are becoming less common.
Currently known Deepfake technology can mimic facial expressions but cannot replicate facial deformations caused by external pressure. Therefore, during a video call, if you suspect you are being defrauded by Deepfake, you can ask the other party to press one side of their nostril or one side of their cheek with their index finger.
Besides visual identification, using AI to recognize AI-generated content is also a popular research direction. For example, synthetic videos may have discontinuities between frames, and temporal consistency analysis could reveal anomalies. Additionally, human heartbeat can cause slight color changes in the skin that are rhythmically consistent with the pulse; Deepfake videos may lack this feature.
However, we must also recognize that the GAN technology used in Deepfake inherently consists of a generator and a discriminator, meaning that any detection method for Deepfake can also be incorporated into the Deepfake discriminator, making it challenging to detect the forged content.
Stock Image, copyright issues may arise from unauthorized use
In recent years, China has implemented regulations such as the “Regulations on the Management of Deep Synthesis in Internet Information Services” and the “Interim Measures for the Management of Generative Artificial Intelligence Services,” but these laws mainly restrict the platforms providing related services. With advancements in software and hardware technology, Deepfake can now be fully executed using models running on personal computers.
Therefore, to address the various issues posed by Deepfake,comprehensive governance mechanisms are needed, requiring multilateral collaboration among technology, platforms, and law in the future.Using Deepfake to impersonate celebrities and deceive others into purchasing products not only potentially violates Articles 1019 and 1024 of the Civil Code, infringing on others’ portrait rights and reputation rights, but could also involve criminal acts under Article 266 of the Criminal Law for fraud and Article 222 for false advertising. Actions involving illegal or criminal use of Deepfake must be prosecuted according to the law.
Planning and Production
Author丨Yu Yang, Head of Tencent Xuanwu Lab
Reviewed by丨Yu Naigong, Head of the Robotics Engineering Program at Beijing University of Technology, Director of the Robotics Research Center at the Beijing Academy of Artificial Intelligence, PhD Supervisor
Zhao Jingwu, Associate Professor at the School of Law at Beihang University, Deputy Director of the Key Laboratory of Industrial and Information Law and Management of the Ministry of Industry and Information Technology, Deputy Director of the International Governance Research Base for Cyberspace
Editor丨Lin Lin
Proofreader丨Xu Lai
Related Articles
1. Stones Actually Grow Hair?! But the truth is scarier than you think…
2. “Drinking More Water” Really Works! Latest research shows: drinking more water helps with weight loss, treats kidney stones…
3. Why Do Women Always Complain of Pain? It’s Not Just Whining, But…
4. A Must-Eat Divine Vegetable in Winter! Nutritious and Delicious, Plus Anti-Cancer Properties, Promise Me You’ll Try It
5. Why I Don’t Recommend You to Be “Open and Honest” with Your Partner in Everything?
The cover image of this article and images within are from copyright stock.
Reproduction may lead to copyright disputes; for original text and images, please reply “Reprint” in the background.
Light Up “Seeing”
Let’s Gain Knowledge Together!
