Source丨National Information Technology Standardization Technical Committee
01.
What is Face Recognition?
Face Recognition is a biometric recognition technology based on the facial feature information of individuals for identity verification. In recent years, with the rapid development of technologies such as Artificial Intelligence, Computer Vision, Big Data, Cloud Computing, and Chips, face recognition technology has made significant progress and been successfully applied in various scenarios.
Broadly speaking, face recognition actually includes a series of related technologies for constructing a face recognition system, including face view acquisition, face localization, face recognition preprocessing, identity confirmation, and identity search; narrowly, face recognition specifically refers to the technology and systems that confirm or search for identity through faces. Additionally, in some application scenarios, quality evaluation and liveness detection may also involve algorithm modules.
The application modes of face recognition mainly include three types:
(1) Face Verification: Determines whether two facial images belong to the same person, commonly used for identity authentication such as identity verification.
(2) Face Identification: Given a facial image, determines whether it exists in a registered database, and if so, returns specific identity information, commonly used for static retrieval or dynamic control.
(3) Face Clustering: Given a batch of facial images, classifies images of the same person into the same category and different people into different categories, commonly applied in smart albums and individual records.
1. Principles of Face Recognition Technology
Current mainstream face recognition algorithms mainly include five major steps: face detection, face preprocessing, feature extraction, comparison recognition, and liveness detection. Among these, face detection, face preprocessing, and feature extraction can collectively be referred to as the face view analysis process, which involves detecting faces from videos and images, selecting suitable face images based on image quality, and extracting facial feature vectors for subsequent comparison recognition; comparison recognition can be divided into face verification (1:1) and face identification (1:N); liveness detection algorithms are used to determine whether the facial images processed in face recognition are collected from real human bodies.
In practical applications, in addition to the aforementioned face recognition algorithms, front-end view acquisition technology, face data storage technology, and application software management technology are also important technical components of face recognition technology applications.
2. Advantages and Limitations of Face Recognition Technology
Technical Advantages. Among various biometric recognition methods, face recognition technology has its unique advantages, thus holding an important position in biometric identification.
(1) Non-intrusiveness: Face recognition can achieve good recognition results without interfering with people’s normal behavior; as long as they stay naturally in front of the camera for a moment, the user’s identity will be correctly recognized.
(2) Convenience: The devices for face recognition collection are simple and quick to use. Generally, common cameras can be used to collect facial images without the need for particularly complex specialized equipment. Image acquisition can be completed within seconds.
(3) Friendliness: The method of identifying identity through face recognition is consistent with human habits, allowing both humans and machines to use facial images for identification.
(4) Non-contact: Face image collection does not require users to have direct contact with devices. Additionally, face images can be collected from relatively far distances. Cameras equipped with optical zoom lenses can increase the focal length by more than ten times, extending the depth of field to over 50 meters, effectively capturing clear images of distant faces.
(5) Scalability: After face recognition, further processing and application of the recognition results can lead to many practical application solutions, such as access control, facial image search, time card swiping, and identification of unauthorized personnel in various fields.
(6) Strong concealment: In security fields, there are strong requirements for system concealment, and face recognition has advantages over fingerprint methods in this respect.
(7) Strong post-event tracking capability: The facial information recorded by the system is an important and easily utilized clue, which is more conducive to post-event tracking applications.
(8) High accuracy: Compared to features like body and gait, facial features have stronger distinguishability and lower false positive rates, allowing for a much larger database scale; currently, ultra-large-scale (billion-level) facial searches are already practical.
Technical Limitations. Face recognition technology faces certain limitations due to issues such as similar faces, age, algorithm bias, diverse scenarios, and the ease of publicly obtaining facial images.
(1) Similar faces are difficult to resolve: Identical twins or very similar faces are prone to misidentification, and currently, no new technology can completely solve this issue. A report from NIST indicates that in most cases, twins can still be distinguished by score differences, but often remain above the threshold, leading to poor application effects in open environments.
(2) Algorithm bias issues: Current facial recognition algorithms largely depend on data samples, but facial data samples from different populations vary, leading to differences in algorithm recognition capabilities for different regions and age groups.
The National Institute of Standards and Technology (NIST) has indicated significant differences in facial recognition software across different regions, races, genders, and ages. For example, children, the elderly, and individuals from underrepresented ethnicities or skin tones have relatively low recognition rates, which is an urgent issue to address.
(3) Recognition rates are easily affected by various factors: Existing facial recognition systems can achieve satisfactory results under ideal user cooperation and collection conditions. However, when users do not cooperate or collection conditions are suboptimal, recognition rates can be affected. For example, according to NIST’s test reports, the error rates of most algorithms increase by an order of magnitude or more when wearing masks, and cross-age and large-angle factors can also cause varying degrees of decline.
(4) Age changes affect recognition: As people age, their facial appearance changes, especially in adolescents, where these changes are more pronounced. Different age groups also exhibit varying recognition rates with facial recognition algorithms.
(5) Security issues: Facial recognition systems face risks of hacking regarding information storage. Therefore, data encryption is crucial. As technology continues to advance, enhancing the security of facial recognition technology is necessary.
3. Development Trends of Face Recognition Technology
With the widespread application of face recognition technology, it continuously promotes the sustained development of the technology itself. Basic algorithm research, face reconstruction technology, mask-wearing face recognition, 3D face recognition technology, new face acquisition technologies, face clustering technology, and low-quality face recognition technology are hot topics of interest in both industry and academia, indicating the development trends of face recognition technology.
Basic algorithm technology hotspots include model structure design, loss function design, unsupervised/semi-supervised learning algorithms, and distributed self-learning algorithms. Currently, model structure design mainly involves manual design and Neural Architecture Search (NAS). The results of the Lightweight Face Recognition competition at ICCV 2019 indicate that while improvements from structural modifications in large model scenarios are limited, enhancements in recognition rates from structural improvements in lightweight scenarios are significant.
The core of loss function design lies in learning features that are both discriminative and sufficiently robust. In recent years, metric learning and various margin-based methods have gradually become mainstream. In terms of feature extraction acceleration, the main methods include lightweight networks, model distillation, and sparse quantization; for feature comparison acceleration, the primary ideas involve quantization and various approximate nearest neighbor retrieval technologies.
Low-quality face recognition technology. In practical dynamic application scenarios, due to uncontrollable factors, the quality of collected images often differs significantly from that of training images, such as face tilting, extreme side profiles, motion blur, and focus blur, as well as occlusions (e.g., masks, sunglasses), low light intensity and contrast, and loss of facial information due to video transmission caused by encoding/decoding processes. These factors lead to a drastic decline in accuracy.
To address these specific issues, researchers have proposed methods to improve the accuracy of face recognition algorithms by comprehensively utilizing various image enhancement and generation technologies, such as employing adversarial generative networks to transfer the style of the camera, using deep learning methods for super-resolution reconstruction of small-sized blurry faces, and employing attention mechanisms for de-blurring face images.
Additionally, 3D face recognition technology can effectively resolve the insufficiency of unimodal robustness in complex scenarios, such as those caused by large angles and occlusions. Common fusion strategies include similarity fusion, feature fusion, and decision fusion.
Mask-wearing face recognition technology. During the COVID-19 pandemic, face recognition while wearing masks has received significant attention. Common solutions include data augmentation, occlusion recovery, and multi-part model fusion, applicable in face control, stranger detection, and contactless passage, all without the need to remove masks. In a database of 300,000 faces, the accuracy of mask-wearing face recognition can exceed 90%.
Face clustering has broad applications in both personal domain collection management and smart city governance. Initially based on traditional clustering methods such as k-means, the results were unsatisfactory. In recent years, face clustering methods based on Graph Convolutional Networks (GCN) have emerged. In practical business, the mining of spatio-temporal information is also a research hotspot.
Specific group recognition technology. For face recognition of children/elderly individuals and people of different skin tones, labeled data is scarce while unlabeled data is more abundant. Researchers suggest utilizing semi-supervised/unlabeled learning methods for further performance improvements. Additionally, adversarial and domain adaptation methods are also of considerable interest to researchers. In specific group recognition, considerations should be made for how to facilitate elderly individuals’ use of face recognition systems.
To prevent attacks on face recognition systems using photos, videos, or masks, attack detection algorithms have also become a research hotspot. The main detection principles include:
a) Discrete image detection, i.e., using one or more images for judgment;
b) Continuous image detection, i.e., using continuous image sequences for judgment, such as detecting edges, frames, screen reflections, pixel points, stripe analysis, etc.;
c) User active cooperation detection, i.e., requiring users to complete corresponding actions through instructions such as nodding, raising their heads, turning their heads left and right, opening their mouths, blinking, and following on-screen prompts for judgment;
d) Detection based on auxiliary hardware devices, i.e., using auxiliary hardware devices to obtain more judgment basis, such as using depth cameras to collect facial depth information or using specific wavelength light sources to project and detect differences in emissivity on skin or non-skin materials;
e) User passive cooperation detection, such as using the absorption characteristics of deoxygenated hemoglobin in veins, muscles, bones, and blood to infrared light to determine whether it comes from a living body; guiding users’ eye movements through specific instructions and tracking eye movements to determine whether it is a real living body.
Multimodal fusion recognition technology. Multimodal fusion recognition technology can effectively solve the unimodal robustness insufficiency issue in complex scenarios, such as those caused by large angles, occlusions, or low pixel counts, especially in applications demanding high reliability and security. Multimodal approaches can enhance the credibility of recognition.
Multimodal recognition has two development directions: one direction involves adding head, shoulder, and body recognition based on facial image feature recognition, which benefits from not requiring additional collection units; the other direction involves integrating other biometric modalities, such as vein texture and voiceprint information. This technology not only improves the robustness of algorithms but also enhances the credibility of living body verification, garnering considerable attention in the industry.
1. Overview of Industry Development
With the rapid development of computer science technologies such as cloud computing, big data, the Internet of Things, and artificial intelligence, along with the continuous maturation of face recognition technology in practical applications, face recognition technology continues to shine in fields such as smart security, smart cities, smart homes, and mobile payments, and new application scenarios for face recognition are continually being explored.
The global face recognition industry scale continues to grow at a very high rate. According to a report by MarketsandMarkets on the global face recognition market, it is expected that the global face recognition market size will grow from $3.2 billion in 2019 to $7.9 billion in 2024, with a compound annual growth rate of 16.6% during the forecast period (2019-2024).
In China, investment in face recognition technology peaked between 2017 and 2018. According to IHS Markit data, in 2018, China accounted for nearly half of the global face recognition market. From 2019 to 2020, the development of face recognition technology became more moderate, entering a rational phase. According to IT Orange data statistics, the total investment in face recognition technology in China has reached 40.6 billion yuan. The Forward Industry Research Institute predicts that the face recognition market size will maintain an average compound growth rate of 23% over the next five years, surpassing 10 billion yuan by 2024.
2. Typical Application Areas
Financial Technology. The application of face recognition in the financial sector is already quite widespread, such as remote bank account opening, identity verification, insurance claims, and face payment. The integration of face recognition technology can effectively enhance the security of financial transactions and improve the convenience of financial services.
In 2013, the Finnish company Uniqul became one of the first to introduce face recognition payment technology globally. In 2015, at the CeBIT exhibition in Hannover, Germany, Jack Ma first demonstrated Alipay’s face recognition payment technology to German Chancellor Angela Merkel. That same year, China Merchants Bank began applying face recognition in some branch counters and ATM services, followed by dozens of banks, including the four major banks, incorporating face recognition products into ATM, STM, counter, branch, and mobile banking services, gradually covering all customers.
Today, face recognition technology has been widely deployed and applied in the domestic financial sector, allowing consumers to access financial services using face recognition technology across various channels. China has significantly outpaced foreign markets in the application of face recognition technology.
Smart Security. Security is the earliest penetrating and most widely applied field for face recognition. According to EEO Research, in 2018, the security industry accounted for 61.2% of the face recognition market in China. Currently, face recognition technology mainly supports software and hardware integrated products for video structuring, facial retrieval, face control, and crowd statistics, focusing on scenarios such as identifying and tracking criminal suspects, locating missing children, and assisting anti-terrorism actions.
Video surveillance systems collect images through extensive monitoring networks, automatically analyze them, and compare faces based on video frames (1:1 and 1:N), allowing analysis of personnel trajectories and travel patterns to achieve identification and tracking of key individuals, fulfilling the goals of early warning, ongoing tracking, and rapid response in public security applications. It has played a significant role in various public security projects such as the Bright Project, Sky Net Project, smart communities, anti-terrorism, and major event security.
Furthermore, in the context of personnel management and security prevention needs in corporate buildings and residential communities, face recognition technology is widely applied. By inputting black and white lists of faces, it can effectively control personnel access to areas, and the high efficiency of machine recognition significantly saves human resources.
Smart Transportation. In foreign public transportation, the application of face recognition technology mainly focuses on security scenarios such as airport security checks and immigration management. Ottawa International Airport in Canada, the local immigration and border protection agency in Australia, and the U.S. Customs and Border Protection have all attempted to deploy face recognition entry and exit systems.
In China, face recognition applications in transportation primarily include 1:1 face verification and 1:N face identification. Currently, face verification technology for security checks has entered a widespread application phase, being promoted extensively in high-speed rail stations, regular train stations, and airports. Meanwhile, the application of 1:N face comparison technology for face payment has mainly been implemented in subways and buses, significantly enhancing commuting efficiency, freeing up substantial human resources, and improving the travel experience. Additionally, face recognition can monitor pedestrian flow at transportation hubs, predict peak traffic times based on travel patterns, and prepare diversion plans in advance.
Moreover, in terms of traffic violation management, face recognition technology can assist law enforcement in quickly and efficiently identifying violators and tracking them using vehicle recognition technology.
Public Services. Government internet platforms, public funds, social security, taxation, online certificates, traffic management, pedestrian red-light violations, traffic fine payments, and housing construction systems have all used or are currently using face recognition systems. The business points in the government service sector mainly involve building private cloud platforms, self-service terminals for government services, and convenient service platforms.
The implementation of face recognition in government systems has improved the efficiency of public services, allowing citizens to avoid waiting in lines and achieve self-service, saving time lost due to low manual efficiency. Some government services can also be conducted online through identity verification via face recognition on mobile devices, alleviating the issues of “running back and forth for services, distant service locations, and scattered service points.”
Education and Examination. As smart education rapidly develops, it increasingly adopts advanced information technologies such as the Internet of Things, cloud computing, and big data, achieving comprehensive collection, storage, and analysis of various educational management and teaching process data, and presenting it intuitively through visualization technologies.
In this process, relevant technology companies, based on their accumulated experience in artificial intelligence, video visualization technology, access control management, big data, and cloud computing, are committed to promoting the development of smart education, creating upgraded smart campuses, smart classrooms, smart dormitories, smart libraries, smart canteens, and smart supermarkets, along with safety control, class attendance, face recognition consumption, and intelligent experiences related to education. At the same time, face recognition technology is also applied in verifying the identities of exam candidates.
Smart Homes. Face recognition in smart homes is mainly applied in security unlocking and personalized home services. In terms of security prevention, face recognition can provide relatively safe and convenient home unlocking technology, potentially gradually replacing traditional password or fingerprint locks. Smart doorbells can identify visitors through face recognition. Additionally, home surveillance cameras can monitor in real time, alerting residents and triggering alarms if a stranger’s face is detected.
In terms of personalized home services, smart TVs can create accounts by inputting facial information, with the machine using face recognition for authentication and targeted content delivery, achieving personalized customization; smart refrigerators can use face recognition technology to push recipes and nutritional suggestions based on different users’ preferences and facial conditions. The application of face recognition technology in the smart home industry brings citizens a more convenient and comfortable lifestyle.
3. Industry Development Trends
Application scenarios are penetrating various industries, and market growth trends are becoming differentiated. With technological advancements and increasing security requirements, face recognition technology is undergoing significant changes in industry applications, rising from industries with lower reliability demands to those with high security and reliability requirements, such as finance, social security, securities, banking, and internet finance. Currently, face recognition technology in China is mainly applied in public security, access control attendance, and financial payment sectors.
When differentiating between different application fields, trends are gradually becoming distinct. In 2019, security, as one of the earliest application fields for face recognition, accounted for about 30% of the market share. With the gradual completion of the Bright Project and smart city construction, face recognition in the security sector is shifting from an incremental market to a stock market.
The application of face recognition in access control attendance is the most mature, accounting for around 42% of the industry market. With the further development of smart buildings, smart communities, and smart homes, the face recognition access control attendance market will also grow accordingly. Finance, as one of the important future application fields for face recognition, currently accounts for about 20% of the industry, and the market size is gradually expanding.
Changes in the global public health environment have brought new demands for face recognition applications. The outbreak of COVID-19 and its global spread pose threats to human life and health, triggering a global public health crisis. Compared to contact-based identity recognition methods like fingerprints and cards, non-contact recognition methods such as face recognition are more suitable for the current global public health environment affected by the coronavirus, reducing the risk of virus transmission through contact.
On one hand, face recognition technology, combined with infrared temperature monitoring technology, can obtain personnel health status information, allowing for timely feedback and control of pandemic sources; on the other hand, comprehensive deployment of monitoring systems can detect and obtain information on the movement of key individuals, helping the government implement targeted control measures.
Currently, the global public health situation remains severe. According to a research report by Zhiyuan, respondents generally agree on enhancing the capabilities of face recognition technology, with 81.9% agreeing on strengthening face recognition for individuals wearing masks. To improve the pandemic prevention system and further block sources of transmission, the new demand for mask-wearing face recognition technology has emerged.
In recent years, the explosive growth of information has put tremendous pressure on data transmission, storage, and centralized computing, leading to the emergence of edge computing. With the rapid development of AI chip technology, the computational power of edge computing devices continues to improve, with more and more calculations being handled at the edge. On one hand, edge computing effectively alleviates bandwidth burdens, enhances computational transmission efficiency, meets real-time response needs, and strengthens data security; on the other hand, model compression and acceleration technologies, along with dedicated AI chips suitable for facial recognition algorithm computations, are continuously improving, leading to sustained improvements in the accuracy of facial recognition algorithms on edge devices, which are now widely applied in communities, schools, hospitals, parks, and transportation scenarios, supporting large-scale applications of face recognition.
The collaborative deployment of cloud-edge-end is bringing new scenarios and models for face recognition applications. The collaborative deployment model of cloud-edge-end distributes face recognition application modules across various parts, enabling control and alarm through front-end edge computing, clustering analysis of facial features at the edge, and gathering effective information in the cloud for big data comparative analysis and scheduling work.
This collaborative deployment method alleviates pressure on the cloud, supports tiered business responses, and combines the agility of edge computing with the global nature of cloud big data, leading to comprehensive performance improvements in face recognition systems across dimensions such as bandwidth, concurrency, and response speed.
In the future, the video encoding capabilities and video feature extraction capabilities of edge devices will further strengthen, and AI applications will increasingly offload more computations to the front end. The cloud will consist of services such as portrait systems, video structuring systems, and facial body clustering analysis, creating various thematic libraries through analysis, clustering, and archiving, connecting with various business applications to meet the intelligent application needs in more complex scenarios.
Intense competition in the domestic market is reflected in the multitude of competing manufacturers, including traditional security enterprises, AI startups, and platform ecosystem companies. Traditional security enterprises understand the pain points and customer needs of the security video industry deeply, leveraging their product + integration advantages to establish strong scale effect barriers.
AI startups are primarily emerging computer vision (CV) companies focusing on algorithms, with AI algorithms as their core advantage while also considering hardware implementation and productization. Platform ecosystem companies rely on their powerful cloud platforms and cloud computing capabilities, horizontally integrating application solutions from partners to build a unified ecosystem and form differentiated competition.
In the domestic market, intense competition also manifests in competition across the entire industry chain, extending from algorithm competition to chip and platform competition. Major market participants have joined the AI chip competition, with security enterprises focusing on edge-side and end-side inference chips, startups emphasizing edge-side inference chips, and platform ecosystem companies prioritizing end/cloud integration, building a full-stack AI ecosystem from training to inference. Downstream competition primarily revolves around application layers, ecosystem competition, and deep industry engagement.
Overall, China’s face recognition technology and applications are currently at the forefront internationally, with widespread applications in sectors such as financial technology, smart security, smart transportation, public services, education, and smart homes. However, in the past year, face recognition technology has also faced many negative impacts, with issues such as “facial recognition hidden in property sales offices” and facial information leakage becoming increasingly common. As the technical threshold gradually lowers, strengthening the research and application of security technologies and improving relevant laws and regulations have become particularly important.
This article is transferred from: Chengmai Technology
How to Join the Society
Register as a Society Member:
Individual Members:
Follow the Society’s WeChat: China Command and Control Society (c2_china), reply “Individual Member” to obtain the membership application form, fill it out as required, and if there are any questions, you can leave a message in the public account. You can only pay membership fees online through Alipay after passing the society’s review.
Unit Members:
Follow the Society’s WeChat: China Command and Control Society (c2_china), reply “Unit Member” to obtain the membership application form, fill it out as required, and if there are any questions, you can leave a message in the public account. You can only pay membership fees after passing the society’s review.
Long press the QR code below to follow the Society’s WeChat
