Click on the above “Beginner Learns Vision“, choose to add “Star Mark” or “Top“
Essential Insights Delivered Promptly
Source | Intelligent Manufacturing Bureau
Introduction
Defect detection is a crucial aspect of the industrial production process, and the quality of its detection results directly affects product quality. In real-world scenarios, when the defect rate is very low or even nonexistent, the insufficiency of defect samples leads to low accuracy in deep learning defect detection models. How can high-precision detection be achieved with few defect samples? Currently, there are two methods: one is few-shot learning, and the other is using GANs. This article will introduce a method using GAN for surface defect detection of products with no defect samples.
Deep learning has been well applied in mainstream fields of computer vision, but in industrial fields, such as product surface defect detection, it seems that the strong capabilities of deep learning have not been fully utilized. In recent years, research on surface defects has mainly focused on various mainstream neural network frameworks, from CNN to YOLO, SSD, and even semantic segmentation with FCN-related papers. Through some techniques, these frameworks have been lightweighted to classify or detect defects. However, there is always a problem: there must be defect samples available for training, and the quantity cannot be too small! Of course, some research groups have used sparse coding, dictionary learning, and sparse autoencoders to detect surface defects, but these methods are very limited and mainly target images with periodic background textures, such as silk and printed materials. Many research groups and industrial software companies both domestically and internationally want to develop practical algorithm software. Some notable companies in the defect detection field include: VIDI, Halcon, and it is said that Hikvision is also conducting algorithm research for industrial products.Paper Title: A Surface Defect Detection Method Based on Positive Samples
Paper Link: https://doi.org/10.1007/978-3-319-97310-4_54
The authors propose to detect defect samples solely based on existing normal surface image samples, using certain technical means. They effectively apply the recently popular GAN in the framework. Throughout this year, the teachers in the research group have been discussing the feasibility of this method. Is it necessary to have defect samples for defect detection? From sparse autoencoders, few-shot learning to zero-shot learning, one of the hot topics in computer vision research, the conclusion is drawn: most industrial product surface defect detection requires defect samples or artificially created defect samples. Although the paper does not directly use defect samples from production lines, it artificially generates defect samples through algorithms and effectively integrates the powerful capabilities of GAN in the field of image restoration. The entire framework design is very clever.Article Idea:The overall idea of the paper is that GAN has a strong capability in image restoration and reconstruction. By artificially adding some defects to normal samples during the training phase, GAN learns a network that can repair these defect areas. In the detection phase, a real defect sample is input, and the trained GAN will repair it. Then, based on LBP, defect detection can be completed. The entire algorithm framework does not require real defect samples and manual labels, but in the framework, artificial defects are generated (for example, using Photoshop).In Simple Terms:The authors utilize GAN’s capability in image restoration (reconstruction) by collecting some normal (defect-free) samples from industrial sites and artificially adding some defects, such as lines and spots.During training, the artificially created defect images and original images are input as training samples for GAN, resulting in a network with image restoration capability.During testing, the trained GAN is directly used to reconstruct and repair the collected images. If there are defect areas in the samples, according to the network design, those areas will need to be repaired. The repaired images and the original defect images are then analyzed using LBP to identify the significant difference areas, which are the defect areas.
01
Main Content
The main framework idea of the paper is based on the structure of the GAN network. GAN mainly includes two parts: the generator G and the discriminator D. The generator is mainly used to learn the distribution of real images so that the images it generates appear more realistic, in order to “fool” the discriminator. The discriminator, on the other hand, needs to determine the authenticity of the received images. Throughout the process, the generator strives to make the generated images more realistic, while the discriminator works hard to identify the authenticity of the images. This process is akin to a game, where over time, the generator and discriminator continuously engage in adversarial training until both networks reach a dynamic equilibrium: The images generated by the generator closely resemble the distribution of real images, while the discriminator cannot distinguish between real and fake images, with the predicted probability of a given image being real approaching 0.5 (this statement is quoted from Professor Li Hongyi, paying tribute to him).
Training Phase
During the training phase, the model uses some image processing techniques to artificially generate defects on normal sample images (indicated by the red box module in the illustration). The G module, composed of autoencoders, is used for defect repair learning. The learning objective is to minimize the L1 norm between the generated images and the normal samples. By training with a certain number of samples, a G module capable of defect repair can be obtained. Some resources on using GAN for image repair can be referenced [3][4], and of course, the references in the paper can also be consulted.Training Phase
Testing Phase
In the testing phase, the trained G module from the previous step is used as the image repair module. For the output image samples, if defect areas exist, the repair module G will provide the repaired image. This repaired image, along with the original defect sample image, is used as input for the LBP algorithm, which accurately locates the defect areas.Testing Phase
02
Other Details
2.1 Defect GenerationIn actual training, the authors manually generate some defect samples, as shown in Figure 3, allowing the training network to automatically repair defects. Additionally, the authors employed techniques to augment samples, such as adding Gaussian noise and random resizing.Defect Generation3.2 Defect Image ReconstructionThe main purpose of the defect image reconstruction part is to make the reconstructed defect images as similar as possible to normal samples. The authors modified the framework based on references [5][6], using L1 distance as the objective function to measure reconstruction differences.Then, in experiments, the authors found that using only L1 was not sufficient; details such as image edges might be inaccurately measured, so they incorporated GAN loss to enhance the network’s reconstruction effects.Finally, the following objective function was obtained.2.3 Defect DetectionBecause there are some pixel-level details that differ between the images repaired using GAN and the original defect sample images, the authors used the LBP algorithm, which has been well applied in the field of facial recognition in recent years, for defect area detection. The details of the algorithm will not be introduced here; the illustration is as follows.
03
Experiments
The article conducted validation experiments on the DAGM 2007 dataset and fabric dense images. The experiments show that the proposed GAN+LBP algorithm has a high detection accuracy comparable to supervised training algorithms with sufficient training samples. The experiments used two types of datasets, 4.1 is printed pattern surfaces, and 4.2 is fabric surfaces.
4.1 Texture SurfaceTest SamplesResultsa. Original Image, b. Repaired Image, c. Paper Method, d. FCN Method, e. True Label3.2 Fabric PictureIn the experiments, there were five types of defect samples. The experimental samples were categorized by background into three types, each containing five defect samples and twenty-five normal samples.Test SamplesResultsa. Original Image, b. Repaired Image, c. Paper Method, d. FCN Method, e. True LabelI have limited abilities; please point out any unclear or incorrect expressions so we can improve together!
References
[1] Zhao Z, Li B, Dong R, et al. A Surface Defect Detection Method Based on Positive Samples[C]// Pacific Rim International Conference on Artificial Intelligence. Springer, Cham, 2018:473-481.[2] Wu X. Fully Convolutional Networks for Semantic Segmentation[J]. Computer Science, 2015.[3] Li Y, Liu S, Yang J, et al. Generative Face Completion[J]. 2017.[4] Yang C, Lu X, Lin Z, et al. High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis[J]. arXiv preprint arXiv:1611.09969, 2016.[5] Image-to-Image Translation with Conditional Adversarial Networks.[6] Unsupervised representation learning with deep convolutional generative adversarial networks.
This article is for academic sharing only. If there is any infringement, please contact us to delete the article.
Download 1: OpenCV-Contrib Extension Module Chinese Tutorial
Reply with: OpenCV Extension Module Chinese Tutorial on the "Beginner Learns Vision" WeChat account to download the first OpenCV extension module tutorial in Chinese, covering installation, SFM algorithms, stereo vision, target tracking, biological vision, super-resolution processing, and more than twenty chapters of content.
Download 2: Python Vision Practical Projects 52 Lectures
Reply with: Python Vision Practical Projects on the "Beginner Learns Vision" WeChat account to download 31 visual practical projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, and facial recognition to help quickly learn computer vision.
Download 3: OpenCV Practical Projects 20 Lectures
Reply with: OpenCV Practical Projects 20 Lectures on the "Beginner Learns Vision" WeChat account to download 20 practical projects based on OpenCV, achieving advanced learning of OpenCV.
Discussion Group
You are welcome to join the reader group of the WeChat account to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided in the future). Please scan the WeChat ID below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiao Tong University + Vision SLAM". Please follow the format; otherwise, you will not be approved. After successful addition, you will be invited to the relevant WeChat group based on your research direction. Please do not send advertisements in the group; otherwise, you will be removed from the group. Thank you for your understanding~
