Click on the “Beginner Learning Vision” above, and select the “Starred” public account
Essential Knowledge Delivered Immediately
This article is reprinted from: New Machine Vision
This article takes Dalsa Sherlock software as an example to explore the image processing methods for smooth and blurred images in visual inspection.
1. Observing the gray distribution to describe an image is called the spatial domain, while observing the frequency of changes in the image is called the frequency domain.
2. Frequency domain analysis: Low frequency corresponds to slow changes in image intensity, while high frequency corresponds to fast changes. Low-pass filters remove the high-frequency components of the image, and high-pass filters remove the low-frequency components of the image.
Smoothing and Blurring Processing (Low-pass)
Gaussian filtering, median filtering, and mean filtering all belong to low-pass filtering.
The edges, abrupt parts, and granular noise of an image represent the high-frequency components of the image signal, while large background areas represent the low-frequency signal of the image signal.
Using filtering to remove the high-frequency parts can eliminate noise.
In Sherlock, the algorithms used for low-pass processing to smooth images include: Lowpass, Lowpass5X5, Gaussian, Gaussian5X5, GaussianWXH, Median, Smooth.
Low-pass Filtering: Lowpass Lowpass5X5
These two algorithms in Sherlock can be understood as low-pass filtering. According to the documentation, these two algorithms perform mean smoothing on images within sizes of 3×3 and 5×5, and can be executed multiple times. The difference from the smooth algorithm is not clear.
(1) Mean Filtering: Smooth
Mean filtering is the simplest low-pass filtering method. Based on the specified size, it takes the average of adjacent pixels. In Sherlock, a size of 3×3 is used, and each pixel value is replaced by the average of its original pixel value and the values of its surrounding 8 pixels.
For example, in the following image, if the original pixel value of the center point is 1 and the average of adjacent pixels is 2, then after mean filtering, the pixel value of the center point becomes 2.
(2) Median Filtering: Median
According to the specified size, the pixels in the area are sorted, and the pixel value of the center point is replaced by the median pixel value within the filtering size.
Median filtering is very effective for removing small noise points or impulse noise.
Median filtering alters the structure of the image, changing the intensity of the image.
(3) Gaussian Filtering: Gaussian Gaussian5X5 GaussianWXH
Gaussian filtering is the process of performing a weighted average across the entire image. The value of each pixel is obtained by averaging its own value and the values of other neighboring pixels based on weights, blurring the details of the image. It is commonly used to remove noise and smooth edges.
The specific operation of Gaussian filtering is: using a template (or called convolution, mask) to scan each pixel in the image, replacing the value of the center pixel with the weighted average gray value of the neighboring pixels determined by the template.
Gaussian:
Using a 3X3 size, with filter coefficients based on a standard deviation of 0.85, can be executed multiple times.
Gaussian5X5:
Using a 5X5 size, with filter coefficients based on a standard deviation of 1, can be executed multiple times. However, increasing the number of executions will increase the value of the standard deviation, approximately the square root of the number of repetitions.
Note: Each time it is reused, it will leave 2 pixels at the image boundary unchanged. If the boundary of the image is affected, pay attention to setting the gray value of the boundary.
GaussianWXH:
Using a size that can be set, the weight of the standard deviation can also be set.
A large Gaussian filter can be approximated by repeatedly executing smaller Gaussian filters multiple times.
For example, executing a 3×3 or 5×5 Gaussian filter N*1.4 or N*2.8 times will yield a filter size. For instance, executing a 3×3 filter 9 times will yield a filter size of about 9*1.4, which is similar to a 13X13 Gaussian.
Standard Deviation σ
The larger the sigma, the more uniform the distribution, the greater the surrounding weight, and the greater the blurring effect.
The smaller the sigma, the more concentrated the distribution, the greater the weight near the center, and the smaller the blurring effect.
Mean Gaussian
[Sharpening Enhancement Processing]
The sharpening processing of the image makes the edges clearer and enhances details.
The algorithms used in Sherlock for sharpening images include various convolutions: 1X3, 1X5, 1X7, 3X3, 5X1, 5X5, 7X1, Chatter Edge, Gradient, Highpass, Sharpen.
◆ 1X3, 1X5, 1X7, 3X3, 5X1, 5X5, 7X1
These select different sizes of convolution kernels, where each pixel at each position is convolved with adjacent pixels using the specified convolution kernel.
If the target pixel point and its surrounding values (up, down, left, right, front, and back) have significant differences, this operator can perform convolution on this position in the original image matrix. The resulting value will have a significant difference from the original gray value of that pixel. When this difference exceeds the preset range, this pixel is marked as 255 (white), while other points are marked as 0 (black), thus obtaining an effect image with a black background and white lines as the edge or shape extraction effect. The sharpening operator: through convolution, can increase the variance of each element in the matrix with surrounding elements, lightly sharpening, or heavily extracting edges. Conversely, it is the denoising process.
The above images show the parameters of the algorithm, which are the parameters for setting the convolution kernel.
◆ Chatter Edge
This is used for extracting noisy or blurred edges, and the filter size can be customized. The output is a binarized or trimmed gray-level image. The image jump caused by mechanical vibration is shown below:
Using a differential filter with a customizable kernel size, for example, a kernel size of 4 is {-1,0,0,1} or a kernel size of 6 is {-1,0,0,0,0,1}. This filter scans the input image in the horizontal, vertical, or both directions to obtain the output image.
Applying a filter size of 16, a filter in the “X” direction enhances edges with lower contrast in the image.
Any output below the edge threshold is set to 0. Values equal to or greater than the edge threshold are output as all white. If keep gray above is set to True, the original values are retained.
The filter size (convolution kernel size) can be set from 2-255. Since convolution is a linear operation, we can divide this differential filter into two rectangular (sampling integrator) filters. One filter is offset by one pixel relative to the other filter. One rectangular filter is a low-pass filter used to remove noise. -1, …., 1 are used to enhance edges.
The filter direction sets the direction of the filter. “X” indicates that the filter is horizontal, “Y” indicates that the filter is vertical, both will cause the filter to scan through the input image to obtain the output image. If set to “either”, filtering in both directions is applied. If keep gray above is set to false, the results of X and Y filtering are merged through a logical OR. If keep gray above is set to true, the output is the maximum value of the X and Y directions.
The transition type sets the edge contrast change. “Dark-to-light” amplifies edges from dark to bright, “light-to-dark” amplifies edges from bright to dark, and “either” amplifies both types of contrast changes.
The image below shows a very slow and diffused transition edge, while the right image shows the result of applying a filter size of 10 to chatter edges.
◆Gradient Sharpening, Used in Linear ROI
Enhances edges based on the distance-separated pixels.
output = abs[ (n – separation/2) – (n + separation/2) ]
◆Highpass Highpass5x5 Highpass Filtering
◆Sharpen
During image enhancement, various image smoothing algorithms are usually used to eliminate noise. Generally, the energy of the image is mainly concentrated in its low-frequency part, while the noise is mainly in the high-frequency band, and the edge information of the image is also mainly concentrated in its high-frequency part. This leads to the situation where the original image becomes blurred after smoothing processing (smoothing can be considered as removing noise, which also blurs the edge information of the image). To reduce the adverse effects of this, image sharpening techniques need to be used to make the edges, contours, and details of the image clear. The purpose of image sharpening processing is to make the edges, contour lines, and details of the image clear. The fundamental reason for the blurring of the smoothed image is that it has undergone averaging or integration operations, so it can be clarified by performing inverse operations (such as differential operations). Differential operations seek the rate of change of the signal, and from the differential properties of Fourier transform, it is known that differential operations have a strong high-frequency component effect. From the frequency domain perspective, the essence of image blurring is that its high-frequency components are attenuated, so high-pass filters can be used to clarify the image. However, it should be noted that images that can be sharpened must have a high signal-to-noise ratio; otherwise, the sharpened image will have a lower signal-to-noise ratio, leading to an increase in noise that exceeds the signal. Therefore, it is generally advisable to first remove or reduce noise before sharpening.
[Edge Detection]
The general steps for edge detection are:
1. Filtering: Edge detection algorithms are mainly based on the first and second derivatives of image intensity, but the calculation of derivatives is very sensitive to noise, so filters must be used to improve the performance of edge detectors related to noise. It should be noted that most filters reduce noise while also causing a loss of edge intensity, so a trade-off is needed between enhancing edges and reducing noise.
2. Enhancement: The basis for enhancing edges is to determine the change values of the intensity in the neighborhood of each point in the image. Enhancement algorithms can highlight points with significant changes in neighborhood (or local) intensity values. Edge enhancement is generally accomplished by calculating the gradient magnitude.
3. Detection: There are many points in the image with large gradient magnitudes, but not all of these points are edge points in specific application fields, so some method should be used to determine which points are edge points. The simplest edge detection criterion is the gradient magnitude threshold criterion.
4. Localization: If a specific application requires determining the edge position, the position of the edge can be estimated at sub-pixel resolution, and the orientation of the edge can also be estimated. In edge detection algorithms, the first three steps are widely used. This is because in most cases, it is only necessary for the edge detector to indicate that an edge appears near a certain pixel point in the image, and there is no need to specify the exact position or direction of the edge.
The algorithms used in Sherlock for edge detection include: Canny, Compass, First Diff, Kirsch, Laplace, LineEnhance, Prewitt, Roberts, Second Diff, Sobel.
◆Sobel
Since the Sobel operator combines Gaussian smoothing and differentiation, its results are somewhat robust to noise.
The main method is to convolve each point of the image with the Sobel operator: one for detecting vertical edges, and one for detecting horizontal edges, and the maximum of the last two convolutions will be used as the output for that point, i.e., the detected gray level.
Sobel operator:
It can be seen that the Sobel operator includes two sets of 3*3 matrices, the left representing vertical and the right representing horizontal. Convolving it with the image will yield the approximate values of vertical and horizontal brightness differences.
The Sobel operator detects edges based on the weighted differences in gray levels between adjacent pixels. It has a smoothing effect on noise and provides relatively accurate edge direction information, although the precision of edge localization is not very high. It is a commonly used edge detection method when high precision is not required.
◆Canny
The three-level operation detects edges in all directions. A SobelXY detector is used to find the edge gradient, and points perpendicular to the edge contour are rejected. Finally, a hysteresis threshold method is applied, which uses a high threshold and a low threshold to distinguish edge pixels. If the gradient value of an edge pixel is greater than the high threshold, it is considered a strong edge point. If the gradient value of an edge pixel is less than the high threshold but greater than the low threshold, it is marked as a weak edge point. Points below the low threshold are suppressed.
Strong edge points can be considered as true edges. Weak edge points may be true edges or may be caused by noise or color changes. To obtain accurate results, weak edge points caused by the latter should be removed. It is generally believed that weak edge points caused by true edges are connected to strong edge points, while weak edge points caused by noise are not. The so-called hysteresis boundary tracking algorithm checks the 8-connected neighborhood pixels of a weak edge point; as long as there is a strong edge point present, this weak edge point is considered a true edge and retained.
The recommended high-to-low threshold ratio for Canny is between 2:1 and 3:1.
Example Steps for Canny Edge Detection:
1. Noise elimination. Use a Gaussian smoothing filter for convolution to reduce noise. For example, the following 5×5 Gaussian kernel:
2. Calculate gradient magnitude and direction. Here, according to the steps of the Sobel filter:
A. Use a pair of convolution arrays (acting on the x and y directions, respectively):
B. Use the following formula to calculate gradient magnitude and direction:
Gradient direction is approximated to one of four possible angles (generally 0, 45, 90, 135).
3. Non-maximum suppression. This step eliminates non-edge pixels, retaining only some fine lines (candidate edges).
4. Hysteresis threshold: In the last step, Canny uses hysteresis thresholding, which requires two thresholds (high threshold and low threshold):
A. If the magnitude at a certain pixel location exceeds the high threshold, that pixel is retained as an edge pixel.
B. If the magnitude at a certain pixel location is less than the low threshold, that pixel is excluded.
C. If the magnitude at a certain pixel location is between the two thresholds, that pixel is retained only if it is connected to a pixel above the high threshold.
◆Laplace, Laplace5x5, Laplacian edge detectors, using filtering sizes of 3×3 and 5×5, belong to high-pass filtering of second-order derivatives.
Laplacian sharpening of an image is related to the degree of mutation of a certain pixel concerning its surrounding pixels.
From the two templates, it can be seen that if there is a white point in a black plane, the template matrix can make this white point brighter. Since the edges of an image are areas where gray levels change, the Laplacian template is very useful for edge detection.
The first derivative of a function describes the direction in which the function image changes, whether it increases or decreases; while the second derivative describes the speed of change of the image, whether it increases or decreases sharply or gently. Based on this, we can infer that the second derivative can find the transition degree of the image’s pigment, such as the transition from white to black being relatively sharp.
Or: when the gray level of the central pixel in the neighborhood is lower than the average gray level of the other pixels in that neighborhood, the gray level of the central pixel should be further reduced. When the gray level of the central pixel in the neighborhood is higher than the average gray level of the other pixels in that neighborhood, the gray level of the central pixel should be further increased, thus achieving image sharpening.
◆Compass
A directional edge filter enhances edges that are perpendicular to the selected direction. Edges parallel to the direction are set to 0.
Performs first-order differential filtering.
◆First Diff X Quickly enhances vertical edges.
The image at this point is determined by the absolute difference between itself and the pixel to its right.
◆First Diff Y Quickly enhances vertical edges.
The image at this point is determined by the absolute difference between itself and the pixel below.
◆First Diff XY Quickly enhances edges in any direction.
The image at this point is determined by the sum of the differences in the X and Y directions.
◆Second Diff X, Second Diff Y, Second Diff XY, similar to the above algorithms, but the pixel values are determined by the differences between the center pixel and the surrounding pixels in the left and right (or up and down) directions. output = abs(west – 2*center + east), output = abs(north – 2*center + south).
output = abs(west – 2*center + east) + abs(north – 2*center + south).
◆Kirsch, enhances changes from dark to light using 3×3 convolution based on the selected direction.
◆LineEnhance, a directional edge detector, enhances edges or lines that are perpendicular to a specified direction.
Uses second-order differentials to find changes in the image gradient, replacing the gray values of the parts that belong to the edge with the maximum gray value, while setting the parts that do not belong to the edge to 0.
◆PrewittX, PrewittY, PrewittXY, use 3×3 filters to enhance edges in horizontal, vertical, and any direction. PrewittX and PrewittY retain positive edges in horizontal and vertical directions, while negative ones are set to 0 and are not sensitive to noise.
◆Roberts, the edge operator uses the difference between two diagonally adjacent pixels. From the practical effect of image processing, it has precise edge localization, but is sensitive to noise. It is an operator that uses local differential operators to find edges.
Source: Zhiyu Image
Good news!
Beginner Learning Vision Knowledge Group
Is now open to the public👇👇👇
Download 1: OpenCV-Contrib Extension Module Chinese Tutorial
Reply "Extension Module Chinese Tutorial" in the "Beginner Learning Vision" public account backend to download the first Chinese version of the OpenCV extension module tutorial available online, covering installation of extension modules, SFM algorithms, stereo vision, target tracking, biological vision, super-resolution processing, and more than twenty chapters of content.
Download 2: Python Vision Practical Project 52 Lectures
Reply "Python Vision Practical Project" in the "Beginner Learning Vision" public account backend to download 31 practical vision projects including image segmentation, mask detection, lane line detection, vehicle counting, adding eyeliner, license plate recognition, character recognition, emotion detection, text content extraction, and face recognition, to help quickly learn computer vision.
Download 3: OpenCV Practical Project 20 Lectures
Reply "OpenCV Practical Project 20 Lectures" in the "Beginner Learning Vision" public account backend to download 20 practical projects based on OpenCV to advance OpenCV learning.
Group Chat
Welcome to join the reader group of the public account to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided in the future). Please scan the WeChat number below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiao Tong University + Vision SLAM". Please follow the format; otherwise, you will not be approved. After successful addition, you will be invited into related WeChat groups according to your research direction. Please do not send advertisements in the group; otherwise, you will be removed from the group. Thank you for your understanding~