CN117094975A

CN117094975A - Method and device for detecting surface defects of steel and electronic equipment

Info

Publication number: CN117094975A
Application number: CN202311068561.0A
Authority: CN
Inventors: 郝亮; 李毅仁; 李玉涛; 潘志威; 王建业; 苗志伟; 杜利达; 金皓宁; 张宇; 李宏鹏
Original assignee: Harbin Institute Of Technology shenzhen Shenzhen Institute Of Science And Technology Innovation Harbin Institute Of Technology; Hengshui Board Packaging Materials Technology Co ltd; Hegang Digital Technology Co ltd; HBIS Co Ltd
Current assignee: Harbin Institute Of Technology shenzhen Shenzhen Institute Of Science And Technology Innovation Harbin Institute Of Technology; Hengshui Board Packaging Materials Technology Co ltd; Hegang Digital Technology Co ltd; HBIS Co Ltd
Priority date: 2023-08-23
Filing date: 2023-08-23
Publication date: 2023-11-21

Abstract

The invention provides a method and a device for detecting surface defects of steel and electronic equipment. The method comprises the following steps: generating a binarization segmentation threshold value of the steel surface image to be detected according to the front Jing Huidu mean value and the background gray level mean value of the steel surface image to be detected; sequentially carrying out gray level equalization treatment, filtering treatment and edge enhancement treatment on the steel surface image to be detected to obtain a first image; performing binarization processing on the first image by using a binarization segmentation threshold value to obtain a second image; generating a defect detection result of the steel surface image to be detected according to the second image and a pre-trained defect detection model; the defect detection model is obtained based on training of a preset training set, the preset training set comprises a plurality of steel surface defect images and steel surface defect-free images, and the steel surface defect images correspond to defect types. The invention can solve the problem that the existing defect detection method cannot meet the high-speed requirement on the detection of the defects on the steel surface in actual industrial production.

Description

Method and device for detecting surface defects of steel and electronic equipment

Technical Field

The invention relates to the technical field of steel detection, in particular to a method and a device for detecting surface defects of steel and electronic equipment.

Background

Iron and steel is an important industrial raw material and is widely applied to production and preparation scenes of aerospace equipment, automobile components, industrial equipment and the like. Therefore, the quality of steel directly determines the quality of industrial costs of raw materials. To ensure the quality of steel, it is common to detect whether the surface of steel is defective or not and evaluate the quality of steel by the degree of the defect of the surface of steel.

Currently, the detection of steel surface defects is generally performed by manual spot inspection. However, the manual spot inspection mode cannot meet the high-speed requirement of steel surface defect detection in actual industrial production.

Disclosure of Invention

The embodiment of the invention provides a method, a device and electronic equipment for detecting steel surface defects, which are used for solving the problem that the existing defect detection method cannot meet the high-speed requirement for detecting the steel surface defects in actual industrial production.

In a first aspect, an embodiment of the present invention provides a method for detecting a surface defect of steel, including:

generating a binarization segmentation threshold value of the steel surface image to be detected according to the front Jing Huidu mean value and the background gray level mean value of the steel surface image to be detected;

sequentially carrying out gray level equalization treatment, filtering treatment and edge enhancement treatment on the steel surface image to be detected to obtain a first image;

Performing binarization processing on the first image by using a binarization segmentation threshold value to obtain a second image;

generating a defect detection result of the steel surface image to be detected according to the second image and a pre-trained defect detection model; the defect detection model is obtained based on training of a preset training set, the preset training set comprises a plurality of steel surface defect images and steel surface defect-free images, and the steel surface defect images correspond to defect types.

In one possible implementation manner, generating a binary segmentation threshold of the steel surface image to be measured according to the front Jing Huidu mean value and the background gray level mean value of the steel surface image to be measured includes:

performing foreground and background segmentation operation on the steel surface image to be detected to obtain a foreground region and a background region;

calculating an average gray value of the steel surface image to be detected, a first pixel ratio of a foreground area to the steel surface image to be detected, and a second pixel ratio of a background area to the steel surface image to be detected;

obtaining a front Jing Huidu average value according to the average gray value and the first pixel ratio;

obtaining a background gray average value according to the average gray value and the second pixel ratio;

the average of the front Jing Huidu average and the background grayscale average was calculated and defined as the binarized segmentation threshold.

In one possible implementation, binarizing the first image using a binarization segmentation threshold includes:

setting a gray value corresponding to a pixel point larger than a binarization segmentation threshold value in the first image as a first gray value;

and setting the gray value corresponding to the pixel point smaller than the binarization segmentation threshold in the first image as a second gray value.

In one possible implementation manner, generating a defect detection result of the steel surface image to be detected according to the second image and the pre-trained defect detection model includes:

performing closed operation processing on the second image to obtain a closed operation image;

performing defect area identification on the closed operation image;

when a defect area with the size larger than a preset threshold exists in the closed operation image, marking an image area corresponding to the defect area with the size larger than the preset threshold in the closed operation image in the steel surface image to be detected, and inputting the marked steel surface image to be detected into a defect detection model to obtain a defect detection result of a defect type corresponding to the steel surface image to be detected;

and when a defect area with the size larger than a preset threshold value does not exist in the closed operation image, generating a defect detection result that the steel surface image to be detected is defect-free.

In one possible implementation manner, gray level equalization processing, filtering processing and edge enhancement processing are sequentially performed on the steel surface image to be tested to obtain a first image, including:

acquiring a gray mapping relation for gray level equalization according to a gray level histogram of the steel surface image to be detected, and carrying out gray level equalization on the steel surface image to be detected according to the gray level mapping relation to obtain a third image;

acquiring a convolution weight value for filtering according to the spatial proximity of the third image and the pixel value similarity of the third image, and performing filtering on the third image according to the convolution weight value to obtain a fourth image;

a horizontal gradient image and a vertical gradient image of a fourth image for edge enhancement processing are acquired, and a first image is obtained from the horizontal gradient image and the vertical gradient image.

In one possible implementation manner, the method includes obtaining a gray mapping relation for gray equalization according to a gray histogram of an iron and steel surface image to be measured, and performing gray equalization on the iron and steel surface image to be measured according to the gray mapping relation to obtain a third image, including:

dividing the steel surface image to be detected into a plurality of partial images, and obtaining gray histograms corresponding to all the partial images; each local image corresponds to a gray level histogram, the gray level histogram is a probability density function corresponding to each gray level in the local image, the abscissa of the gray level histogram is the gray level of each pixel point in the local image, and the ordinate of the gray level histogram is the frequency of the pixel point of each gray level in the local image;

Performing contrast limiting treatment on the gray level histogram to obtain a second gray level histogram;

acquiring an accumulated probability density function corresponding to the second gray level histogram, and calculating the accumulated probability density function and the total gray level number to obtain a gray level mapping relation for gray level equalization processing; the cumulative probability density function is the integral of a second probability density function corresponding to the second gray level histogram;

carrying out gray level equalization treatment on the partial images according to the gray level mapping relation until all the partial images are processed;

and integrating all the processed partial images to obtain a third image.

In one possible implementation manner, according to the spatial proximity of the third image and the pixel value similarity of the third image, a convolution weight for filtering processing is obtained, and according to the convolution weight, the filtering processing is performed on the third image to obtain a fourth image, including:

defining a center point in the third image;

calculating the spatial proximity degree between each pixel point in the third image and the center point, and the pixel value similarity between each pixel point and the center point;

multiplying the spatial proximity of each pixel point by the similarity of the pixel values to obtain a convolution weight value of each pixel point;

And obtaining a fourth image according to the convolution weight value of each pixel point and the pixel value of each pixel point.

In one possible implementation, acquiring a horizontal gradient image and a vertical gradient image of a fourth image for edge enhancement processing, and obtaining a first image from the horizontal gradient image and the vertical gradient image includes:

acquiring a horizontal matrix corresponding to the fourth image in the horizontal direction and a vertical matrix corresponding to the fourth image in the vertical direction;

carrying out plane convolution calculation on the horizontal matrix and the fourth image to obtain a horizontal gradient image;

carrying out plane convolution calculation on the vertical matrix and the fourth image to obtain a vertical gradient image;

and performing bit OR operation on the horizontal gradient image and the vertical gradient image to obtain a first image.

In a second aspect, an embodiment of the present invention provides a steel surface defect detection apparatus, including:

the generation module is used for generating a binarization segmentation threshold value of the steel surface image to be detected according to the front Jing Huidu mean value and the background gray level mean value of the steel surface image to be detected;

the first processing module is used for sequentially carrying out gray level equalization processing, filtering processing and edge enhancement processing on the steel surface image to be detected to obtain a first image;

The second processing module is used for carrying out binarization processing on the first image by utilizing a binarization segmentation threshold value to obtain a second image;

the detection module is used for generating a defect detection result of the steel surface image to be detected according to the second image and the pre-trained defect detection model; the defect detection model is obtained based on training of a preset training set, the preset training set comprises a plurality of steel surface defect images and steel surface defect-free images, and the steel surface defect images correspond to defect types.

In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method as described above in the first aspect or any one of the possible implementations of the first aspect when the computer program is executed.

The embodiment of the invention provides a method, a device and electronic equipment for detecting steel surface defects, which are characterized in that a binarization segmentation threshold value of a steel surface image to be detected is generated according to a pre-obtained front Jing Huidu mean value and a background gray level mean value of the steel surface image to be detected, then a first image is obtained by carrying out pretreatment such as gray level equalization treatment, filtering treatment, edge enhancement treatment and the like on the steel surface image to be detected in sequence, the first image is subjected to binarization treatment through the obtained binarization segmentation threshold value, a second image with black and white pixels is obtained, and finally a defect detection result of the steel surface image to be detected is obtained according to the second image and a defect detection model which is trained in advance.

Therefore, the steel surface image to be detected is subjected to binarization processing through the proper binarization segmentation threshold value, so that pixels in the first image can be converted into black or white, the contrast of the first image is enhanced, the definition of the first image is enhanced, the defect position identification time is shortened, and the technical effect of the defect detection process is accelerated. And because the second image has only two colors, and the small composition colors of the second image also laterally indicate that the data size of the second image is small, the subsequent image processing speed is obviously faster than that of the image which is not subjected to the binarization processing. In addition, the detection result obtained by adopting the defect detection model obtained based on a large amount of data training is more accurate. Therefore, the invention can detect the defects of the steel surface at high speed and accurately in actual industrial production.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of an implementation of a method for detecting defects on a steel surface according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a gray level histogram of a partial image according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of setting a gray threshold and updating a gray histogram according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of comparison between the image morphology and the processing before and after the processing according to the embodiment of the present invention;

FIG. 5 is a schematic diagram of a binary matrix according to an embodiment of the present invention;

FIG. 6 is a flowchart of a specific implementation of a method for detecting defects on a steel surface according to an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a steel surface defect detecting device according to an embodiment of the present invention;

fig. 8 is a schematic diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the following description will be made by way of specific embodiments with reference to the accompanying drawings.

The surface quality of steel is one of the most important quality factors of steel, and the quality of the surface quality of steel directly influences the performance and quality of the final product. However, during the steel processing process, different types of defects such as scabs, cracks, scratches, holes or pits may occur on the steel surface due to raw materials or processes. These defects not only affect the appearance of the product, but also reduce the properties of corrosion resistance, wear resistance, fatigue strength, etc. of the product. Therefore, how to detect the surface defects of steel to improve the surface quality of steel is a very important issue for steel processing enterprises.

However, in the prior art, the defect detection is performed on the steel surface by adopting a manual spot check mode, so that the high-speed requirement on the defect detection on the steel surface in actual industrial production cannot be met, and the defect can be omitted in the detection process. As can be seen, the prior art does not meet the high accuracy of defect detection in high speed scenarios. Therefore, the invention provides a high-precision steel surface defect detection method which can solve the problems in the prior art and is suitable for a high-speed scene.

Fig. 1 is a flowchart of an implementation of a method for detecting a surface defect of steel according to an embodiment of the present invention, which is described in detail below:

and step 101, generating a binarization segmentation threshold value of the steel surface image to be detected according to the front Jing Huidu mean value and the background gray level mean value of the steel surface image to be detected.

In some embodiments, acquiring an image of the steel surface to be measured may be performed by 1, dividing the steel surface. 2. And shooting each divided area by adopting a common camera. 3. And integrating all the acquired images to obtain the surface image of the steel to be detected. Besides the method, the image of the surface of the steel to be detected can be obtained through a line scanning camera.

The principle of the line scanning camera is that the camera and the photographed object do relative uniform motion, so that a complete and clear image of the photographed object is obtained. Because the line scanning camera has simple structure, low cost and high flexibility, the invention preferentially adopts the line scanning camera to acquire the surface image of the steel to be detected.

In some embodiments, a front background segmentation model may be used to perform front background segmentation on the steel surface image to be detected, so as to obtain a foreground area and a background area of the steel surface image to be detected, and further calculate a front Jing Huidu mean value and a background gray mean value.

In some embodiments, the front background segmentation model may be trained using a plurality of existing steel surface images, foreground regions of the steel surface images, and background regions of the steel surface images to ensure accuracy of the front background segmentation model.

In one possible implementation, the present invention is not limited to the type of algorithm employed by the foreground segmentation model, for example, a full convolutional neural network may be employed as the algorithmic framework of the foreground segmentation model.

In some embodiments, after the steel surface image to be measured is divided into the foreground region and the background region, the front Jing Huidu mean value and the background gray mean value can be obtained through the total average gray value of the whole image of the steel surface image to be measured and the duty ratio relationship of the foreground region, the background region and the whole image. The specific implementation modes of the method can be as follows:

1. and calculating the total average gray value of the whole image of the steel surface image to be measured.

2. And calculating a first pixel ratio of the pixel points in the foreground region to the total pixel points.

3. And calculating a second pixel ratio of the pixel points in the background area to the total pixel points.

4. And multiplying the total average gray value by the first pixel ratio and the second pixel ratio respectively to obtain a front Jing Huidu mean value and a background gray mean value of the steel surface image to be detected.

In some embodiments, the binarized segmentation threshold may be an average of a foreground gray-scale average and a background gray-scale average.

In some embodiments, the parameter data can be provided for subsequent binarization processing by acquiring the binarization segmentation threshold value, so that subsequent image processing is facilitated, and the defect detection speed of the steel surface image to be detected is improved.

Specifically, the implementation step of step 101 may include:

1. and training the front background segmentation model to obtain a trained front background segmentation model.

2. And acquiring an image of the surface of the steel to be detected by using a line scanning camera.

3. And dividing the steel surface image to be detected into a foreground region and a background region by adopting a foreground-background division model.

4. And calculating the total average gray value of the image of the steel surface to be detected, and calculating a first pixel ratio of the pixel points in the foreground area to the total pixel points and a second pixel ratio of the pixel points in the background area to the total pixel points. For example, assuming that the pixel point in the foreground region is 50, the pixel point in the background region is 100, and the total pixel point is 200, the first pixel ratio is 0.25, and the second pixel ratio is 0.5.

5. The first pixel ratio is multiplied by the total average gray value to obtain a first Jing Huidu mean value, and the second pixel ratio is multiplied by the total average gray value to obtain a background gray mean value. For example, assuming that the total average gray value is 50, the foreground gray average value is 12.5 and the background gray average value is 25.

6. The average of the front Jing Huidu average and the background gray average was calculated and used as a binarization segmentation threshold. For example, according to the above calculation, the binarized segmentation threshold isI.e. 18.75.

And 102, sequentially carrying out gray level equalization treatment, filtering treatment and edge enhancement treatment on the image of the surface of the steel to be detected to obtain a first image.

Specifically, the main technical feature of the gray level equalization processing provided by the invention is that the histogram distribution of an image is converted into approximately uniform distribution, so that the contrast of the image is increased.

The gradation equalization processing will be described first. Specifically, the gray-scale equalization process may be performed in steps 201-204:

and step 201, dividing the steel surface image to be detected into a plurality of partial images, and obtaining gray histograms corresponding to all the partial images.

In some embodiments, the steel surface image to be measured is divided into a plurality of partial images, and each partial image can be correspondingly processed, so that a more accurate and attractive image after gray level equalization processing is obtained.

In some embodiments, the present invention is not limited to a specific size of the partial image, but the size of the partial image obtained by division should be in a moderate range, and should not be too small or too large.

In some embodiments, the gray level histogram reflects the frequency of occurrence of each gray level pixel in an image as a function of gray level. Therefore, the abscissa of the gray level histogram may be the gray level of each pixel point in the partial image, the ordinate of the gray level histogram may be the frequency of occurrence of each pixel point of the gray level in the partial image, and each partial image corresponds to one gray level histogram.

The gray level histogram may be regarded as a probability density function corresponding to each gray level in the partial image.

Thus, it can be based onCalculating respective gray levelsA corresponding probability density. Wherein P represents a probability density function, i represents a current gray level, n _i The number of pixel points representing the gray level as the current gray level, L represents the total number of gray levels, and n represents the total number of pixel points.

To more precisely express the probability of a gray histogram, the gray histogram is explained in a specific embodiment as follows:

in some embodiments, it is assumed that there are 36 pixels in a partial image, where there are 6 pixels with a gray level of 1, 4 pixels with a gray level of 2, 6 pixels with a gray level of 3, 6 pixels with a gray level of 4, 2 pixels with a gray level of 5, and 12 pixels with a gray level of 6. It can be seen that the frequency of the pixel point with the gray level of 1 is The frequency of the pixel with gray level 2 is +.>The frequency of the pixel with gray level 3 is +.>The frequency of the pixel with gray level 4 is +.>The frequency of the pixel with gray level 5 is +.>The frequency of the pixel with gray level 6 is +.>A gray histogram of the partial image under the current assumption as shown in fig. 2 is obtained from the frequencies corresponding to the respective gray levels.

Step 202, performing contrast limitation processing on the gray level histogram to obtain a second gray level histogram.

In some embodiments, as shown in fig. 3, the contrast limiting process may be understood as setting a gray threshold in the gray histogram, and equally dividing pixels into gray levels when there are pixels in the gray histogram having a probability density greater than the gray threshold. And updating the gray level histogram based on the frequency of the pixel points of each gray level in the local image after distribution to obtain a second gray level histogram.

In one possible implementation, the number of pixels within a certain gray level range can be made approximately the same by reassigning pixels of the partial image, so as to facilitate the subsequent equalization operation.

The present invention is not limited to the specific value of the gray threshold, and may be, for example, 25%, 30%, 27%, or the like.

Step 203, an accumulated probability density function corresponding to the second gray level histogram is obtained, and the accumulated probability density function and the total gray level number are calculated to obtain a gray level mapping relation for gray level equalization processing.

In some embodiments, the cumulative probability density function is an integral of a second probability density function corresponding to the second gray level histogram.

After the cumulative probability density function corresponding to the second gray level histogram is obtained, the cumulative probability density function is normalized to a range between (0, l-1).

In one possible implementation, changing the second histogram from a certain gray level interval in the comparison set to a uniform distribution over the full gray level range can be achieved by normalizing the cumulative probability density function between the (0, l-1) ranges.

In some embodiments, the gray scale mapping may be obtained by multiplying the cumulative probability density function by the total number of gray scale levels.

And 204, carrying out gray level equalization processing on the local images according to the gray level mapping relation until all the local image processing is completed.

In some embodiments, step 201 has illustrated that each partial image corresponds to a gray level histogram, and thus, each partial image also corresponds to a gray level mapping relationship. It can be seen that the gray mapping relationship corresponding to each partial image may be different. Therefore, in practical application, the local image needs to be operated with the corresponding gray mapping relationship to obtain the processed local image.

And step 205, integrating all the processed partial images to obtain a third image.

In some embodiments, all the processed partial images are integrated according to the rule of division in step 201, so as to obtain the gray-level-balanced steel surface image to be measured.

The foregoing describes the gradation equalization processing, and the following describes the filtering processing.

In some embodiments, the image tends to be contaminated with various noise during its transmission recording due to imperfections in the transmission medium and recording equipment, etc. Therefore, it is necessary to filter the image during image processing. The filtering process may be regarded as a denoising process, and is a technical means for denoising and smoothing an image.

Specifically, the specific implementation steps of the filtering process may be steps 301-304:

in step 301, a center point is defined in the third image.

In some embodiments, the invention is not limited to a specific selection of the location of the center point, but the center point should be selected from a moderately located region of the third image and should not be excessively biased in a certain direction.

In step 302, the spatial proximity of each pixel point to the center point in the third image and the pixel value similarity between each pixel point and the center point are calculated.

In some embodiments, it may be according toAnd calculating the spatial proximity of each pixel point to the center point and the pixel value similarity of each pixel point and the center point. Wherein w is _s Representing spatial proximityDegree, w _r The pixel value similarity is represented by (k, l), the pixel point is represented by (x, y), the center point is represented by f (k, l), the pixel value of the pixel point is represented by f (x, y), and the base of the natural logarithm is represented by e.

Step 303, multiplying the spatial proximity of each pixel point by the similarity of the pixel values to obtain a convolution weight of each pixel point;

in some embodiments, w (x, y, k, l) =w _s (x,y,k,l)×w _r And (x, y, k, l) obtaining the convolution weight of each pixel point. Where w represents the convolution weight.

And step 304, obtaining a fourth image according to the convolution weight value of each pixel point and the pixel value of each pixel point.

In some embodiments, the fourth image may be obtained by: 1. calculating products of pixel values of all pixel points in the third image and convolution weights, and adding all the products to obtain a product sum; 2. calculating the sum of all convolution weights; 3. and comparing the product sum with the sum of the convolution weights to obtain a fourth image.

Specifically, the formula of the above steps may be: Where g (x, y) represents a fourth image, and S (x, y) represents a region within a range of (2n+1) x (2n+1) centered on (x, y).

In the present invention, S (x, y) may refer to an area where the third image is located.

In some embodiments, the edges of the defects are not strong, as the filtered fourth image may be smoother. Therefore, the edge enhancement processing and the filtering processing can be combined, and the technical effects of removing noise in the image and enhancing the edge of the defect area are achieved.

The foregoing describes the filtering process, and the edge enhancement process is described below.

In some embodiments, the main technical content of the edge enhancement processing provided by the invention can be that an operator is used for convolution calculation in the horizontal direction and the vertical direction of an image to obtain gradient results in the two directions, and the edge enhancement of a defect area is realized by combining the gradient results in the two directions.

Specifically, the specific implementation steps of the edge enhancement process may be steps 401-404:

step 401, acquiring a horizontal matrix corresponding to the fourth image in the horizontal direction and a vertical matrix corresponding to the fourth image in the vertical direction.

In some embodiments, since the edge enhancement process uses operators to convolve the images, the internal parameters of the horizontal matrix and the vertical matrix corresponding to the fourth image are determined by the type of operators.

In some embodiments, taking the Sobel operator as an example, the horizontal matrix corresponding to the fourth image may beThe vertical matrix corresponding to the fourth image may be +.>

In step 402, a plane convolution calculation is performed on the horizontal matrix and the fourth image, so as to obtain a horizontal gradient image.

In some embodiments, it may be according toA horizontal gradient image is obtained. Where X (X, y) represents a horizontal gradient image.

In some embodiments, performing a planar convolution calculation on the horizontal matrix with the fourth image corresponds to extracting edges in the horizontal direction using the horizontal matrix.

And step 403, performing plane convolution calculation on the vertical matrix and the fourth image to obtain a vertical gradient image.

In some embodiments, it may be according toA vertical gradient image is obtained. Wherein Y (x, Y) represents verticalGradient images.

In some embodiments, performing a planar convolution calculation on the vertical matrix with the fourth image is equivalent to extracting edges in the vertical direction using the vertical matrix.

Step 404, performing bit-wise OR operation on the horizontal gradient image and the vertical gradient image to obtain a first image.

In some embodiments, it may be according toA first image is obtained. Wherein G (x, y) represents a first image, ">Representing a bitwise or operation.

In some embodiments, the edge information of the horizontal direction and the vertical direction is integrated to obtain the edge of the whole image, so that the boundary between the defect area and the background is highlighted, and the subsequent positioning and marking of the defect area are simpler.

And 103, performing binarization processing on the first image by using the binarization segmentation threshold value to obtain a second image.

In some embodiments, the specific steps of the binarization process may be: 1. and setting the gray value corresponding to the pixel point larger than the binarization segmentation threshold value in the first image as a first gray value. 2. And setting the gray value corresponding to the pixel point smaller than the binarization segmentation threshold in the first image as a second gray value.

In order to make the contrast of the binarized first image higher, and to refer to the range of values of the gray scale values, the value of the first gray scale value may be 255, and the value of the second gray scale value may be 0.

In one possible implementation, the binarizing process is performed on the first image, so that pixels in the first image can be converted into black or white, and thus contrast of the first image is enhanced, and definition of the first image is enhanced. Further, since there are only two colors in the second image, the data amount of the second image is also reduced as compared with the previous image.

It should be noted that the small data size of the second image also side-illustrates that the subsequent image processing speed is faster than the image that is not binarized. Therefore, the technical effect of the invention suitable for the high-speed scene can be realized by performing binarization processing on the first image.

Step 104, generating a defect detection result of the steel surface image to be detected according to the second image and a pre-trained defect detection model; the defect detection model is obtained based on training of a preset training set, the preset training set comprises a plurality of steel surface defect images and steel surface defect-free images, and the steel surface defect images correspond to defect types.

The foregoing describes edge enhancement processing, and morphological closing processing is described below.

In some embodiments, to make the image input into the defect detection model more accurate, morphological close operation processing may be further performed on the second image to obtain a close operation image. As a comparison schematic diagram before and after the morphological closing operation of the image shown in fig. 4, fig. a is an image before the morphological closing operation, fig. b is an image after the morphological closing operation, white portions in fig. a and b are defective areas, and black portions are non-defective areas. Therefore, the morphological closing operation processing can remove small noise in the second image background, fill small holes in the communication area and disconnect less connected adjacent objects.

Specifically, the specific implementation steps of the morphological closing operation process may be steps 501-503:

step 501, defining a binary matrix, the binary matrix being composed of structural elements. As shown in the schematic structure of the binary matrix in fig. 5, the black part in the figure represents the effective part of the structural element, the value is 1, and the white part in the figure represents the ineffective part of the structural element, the value is 0.

Step 502, performing a dilation operation on the second image using the structural elements. I.e. the second image is convolved with the structural element moving from each pixel point in the second image.

For each pixel point during the expansion operation, the pixel value of at least one pixel point is set to 1 as long as the pixel value of at least one pixel point is non-zero in the portion where the structural element intersects with the surrounding pixel points of the certain pixel point.

At step 503, the expanded second image is subjected to an etching operation using the structural element. I.e. the structure elements are used to convolve the expanded second image starting from each pixel point in the expanded second image.

It should be noted that, for each pixel point in the etching operation process, only if the pixel values of all the pixel points in the intersection portion of the adjacent area of a certain pixel point and the structural element are non-zero, the pixel value of the pixel point may be set to 1; otherwise, the pixel value of the pixel point is set to 0.

In some embodiments, the training process of the defect detection model may be: 1. and training the defect detection model based on a preset training set. The preset training set comprises a plurality of steel surface defect images, steel surface defect-free images and defect types corresponding to the steel surface defect images. 2. And adjusting and optimizing parameters of the defect detection model based on a preset training set and an application scene to obtain the trained defect detection model.

It should be noted that the defect detection model may be a deep learning model, and the training process of the defect detection model may be shortened according to the transfer learning under the condition of insufficient training time.

It should be noted that, assuming that the defect detection model is a deep learning model, in the actual training process, since the difference of the sizes of the individual defect regions is large, in order to improve the feature extraction capability of the defect detection model for the defect regions with different sizes, the neck structure of the deep learning model, that is, the feature pyramid network, may be replaced by an acceptance block.

In some embodiments, after the closed-loop image is obtained, defect region identification may be performed on the closed-loop image.

Specifically, when a defect area with a size larger than a preset threshold value does not exist in the closed operation image, and the similarity between the steel surface image to be detected corresponding to the closed operation image and the steel surface defect-free image is larger than a first preset value, determining that the steel surface image to be detected corresponding to the closed operation image is defect-free.

In one possible implementation, the present invention is not limited to the value of the first preset value. For example, the first preset value may be 90%, 95%, 96%, or the like.

In some embodiments, when a defect area with a size greater than a preset threshold exists in the closed operation image, the position of the defect area in the closed operation image can be referred to mark the position of the defect area in the steel surface image to be detected, the marked steel surface image to be detected is stored in a suspicious image queue, and then the suspicious image queue is input into a defect detection model to obtain a defect detection result of a defect type corresponding to the steel surface image to be detected.

In one possible implementation mode, the defect area is marked in the original image and then is input into the defect detection model, so that the initial positioning of the position of the defect can be realized, the defect positioning time in the detection process is saved, the accuracy of the image and the defect result can be ensured from the source, and the problem of detection errors possibly caused by inputting the processed image into the defect detection model is solved.

Specifically, when a defect area with the size larger than a preset threshold exists in the closed operation image, and the marked defect area is a defect detection result of a defect type corresponding to the steel surface image to be detected and corresponding to the closed operation image when the similarity of the marked defect area is larger than a second preset value.

In one possible embodiment, the present invention is not limited to the value of the second preset value. For example, the second preset value may be 90%, 95%, 96%, or the like.

In some embodiments, the types of defects may include cracks, scratches, folds, ears, scars, rust, bubbles, pits, porosity, nonmetallic inclusions, overburden, white spots, weld marks, or end burrs, etc.

It should be noted that when a defect region having a size larger than a preset threshold exists in the closed operation image and a steel surface defect image similar to the marked defect region is not found in the steel surface defect image, it is determined that the marked defect region is not a defect but is also of a defect-free type.

It should be noted that, there may be multiple defect areas in the surface image of the steel to be detected, and when the marked surface image of the steel to be detected is input into the defect detection model for defect detection, the defect detection model performs defect detection on the multiple defect areas simultaneously. Therefore, the defect detection is not required to be carried out on the defect areas one by one, and the detection process can meet the high-speed requirement on the detection of the defects on the steel surface in actual industrial production.

After determining that the marked iron and steel surface image to be measured has defects, the iron and steel surface image to be measured can be stored in a preset training set, and training data is newly added for the preset training set.

In some embodiments, in practical application, monitoring may be set in the suspicious image queue, and if the suspicious image queue has a marked iron and steel surface image to be detected newly added, the marked iron and steel surface image to be detected is immediately input into the defect detection model for detection, so as to obtain a detection result of the iron and steel surface image to be detected.

It is worth mentioning that when the suspicious image queue is monitored, it can be guaranteed that no other images to be detected exist in the suspicious image queue, the images to be detected newly added into the suspicious image queue can be input into the defect detection model at the first time, and therefore the problem of time waste caused by no reaction of the images to be detected after the images to be detected are added into the suspicious image queue is solved.

As shown in the specific implementation flowchart of the steel surface defect detection method shown in fig. 6, the present invention can satisfy both the processing speed requirement and the defect detection accuracy requirement in the high-speed steel defect detection task.

The embodiment of the invention provides a steel surface defect detection method, which comprises the steps of firstly generating a binarization segmentation threshold value of a steel surface image to be detected according to a pre-obtained front Jing Huidu mean value and a background gray level mean value of the steel surface image to be detected, then sequentially carrying out preprocessing such as gray level equalization processing, filtering processing, edge enhancement processing and the like on the steel surface image to be detected to obtain a first image, carrying out binarization processing on the first image through the obtained binarization segmentation threshold value to obtain a second image with black and white pixels, and finally obtaining a defect detection result of the steel surface image to be detected according to the second image and a defect detection model which is trained in advance.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.

The following are device embodiments of the invention, for details not described in detail therein, reference may be made to the corresponding method embodiments described above.

Fig. 7 is a schematic structural diagram of a steel surface defect detecting device according to an embodiment of the present invention, and for convenience of explanation, only the portions relevant to the embodiment of the present invention are shown, and the details are as follows:

as shown in fig. 7, the steel surface defect detecting device 7 includes:

the generating module 71 is configured to generate a binary segmentation threshold of the steel surface image to be tested according to the front Jing Huidu mean value and the background gray level mean value of the steel surface image to be tested;

a first processing module 72, configured to sequentially perform gray level equalization processing, filtering processing, and edge enhancement processing on the steel surface image to be detected, so as to obtain a first image;

a second processing module 73, configured to perform binarization processing on the first image by using a binarization segmentation threshold value, so as to obtain a second image;

the detection module 74 is configured to generate a defect detection result of the steel surface image to be detected according to the second image and the pre-trained defect detection model; the defect detection model is obtained based on training of a preset training set, the preset training set comprises a plurality of steel surface defect images and steel surface defect-free images, and the steel surface defect images correspond to defect types.

In one possible implementation, the generating module 71 is specifically configured to:

In one possible implementation, the second processing module 73 is specifically configured to:

In one possible implementation, the detection module 74 is specifically configured to:

performing defect area identification on the closed operation image;

In one possible implementation, the first processing module 72 is specifically configured to:

and integrating all the processed partial images to obtain a third image.

defining a center point in the third image;

The embodiment of the invention provides a steel surface defect detection device, which firstly generates a binarization segmentation threshold value of a steel surface image to be detected according to a pre-obtained front Jing Huidu mean value and a background gray level mean value of the steel surface image to be detected, then sequentially carries out preprocessing such as gray level equalization processing, filtering processing, edge enhancement processing and the like on the steel surface image to be detected to obtain a first image, carries out binarization processing on the first image through the obtained binarization segmentation threshold value to obtain a second image with black and white pixels, and finally obtains a defect detection result of the steel surface image to be detected according to the second image and a defect detection model which is trained in advance.

Fig. 8 is a schematic diagram of an electronic device according to an embodiment of the present invention. As shown in fig. 8, the electronic device 8 of this embodiment includes: a processor 80, a memory 81 and a computer program 82 stored in the memory 81 and executable on the processor 80. The processor 80, when executing the computer program 82, implements the steps of the various embodiments of the steel surface defect detection method described above, such as steps 101 through 104 shown in fig. 1. Alternatively, the processor 80, when executing the computer program 82, performs the functions of the modules/units of the apparatus embodiments described above, such as the functions of the modules/units 71-74 shown in fig. 7.

By way of example, the computer program 82 may be partitioned into one or more modules/units that are stored in the memory 81 and executed by the processor 80 to complete the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing the specified functions, which instruction segments are used to describe the execution of the computer program 82 in the electronic device 8. For example, the computer program 82 may be split into modules/units 71 to 74 shown in fig. 7.

The electronic device 8 may include, but is not limited to, a processor 80, a memory 81. It will be appreciated by those skilled in the art that fig. 8 is merely an example of an electronic device 8 and is not meant to be limiting as to the electronic device 8, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the electronic device may also include input-output devices, network access devices, buses, etc.

The processor 80 may be a central processing unit (Central Processing Unit, CPU), other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or a memory of the electronic device 8. The memory 81 may also be an external storage device of the electronic device 8, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device 8. Further, the memory 81 may also include both an internal storage unit and an external storage device of the electronic device 8. The memory 81 is used for storing the computer program and other programs and data required by the electronic device. The memory 81 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/electronic device and method may be implemented in other manners. For example, the apparatus/electronic device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the procedures in the above-described embodiments of the method, or may be implemented by instructing the relevant hardware by a computer program, where the computer program may be stored in a computer readable storage medium, and the computer program may implement the steps of the above-described embodiments of the method for detecting a surface defect of steel when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth.

The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.

Claims

1. A method for detecting surface defects of steel, comprising:

generating a binarization segmentation threshold value of the steel surface image to be detected according to a front Jing Huidu mean value and a background gray level mean value of the steel surface image to be detected;

performing binarization processing on the first image by using the binarization segmentation threshold value to obtain a second image;

generating a defect detection result of the steel surface image to be detected according to the second image and a pre-trained defect detection model; the defect detection model is obtained by training based on a preset training set, the preset training set comprises a plurality of steel surface defect images and steel surface defect-free images, and the steel surface defect images are corresponding to defect types.

2. The method for detecting a steel surface defect according to claim 1, wherein the generating the binarized segmentation threshold of the steel surface image to be detected according to the front Jing Huidu mean value and the background gray-level mean value of the steel surface image to be detected comprises:

calculating an average gray value of the steel surface image to be detected, a first pixel ratio of the foreground region to the steel surface image to be detected, and a second pixel ratio of the background region to the steel surface image to be detected;

obtaining the front Jing Huidu average value according to the average gray value and the first pixel ratio;

obtaining the background gray average value according to the average gray value and the second pixel ratio;

an average of the front Jing Huidu average and the background grayscale average is calculated and defined as the binarized segmentation threshold.

3. The method for detecting a surface defect of steel according to claim 1, wherein the binarizing the first image using the binarized segmentation threshold value comprises:

Setting a gray value corresponding to a pixel point larger than the binarization segmentation threshold value in the first image as a first gray value;

4. The method for detecting the defects of the steel surface according to claim 1, wherein the generating the defect detection result of the image of the steel surface to be detected according to the second image and a pre-trained defect detection model comprises:

performing a closed operation process on the second image to obtain a closed operation image;

performing defect area identification on the closed operation image;

when a defect area with the size larger than a preset threshold exists in the closed operation image, marking an image area corresponding to the defect area with the size larger than the preset threshold in the closed operation image in the steel surface image to be detected, and inputting the marked steel surface image to be detected into the defect detection model to obtain a defect detection result of a defect type corresponding to the steel surface image to be detected;

and when the defect area with the size larger than a preset threshold value does not exist in the closed operation image, generating a defect detection result that the steel surface image to be detected is defect-free.

5. The method for detecting a surface defect of steel according to claim 1, wherein the sequentially performing gray-scale equalization processing, filtering processing and edge enhancement processing on the steel surface image to be detected to obtain a first image comprises:

acquiring a gray mapping relation for gray level equalization processing according to the gray level histogram of the steel surface image to be detected, and carrying out gray level equalization processing on the steel surface image to be detected according to the gray level mapping relation to obtain a third image;

acquiring a convolution weight for filtering according to the spatial proximity of the third image and the pixel value similarity of the third image, and filtering the third image according to the convolution weight to obtain a fourth image;

acquiring a horizontal gradient image and a vertical gradient image of the fourth image for edge enhancement processing, and obtaining the first image according to the horizontal gradient image and the vertical gradient image.

6. The method for detecting a defect on a steel surface according to claim 5, wherein the step of obtaining a gray mapping relation for gray equalization according to a gray histogram of the steel surface image to be detected, and performing gray equalization on the steel surface image to be detected according to the gray mapping relation, to obtain a third image, comprises:

Dividing the steel surface image to be detected into a plurality of partial images, and obtaining gray histograms corresponding to all the partial images; each local image corresponds to a gray level histogram, the gray level histogram is a probability density function corresponding to each gray level in the local image, the abscissa of the gray level histogram is the gray level of each pixel point in the local image, and the ordinate of the gray level histogram is the frequency of occurrence of each gray level pixel point in the local image;

performing contrast limiting processing on the gray level histogram to obtain a second gray level histogram;

acquiring an accumulated probability density function corresponding to the second gray level histogram, and calculating the accumulated probability density function and the total gray level number to obtain a gray level mapping relation for gray level equalization processing; wherein the cumulative probability density function is an integral of a second probability density function corresponding to the second gray level histogram;

carrying out gray level equalization processing on the partial images according to the gray level mapping relation until all partial image processing is completed;

and integrating all the processed partial images to obtain a third image.

7. The method for detecting a surface defect of steel according to claim 5, wherein the obtaining a convolution weight for filtering according to the spatial proximity of the third image and the similarity of pixel values of the third image, and the filtering the third image according to the convolution weight to obtain a fourth image comprises:

defining a center point in said third image;

calculating the spatial proximity of each pixel point in the third image to the central point and the pixel value similarity of each pixel point and the central point;

multiplying the spatial proximity of each pixel point by the pixel value similarity to obtain a convolution weight of each pixel point;

8. The steel surface defect detection method according to claim 5, wherein the acquiring the horizontal gradient image and the vertical gradient image of the fourth image for edge enhancement processing, and obtaining the first image from the horizontal gradient image and the vertical gradient image, comprises:

Carrying out plane convolution calculation on the horizontal matrix and the fourth image to obtain the horizontal gradient image;

carrying out plane convolution calculation on the vertical matrix and the fourth image to obtain the vertical gradient image;

and carrying out bit-wise OR operation on the horizontal gradient image and the vertical gradient image to obtain the first image.

9. A steel surface defect detection apparatus, comprising:

the second processing module is used for carrying out binarization processing on the first image by utilizing the binarization segmentation threshold value to obtain a second image;

the detection module is used for generating a defect detection result of the steel surface image to be detected according to the second image and a pre-trained defect detection model; the defect detection model is obtained by training based on a preset training set, the preset training set comprises a plurality of steel surface defect images and steel surface defect-free images, and the steel surface defect images are corresponding to defect types.

10. An electronic device comprising a memory for storing a computer program and a processor for calling and running the computer program stored in the memory, characterized in that the processor implements the steps of the method according to any of the preceding claims 1-8 when the computer program is executed.