WO2023087741A1 - Defect detection method and apparatus, and electronic device, storage medium and computer program product - Google Patents

Abstract

The present disclosure relates to a defect detection method and apparatus, and an electronic device, a storage medium and a computer program product. The method comprises: extracting an area to be subjected to detection from an image to be subjected to detection; dividing said area into a plurality of image blocks to be subjected to detection; respectively performing defect detection on each of said image blocks, so as to obtain a corresponding defect detection result, which is used for representing whether there is a defect in the corresponding one of said image blocks; and finally, determining an image detection result on the basis of each defect detection result. By means of the embodiments of the present disclosure, during defect detection, an interference area in an image is removed by means of extracting an area to be subjected to detection, and an area requiring detection is detected, thereby improving the precision of the defect detection. Moreover, the defect detection is performed in parallel by means of dividing the area to be subjected to detection into a plurality of image blocks to be subjected to detection, thereby improving the efficiency of a detection process.

Description

Defect detection method and device, electronic device, storage medium and computer program product

Cross References to Related Applications

The embodiment of the present disclosure is based on the Chinese patent application with the application number 202111371813.8, the application date is November 18, 2021, and the application name is "Defect detection method and device, electronic equipment and storage medium", and the priority of the Chinese patent application is required Right, the entire content of this Chinese patent application is hereby incorporated into this disclosure by way of introduction.

technical field

Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a defect detection method and device, electronic equipment, storage media, and computer program products.

Background technique

Battery defect detection is an important problem in computer vision and industrial vision inspection. Battery defect inspection includes a variety of applications, such as roof weld inspection, sealing nail defect inspection, battery tab deformation defect inspection, and coated surface inspection. Since the structure on the battery is relatively complex and the defects are often small, false detection or missed detection usually occurs during defect detection. At the same time, since the images collected during battery inspection are usually high-resolution industrial images, the amount of calculation in the defect detection process is large and the detection speed is slow.

Contents of the invention

Embodiments of the present disclosure propose a defect detection method and device, electronic equipment, storage media, and computer program products, aiming at improving the efficiency of the defect detection process and the accuracy of defect detection results.

In a first aspect, an embodiment of the present disclosure provides a defect detection method, the method comprising:

Extract the region to be detected in the image to be detected;

dividing the region to be detected into a plurality of image blocks to be detected;

performing defect detection on each of the image blocks to be detected, to obtain a defect detection result corresponding to each of the image blocks to be detected, and the defect detection results are used to indicate whether there is a defect in the corresponding image block to be detected;

An image detection result is determined based on each of the defect detection results.

In a second aspect, an embodiment of the present disclosure provides a defect detection device, the device comprising:

An area extraction module configured to extract an area to be detected in the image to be detected;

An image division module configured to divide the region to be detected into a plurality of image blocks to be detected;

The defect detection module is configured to perform defect detection on each of the image blocks to be detected, and obtain a defect detection result corresponding to each of the defect detection image blocks, and the defect detection result is used to represent the defects in the corresponding image block to be detected whether there are defects;

A result determination module configured to determine an image detection result according to each of the defect detection results.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.

In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer-readable storage medium storing program code, and when instructions included in the program code are executed by a processor of a computer device, the steps in the above method are implemented. .

In the embodiment of the present disclosure, when performing defect detection, firstly, the interference region in the image is eliminated by extracting the region to be detected, and the region to be detected is detected, thereby improving the accuracy of defect detection. In the second aspect, by dividing the area to be detected into a plurality of image blocks to be detected, and then obtaining the position information of the image blocks, the location of the defect is realized; in the third aspect, by Defect detection is performed on the detected image blocks, which reduces the missed detection rate and false detection rate; in the fourth aspect, the efficiency of the detection process is improved by performing defect detection on multiple image blocks to be detected in parallel.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory in nature and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings here are incorporated into the description and constitute a part of the present description. These drawings show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.

FIG. 1 shows a flowchart of a defect detection method provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram of an image to be detected provided by an embodiment of the present disclosure;

FIG. 3 shows a schematic diagram of extracting a region to be detected provided by an embodiment of the present disclosure;

Fig. 4 shows a schematic flow chart of extracting a region to be detected provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of determining an image block to be detected provided by an embodiment of the present disclosure;

FIG. 6A shows a schematic diagram of determining an image detection result provided by an embodiment of the present disclosure;

FIG. 6B shows a flow chart of another defect detection method provided by an embodiment of the present disclosure;

FIG. 6C shows a schematic diagram of the process of an image preprocessing method provided by an embodiment of the present disclosure;

FIG. 6D shows a schematic diagram of the process of a model classification method provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic diagram of a defect detection device provided by an embodiment of the present disclosure;

FIG. 8 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure;

Fig. 9 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without some of the specific details. In some instances, methods, means, components and circuits that are well known to those skilled in the art have not been described in detail so as to obscure the gist of the present disclosure.

FIG. 1 shows a flowchart of a defect detection method provided by an embodiment of the present disclosure. In a possible implementation manner, the defect detection method in the embodiment of the present disclosure may be executed by an electronic device such as a terminal device or a server. Wherein, the terminal device may be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, Mobile or fixed terminal devices such as wearable devices. The terminal device can implement the defect detection method by calling the computer-readable instructions stored in the memory by the processor. In some embodiments, the defect detection method of the embodiments of the present disclosure can also be executed by a server, and the server can be a single server or a server cluster composed of multiple servers.

The embodiments of the present disclosure may be applied, for example, to the scene of initial detection of items such as batteries and product packaging shells. When testing the battery, it can also be used for roof weld detection, sealing nail defect detection, battery tab deformation defect detection, and coating surface detection. The embodiments of the present disclosure do not limit the application scenarios of defect detection. .

As shown in Figure 1, a defect detection method provided by an embodiment of the present disclosure may include the following steps:

Step S10, extracting the region to be detected in the image to be detected.

In a possible implementation manner, the defect detection method in the embodiment of the present disclosure is used to detect a defect of a target object, and the image to be detected is an image obtained by collecting the target object. In different application scenarios, the target audience is different. For example, when performing battery defect detection in the embodiment of the present disclosure, the target object may be a battery, that is, the defect detection method is used to detect the battery. When the product shell defect detection is performed in the embodiment of the present disclosure, the target object may be the product shell. In some embodiments, the content of the image to be detected can also be determined according to the type of defect to be detected. For example, when the defect to be detected is a weld defect of a battery, the image to be detected needs to include the weld of the battery, and the image to be detected may be an image of the battery top cover obtained by collecting the battery top cover.

The area to be inspected refers to an area that may have defects that need to be inspected. In some embodiments, the region to be inspected may include a roof weld region in the battery image. Because the image to be detected usually includes not only the region to be detected, but also some interference regions that do not need to be detected. In order to avoid increasing the amount of calculation and increasing the possibility of false detection due to the detection of the interference area that does not need to be detected, the interference area can be excluded by extracting the area to be detected in the image to be detected.

Fig. 2 shows a schematic diagram of an image to be detected provided by an embodiment of the present disclosure. As shown in FIG. 2 , in the application scenario of detecting a weld defect of a battery, the target object to be detected in the embodiment of the present disclosure is a battery, and the image 20 to be detected is a battery image obtained by collecting a top cover of the battery. The image of the battery includes the roof weld area that needs to be detected and may have the weld defect 21 , and other areas where the weld defect 21 may not exist. Therefore, the region to be detected can be obtained by extracting the roof weld region of the image to be detected.

In a possible implementation, since the area where the weld seam of the battery is located is located at the edge of the top cover of the battery, when performing defect detection on the weld seam of the battery, the embodiments of the present disclosure can obtain the image to be detected by extracting the edge of the image to be detected area.

Fig. 3 shows a schematic diagram of extracting a region to be detected provided by an embodiment of the present disclosure. As shown in FIG. 3 , the image 30 to be detected is an image of the battery top cover, where the weld 31 is located at the edge of the battery top cover. In some embodiments, the image to be detected 30 also includes the inner area of the battery top cover that is far away from the welding seam 31 . In order to prevent the internal area of the battery top cover from affecting the detection efficiency and interfering with the detection results, the interference area inside the battery top cover can be removed by means of edge extraction, and the welding seam area of the top cover can be obtained as the area 32 to be detected.

In a possible implementation manner, the region to be detected may be obtained by performing binarization processing and maximum contour extraction on the image to be detected. That is to say, the image to be detected may be binarized first by using a preset threshold parameter (for example, the threshold parameter may be 50), so as to highlight contour edges in the image to be detected. In some embodiments, a contour extraction function such as findContours is used to extract the maximum contour, remove regions other than the extracted maximum contour, and retain the extracted maximum contour region as the region to be detected.

In some embodiments, since the collected image to be detected may have problems such as deformation, in order to improve the image processing effect, other processing processes such as size conversion may also be added before extracting the region to be detected. For example, if there is a deformation problem in the collected image to be detected, before binarizing the image to be processed, the image to be detected can be resized by dimension transformation method to restore its original size. Fig. 4 shows a schematic flow chart of extracting a region to be detected provided by an embodiment of the present disclosure. As shown in Figure 4, the process of determining the area to be processed by image processing can be to first perform size transformation 41 on the image to be detected 40 through a dimension transformation method (for example, the height of the image to be detected to be collected is elongated by 5 times, and the width does not change, then The height of the image to be processed can be reduced by 1/5 through the dimension transformation method, and the width remains unchanged to obtain the image to be detected after the size transformation); after obtaining the image to be detected after the size transformation, the preset threshold parameter Perform binarization 42 on the size-changed image to be detected to highlight the contour edge to obtain the maximum contour; finally remove the interference area of the non-detected region in the to-be-detected image 40 by means of maximum contour extraction 43 to obtain the to-be-detected region 44 .

Step S20, dividing the region to be detected into a plurality of image blocks to be detected.

In a possible implementation manner, after the region to be detected in the image to be detected is extracted, the region to be detected is divided to obtain multiple image blocks to be detected, and then defect detection is performed on each image block to be detected to improve detection efficiency. In some embodiments, the manner of dividing the region to be detected may be, for example, dividing the region to be inspected evenly into a preset number of regions with the same size as image blocks to be detected. Alternatively, a plurality of image blocks to be detected may be determined by sliding on the region to be detected through a preset sliding window, and the width of the sliding window is the same as the width of the region to be detected. Wherein, when the region to be detected is a rectangle, the width of the sliding window may be the width of the rectangle. When the area to be detected is a rectangular ring, the width of the sliding window may be the width of the ring.

That is to say, in a possible implementation, after the area to be detected is determined, the preset sliding step size and the sliding window with the same width as the area to be detected can be used to specify the position in the upper left corner, lower right corner, etc. of the area to be detected Start to slide to the preset direction with the sliding step as the unit, and each sliding process determines an image block to be detected. In some embodiments, there is an undetermined area after the last sliding, that is, when the length of the remaining area is less than the length of the sliding window, it can be directly determined that this area is also an image block to be detected. The manner of detecting the region is not limited.

Fig. 5 shows a schematic diagram of determining an image block to be detected provided by an embodiment of the present disclosure. As shown in Figure 5, for the region to be detected 50, the width of the region to be detected 50 can be set as the width of the sliding window according to the width of the sliding window is the same as the width of the region to be detected 50, starting from the upper left corner in a clockwise direction slide. After each sliding, determine the area inside the current sliding window as an image block to be detected 51 , and determine the coordinates of the central point of the image block to be detected as the coordinates of the image block to be detected. In some embodiments, the coordinates of the upper left corner of the area to be detected can be defined as (0,0), and the coordinates of the center point of the image block to be detected can be determined according to the size of the sliding window and the sliding step, and then the image to be detected can be obtained The coordinates of the block, for example, the coordinates of the central point of the image block to be detected are (40, 50), then the coordinates of the image block to be detected are (40, 50). In the embodiment of the present disclosure, by dividing the area to be detected into multiple image blocks, it is convenient to obtain the position information corresponding to the defect classification result obtained after the classification processing after the image classification processing is performed on the image block to be detected.

Step S30 , performing defect detection on each of the image blocks to be detected, and obtaining a defect detection result corresponding to each of the image blocks to be detected.

In a possible implementation manner, after determining at least one image block to be detected that may have a defect in the image to be detected, defect detection is performed on each image block to be detected to obtain a defect detection result of each image block to be detected. Wherein, each defect detection result is used to represent whether there is a defect of the target object in the corresponding image block to be detected. For example, when the embodiment of the present disclosure is used to detect a battery weld defect, the defect detection result is used to indicate whether there is a battery weld defect in the corresponding image block to be detected.

In some embodiments, the manner of performing defect detection on the image block to be detected may be: for example, the defect detection may be performed by means of image processing, or the defect detection may also be performed by using a trained defect detection model. Wherein, the image processing method may be to determine a standard image block, that is, a standard image block with the same position as the image block to be detected in an image without defects, and obtain a defect detection result by comparing the difference between the standard image block and the image block to be detected. When performing defect detection through the defect detection model obtained through training, each image block to be detected may be input into the defect detection model, and a corresponding defect detection result may be output.

In a possible implementation manner, the defect detection result of the image block to be detected may include a position attribute, at least one defect category, and a confidence level of each defect category. Among them, the position attribute is used to represent the position of the image block to be detected corresponding to the defect detection result in the image to be detected, the defect category is used to represent the possible defect types in the image block to be detected, and the confidence level represents the corresponding The probability of a defect class defect. For example, when the image to be detected is a battery image and is used to detect the weld defect of the battery top cover, the defect detection result obtained from the detection includes at least one weld defect category and the corresponding confidence level, as well as the location attribute. The position attribute is used to characterize the position of the corresponding image block to be detected in the battery image, and the confidence degree is used to characterize the probability of welding seam defects including the corresponding defect category in the figure, wherein the position attribute can be the to-be-detected position obtained after step S20 The coordinates of the image block.

The embodiment of the present disclosure detects the obtained image block to be detected by extracting the region to be detected and dividing the region to be detected, so that the defect detection process of an image to be detected with a relatively large resolution can be converted into a defect detection process for multiple resolutions. The process of performing defect detection on smaller image blocks to be detected reduces the calculation amount of the defect detection process. In some embodiments, the detection process of each image block to be detected can also be executed in parallel, thereby increasing the detection speed.

Step S40, determining an image detection result according to each defect detection result.

In a possible implementation manner, after determining the defect detection result corresponding to each image block to be detected, the image detection result of the image to be detected may be determined according to each defect detection result. In some embodiments, the image detection result may be determined by determining the image confidence level according to the confidence level of each defect category, and then determining the defect detection result whose image confidence level is greater than the first threshold as the image detection result. Image confidence is used to characterize the possibility of defects in the image to be detected. The image detection result is used to indicate whether there is a defect of the target object in the image to be detected. For example, when the image to be detected is an image of the battery top cover, the image detection result is used to indicate whether there is a weld defect of the top cover in the image to be detected. In some embodiments, when the image detection result indicates that there is a defect in the image to be detected, it may also include the type of defect and the position information of the defect.

FIG. 6A shows a schematic diagram of determining an image detection result provided by an embodiment of the present disclosure. As shown in FIG. 6A, in a possible implementation, after determining the N image blocks to be detected (including the image block to be detected 1, the image block to be detected 2...the image block to be detected N) 60 Afterwards, defect detection is performed on each image block 60 to be detected through the detection model 61 , and a defect detection result corresponding to each image block 60 to be detected is obtained. In some embodiments, the image detection result 63 is determined according to N defect detection results (including defect detection result 1, defect detection result 2 . . . defect detection result N) 62 corresponding to N image blocks 60 to be detected. Wherein, the image detection result 63 may be determined based on the defect category and confidence included in each defect detection result 62 .

In a possible implementation manner, the process of determining the image confidence of the corresponding image block to be detected according to each defect detection result may be: determine the confidence of the target defect category as the target confidence, and may respond to the target confidence being less than The second threshold is to determine the image confidence level according to the confidence level of each defect category other than the target defect category, or determine that the image confidence level is 0 in response to the target confidence level being not less than the second threshold value. That is, when there is a defect that needs to be ignored, the defect category of the defect is set as the target defect category. When the confidence of the target defect category is less than the preset second threshold, it is considered that the corresponding defect in the image block to be detected has a small probability of the target defect category, so it can be based on the Confidence determines the image confidence. When the confidence degree of the target defect category is not less than the preset second threshold, it is considered that the defect of the corresponding image block to be detected has a higher probability of the target defect category, so it is necessary to ignore the defect detection result and directly determine the image confidence degree is 0. In some embodiments, the target defect category can be set according to actual application scenarios. For example, the starting point position is often misrecognized as a defect during the detection of weld defects on the battery top cover, so the starting point can be set as the target defect category, so that when the defect type is detected as the starting point Ignore the defect detection result and determine the image confidence as 0.

In some embodiments, the manner of determining the image confidence level according to the confidence level of each defect category except the target defect category may be to calculate the sum of the confidence levels of each defect category to obtain the image confidence level. Alternatively, the confidence of each defect category can also be input into a logistic regression function (softmax function) to obtain the image confidence. For example, the confidence of each defect category can be input into the softamx function and then summed to obtain the image confidence representing the possibility of defects in the image to be detected.

In the embodiments of the present disclosure, when determining the image confidence, the target defect category that is likely to cause false detection results is preset, and the image confidence is determined according to the confidence of the target defect category. The above method can reduce the frequency of false detection results and improve the accuracy of the finally obtained image detection results.

In a possible implementation manner, after determining the image confidence of each image block to be detected, the image detection result is determined according to the defect detection results whose image confidence is greater than a first threshold. In some embodiments, it may be determined that the defect detection result whose image confidence is greater than the first threshold is the target detection result, and the defect category with the highest confidence in the target detection result is determined as the target category. According to the position attribute and Object classes determine image detection results. Wherein, there may be one or more target categories, that is, after the defect categories are sorted in descending order of confidence, the top N defect categories are obtained as target categories. That is to say, the target category is screened according to the confidence of each defect category in the target detection result, and finally an image detection result including multiple position attributes and target categories corresponding to each position attribute is obtained. In some embodiments, the image confidence indicates the possibility of defects in the image to be detected, and the greater the image confidence, the higher the possibility of defects. When the image confidence is not greater than the first threshold, it can be directly determined that the image detection result is no defect.

In the embodiment of the present disclosure, when performing defect detection, firstly, the interference region in the image is eliminated by extracting the region to be detected, and the region to be detected is detected, thereby improving the accuracy of defect detection. In the second aspect, by dividing the area to be detected into a plurality of image blocks to be detected, and then obtaining the position information of the image blocks, the location of the defect is realized; in the third aspect, by Defect detection is performed on the detected image blocks, which reduces the missed detection rate and false detection rate; in the fourth aspect, the efficiency of the detection process is improved by performing defect detection on multiple image blocks to be detected in parallel.

The embodiment of the present disclosure also provides a defect detection method, which is applied to the defect detection of the battery welding position, as shown in FIG. 6B , the method includes the following steps:

Step 201: Obtain the original image of the battery sample (that is, the image to be detected);

Step 202: Perform image preprocessing on the original image of the battery sample, wherein, as shown in Figure 6C, the input of step 202 is the original image of the battery sample 301, and the output is the image block to be detected (image block to be detected) 302 of the weld seam and the corresponding The location coordinates of (that is, the location attribute).

In some embodiments, the implementation of step 202 includes:

First: detect the edge of the weld seam on the original image of the battery sample: since the region of interest (Region Of Interest, ROI) (that is, the area to be detected) is the battery weld seam, and the battery seam is the edge area of the battery, therefore, it can pass the edge The extracted method locates the ROI. In some embodiments, the implementation of locating the ROI by means of edge extraction may include:

1. Transform the image size of the original image of the battery sample, change the height of the original image of the battery sample to 1/5 of the original, and keep the width unchanged (due to the sampling of the original image of the battery sample, in order to speed up the sampling, frame skipping sampling is performed, at the height There is some loss on the image, in order to restore the original image, the height of the original image of the battery sample is changed to 1/5 of the original).

2. Binarize the original image of the battery sample after size transformation, and set the threshold parameter to 50.

3. Extract the maximum contour of the weld seam in the original image of the battery sample after binarization processing, and obtain the contour line 303 as shown in FIG. The area to be detected) to obtain the ROI.

Secondly: cut the weld seam along the edge mask map on the original brightness map with equal steps (i.e. the sliding step length) in the form of a sliding window (i.e. the sliding window), to obtain the image block to be detected of the welding seam, And retain the position coordinates (that is, the position attribute) of each image block to be detected, and generate a set of image blocks to be detected and a list of corresponding position coordinates. Among them, the size of the image block to be detected is selected as 1500x1500, and the step size is 700.

Step 203: Use the network model to classify the image blocks to be detected. The implementation of step 203 may include the following steps 2031 and 2032:

Step 2031: labeling the extracted image block set to be detected with a category label, for example, labeling can be done manually;

Step 2032: As shown in FIG. 6D , input the image block set 401 to be detected as a training set into the ResNet18 network 402 for training.

Among them, the key parameters used in training the ResNet18 network model are as follows:

Optimizer: use Stochastic Gradient Descent (SGD), where learning rate lr=0.001, momentum=0.9, weight decay weight_decay=0.0005;

Scheduler (Scheduler): Use equal intervals to adjust the learning rate StepLR, where the number of learning rate drop intervals step_size=16, learning rate adjustment multiple gamma=0.9;

Loss and weight: Cross-entropy loss is used, where the weight of the classification type pinhole and starting point is 1.1, and the weight of other classification types is 1; here, the weight of each type can be set according to the degree of detection and importance.

Input size: 250x250.

Step 204: Result output (that is, the defect detection result). Wherein, the input of step 204 is the set of image blocks to be detected extracted in step 202 and the category labels manually marked in step 203, and the output is the category and confidence level of each image block to be detected. As shown in FIG. 6D , the detection frame 403 is an output result, and the output result includes the category, confidence level and location information of the image block to be detected, wherein the location information can be obtained through step 202 .

Step 205: Outputting the category and position of the image block to be detected for defects (ie the image detection result). Wherein, the input of step 205 is the category, position and confidence level of each image block to be detected, and the output is the category and position of the image block to be detected containing defects screened by the threshold. The implementation of step 205 includes the following steps 2051 and 2052:

Step 2051: According to the confidence of all the image blocks to be detected, filter out the image blocks to be detected that contain defects (the starting point is non-defective, which is divided into one category separately), wherein:

1) The starting point (that is, the target defect category) and the non-defective category are not output (a total of seven categories, five types of defects after removing the starting point and the non-defective category), only the image blocks to be detected containing defects need to be output.

2) Confidence degree of defect: the confidence degree of the defect in the image block to be detected (that is, the image confidence degree), which is the normalized function of the image block to be detected corresponding to each category output by the model. For example: after softmax, five types of defects The sum of the probabilities, the confidence threshold range of the defect-containing image block to be detected is ≥0.6-0.7 (ie, the first threshold), that is, when the calculated sum of the probabilities of the five types of defects is greater than or equal to the threshold range, It is considered that the image block to be detected contains defects.

3) Confidence of the starting point: the confidence that the image block to be detected is the starting point, which is the confidence of the starting point category after the scores of the corresponding categories of the image block to be detected output by the model are normalized by a function such as softmax. In some embodiments, when the confidence of the starting point category is less than 0.1, the confidence that the image block to be detected is the starting point is less than 0.1 (that is, the second threshold), and the image block to be detected is considered not to be Starting point: when the confidence degree of the starting point category is greater than or equal to 0.1, the image block to be detected is considered as the starting point, and the confidence degree of determining that the image block to be detected contains defects is 0. Since the starting point position is often misidentified as a defect, the defect detection result is ignored when the defect category is detected as the starting point, thereby reducing the probability of misjudgment.

Step 2052: Output the category and position coordinates of the image block containing defects to be detected.

It can be understood that the above-mentioned methods mentioned in the embodiments of the present disclosure can be combined with each other to form a combined embodiment without violating the principle and logic. Those skilled in the art can understand that, in the above method in the specific implementation manner, the execution sequence of each step should be determined by its function and possible internal logic.

It should be noted that the above descriptions of the various embodiments tend to emphasize the differences between the various embodiments, and the same or similar points can refer to each other.

In addition, the embodiments of the present disclosure also provide a defect detection device, electronic equipment, computer-readable storage media, and computer program products, all of which can be used to implement any defect detection method provided by the embodiments of the present disclosure, corresponding technical solutions and The description can be found in the corresponding notes in the Methods section.

FIG. 7 shows a schematic diagram of a defect detection device provided by an embodiment of the present disclosure. As shown in FIG. 7 , the device includes an area extraction module 70 , an image division module 71 , a defect detection module 72 and a result determination module 73 .

The area extraction module 70 is configured to extract the area to be detected in the image to be detected;

The image division module 71 is configured to divide the region to be detected into a plurality of image blocks to be detected;

The defect detection module 72 is configured to perform defect detection on each of the image blocks to be detected, and obtain a defect detection result corresponding to each of the defect detection image blocks, and the defect detection result is used to represent the corresponding image block to be detected whether there are defects in

The result determination module 73 is configured to determine an image detection result according to each of the defect detection results.

In a possible implementation manner, the region extraction module 70 includes:

The edge extraction sub-module is configured to perform edge extraction on the image to be detected to obtain a region to be detected.

In a possible implementation manner, the edge extraction submodule includes:

The region extraction unit is configured to obtain the region to be detected by performing binarization processing and maximum contour extraction on the image to be detected.

In a possible implementation, the image division module 71 includes:

The sliding division sub-module is configured to determine a plurality of image blocks to be detected by sliding on the region to be detected through a preset sliding window, and the width of the sliding window is the same as the width of the region to be detected.

In a possible implementation manner, the defect detection result includes a position attribute and a defect category, and the position attribute is used to characterize the position of the image block to be detected corresponding to the defect detection result in the image to be detected.

In a possible implementation manner, the defect detection result further includes the confidence level of each defect category, and the result determination module 73 includes:

a confidence degree determination submodule configured to determine an image confidence degree according to the confidence degree of each of the defect categories, the image confidence degree being configured to characterize the possibility of a defect in the image to be detected;

The detection result determining sub-module is configured to determine the image detection result according to the defect detection result whose image confidence is greater than a first threshold.

In a possible implementation manner, the confidence determination submodule includes:

The first confidence degree determination unit is configured to determine the confidence degree of the target defect category as the target confidence degree;

The second confidence level determination unit is configured to determine the image confidence level by at least one of the following: in response to the target confidence level being less than a second threshold, according to each of the defect categories other than the target defect category The confidence level determines the image confidence level; in response to the target confidence level being not less than a second threshold, it is determined that the image confidence level is 0.

In a possible implementation manner, the detection result determining submodule includes:

The first result determining unit is configured to determine the defect detection result whose image confidence is greater than the first threshold as the target detection result;

A category screening unit configured to determine the defect category with the highest confidence in the target detection result as the target category;

The second result determining unit is configured to determine the image detection result according to the position attribute and the target category in each of the target detection results.

In a possible implementation manner, the defect detection device is used to detect a battery, the image to be inspected includes a battery image, and the area to be inspected includes a top cover weld area in the battery image.

In some embodiments, the functions or modules included in the apparatus provided by the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments, and the implementation manner may refer to the descriptions of the above method embodiments.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. Computer readable storage media may be volatile or nonvolatile computer readable storage media.

An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

Electronic devices may be provided as terminals, servers, or other forms of devices.

FIG. 8 shows a schematic diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

8, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and a communication component 816 .

The processing component 802 generally controls the overall operations of the electronic device 800, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802 .

The memory 804 is configured to store various types of data to support operations at the electronic device 800 . Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be realized by any type of volatile or non-volatile storage device or their combination, such as Static Random-Access Memory (Static Random-Access Memory, SRAM), Electrically Erasable Programmable Read-Only Memory (Electrically -Erasable Programmable Read-Only Memory, EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-only Memory (PROM), Read-Only Memory (Read- only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The power supply component 806 provides power to various components of the electronic device 800 . Power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 800 .

The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (Liquid Crystal Display, LCD) and a touch panel (Touch Panel, TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes at least one of the following: a front camera; a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, at least one of the front camera and the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 810 is configured to be at least one of the following: outputting, inputting an audio signal. For example, the audio component 810 includes a microphone (Microphone, MIC), and when the electronic device 800 is in an operation mode, such as a calling mode, a recording mode and a voice recognition mode, the microphone is configured to receive an external audio signal. Received audio signals may be stored in memory 804 or transmitted by communication component 816 . In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of electronic device 800 . For example, the sensor component 814 can detect the open/closed state of the electronic device 800, the relative positioning of components, such as the display and the keypad of the electronic device 800, the sensor component 814 can also detect the electronic device 800 or a Changes in position of components, presence or absence of user contact with electronic device 800 , electronic device 800 orientation or acceleration/deceleration and temperature changes in electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 814 may also include an optical sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge-Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (Infrared Data Association, IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BlueTooth, BT) technology and other technology to achieve.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (Application Specific Integrated Circuit, ASIC), Digital Signal Processor (Digital Signal Processor, DSP), Digital Signal Processing Device (Digital Signal Process Device , DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), controller, microcontroller, microprocessor or other electronic components for implementation the above method.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to implement the above method.

FIG. 9 shows a schematic diagram of an electronic device 1900 provided by an embodiment of the present disclosure. For example, electronic device 1900 may be a server. Referring to FIG. 9 , electronic device 1900 includes processing component 1922 , which includes one or more processors, and memory resources represented by memory 1932 for storing instructions executable by processing component 1922 , such as application programs. The application program stored in the memory 1932 may include one or more modules, wherein each module corresponds to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input-output (I/O) Interface 1958. The electronic device 1900 can operate based on the operating system stored in the memory 1932, such as the Microsoft server operating system (Windows Server ^™ ), the operating system based on the graphical user interface (Mac OS X ^™ ) introduced by Apple Inc., multi-user multi-process computer operation system (Unix ^™ ), a free and open-source Unix-like operating system (Linux ^™ ), an open-source Unix-like operating system (FreeBSD ^™ ), or the like.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to implement the above method.

An embodiment of the present disclosure may be a computer program product. A computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of embodiments of the present disclosure.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disk, hard disk, Random Access Memory (RAM), ROM, EPROM, SRAM, portable compact disk read-only memory ( Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD), memory stick, floppy disk, mechanically encoded devices such as punched cards or raised structures in grooves on which instructions are stored , and any suitable combination of the above. Computer-readable storage media as used herein is not to be interpreted as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over at least one of a network, such as the Internet, a local area network, a wide area network, and a wireless network. The network may include at least one of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing operations of embodiments of the present disclosure may be assembly instructions, instruction set architecture (Instruction Set Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in the form of a or any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as “C” or similar programming languages language. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer such as use an Internet service provider to connect via the Internet). In some embodiments, by using the state information of computer-readable program instructions to personalize and customize electronic circuits, such as programmable logic circuits, FPGAs, or programmable logic arrays (Programmable Logic Array, PLA), the electronic circuits can execute computer-readable Read program instructions, thereby implementing various aspects of the present disclosure.

Aspects of embodiments of the present disclosure are described herein with reference to at least one of flowchart illustrations and block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of at least one of the flowchart and block diagrams, and combinations of blocks in at least one of the flowchart and block diagrams, can be implemented by computer readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing a device for realizing the functions/actions specified in one or more blocks of at least one of the flowchart and the block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause at least one of computers, programmable data processing devices and other devices to work in a specific way, so that the computer-readable medium storing the instructions An article of manufacture is then included, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks of at least one of the flowcharts and block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in one or more blocks of at least one of the flowcharts and block diagrams.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also to be noted that each block of at least one of the block diagrams and flowcharts, and combinations of blocks of at least one of the block diagrams and flowcharts, may be implemented with dedicated hardware-based devices that perform specified functions or actions. system, or it may be implemented by a combination of special purpose hardware and computer instructions.

The computer program product can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

If the disclosed technical solution involves personal information, the products applying the disclosed technical solution have clearly notified the personal information processing rules and obtained the individual's independent consent before processing personal information. If the disclosed technical solution involves sensitive personal information, the products applying the disclosed technical solution have obtained individual consent before processing sensitive personal information, and at the same time meet the requirement of "express consent". For example, at a personal information collection device such as a camera, a clear and prominent sign is set up to inform that it has entered the scope of personal information collection, and personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed to agree to the collection of his personal information; or On the personal information processing device, when the personal information processing rules are informed with obvious signs/information, personal authorization is obtained through pop-up information or by asking individuals to upload their personal information; among them, the personal information processing rules may include Information processor, purpose of personal information processing, processing method, type of personal information processed and other information.

Having described various embodiments of the present disclosure above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Claims (13)

1. A defect detection method, the method comprising:

   Extract the region to be detected in the image to be detected;

   dividing the region to be detected into a plurality of image blocks to be detected;

   performing defect detection on each of the image blocks to be detected, to obtain a defect detection result corresponding to each of the image blocks to be detected, and the defect detection results are used to indicate whether there is a defect in the corresponding image block to be detected;

   An image detection result is determined based on each of the defect detection results.
2. The method according to claim 1, wherein said extracting the region to be detected in the image to be detected comprises:

   Edge extraction is performed on the image to be detected to obtain a region to be detected.
3. The method according to claim 2, wherein said performing edge extraction on the image to be detected to obtain the region to be detected comprises:

   The region to be detected is obtained by performing binarization processing and maximum contour extraction on the image to be detected.
4. The method according to any one of claims 1 to 3, wherein said dividing the region to be detected into a plurality of image blocks to be detected comprises:

   A plurality of image blocks to be detected are determined by sliding on the region to be detected through a preset sliding window, and the width of the sliding window is the same as the width of the region to be detected.
5. The method according to any one of claims 1 to 4, wherein the defect detection result includes a position attribute and a defect category, and the position attribute is used to characterize the location of the image block to be detected corresponding to the defect detection result. Describe the position in the image to be detected.
6. The method according to claim 5, wherein the defect detection result further includes a confidence level of each of the defect categories, and determining the image detection result according to each of the defect detection results comprises:

   determining an image confidence level according to the confidence level of each defect category, the image confidence level being used to characterize the possibility of a defect in the image to be detected;

   The image detection result is determined according to the defect detection result whose image confidence is greater than a first threshold.
7. The method according to claim 6, wherein said determining the image confidence level according to the confidence level of each said defect category comprises:

   Determining the confidence level of the target defect category as the target confidence level;

   The method for determining the image confidence includes at least one of the following: in response to the target confidence being less than a second threshold, determining the image confidence according to the confidence of each of the defect categories other than the target defect category; in response to The target confidence level is not less than the second threshold, and the image confidence level is determined to be 0.
8. The method according to claim 6 or 7, wherein said determining the image detection result according to the defect detection result whose image confidence is greater than a first threshold comprises:

   Determining that a defect detection result whose image confidence is greater than a first threshold is a target detection result;

   Determining the defect category with the highest confidence in the target detection result as the target category;

   The image detection result is determined according to the position attribute and the target category in each of the target detection results.
9. The method according to any one of claims 1 to 8, wherein the defect detection method is used to detect a battery, the image to be inspected includes a battery image, and the region to be inspected includes a top cover in the battery image Weld area.
10. A defect detection device, said device comprising:

    An area extraction module configured to extract an area to be detected in the image to be detected;

    An image division module configured to divide the region to be detected into a plurality of image blocks to be detected;

    The defect detection module is configured to perform defect detection on each of the image blocks to be detected, and obtain a defect detection result corresponding to each of the defect detection image blocks, and the defect detection result is used to represent the defects in the corresponding image block to be detected whether there are defects;

    A result determination module configured to determine an image detection result according to each of the defect detection results.
11. An electronic device comprising:

    processor;

    memory for storing processor-executable instructions;

    Wherein, the processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1-9.
12. A computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.
13. A computer program product, comprising a computer-readable storage medium storing program code, when the instructions included in the program code are executed by the processor of the computer device, the steps in the method described in any one of claims 1 to 9 are realized .

Images