CN117496299A - A method, device, terminal equipment and medium for augmenting defect image data - Google Patents

Abstract

The application is applicable to the technical field of image processing, and provides a method, a device, terminal equipment and a medium for amplifying defect image data, wherein original defect image training data are obtained; training a pre-constructed denoising diffusion probability model by using original defect image training data to obtain a trained denoising diffusion probability model, and inputting the original defect image training data into the trained denoising diffusion probability model to generate surface defect image data; evaluating high-quality image data from the surface defect image data, and constructing new surface defect image data according to the high-quality image data and the original defect image training data; progressively training a denoising diffusion probability model by utilizing new defect image numbers meeting preset quality requirements, and amplifying acquired target defect image data by utilizing the trained denoising diffusion probability model; the method and the device can improve the generation quality of the defect image.

Description

A method, device, terminal equipment and medium for augmenting defect image data

Technical field

The present application belongs to the field of image processing technology, and in particular relates to a method, device, terminal equipment and medium for augmenting defect image data.

Background technique

In the modern industrial production process, affected by factors such as raw materials, heat treatment, and transportation, defects such as scratches, plaques, and mill scale will appear on the surface of steel, steel plates, and other industrial products. In order to ensure product quality and save production resources, product surface inspection Defect detection is a key step to ensure product quality. Among them, surface defect detection technology is an important technical means for realizing unmanned and intelligent detection in modern industry. However, in the actual production process, the lack of defect data is the biggest obstacle restricting the development of surface defect detection technology. Therefore, achieving defect image data augmentation with limited defect data samples is an important direction for the development of intelligent surface defect detection technology.

Data augmentation technology is an important way to expand data samples. Traditional data augmentation algorithms convert the original defect data into geometric and color differences based on prior knowledge. This method is simple and effective, but has poor scalability in actual industrial applications. With the development of deep learning technology, many researchers learn the deep features of defect data from limited data samples, and use deep models to approximate the true distribution of defect data features, and then generate models that satisfy the defect feature distribution by sampling within the real distribution. Defect data.

At present, among the data augmentation technologies based on deep learning, the most widely used one is the generative adversarial network. However, the main flaw of this network is that when faced with the problem of small sample data sets with large distribution differences, the generated adversarial network generates There are extreme cases of sample quality, such as situations where the defect characteristics are not obvious, and the data structural parameters are greatly different from the original data set. In addition, when faced with the problem of small sample data sets with large distribution differences, generative adversarial networks are prone to instability in the training process and model collapse.

Therefore, a new defect image data augmentation method is urgently needed to solve the problem of low quality of new defect image data generated by existing defect image data augmentation methods.

Contents of the invention

The present application provides a defect image data augmentation method, device, terminal equipment and medium, which can solve the problem of low quality of new defect image data generated by the existing defect image data augmentation method.

In the first aspect, this application provides a method for augmenting defect image data, including:

Step 1: Obtain original defect image training data; the original defect image training data includes multiple original defect images;

Step 2: Use the original defect image training data to train the pre-built denoising diffusion probability model to obtain the trained denoising diffusion probability model, and input the original defect image training data into the trained denoising diffusion probability model to generate the surface Defect image data; surface defect image data includes multiple surface defect images;

Step 3: Evaluate high-quality image data from the surface defect image data, construct new surface defect image data based on the high-quality image data and original defect image training data, and determine whether the new surface defect image data meets the preset quality requirements; High Quality image data includes multiple high-quality images;

Step 4. If the new surface defect image data meets the preset quality requirements, use the denoising diffusion probability model trained in step 2 as the final denoising diffusion probability model, and use the final denoising diffusion probability model to analyze the collected target defect images. The data is augmented; otherwise, the intermediate original defect image training data is constructed based on the new surface defect image data and the original defect image training data, and the intermediate original defect image training data is used as the original defect image training data in step 2, and returns to the execution step 2.

Optionally, evaluate high-quality image data from surface defect image data, including:

Construct a deep feature evaluator based on LPIPS, and calculate the LPIPS score of each surface defect image based on the deep feature evaluator;

Calculate the SSIM score for each surface defect image;

Surface defect images with LPIPS scores less than the preset score threshold and SSIM scores greater than the preset index threshold are evaluated as high-quality images to obtain high-quality image data.

Optionally, build a deep feature evaluator based on LPIPS and calculate the LPIPS score of each surface defect image based on the deep feature evaluator, including:

Step i, determine the low-quality image from the surface defect image data to obtain low-quality image data P ₁ ^low ; the low-quality image represents a surface defect image whose similarity to the original defect image is lower than the preset similarity threshold;

Step ii, use the pre-built initial AlexNet network to extract defect features of the original defect image training data and defect features of the low-quality image data respectively;

Step iii, if the feature similarity between the defect features of the original defect image training data and the defect features of the low-quality image data is less than the preset feature similarity threshold, fine-tune the parameters of the initial AlexNet network to obtain the fine-tuned AlexNet network, and Use the fine-tuned AlexNet network as the initial AlexNet network in step ii, and return to step ii; otherwise, perform step iv;

Step iv, use the fine-tuned AlexNet network to extract the image features of each surface defect image, and calculate the LPIPS score of the surface defect image based on the image features.

Optionally, in step 3, determine whether the new surface defect image data meets the preset quality requirements, including:

Step a, if the LPIPS score difference between the new surface defect image data and the original defect image training data is less than the preset LPIPS score difference threshold, and the SSIM score difference of the new defect image data is greater than the preset SSIM score difference threshold, then Determine that the new defect image data meets the preset quality requirements; otherwise, perform step b;

Step b, if the difference between the LPIPS score of the new defect image data constructed for the v+1th time and the LPIPS score of the new defective image data constructed for the vth time is less than the preset threshold, then determine the LPIPS score of the new defect image data constructed for the v+1th time. The new defect image data meets the preset quality requirements; otherwise, it is determined that the new defect image data does not meet the preset quality requirements.

In the second aspect, this application provides an augmentation device for defect image data, including:

The image acquisition module is used to obtain original defect image training data; the original defect image training data includes multiple original defect images;

The surface defect image generation module is used to train the pre-built denoising diffusion probability model using the original defect image training data to obtain the trained denoising diffusion probability model, and input the original defect image training data into the trained denoising diffusion probability model. Probabilistic model generates surface defect image data; surface defect image data includes multiple surface defect images;

The high-quality image evaluation module is used to evaluate high-quality image data from surface defect image data, construct new surface defect image data based on high-quality image data and original defect image training data, and determine whether the new surface defect image data meets the predicted requirements. Set quality requirements; high-quality image data includes multiple high-quality images;

The model augmentation module is used to use the denoising diffusion probability model trained in the surface defect image generation module as the final denoising diffusion probability model if the new surface defect image data meets the preset quality requirements, and use the final denoising diffusion probability model , augment the collected target defect image data; otherwise, construct intermediate original defect image training data based on the new surface defect image data and original defect image training data, and use the intermediate original defect image training data as the surface defect image generation module The original defect image training data is returned to execute the surface defect image generation module.

In a third aspect, the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned augmented method is implemented.

In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned augmentation method is implemented.

The above solution of this application has the following beneficial effects:

The augmentation method of defect image data provided by this application generates surface defect image data by inputting original defect image training data into the trained denoising diffusion probability model, and then evaluates high-quality image data from the surface defect image data, and Based on high-quality image data and original defect image training data, new surface defect image data is constructed, and the new surface defect image data is used to train the denoising diffusion probability model, which enriches the diversity of the denoising diffusion probability model training data and improves the denoising efficiency. The quality of the diffusion probability model training data improves the accuracy of the denoising diffusion probability model, which is beneficial to improving the quality of the new defect image data generated.

Other beneficial effects of the present application will be described in detail in the subsequent specific embodiments section.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the purpose of the present application. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a flow chart of a method for augmenting defect image data provided by an embodiment of the present application;

Figure 2 is a schematic structural diagram of a denoising diffusion model in an embodiment of the present application;

Figure 3 is a schematic flow chart of the calculation process of Lpips score in an embodiment of the present application;

Figure 4 is a schematic flowchart of the calculation process of SSIM score in an embodiment of the present application;

Figure 5 is a schematic structural diagram of an augmentation device for defect image data provided by an embodiment of the present application;

Figure 6 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.

Detailed ways

In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures and technologies are provided to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and/or components but does not exclude one or more other The presence or addition of features, integers, steps, operations, elements, components and/or collections thereof.

It will also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. ". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, to mean "once determined" or "in response to a determination" or "once the [described condition or event] is detected ]" or "in response to detection of [the described condition or event]".

In addition, in the description of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

Reference in this specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments", etc. appearing in different places in this specification are not necessarily References are made to the same embodiment, but rather to "one or more but not all embodiments" unless specifically stated otherwise. The terms "including," "includes," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

In order to solve the problem of low quality of new defect image data generated by the existing defect image data augmentation method, this application provides a defect image data augmentation method, device, terminal equipment and medium, wherein the method uses the original defect image to The training data is input into the trained denoising diffusion probability model to generate surface defect image data, and then high-quality image data is evaluated from the surface defect image data, and a new surface defect image is constructed based on the high-quality image data and original defect image training data. data, and use the new surface defect image data to train the denoising diffusion probability model, which enriches the diversity of the denoising diffusion probability model training data, improves the quality of the denoising diffusion probability model training data, and improves the accuracy of the denoising diffusion probability model. properties, thereby benefiting to improve the quality of the new defect image data generated.

As shown in Figure 1, the augmentation method of defect image data provided by this application includes the following steps:

Step 1: Obtain original defect image training data.

The above original defect image training data includes a plurality of original defect images.

Illustratively, in an embodiment of the present application, in order to generate accurate and effective industrial surface defect image data, the original defect image can be obtained through the open source data set NEU-CLS (an open source data set) based on contrast, brightness and sharpness. Data, the original defect image data not only includes the original defect image, but also includes the category of the defect to which each image belongs (such as surface scratch defects, surface patch defects and surface scale defects), size parameters, grayscale mean value and grayscale standard Difference.

Step 2: Use the original defect image training data to train the pre-built denoising diffusion probability model to obtain the trained denoising diffusion probability model, and input the original defect image training data into the trained denoising diffusion probability model to generate the surface Defect image data.

The above-mentioned surface defect image data includes a plurality of reconstructed defect images.

It should be understood that the Probabilistic Denoising Diffusion Model is a probability-based image denoising method that gradually reduces the noise level in the image by iteratively applying a series of random transformations to achieve the effect of improving image quality. In the embodiment of the present application, the model structure of the denoising diffusion probability model is shown in Figure 2.

The following is an exemplary explanation of the process of inputting the original defect image training data into the trained denoising diffusion probability model to generate surface defect image data in step 2, as follows:

Randomly generate a normally distributed noise matrix with the same size parameters as the original defect image and a mean of 0 and a variance of 1. Among them, n represents the serial number of the noise matrix, n=1, 2,...,N, and N represents the total number of noise matrices.

For each noise matrix respectively, calculate the formula

β _t =1-α _t , n = 1, ..., N, t = 1, ..., T

Obtain the nth surface defect image at the t-1th time step Among them,/> represents the prediction noise of the nth surface defect image at the tth time step, β _t represents the hyperparameter, and α _t represents the hyperparameter.

Step 3: Evaluate high-quality image data from the surface defect image data, construct new surface defect image data based on the high-quality image data and original defect image training data, and determine whether the new surface defect image data meets the preset quality requirements.

LPIPS (Learned Perceptual Image Patch Similarity) score is an indicator used to evaluate image quality and visual perception differences. Its value usually ranges from 0 to 1. The smaller the LPIPS score, the more similar the images are. The larger the LPIPS score, the greater the difference between the images.

SSIM (Structural Similarity Index) indicator is an indicator used to evaluate image quality. It calculates the similarity between images based on aspects such as image brightness, contrast, and structure, and takes into account the perceptual characteristics of the human visual system. The higher the SSIM index, the higher the similarity between the two images.

It is worth mentioning that this application calculates the LPIPS score and SSIM index of the reconstructed defect image, and selects high-quality images with high LPIPS score and good SSIM index to participate in the generation of denoising diffusion probability model, which reduces the problems caused by low-quality images. Negative impact, it helps to improve the accuracy of the denoising diffusion probability model, thereby improving the quality of new defect image data generated by the denoising diffusion probability model.

The above-mentioned process of constructing new defect image data based on high-quality images and original defect image training data is: merging all high-quality images and original defect image training data to form a new image set, which includes all original defects. images and high quality images. This can enrich the diversity of generated samples and help improve the quality of new defect image data.

Step 4. If the new surface defect image data meets the preset quality requirements, use the denoising diffusion probability model trained in step 2 as the final denoising diffusion probability model, and use the final denoising diffusion probability model to analyze the collected target defect images. The data is augmented; otherwise, the intermediate original defect image training data is constructed based on the new surface defect image data and the original defect image training data, and the intermediate original defect image training data is used as the original defect image training data in step 2, and returns to the execution step 2.

The following is an exemplary explanation of the process of evaluating high-quality image data from surface defect image data in step 3.

Specifically including steps A ~ C:

Step A, build a deep feature evaluator based on LPIPS, and calculate the LPIPS score of each surface defect image based on the deep feature evaluator.

Specifically, step i determines the low-quality image from the surface defect image data to obtain low-quality image data P ₁ ^low ; the low-quality image represents surface defects whose similarity to the original defect image is lower than the preset similarity threshold image;

Step ii, use the pre-built initial AlexNet network to extract defect features of the original defect image training data and defect features of the low-quality image data respectively;

Step iii, if the feature similarity between the defect features of the original defect image training data and the defect features of the low-quality image data is less than the preset feature similarity threshold, fine-tune the parameters of the initial AlexNet network to obtain the fine-tuned AlexNet network, and Use the fine-tuned AlexNet network as the initial AlexNet network in step ii, and return to step ii; otherwise, perform step iv;

Step iv, use the fine-tuned AlexNet network to extract the image features of each surface defect image, and calculate the LPIPS score of the surface defect image based on the image features.

Specifically, through the calculation formula

Get the residual of the qth layer feature vector Among them/> Represents the nth reconstructed defective image/> The qth layer feature vector of ,/> Represents the k-th original defect image/> The qth layer feature vector of , |·| means taking the absolute value, q=1, 2,...,5.

By calculation formula

Get the nth reconstructed defect image With the kth original defect image/> LPIPS score.

Among them, w represents the feature dimensionality reducer, which is used to convert high-dimensional features into low-dimensional features.

By calculation formula

Get the nth reconstructed defect image LPIPS score/>

The process is specifically shown in Figure 3.

Step B, calculate the SSIM score of each surface defect image.

Specifically, through the calculation formula

Get the nth reconstructed defect image Grayscale mean/> Among them, H and W represent the nth reconstructed defect image pixel size,/> Represents the nth reconstructed defective image/> Pixels (i, j), i=1, 2,..., H, j=1, 2,..., W;

By calculation formula

Get the nth reconstructed defect image Grayscale standard deviation/>

By calculation formula

Get the nth reconstructed defect image With the kth original defect image/> The covariance matrix />

By calculation formula

Get the nth reconstructed defect image With the kth original defect image/> brightness similarity; where, C ₁ represents a non-zero constant, used to ensure that the denominator is not 0;

By calculation formula

Get the nth reconstructed defect image With the kth original defect image/> Contrast similarity/> Among them, C2 represents a non-zero constant, used to ensure that the denominator is not 0;

By calculation formula

Get the nth reconstructed defect image With the kth original defect image/> Structural similarity/> Among them, C ₃ represents a non-zero constant, used to ensure that the denominator is not 0;

By calculation formula

Get the nth reconstructed defect image With the kth original defect image/> SSIM similarity/> Among them, α, β, and γ are all equilibrium parameters;

By calculation formula

Get the nth reconstructed defect image SSIM indicator/>

The process is specifically shown in Figure 4.

Step C: Evaluate surface defect images whose LPIPS score is less than the preset score threshold and whose SSIM score is greater than the preset index threshold as high-quality images to obtain high-quality image data.

The following is an exemplary description of the process of determining whether the new surface defect image data meets the preset quality requirements in step 3, as shown in steps a to b.

Step a, if the LPIPS score difference between the new surface defect image data and the original defect image training data is less than the preset LPIPS score difference threshold, and the SSIM score difference of the new defect image data is greater than the preset SSIM score difference threshold, then Determine that the new defect image data meets the preset quality requirements; otherwise, perform step b;

Step b, if the difference between the LPIPS score of the new defect image data constructed for the v+1th time and the LPIPS score of the new defective image data constructed for the vth time is less than the preset threshold, then determine the LPIPS score of the new defect image data constructed for the v+1th time. The new defect image data meets the preset quality requirements; otherwise, it is determined that the new defect image data does not meet the preset quality requirements.

The following is an exemplary explanation of the process of augmenting the collected target defect image data using the final denoising diffusion probability model, as follows:

The collected target defect image data is input into the final denoising diffusion probability model, and denoising is performed through the final denoising diffusion probability model to generate new defect image data corresponding to the target defect image data. In an embodiment of the present application, in the open source data The generation effects of the defect image data augmentation method provided by this application and other commonly used defect image data augmentation methods are compared on NEU-CLS, as shown in the following table:

The values in the table represent the Fréchet-Inception Distance (FID) of various types of defect images generated. It is an indicator used to evaluate the quality of images generated by generative adversarial networks and characterizes the generated images and the original images. The smaller the value, the more similar it is. It can be seen that as the number of executions of the defect image data augmentation method provided by this application increases, the FID values of various types of defect images gradually decrease, indicating that the generated new defect images are closer to the original defect images. This is exactly what defect image augmentation means. The purpose pursued is that the smaller the generated new defect image and the original defect image are, the higher the quality of the generated new defect image is, which is more conducive to improving the accuracy of product surface defect detection. Therefore, the augmentation method of defect image data provided by this application can effectively alleviate the impact of the lack of defect data on surface defect detection technology, and the quality of the new defect image generated by the augmentation method of defect image data provided by this application is better than Other existing technologies.

The following is an exemplary explanation of the process of fine-tuning the pre-constructed denoising diffusion probability model using the original defect image training data in Step 2 of this application. Specifically, it includes steps i~iv:

Step i, by calculating the formula

Get the kth original defect image Diffusion image at time step t/>

Among them, k = 1, 2, ..., K, K represents the total number of original defect images, α _t represents the diffusion coefficient at the t time step, and ε represents a random distribution subject to a normal distribution with a mean of 0 and a variance of 1. Noise matrix, t = 1, 2, ..., T, T represents the total time steps of the preset diffusion process.

Step ii, use the noise prediction network ε _θ (t) corresponding to the t-th time step to Make a prediction and get/> Prediction noise at time step t/>

Step iii, by calculating the formula Get the training loss value L.

Among them, ||·|| represents L2 normal form.

Step iv, if the training loss value is less than the preset loss threshold, use the weight parameters of the noise prediction network corresponding to each time step as the final weight parameters to obtain the trained denoising diffusion probability model; otherwise, proceed based on the training loss value Stochastic gradient descent operation, backpropagation updates the weight parameters of the noise predictor network corresponding to each time step, and returns to execution step i.

It can be seen from the above steps that the augmentation method of defect image data provided by this application generates surface defect image data by inputting the original defect image training data into the trained denoising diffusion probability model, and then evaluates the surface defect image data from the surface defect image data. High-quality image data, and based on the high-quality image data and original defect image training data, new surface defect image data is constructed, and the new surface defect image data is used to train the denoising diffusion probability model, which enriches the diversity of denoising diffusion probability model training data. It improves the quality of the denoising diffusion probability model training data and improves the accuracy of the denoising diffusion probability model, which is beneficial to improving the quality of the new defect image data generated.

The following is an exemplary description of the augmentation device for defect image data provided by this application.

As shown in Figure 5, the defect image data augmentation device 500 includes the following modules:

The image acquisition module 501 is used to acquire original defect image training data; the original defect image training data includes multiple original defect images;

The surface defect image generation module 502 is used to train the pre-constructed denoising diffusion probability model using the original defect image training data, obtain the trained denoising diffusion probability model, and input the original defect image training data into the trained denoising diffusion probability model. The diffusion probability model generates surface defect image data; the surface defect image data includes multiple surface defect images;

The high-quality image evaluation module 503 is used to evaluate high-quality image data from the surface defect image data, construct new surface defect image data based on the high-quality image data and original defect image training data, and determine whether the new surface defect image data meets Preset quality requirements; high-quality image data includes multiple high-quality images;

The model augmentation module 504 is used to use the denoising diffusion probability model after training in the surface defect image generation module as the final denoising diffusion probability model if the new surface defect image data meets the preset quality requirements, and use the final denoising diffusion probability model to augment the collected target defect image data; otherwise, construct intermediate original defect image training data based on the new surface defect image data and original defect image training data, and use the intermediate original defect image training data as the surface defect image generation module The original defect image training data in is returned to execute the surface defect image generation module.

It should be noted that the information interaction, execution process, etc. between the above-mentioned devices/units are based on the same concept as the method embodiments of the present application. For details of their specific functions and technical effects, please refer to the method embodiments section. No further details will be given.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional units and modules is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs. Module completion means dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be hardware-based. It can also be implemented in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present application. For the specific working processes of the units and modules in the above system, please refer to the corresponding processes in the foregoing method embodiments, and will not be described again here.

The following is an exemplary description of the terminal equipment provided by this application.

As shown in Figure 6, an embodiment of the present application provides a terminal device. As shown in Figure 6, the terminal device D10 of this embodiment includes: at least one processor D100 (only one processor is shown in Figure 6), Memory D101 and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the steps in any of the above method embodiments are implemented.

Specifically, when the processor D100 executes the computer program D102, it obtains the original defect image training data and uses the original defect image training data to train the pre-constructed denoising diffusion probability model to obtain the trained denoising diffusion probability. model, and input the original defect image training data into the trained denoising diffusion probability model to generate surface defect image data. The denoising diffusion probability model that meets the preset quality requirements is used as the final denoising diffusion probability model, and the final denoising diffusion is used Probabilistic model is used to augment the collected target defect image data. Among them, the original defect image training data is input into the trained denoising diffusion probability model to generate surface defect image data, and then the high-quality image data is evaluated from the surface defect image data and trained based on the high-quality image data and the original defect image. data, construct new surface defect image data, and use the new surface defect image data to train the denoising diffusion probability model, which enriches the diversity of the denoising diffusion probability model training data, improves the quality of the denoising diffusion probability model training data, and improves The accuracy of the denoised diffusion probability model is thereby beneficial to improve the quality of the new defect image data generated.

The processor D100 may be a central processing unit (CPU). The processor D100 may also be other general-purpose processors, digital signal processors (DSP), or application specific integrated circuits (ASICs). Integrated Circuit), off-the-shelf programmable gate array (FPGA, Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), or a secure digital (SMC) device equipped on the terminal device D10. SD (Secure Digital) card, Flash Card, etc. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store operating systems, application programs, boot loaders (Boot Loaders), data and other programs, such as program codes of the computer programs. The memory D101 can also be used to temporarily store data that has been output or will be output.

Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps in each of the above method embodiments can be implemented.

Embodiments of the present application provide a computer program product. When the computer program product is run on a terminal device, the steps in each of the above method embodiments can be implemented when the terminal device executes it.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, this application can implement all or part of the processes in the above embodiment methods by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program When executed by a processor, the steps of each of the above method embodiments may be implemented. Wherein, the computer program includes computer program code, which may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code to an augmentation device/terminal device of defective image data, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory) , random access memory (RAM, Random Access Memory), electrical carrier signals, telecommunications signals and software distribution media. For example, U disk, mobile hard disk, magnetic disk or CD, etc. In some jurisdictions, subject to legislation and patent practice, computer-readable media may not be electrical carrier signals and telecommunications signals.

In the above embodiments, each embodiment is described with its own emphasis. For parts that are not detailed or documented in a certain embodiment, please refer to the relevant descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed devices/network devices and methods can be implemented in other ways. For example, the apparatus/network equipment embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units. Or components can be combined or can be integrated into another system, or some features can be omitted, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above is the preferred embodiment of the present application. It should be pointed out that for those of ordinary skill in the art, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications can also be made. should be regarded as the scope of protection of this application.

Claims (7)

1. An augmentation method for defect image data, characterized by including: Step 1: Obtain original defect image training data; the original defect image training data includes multiple original defect images; Step 2: Use the original defect image training data to train the pre-constructed denoising diffusion probability model to obtain the trained denoising diffusion probability model, and input the original defect image training data into the trained denoising probability model. A diffusion probability model generates surface defect image data; the surface defect image data includes multiple surface defect images; Step 3: Evaluate high-quality image data from the surface defect image data, construct new surface defect image data based on the high-quality image data and the original defect image training data, and judge the new surface defect image data Whether it meets the preset quality requirements; the high-quality image data includes multiple high-quality images; Step 4: If the new surface defect image data meets the preset quality requirements, use the denoising diffusion probability model after training in step 2 as the final denoising diffusion probability model, and use the final denoising diffusion probability model model to augment the collected target defect image data; otherwise, construct intermediate original defect image training data based on the new surface defect image data and the original defect image training data, and use the intermediate original defect image training data to As the original defect image training data in step 2, return to step 2. 2. The augmentation method according to claim 1, characterized in that estimating high-quality image data from the surface defect image data includes: Constructing a deep feature evaluator based on LPIPS, and calculating a LPIPS score for each of the surface defect images based on the deep feature evaluator; Calculate the SSIM score for each of the surface defect images; Surface defect images whose LPIPS score is less than a preset score threshold and whose SSIM score is greater than a preset index threshold are evaluated as high-quality images to obtain the high-quality image data. 3. The augmentation method according to claim 2, characterized in that said constructing a deep feature evaluator based on LPIPS and calculating the LPIPS score of each of the surface defect images according to the deep feature evaluator includes: Step i, determine a low-quality image from the surface defect image data to obtain low-quality image data P ₁ ^low ; the similarity between the low-quality image and the original defect image is lower than a preset similarity threshold images of surface defects; Step ii: Use the pre-constructed initial AlexNet network to respectively extract the defect features of the original defect image training data and the defect features of the low-quality image data; Step iii, if the feature similarity between the defect features of the original defect image training data and the defect features of the low-quality image data is less than the preset feature similarity threshold, fine-tune the parameters of the initial AlexNet network to obtain fine-tuning The final AlexNet network, and use the fine-tuned AlexNet network as the initial AlexNet network in step ii, return to step ii; otherwise, perform step iv; Step iv: Use the fine-tuned AlexNet network to extract image features of each surface defect image, and calculate the LPIPS score of the surface defect image based on the image features. 4. The augmentation method according to claim 3, wherein determining whether the new surface defect image data meets preset quality requirements in step 3 includes: Step a, if the LPIPS score difference between the new surface defect image data and the original defect image training data is less than the preset LPIPS score difference threshold, and the SSIM score difference of the new defect image data is greater than the preset SSIM If the score difference threshold is reached, it is determined that the new defect image data meets the preset quality requirements; otherwise, step b is performed; Step b, if the difference between the LPIPS score of the new defect image data constructed for the v+1th time and the LPIPS score of the new defective image data constructed for the vth time is less than the preset threshold, then determine the LPIPS score of the new defect image data constructed for the v+1th time. The new defect image data meets the preset quality requirements; otherwise, it is determined that the new defect image data does not meet the preset quality requirements. 5. An augmentation device for defect image data, characterized in that it includes: An image acquisition module, used to acquire original defect image training data; the original defect image training data includes multiple original defect images; A surface defect image generation module is used to train a pre-constructed denoising diffusion probability model using the original defect image training data, obtain a trained denoising diffusion probability model, and input the original defect image training data into the The trained denoising diffusion probability model generates surface defect image data; the surface defect image data includes multiple surface defect images; A high-quality image evaluation module, used to evaluate high-quality image data from the surface defect image data, and construct new surface defect image data based on the high-quality image data and the original defect image training data, and determine the Whether the new surface defect image data meets the preset quality requirements; the high-quality image data includes multiple high-quality images; A model augmentation module, configured to use the trained denoising diffusion probability model in the surface defect image generation module as the final denoising diffusion probability model if the new surface defect image data meets the preset quality requirements, using The final denoising diffusion probability model augments the collected target defect image data; otherwise, the intermediate original defect image training data is constructed based on the new surface defect image data and the original defect image training data, and the obtained The intermediate original defect image training data is used as the original defect image training data in the surface defect image generation module, and is returned to execute the surface defect image generation module. 6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, the processor implements the claims as claimed in The augmentation method described in any one of 1 to 4. 7. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, the augmentation as claimed in any one of claims 1 to 4 is achieved. method.