CN115049627B - Steel Surface Defect Detection Method and System Based on Domain Adaptive Deep Migration Network - Google Patents

Abstract

The invention provides a steel surface defect detection method and system based on a domain-adaptive deep migration network. The method includes: acquiring a typical defect image sample on the strip steel surface, and preprocessing the sample; constructing a confrontational domain separation and self-adaptive network model; Embed the new sample features into the shared features of the source domain image samples, and calculate the task classification loss and embedding classification loss; by adding weights to multiple losses, dynamically optimize the dynamic classification loss and dynamic adaptation loss, and update the model parameters; when When the number of iterations reaches the optimum, save the model parameters and input the test set in the target field to obtain the detection accuracy of steel surface defects in the target field. The invention introduces adaptive mining sample hidden information and adds dynamic weight optimization loss algorithm on the basis of confrontational domain separation and self-adaptive deep migration network, improves the generalization ability of the network model, and finally realizes more accurate detection of steel surface defects.

Description

Steel Surface Defect Detection Method and System Based on Domain Adaptive Deep Migration Network

technical field

The invention belongs to the technical field of image detection, and in particular relates to a steel surface defect detection method and system based on a domain-adaptive deep migration network.

Background technique

Steel is widely used in automobile manufacturing, aerospace, daily necessities and other fields, and is an indispensable and important raw material in the development of the national economy. With the rapid development of society, its quality requirements are becoming more and more stringent. In the process of production and processing, due to the influence of factors such as the environment, equipment performance, and processing technology, various defects, such as holes and scratches, will occur on the surface of the steel. These observable defects can lead to changes in steel properties and greatly reduce product quality, resulting in large negative impacts and economic losses on manufacturing companies. Therefore, steel surface defect detection, as an important part of steel quality monitoring, has received extensive attention.

The primary goal of steel surface inspection is to accurately predict the type of defect. Since manual inspection was mainly used in the early stage, there were deficiencies such as high cost, low efficiency, and subjective judgment. For this reason, machine learning technology provides an efficient and objective image detection method, which improves the accuracy and efficiency of defect recognition to a certain extent. However, this kind of traditional machine learning requires professionals with rich experience and domain knowledge to extract more suitable features, and the detection performance largely depends on the selection of features. Different from traditional machine learning, deep learning does not require manual feature extraction, which can avoid the trial-and-error experiment of combining a large number of features with classifiers, and achieve in-depth and multi-angle characterization of image features. However, there are still many problems to be solved in the practical application of deep learning, such as huge amount of calculation, difficulty in obtaining labeled samples, and time-consuming and laborious training samples.

With the development of artificial intelligence technology, transfer learning has become one of the more important research contents of the current surface defect detection method based on machine vision. The goal of transfer learning is to transfer the knowledge obtained from the source task to the target task to assist the target task in learning. Solve the problems of insufficient sample size and low training efficiency in deep learning. However, due to the existence of domain differences (differences in feature distribution), direct transfer will degrade the model performance. Domain adaptation can align the feature information of the source domain and the target domain through feature transformation, and solve the problems caused by domain differences. Deep domain adaptive learning breaks the limitation of the same distribution of samples in traditional deep learning by transferring feature information, and accelerates the convergence speed of the model. However, in practical applications, the deep domain adaptive network still has weak image feature extraction capabilities and poor model stability. And it is not easy to converge and other problems. Therefore, it is necessary to optimize the deep domain adaptive network to further improve the training performance of the model.

Contents of the invention

Embodiments of the present invention provide a steel surface defect detection method and system based on a domain-adaptive deep migration network, which is used to solve the problem of poor generalization ability of detection models in the prior art and even technical defects of low recognition rate.

An embodiment of the present invention provides a method for detecting steel surface defects based on a domain-adaptive deep migration network, the method comprising:

S1: Obtain image samples of typical defects on the surface of the strip, and preprocess the samples;

S2: Construct an adversarial domain separation and adaptive network model based on the preprocessed samples;

S3: Embedding the new sample feature into the shared feature of the source domain image sample obtained after preprocessing, and inputting it into the adversarial domain separation and adaptive network model, and calculating the task classification loss and embedding classification loss;

S4: Dynamically optimize dynamic classification loss and dynamic adaptation loss by adding weights to multiple losses, wherein the dynamic classification loss includes task classification loss and embedding classification loss, and the dynamic adaptation loss includes domain adaptation loss and domain separation loss, And update the parameters of the confrontational domain separation and adaptive network model;

S5: Determine whether the number of iterations in the update reaches the optimal number of iterations, if so, execute step S6, otherwise, return to execute step S3;

S6: Save the parameters, obtain an optimized adversarial domain separation and adaptive network model, and detect the target domain sample test set to obtain the steel surface defect detection accuracy.

Preferably, the method for preprocessing the sample in the step S1 is:

First, all image samples are divided and unified in size, and N source domain image samples and N target domain image samples are selected, wherein the source domain image samples and the target domain image samples both include qualified image samples and defective images Sample, N is a positive integer;

Then, dividing the source domain image sample and the target domain image sample into a training set and a test set according to the same ratio;

Finally, the source domain image sample is input into a deep extraction network model, and the deep extraction network model is trained to obtain trained model parameters.

Preferably, the method of constructing the confrontational domain separation and adaptive network model according to the preprocessed samples in the step S2 is as follows:

First, the source domain image samples and target domain image sample training sets are input into multiple encoder network models based on deep convolutional neural networks, and based on the multiple encoder network models, the private parts of the source domain and the target domain and the source domain are separated. The shared part between the domain and the target domain realizes the separation of domain information, and the plurality of encoder network models include a shared encoder, a source domain private encoder and a target domain private encoder network model;

Then, initialize the plurality of encoder network models using the model parameters trained by the source domain image samples;

Finally, the outputs of multiple encoder network models after initialization are input into the task classifier, domain adaptation discriminator and domain separation discriminator through a multi-layer fully connected network.

Preferably, the method of embedding the new sample features into the shared features of the source domain image samples obtained after preprocessing in the step S3 is as follows:

Adaptively adjust the inter-class distance of new sample features according to the training state of the confrontational domain separation and adaptive network model, and use the spatial linear interpolation method to realize the embedding of new samples. The classification loss measure of the task classifier in the process;

Wherein, the new sample features are expressed as follows:

为嵌入的新样本特征，其标签对应相应的异类样本标签，X为同类样本特征，X^-为异类样本特征，L_task为任务分类损失，λ为调整嵌入新样本特征类间距离的参数；in,

is the embedded new sample feature, and its label corresponds to the corresponding heterogeneous sample label, X is the same sample feature, X ^- is the heterogeneous sample feature, L _task is the task classification loss, and λ is a parameter to adjust the inter-class distance of the embedded new sample feature;

The new feature is optimized, and its expression is as follows:

D ^E (X, X ⁺ ) = ‖X, X ⁺ ‖ ²

D ^E (X, X ⁺ )<D ^E (X, X ^- )

为嵌入的新样本特征，其标签对应相应的异类样本标签，X为同类样本特征，X^-为异类样本特征，X⁺为原始样本特征，L_task为任务分类损失，λ为调整嵌入新样本特征类间距离的参数，D^E(X，X⁺)为同类样本之间的距离，D^E(X，X^-)为同类样本与异类样本之间的距离。in,

is the embedded new sample feature, its label corresponds to the corresponding heterogeneous sample label, X is the same sample feature, X ^- is the heterogeneous sample feature, X ⁺ is the original sample feature, L _task is the task classification loss, λ is the adjusted embedding new sample feature The parameters of the distance between classes, D ^E (X, X ⁺ ) is the distance between samples of the same class, and D ^E (X, X ^- ) is the distance between samples of the same class and samples of different classes.

Preferably, in the step S4, the dynamic classification loss and the dynamic adaptation loss are dynamically optimized by adding weights to multiple losses, specifically including:

The dynamic classification loss is the result of dynamically adjusting the weights of the task classification loss and the embedded classification loss, expressed as follows:

Among them, L _{dynamic-class} is the dynamic classification loss, L _task is the task classification loss, and L _embedded is the embedded classification loss;

The dynamic adaptation loss is the result of dynamically adjusting the weights of the domain adaptation loss and the domain separation loss, expressed as follows:

Among them, L _dynamic-ad is the dynamic adaptation loss, L _adapt is the domain adaptation loss, and L _sep is the domain separation loss.

Preferably, the task classification loss L _task is calculated according to cross entropy, expressed as follows:

为任务分类器的权值参数，

为共享编码器的权值参数，x_s为源域图像样本。Among them, C _task is the task classifier, En _j is the shared encoder,

is the weight parameter of the task classifier,

is the weight parameter of the shared encoder, and x _s is the image sample in the source domain.

Preferably, the domain adaptation loss L _adapt is generated according to the domain adaptation discriminator confusion domain features, expressed as follows:

为共享编码器的权值参数，

为域适应鉴别器的权值参数，x_s为源域图像样本，x_t为目标域图像样本，E_x图像样本数学期望。Among them, En _j is the shared encoder, D _adapt is the domain adaptive discriminator,

is the weight parameter of the shared encoder,

is the weight parameter of the domain adaptation discriminator, x _s is the image sample in the source domain, x _t is the image sample in the target domain, and E _{x is} the mathematical expectation of the image sample.

Preferably, the domain separation loss L _sep is generated according to the separated domain features of the domain separation discriminator, expressed as follows:

为源域私有编码器，/>

为目标域私有编码器，D_sep为域分离鉴别器，/>

为共享编码器的权值参数，/>

为源域私有编码器的权值参数，

为目标域私有编码器的权值参数，/>

为域分离鉴别器的权值参数，x_s为源域图像样本，x_t为目标域图像样本，E_x图像样本数学期望。Among them, En _j is the shared encoder,

private encoder for the source domain, />

is the target domain private encoder, D _sep is the domain separation discriminator, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain,

is the weight parameter of the private encoder in the target domain, />

is the weight parameter of the domain separation discriminator, x _s is the source domain image sample, x _t is the target domain image sample, E _x the mathematical expectation of the image sample.

Preferably, the method for updating the confrontational domain separation and adaptive network model in the step S4 is:

Iteratively update the model parameters of multiple encoders, domain adaptation discriminators and domain separation discriminators through backpropagation through dynamic classification loss, domain adaptation loss and domain separation loss, including:

Initialize parameter θ:

为域适应鉴别器的权值参数，/>

为域分离鉴别器的权值参数，

为任务分类器的权值参数，/>

为共享编码器的权值参数，/>

为源域私有编码器的权值参数，/>

为目标域私有编码器的权值参数；in,

weight parameter for the domain adaptation discriminator, />

is the weight parameter of the domain separation discriminator,

is the weight parameter of the task classifier, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain, />

is the weight parameter of the private encoder of the target domain;

The dynamic classification loss updates each network parameter as follows:

为共享编码器的权值参数，

为任务分类器的权值参数，L_{dynamic-class}为动态分类损失，η为学习率，/>

为微分运算符；Among them, En _j is the shared encoder, C _task is the task classifier,

is the weight parameter of the shared encoder,

is the weight parameter of the task classifier, L _{dynamic-class} is the dynamic classification loss, η is the learning rate, />

is a differential operator;

The domain adaptation loss updates the parameters of each network model as follows:

为域适应鉴别器的权值参数，/>

为共享编码器的权值参数，L_adapt为域适应损失，η为学习率，/>

为微分运算符；Among them, D _adapt is the domain adaptive discriminator, En _j is the shared encoder,

weight parameter for the domain adaptation discriminator, />

is the weight parameter of the shared encoder, L _adapt is the domain adaptation loss, η is the learning rate, />

is a differential operator;

The domain separation loss updates the parameters of each network model as follows:

为源域私有编码器，/>

为目标域私有编码器，D_sep为域分离鉴别器，/>

为共享编码器的权值参数，/>

为源域私有编码器的权值参数，

为目标域私有编码器的权值参数，/>

为域分离鉴别器的权值参数，L_adapt为域适应损失，L_sep为域分离损失，η为学习率，/>

为微分运算符。Among them, En _j is the shared encoder,

private encoder for the source domain, />

is the target domain private encoder, D _sep is the domain separation discriminator, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain,

is the weight parameter of the private encoder in the target domain, />

is the weight parameter of the domain separation discriminator, L _adapt is the domain adaptation loss, L _sep is the domain separation loss, η is the learning rate, />

is a differentiation operator.

An embodiment of the present invention provides a steel surface defect detection system based on a domain-adaptive deep migration network, the system includes:

The sample preprocessing module is used to obtain image samples of typical defects on the surface of the strip and preprocess the samples;

Build a network model module, which is used to build an adversarial domain separation and adaptive network model based on preprocessed samples;

Optimizing the network model module for embedding the new sample features into the shared features of the source domain image samples obtained after preprocessing, and inputting them into the confrontational domain separation and adaptive network model, calculating task classification loss and embedding classification loss; Dynamically optimize dynamic classification loss and dynamic adaptation loss by adding weights to multiple losses, wherein the dynamic classification loss includes task classification loss and embedding classification loss, the dynamic adaptation loss includes domain adaptation loss and domain separation loss, and updates The parameters of the confrontational domain separation and adaptive network model; judging whether the number of iterations in the update reaches the optimal number of iterations, if so, input the optimization result into the sample detection module, otherwise, continue the iterative calculation;

The sample detection module is used to save the parameters, obtain the optimized confrontation domain separation and self-adaptive network model, and detect the target domain sample test set to obtain the steel surface defect detection accuracy.

The system is used to realize the above-mentioned steel surface defect detection method based on domain adaptive deep migration network.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention provides a steel surface defect detection method and system based on a domain-adaptive deep migration network. The present invention introduces adaptive mining sample hidden information and adds a dynamic weight optimization loss algorithm on the basis of confrontational domain separation and adaptive deep migration network . Adaptive mining of sample hidden information can improve the convergence speed and recognition accuracy in the network model training process, and adding dynamic weight optimization loss can make the network model trained in the source domain perform well in the target domain, making the network model adaptive domain The change of the network model improves the generalization ability of the network model, and finally achieves more accurate steel surface defect detection.

Description of drawings

In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the accompanying drawings that need to be used in the embodiments will be briefly introduced below, and the features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings. The accompanying drawings are schematic and should not be construed as limiting the present invention in any way. Those skilled in the art can obtain other drawings according to these drawings without creative work. in:

Fig. 1 is a flow chart of a steel surface defect detection method based on a domain-adaptive deep migration network according to an embodiment of the present invention;

FIG. 2 is a structural diagram of a network model based on confrontational domain separation and adaptive deep migration according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of the principle of embedding new sample features according to an embodiment of the present invention;

Fig. 4 is a block diagram of a steel surface defect detection system based on a domain-adaptive deep migration network according to an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Embodiment one

An embodiment of the present invention provides a steel surface defect detection method based on a domain-adaptive deep migration network. As shown in FIG. 1 , the method of this embodiment includes:

S101: Obtain an image sample of a typical defect on the surface of the strip, and preprocess the sample;

S102: Construct an adversarial domain separation and adaptive network model according to the preprocessed samples;

S103: Embedding the new sample feature into the shared feature of the source domain image sample obtained after preprocessing, and inputting it into the adversarial domain separation and adaptive network model, and calculating task classification loss and embedding classification loss;

S104: Dynamically optimize dynamic classification loss and dynamic adaptation loss by adding weights to multiple losses, wherein the dynamic classification loss includes task classification loss and embedding classification loss, and the dynamic adaptation loss includes domain adaptation loss and domain separation loss, And update the parameters of the confrontational domain separation and adaptive network model;

S105: Determine whether the number of iterations in the update reaches the optimal number of iterations, if so, execute step S106, otherwise, return to execute step S103;

S106: Save the parameters, obtain an optimized adversarial domain separation and self-adaptive network model, and detect a target domain sample test set to obtain steel surface defect detection accuracy.

The invention provides a steel surface defect detection method based on a domain-adaptive deep migration network. The invention establishes an adversarial domain separation based on a classification loss and an adaptive model performance evaluation mechanism; at the same time, a spatial linear interpolation method is used to adaptively mine migration space hidden The sample information of the mined new sample is used as the main metric to measure its contribution to the network performance, and the contribution performance is applied as a weight to the classification loss, aiming to eliminate the influence of noise samples on the model; in confrontation training In the process, the discriminative performance of the model is improved by adding dynamic weights to optimize the confrontation loss and smooth the network parameters. The invention can improve the training performance of the network model, and greatly improves the accuracy of steel surface defect detection under the domain offset environment.

Further, the method for preprocessing the sample in step S101 is:

First, all the image samples are divided and unified in size, and N source domain image samples {X _s } and N target domain image samples {X _t } are selected, wherein, the source domain image samples and the target domain image samples Both include qualified image samples and defective image samples, and N is a positive integer;

Then, dividing the source domain image sample and the target domain image sample into a training set and a test set according to the same ratio;

Finally, the source domain image sample is input into a deep extraction network model, and the deep extraction network model is trained to obtain trained model parameters.

和目标域私有编码器/>

特征迁移部分包括域适应鉴别器D_adapt和域分离鉴别器D_sep，任务分类部分包括任务分类器C_task。Fig. 2 is a structural diagram of a network model based on confrontational domain separation and adaptive deep migration according to an embodiment of the invention. As shown in Fig. 2, the network model consists of three parts: feature extraction part, feature migration part and task classification part. Among them, the feature extraction part includes the shared encoder En _j , the source domain private encoder

and target domain private encoder />

The feature migration part includes a domain adaptation discriminator D _adapt and a domain separation discriminator D _sep , and the task classification part includes a task classifier C _task .

和目标域私有编码器/>

采用AlexNet网络的特征提取结构(即5个卷积层和3个全连接层)，提取源域和目标域的共享特征/>

以及各自的私有特征/>

所述任务分类器C_task由由一个全连接层构成，用于预测任务标签；所述域适应鉴别器D_adapt由两个全连接层组成，用于预测共享特征/>

的域标签，域分离鉴别器D_sep同样由两个全连接层组成，用于预测特征标签。The shared encoder En _j , the source domain private encoder

and target domain private encoder />

Using the feature extraction structure of the AlexNet network (that is, 5 convolutional layers and 3 fully connected layers), extract the shared features of the source domain and the target domain />

and their respective private traits />

The task classifier C _task is composed of a fully connected layer for predicting task labels; the domain adaptation discriminator D _adapt is composed of two fully connected layers for predicting shared features/>

The domain label of the domain separation discriminator D _sep also consists of two fully connected layers for predicting feature labels.

Further, in step S102, the method for constructing the confrontational domain separation and adaptive network model is as follows:

和目标域私有编码器/>

网络模型；First, the source domain image sample {X _s } and the target domain image sample {X _t } training set are input into multiple encoder network models based on deep convolutional neural networks, and the source domain and target domain image samples are separated based on the multiple encoder network models. The private part of the target domain and the shared part between the source domain and the target domain realize domain information separation, and the multiple encoder network models include shared encoder En _j , source domain private encoder

and target domain private encoder />

network model;

Then, initialize the plurality of encoder network models using the model parameters trained by the source domain image samples;

Finally, the outputs of multiple encoder network models after initialization are input into the task classifier C _task , the domain adaptation discriminator D _adapt and the domain separation discriminator D _sep through a multi-layer fully connected network.

表示嵌入的新样本特征/>

X^-表示异类样本特征。Fig. 3 is a schematic diagram of the principle of embedding new sample features according to the embodiment of the present invention. As shown in Fig. 3, X ⁺ represents the original sample feature, and X represents the similar sample feature

represents the embedded new sample features />

X ^- Indicates heterogeneous sample features.

Further, in step S103, the method of embedding the new sample features into the shared features of the source domain image samples obtained after preprocessing is:

Adaptively adjust the inter-class distance of new sample features according to the training state of the confrontational domain separation and adaptive network model, and use the spatial linear interpolation method to realize the embedding of new samples. The classification loss measurement of the task classifier C _task in the process;

Among them, the new sample features are expressed as follows:

为嵌入新样本特征，其标签对应相应的异类样本标签，X为同类样本特征，X^-为异类样本特征，L_task为任务分类损失，λ为调整嵌入新样本特征类间距离的参数；in,

To embed new sample features, its label corresponds to the corresponding heterogeneous sample label, X is the same sample feature, X ^- is the heterogeneous sample feature, L _task is the task classification loss, and λ is the parameter to adjust the inter-class distance of the embedded new sample feature;

和同类样本特征X的类间距离接近零的情况出现，对所述新样本特征/>

进行优化，其表达式如下：New sample features to avoid embedding

When the inter-class distance with the same sample feature X is close to zero, for the new sample feature />

To optimize, its expression is as follows:

D ^E (X, X ⁺ ) = ‖X, X ⁺ ‖ ²

D ^E (X, X ⁺ )<D ^E (X, X ^- )

为嵌入的新样本特征，其标签对应相应的异类样本标签，X为同类样本特征，X^-为异类样本特征，X⁺为原始样本特征，L_task为任务分类损失，λ为调整嵌入新样本特征类间距离的参数，D^E(X，X⁺)为同类样本之间的距离，D^E(X，X^-)为同类样本与异类样本之间的距离。in,

is the embedded new sample feature, its label corresponds to the corresponding heterogeneous sample label, X is the same sample feature, X ^- is the heterogeneous sample feature, X ⁺ is the original sample feature, L _task is the task classification loss, λ is the adjusted embedding new sample feature The parameters of the distance between classes, D ^E (X, X ⁺ ) is the distance between samples of the same class, and D ^E (X, X ^- ) is the distance between samples of the same class and samples of different classes.

Further, in step S104, by adding weights to multiple losses, the dynamic classification loss and dynamic adaptation loss are dynamically optimized, specifically including:

The dynamic classification loss is the result of dynamically adjusting the weights of the task classification loss and the embedded classification loss, expressed as follows:

Among them, L _{dynamic-class} is the dynamic classification loss, L _task is the task classification loss, and L _embedded is the embedded classification loss;

The dynamic adaptation loss is the result of dynamically adjusting the weights of the domain adaptation loss and the domain separation loss, expressed as follows:

Among them, L _dynamic-ad is the dynamic adaptation loss, L _adapt is the domain adaptation loss, and L _sep is the domain separation loss.

Further, the task classification loss L _task is calculated according to the cross entropy, expressed as follows:

为任务分类器的权值参数，

为共享编码器的权值参数，x_s为源域图像样本。Among them, C _task is the task classifier, En _j is the shared encoder,

is the weight parameter of the task classifier,

is the weight parameter of the shared encoder, and x _s is the image sample in the source domain.

Further, the domain adaptation loss L _adapt is generated according to the domain adaptation discriminator confusion domain features, expressed as follows:

为共享编码器的权值参数，

为域适应鉴别器的权值参数，x_s为源域图像样本，x_t为目标域图像样本，E_x图像样本数学期望。Among them, En _j is the shared encoder, D _adapt is the domain adaptive discriminator,

is the weight parameter of the shared encoder,

is the weight parameter of the domain adaptation discriminator, x _s is the image sample in the source domain, x _t is the image sample in the target domain, and E _{x is} the mathematical expectation of the image sample.

Further, the domain separation loss L _sep is generated according to the domain separation discriminator separation domain features, expressed as follows:

为源域私有编码器，/>

为目标域私有编码器，D_sep为域分离鉴别器，/>

为共享编码器的权值参数，/>

为源域私有编码器的权值参数，

为目标域私有编码器的权值参数，/>

为域分离鉴别器的权值参数，x_s为源域图像样本，x_t为目标域图像样本，E_x图像样本数学期望。Among them, En _j is the shared encoder,

private encoder for the source domain, />

is the target domain private encoder, D _sep is the domain separation discriminator, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain,

is the weight parameter of the private encoder in the target domain, />

is the weight parameter of the domain separation discriminator, x _s is the source domain image sample, x _t is the target domain image sample, E _x the mathematical expectation of the image sample.

Further, the method for updating the parameters of the confrontational domain separation and adaptive network model is:

Iteratively update the model parameters of multiple encoders, domain adaptation discriminators and domain separation discriminators through backpropagation through dynamic classification loss, domain adaptation loss and domain separation loss, including:

Initialize parameter θ:

为域适应鉴别器的权值参数，/>

为域分离鉴别器的权值参数，

为任务分类器的权值参数，/>

为共享编码器的权值参数，/>

为源域私有编码器的权值参数，/>

为目标域私有编码器的权值参数；in,

weight parameter for the domain adaptation discriminator, />

is the weight parameter of the domain separation discriminator,

is the weight parameter of the task classifier, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain, />

is the weight parameter of the private encoder of the target domain;

The dynamic classification loss updates each network parameter as follows:

为共享编码器的权值参数，

为任务分类器的权值参数，L_{dynamic-class}为动态分类损失，η为学习率，/>

为微分运算符；Among them, En _j is the shared encoder, C _task is the task classifier,

is the weight parameter of the shared encoder,

is the weight parameter of the task classifier, L _{dynamic-class} is the dynamic classification loss, η is the learning rate, />

is a differential operator;

The domain adaptation loss updates the parameters of each network model as follows:

为域适应鉴别器的权值参数，/>

为共享编码器的权值参数，L_adapt为域适应损失，η为学习率，/>

为微分运算符；Among them, D _adapt is the domain adaptive discriminator, En _j is the shared encoder,

weight parameter for the domain adaptation discriminator, />

is the weight parameter of the shared encoder, L _adapt is the domain adaptation loss, η is the learning rate, />

is a differential operator;

The domain separation loss updates the parameters of each network model as follows:

为源域私有编码器，/>

为目标域私有编码器，D_sep为域分离鉴别器，/>

为共享编码器的权值参数，/>

为源域私有编码器的权值参数，

为目标域私有编码器的权值参数，/>

为域分离鉴别器的权值参数，L_adapt为域适应损失，L_sep为域分离损失，η为学习率，/>

为微分运算符。Among them, En _j is the shared encoder,

private encoder for the source domain, />

is the target domain private encoder, D _sep is the domain separation discriminator, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain,

is the weight parameter of the private encoder in the target domain, />

is the weight parameter of the domain separation discriminator, L _adapt is the domain adaptation loss, L _sep is the domain separation loss, η is the learning rate, />

is a differentiation operator.

Embodiment two

An embodiment of the present invention provides a steel surface defect detection system based on a domain-adaptive deep migration network, as shown in FIG. 4 , including:

A sample preprocessing module 401, configured to obtain image samples of typical defects on the surface of the strip, and preprocess the samples;

Building a network model module 402, used to build an adversarial domain separation and adaptive network model according to the preprocessed samples;

The optimization network model module 403 is used to embed the new sample features into the shared features of the source domain image samples obtained after preprocessing, and input them into the adversarial domain separation and adaptive network model, and calculate the task classification loss and embedding classification loss ; Dynamically optimize dynamic classification loss and dynamic adaptation loss by adding weights to multiple losses, wherein the dynamic classification loss includes task classification loss and embedding classification loss, and the dynamic adaptation loss includes domain adaptation loss and domain separation loss, and Updating the parameters of the adversarial domain separation and adaptive network model; judging whether the number of iterations in the update reaches the optimal number of iterations, if so, input the optimization result into the sample detection module, otherwise, continue the iterative calculation;

The sample detection module 404 is used to save the parameters, obtain an optimized confrontational domain separation and self-adaptive network model, and detect the sample test set in the target domain to obtain the detection accuracy of steel surface defects.

The system is used to realize the steel surface defect detection method based on the domain-adaptive deep migration network described in the first embodiment above, which will not be repeated here.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

Apparently, the above-mentioned embodiments are only examples for clear description, and are not intended to limit the implementation. For those of ordinary skill in the art, other changes or changes in different forms can be made on the basis of the above description. It is not necessary and impossible to exhaustively list all the implementation manners here. However, the obvious changes or changes derived therefrom are still within the scope of protection of the present invention.

Claims (9)

1. A steel surface defect detection method based on domain adaptive deep migration network, characterized in that, comprising: S1: Obtain image samples of typical defects on the surface of the strip, and preprocess the samples; S2: Construct an adversarial domain separation and adaptive network model based on the preprocessed samples; S3: Embedding the new sample feature into the shared feature of the source domain image sample obtained after preprocessing, and inputting it into the adversarial domain separation and adaptive network model, and calculating the task classification loss and embedding classification loss; S4: Dynamically optimize dynamic classification loss and dynamic adaptation loss by adding weights to multiple losses, wherein the dynamic classification loss includes task classification loss and embedding classification loss, and the dynamic adaptation loss includes domain adaptation loss and domain separation loss, And update the parameters of the confrontational domain separation and adaptive network model; S5: Determine whether the number of iterations in the update reaches the optimal number of iterations, if so, execute step S6, otherwise, return to execute step S3; S6: Save the parameters, obtain the optimized adversarial domain separation and adaptive network model, and detect the sample test set in the target domain to obtain the detection accuracy of steel surface defects; In the step S2, the method of constructing the confrontational domain separation and adaptive network model according to the preprocessed samples is as follows: First, the source domain image samples and target domain image sample training sets are input into multiple encoder network models based on deep convolutional neural networks, and based on the multiple encoder network models, the respective private parts of the source domain and the target domain and the source domain are separated. The shared part between the domain and the target domain realizes the separation of domain information, and the plurality of encoder network models include a shared encoder, a source domain private encoder and a target domain private encoder network model; Then, initialize the plurality of encoder network models using the model parameters trained by the source domain image samples; Finally, the outputs of multiple encoder network models after initialization are input into the task classifier, domain adaptation discriminator and domain separation discriminator through a multi-layer fully connected network. 2. A kind of steel surface defect detection method based on domain self-adaptive deep migration network according to claim 1, is characterized in that, the method that sample is carried out preprocessing in described step S1 is: First, all image samples are divided and unified in size, and N source domain image samples and N target domain image samples are selected, wherein the source domain image samples and the target domain image samples both include qualified image samples and defective images Sample, N is a positive integer; Then, dividing the source domain image sample and the target domain image sample into a training set and a test set according to the same ratio; Finally, the source domain image sample is input into a deep extraction network model, and the deep extraction network model is trained to obtain trained model parameters. 3. A method for detecting steel surface defects based on a domain-adaptive deep migration network according to claim 1, wherein in said step S3, the new sample features are embedded into the source domain image samples obtained after preprocessing The methods in the shared trait are: Adaptively adjust the inter-class distance of new sample features according to the training state of the confrontational domain separation and adaptive network model, and use the spatial linear interpolation method to realize the embedding of new samples. The classification loss measure of the task classifier in the process; Wherein, the new sample features are expressed as follows:

in,

is the embedded new sample feature, and its label corresponds to the corresponding heterogeneous sample label, X is the same sample feature, X ^- is the heterogeneous sample feature, L _task is the task classification loss, and λ is a parameter to adjust the inter-class distance of the embedded new sample feature; The new sample features are optimized, and its expression is as follows:

D ^E (X, X ⁺ ) = ‖X, X ⁺ ‖ ²

D ^E (X, X ⁺ )<D ^E (X, X ^- ) in,

is the embedded new sample feature, its label corresponds to the corresponding heterogeneous sample label, X is the same sample feature, X ^- is the heterogeneous sample feature, X ⁺ is the original sample feature, L _task is the task classification loss, λ is the adjusted embedding new sample feature The parameters of the distance between classes, D ^E (X, X ⁺ ) is the distance between samples of the same class, and D ^E (X, X ^- ) is the distance between samples of the same class and samples of different classes. 4. A method for detecting steel surface defects based on a domain-adaptive deep migration network according to claim 1, wherein in said step S4, by adding weights to multiple losses, dynamically optimize the dynamic classification loss and dynamic Adapting to loss, specifically: The dynamic classification loss is the result of dynamically adjusting the weights of the task classification loss and the embedded classification loss, expressed as follows:

Among them, L _{dynamic-class} is the dynamic classification loss, L _task is the task classification loss, and L _embedded is the embedded classification loss; The dynamic adaptation loss is the result of dynamically adjusting the weights of the domain adaptation loss and the domain separation loss, expressed as follows:

Among them, L _dynamic-ad is the dynamic adaptation loss, L _adapt is the domain adaptation loss, and L _sep is the domain separation loss. 5. a kind of steel surface defect detection method based on domain adaptive deep migration network according to claim 4, is characterized in that, described task classification loss L _task obtains according to cross-entropy calculation, expresses as follows:

Among them, C _task is the task classifier, En _j is the shared encoder,

is the weight parameter of the task classifier, />

is the weight parameter of the shared encoder, and x _s is the image sample in the source domain. 6. a kind of steel surface defect detection method based on domain adaptive depth migration network according to claim 4, is characterized in that, described domain adaptation loss L _adapt produces according to domain adaptation discriminator confusion domain feature, expresses as follows:

Among them, En _j is the shared encoder, D _adapt is the domain adaptive discriminator,

is the weight parameter of the shared encoder,

is the weight parameter of the domain adaptation discriminator, x _s is the image sample in the source domain, x _t is the image sample in the target domain, and E _{x is} the mathematical expectation of the image sample. 7. A kind of steel surface defect detection method based on domain self-adaptive deep migration network according to claim 4, it is characterized in that, described domain separation loss L _sep produces according to domain separation discriminator separation domain characteristic, expresses as follows:

Among them, En _j is the shared encoder,

private encoder for the source domain, />

is the target domain private encoder, D _sep is the domain separation discriminator, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain, />

is the weight parameter of the private encoder in the target domain, />

is the weight parameter of the domain separation discriminator, x _s is the source domain image sample, x _t is the target domain image sample, E _x the mathematical expectation of the image sample. 8. A kind of steel surface defect detection method based on domain self-adaptive deep migration network according to claim 1 or 4, it is characterized in that, in the described step S4, update the parameters of the confrontational domain separation and self-adaptive network model The method is: Iteratively update the model parameters of multiple encoders, domain adaptation discriminators and domain separation discriminators through backpropagation through dynamic classification loss, domain adaptation loss and domain separation loss, including: Initialize parameter θ:

in,

weight parameter for the domain adaptation discriminator, />

is the weight parameter of the domain separation discriminator, />

is the weight parameter of the task classifier, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain, />

is the weight parameter of the private encoder of the target domain; The dynamic classification loss updates each network parameter as follows:

Among them, En _j is the shared encoder, C _task is the task classifier,

is the weight parameter of the shared encoder, />

is the weight parameter of the task classifier, L _{dynamic-class} is the dynamic classification loss, η is the learning rate, />

is a differential operator; The domain adaptation loss updates the parameters of each network model as follows:

Among them, D _adapt is the domain adaptive discriminator, En _j is the shared encoder,

is the weight parameter of the domain adaptation discriminator,

is the weight parameter of the shared encoder, L _adapt is the domain adaptation loss, η is the learning rate, />

is a differential operator; The domain separation loss updates the parameters of each network model as follows:

Among them, En _j is the shared encoder,

private encoder for the source domain, />

is the target domain private encoder, D _sep is the domain separation discriminator, />

is the weight parameter of the shared encoder, />

is the weight parameter of the private encoder in the source domain, />

is the weight parameter of the private encoder in the target domain, />

is the weight parameter of the domain separation discriminator, L _adapt is the domain adaptation loss, L _sep is the domain separation loss, η is the learning rate, />

is a differentiation operator. 9. A steel surface defect detection system based on a domain-adaptive deep migration network, characterized in that it includes: The sample preprocessing module is used to obtain image samples of typical defects on the surface of the strip and preprocess the samples; Build a network model module, which is used to build an adversarial domain separation and adaptive network model based on preprocessed samples; Optimizing the network model module for embedding the new sample features into the shared features of the source domain image samples obtained after preprocessing, and inputting them into the confrontational domain separation and adaptive network model, calculating task classification loss and embedding classification loss; Dynamically optimize dynamic classification loss and dynamic adaptation loss by adding weights to multiple losses, wherein the dynamic classification loss includes task classification loss and embedding classification loss, the dynamic adaptation loss includes domain adaptation loss and domain separation loss, and updates The parameters of the confrontational domain separation and adaptive network model; judging whether the number of iterations in the update reaches the optimal number of iterations, if so, input the optimization result into the sample detection module, otherwise, continue the iterative calculation; The sample detection module is used to save the parameters, obtain the optimized confrontation domain separation and self-adaptive network model, and detect the sample test set in the target field to obtain the detection accuracy of steel surface defects; The method of constructing confrontational domain separation and self-adaptive network model according to the sample after preprocessing in the described building network model module is: First, the source domain image samples and target domain image sample training sets are input into multiple encoder network models based on deep convolutional neural networks, and based on the multiple encoder network models, the respective private parts of the source domain and the target domain and the source domain are separated. The shared part between the domain and the target domain realizes the separation of domain information, and the plurality of encoder network models include a shared encoder, a source domain private encoder and a target domain private encoder network model; Then, initialize the plurality of encoder network models using the model parameters trained by the source domain image samples; Finally, the outputs of multiple encoder network models after initialization are input into the task classifier, domain adaptation discriminator and domain separation discriminator through a multi-layer fully connected network.

Images