CN113781492A

CN113781492A - Target element content measuring method, training method, related device and storage medium

Info

Publication number: CN113781492A
Application number: CN202010521698.7A
Authority: CN
Inventors: 林柏洪
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-06-10
Filing date: 2020-06-10
Publication date: 2021-12-10
Anticipated expiration: 2040-06-10
Also published as: CN113781492B

Abstract

The method for measuring the content of the target element comprises the following steps of: acquiring a metallographic image of a target object acquired by a metallographic microscope; cutting the metallographic image into a plurality of image blocks according to a preset cutting size; inputting the image blocks into a trained depth regression network model to extract the characteristics of the image blocks, mapping the characteristics of the image blocks into a real number fixed stage, and obtaining real number measurement results of the image blocks; and weighting the real number measurement results of the plurality of image blocks to obtain the target element content of the metallographic image. By adopting the method disclosed by the embodiment of the specification, the content of the target element can be quickly and effectively measured.

Description

Target element content measuring method, training method, related device and storage medium

Technical Field

The embodiment of the specification relates to the technical field of image processing, in particular to a target element content measuring method, a training method, a related device and a storage medium.

Background

Traditional target element content measurement, for example, sorbite content measurement adopts artifical mode, utilizes metallographic microscope to observe through the naked eye, consumes manpower and inefficiency, in addition, uses metallographic microscope to have great injury to tester's eyes for a long time. In recent years, with the development of image processing technology, different image processing methods are proposed, and corresponding software systems are designed to realize automatic identification of sorbite tissues and complete measurement of sorbite content on the basis of the automatic identification.

However, the conventional grain boundary extraction method is difficult to perfectly identify the sorbite structure, so that errors exist in the sorbite content in the measurement process. Therefore, how to find a quick and effective method for measuring the content of the target element to ensure that the measurement result is not influenced by the sorbite tissue identification effect has great significance.

Disclosure of Invention

In view of the above, embodiments of the present disclosure provide a method for measuring a content of a target element, a training method, a related apparatus, and a storage medium, which are capable of measuring the content of the target element quickly and efficiently.

First, an embodiment of the present specification provides a method for measuring a content of a target element, including:

acquiring a metallographic image of a target object acquired by a metallographic microscope;

cutting the metallographic image into a plurality of image blocks according to a preset cutting size;

inputting the image blocks into a trained depth regression network model to extract the characteristics of the image blocks, mapping the characteristics of the image blocks into a real number fixed stage, and obtaining real number measurement results of the image blocks;

and weighting the real number measurement results of the plurality of image blocks to obtain the target element content of the metallographic image.

Optionally, the deep regression network model includes: the system comprises a preset convolution neural network and a full-connection neural network, wherein the output end of the convolution neural network is connected with the input end of the full-connection neural network.

Optionally, the convolutional neural network comprises: a depth residual error network;

inputting the plurality of image blocks into a trained depth regression network model to extract the features of the plurality of image blocks, mapping the features of the plurality of image blocks into a real number fixed stage, and obtaining real number measurement results of the plurality of image blocks, wherein the method comprises the following steps:

inputting the image blocks into the depth residual error network, and extracting the characteristics of each image block to obtain a real number vector with the characteristics of a preset dimension;

and outputting the real number vector to the fully-connected neural network, mapping the characteristics of each image block into a real number fixed stage, and obtaining real number measurement results of the plurality of image blocks.

Optionally, the depth residual network includes any one of:

a residual network ResNet50 at layer 50 and a residual network ResNet101 at layer 101.

Optionally, the activation functions of the depth residual network and the fully-connected neural network include:

f(x)＝max(x,0)；

where x represents the input variable of the neuron, and f (x) represents the value mapped by the output of the neuron.

Optionally, the weighting the real number measurement results of the plurality of image blocks to obtain the target element content of the metallographic image includes:

and calculating an average value of real number measurement results of the plurality of image blocks, and converting the calculated average value into percentage to obtain the target element content of the metallographic image.

Optionally, the deep regression network model is trained by using the following method:

acquiring a metallographic image of a target object to be trained;

cutting the metallographic image according to the preset cutting size to obtain a plurality of image blocks serving as image blocks to be trained;

and acquiring the target element content artificial measurement results corresponding to the image blocks to be trained, inputting the target element content artificial measurement results corresponding to the image blocks to be trained into the deep regression network model, and inputting the image blocks to be trained into the deep regression network model by adopting a preset training method for training until the deep regression network model reaches a preset stability performance index.

Optionally, the loss function of the deep regression network model includes:

wherein P represents the artificially determined target element content measurement result of the image block of the preset cutting size,

and e is a minimum constant greater than 0, and represents the content of the target elements of the metallographic image output in the process of training by adopting the deep regression network model.

Optionally, the inputting the image block to be trained into the deep regression network model by using a preset training method for training until the deep regression network model reaches a preset stability performance index includes:

the learning rate is set to 10 using an adaptive moment estimation method^-3And converging the output value of the loss function of the deep regression network to a preset threshold value.

Optionally, the method further comprises:

before the image block to be trained is input into the deep regression network model for training, preprocessing a target element content manual measurement result corresponding to the image block to be trained;

and performing post-processing on the real number measurement results of the plurality of image blocks before performing weighting processing on the real number measurement results of the plurality of image blocks.

Optionally, the preprocessing the artificial measurement result of the content of the target element corresponding to the image block to be trained includes: carrying out logarithmic transformation on the artificial measurement result of the content of the target element corresponding to the image block to be trained;

performing post-processing on the real number measurement results of the plurality of image blocks before performing weighting processing on the real number measurement results of the plurality of image blocks, including: and performing exponential transformation on the real number measurement results of the plurality of image blocks.

Optionally, the cutting the metallographic image according to the preset cutting size to obtain a plurality of image blocks, wherein the image blocks to be trained include at least one of the following:

extracting image blocks with the size of the preset cutting size from top to bottom and from left to right according to a preset step length from the metallographic image as the image blocks to be trained;

processing the metallographic image in an up-sampling mode and a down-sampling mode, and cutting the metallographic image obtained after sampling according to the preset cutting size to obtain the image block to be trained;

and carrying out image enhancement treatment on the metallographic image, and cutting the metallographic image subjected to the enhancement treatment according to the preset cutting size to obtain the image block to be trained.

The embodiment of the present specification further provides a training method of a deep regression network model suitable for target element content measurement, including:

acquiring a metallographic image of a target object to be trained;

cutting the metallographic image into a plurality of image blocks according to a preset cutting size, and taking the image blocks as image blocks to be trained;

Optionally, the deep regression network model includes: the device comprises a preset depth residual error network and a fully-connected neural network, wherein the output end of the depth residual error network is connected with the input end of the fully-connected neural network.

The method for inputting the image blocks to be trained into the deep regression network model by adopting a preset training method for training comprises the following steps:

inputting the multiple image blocks to be trained into the deep residual error network, and extracting the characteristics of each image block to be trained to obtain a real number vector with the characteristics as a preset dimensionality;

and outputting the real number vector to the fully-connected neural network, mapping the characteristics of each image block to be trained into a real number fixed stage, and obtaining real number measurement results of the plurality of image blocks to be trained.

Optionally, the loss function of the deep regression network model includes:

and respectively inputting the real number measurement results of the image blocks to be trained and the corresponding target element content manual measurement results into a preset network loss function until the output value of the loss function is converged to a preset threshold value.

by adopting the self-adaptive moment estimation method,set learning rate to 10^-3And gradually converging the loss function of the deep regression network to a preset stable state index.

Optionally, before inputting the image block to be trained into the deep regression network model for training, the method further includes: and preprocessing the manual measurement result of the content of the target element corresponding to the image block to be trained.

Optionally, the preprocessing the artificial measurement result of the content of the target element corresponding to the image block to be trained includes: and carrying out logarithmic transformation on the artificial measurement result of the target element content corresponding to the image block to be trained.

Optionally, the cutting the metallographic image into a plurality of image blocks according to a preset cutting size, as an image block to be trained, includes at least one of the following:

The embodiment of the present specification further provides a method for measuring a content of a target element, including:

acquiring a target object;

preprocessing the target object;

placing the pretreated target object under a metallographic microscope;

automatically shooting a metallographic image of a target object acquired by a metallographic microscope;

and measuring by adopting the method for measuring the content of the target element in any embodiment to obtain the content of the target element in the metallographic image of the target object.

An embodiment of the present specification further provides a target element content measurement apparatus, including:

the metallographic image acquisition unit is suitable for acquiring a metallographic image of a target object acquired by a metallographic microscope;

the image cutting unit is suitable for cutting the metallographic image into a plurality of image blocks according to a preset cutting size;

the measuring unit is suitable for inputting the image blocks into a trained depth regression network model to extract the characteristics of the image blocks, mapping the characteristics of the image blocks into a real number fixed stage to obtain real number measuring results of the image blocks, and performing weighting processing on the real number measuring results of the image blocks to obtain the target element content of the metallographic image.

Optionally, the convolutional neural network comprises: the activation functions of the depth residual error network and the fully-connected neural network are nonlinear activation functions;

the measuring unit is suitable for inputting the image blocks into the depth residual error network, extracting the characteristics of each image block to obtain a real number vector with the characteristics of a preset dimension, outputting the real number vector to the fully-connected neural network, mapping the characteristics of each image block into a real number fixed stage, and obtaining real number measuring results of the image blocks.

Optionally, the measuring unit is adapted to perform exponential transformation on the real number measurement results of the plurality of image blocks, calculate an average value of the real number measurement results after the exponential transformation, and convert the calculated average value into a percentage to obtain the target element content of the metallographic image.

The embodiment of the present specification further provides a training apparatus for a deep regression network model suitable for target element content measurement, including:

the metallographic image acquisition unit is suitable for acquiring a metallographic image of a target object to be trained;

the image cutting unit is suitable for cutting the metallographic image into a plurality of image blocks according to a preset cutting size, and the image blocks are used as image blocks to be trained;

and the training unit is suitable for acquiring the target element content artificial measurement results corresponding to the multiple image blocks to be trained, inputting the target element content artificial measurement results corresponding to the multiple image blocks to be trained into the deep regression network model, and inputting the image blocks to be trained into the deep regression network model by adopting a preset training method for training until the deep regression network model reaches a preset stability performance index.

Optionally, the deep regression network model includes:

the device comprises a preset depth residual error network and a fully-connected neural network, wherein the output end of the depth residual error network is connected with the input end of the fully-connected neural network, and the activation functions of the depth residual error network and the fully-connected neural network are nonlinear activation functions.

Optionally, the training unit is adapted to input the multiple image blocks to be trained into the deep residual error network, extract features of the image blocks to be trained, and obtain a real number vector with features of a preset dimension; and outputting the real number vector to the fully-connected neural network, mapping the characteristics of each image block to be trained into a real number fixed stage, and obtaining real number measurement results of the plurality of image blocks to be trained.

Optionally, the loss function of the deep regression network model includes:

representing said employing said depthAnd e is a minimum constant greater than 0.

Optionally, the training unit is adapted to obtain artificial measurement results of target element contents corresponding to the plurality of image blocks to be trained, and input the real measurement results of the plurality of image blocks to be trained and the artificial measurement results of the corresponding target element contents into a preset network loss function respectively until an output value of the loss function converges to a preset threshold.

Optionally, the training unit is adapted to perform logarithmic transformation on the target element manual measurement results of the plurality of image blocks to be trained, and then input the result into the deep regression network model for training.

An embodiment of the present specification further provides a target element content measurement system, including:

a sample acquiring device adapted to acquire a target object;

the pretreatment device is suitable for pretreating the target object;

the metallographic image shooting device is suitable for automatically shooting a metallographic image of the target object collected by a metallographic microscope after the preprocessed target object is placed under the microscope;

and the target element content measuring device of any one of the preceding embodiments.

The present specification further provides an electronic device, including a memory and a processor, where the memory stores computer instructions executable on the processor, and the processor executes the computer instructions to perform the steps of the method according to any one of the foregoing embodiments.

The present specification also provides a computer readable storage medium, on which computer instructions are stored, and the computer instructions execute the steps of the method of any one of the foregoing embodiments when executed.

Compared with the prior art, the technical scheme of the embodiment of the specification has the following beneficial effects:

on one hand, by adopting the target element content measurement scheme of the embodiment of the specification, the metallographic image of the target object is cut into a plurality of image blocks according to the preset cutting size, the image blocks are input into the trained depth regression network to automatically extract the characteristics of the image blocks, the characteristics of the image blocks are mapped into a real number fixed level to obtain the real number measurement results of the image blocks, then the real number measurement results of the image blocks are weighted to obtain the target element content of the metallographic image, the target element content measurement scheme does not need to identify the target element tissue in the metallographic image in advance, the whole measurement process is automatically completed based on the trained depth regression network without manual measurement, the measurement precision does not depend on the identification precision of the target element content, and the stability of the measurement results can be ensured, the content of the target element in the target object can be measured efficiently, stably and accurately.

On the other hand, by using the training scheme in the embodiment of the present specification, the depth regression network model is trained by obtaining the target element content artificial measurement results corresponding to the multiple image blocks to be trained of the target object and combining the target element content artificial measurement results corresponding to the multiple image blocks to be trained until the depth regression network model reaches the preset stability performance index, so that the trained depth regression network model can be used for automatically measuring the target element content in the target object, and the depth regression network model is used for measuring without identifying the target element tissue in the target object to be measured in advance, so that the accuracy and stability of the measurement result can be ensured, and the depth regression network model can be used for automatically measuring, therefore, the depth regression network model trained by using the training method in the embodiment of the present specification is used for measuring the target element content in the target object The content of the target element in the target object can be measured quickly and effectively.

Drawings

FIG. 1 is a block diagram of a process for measuring the content of a target element in an embodiment of the present disclosure;

FIG. 2 is a flow chart of a method for measuring the content of a target element in an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a residual structure in an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a specific application of a target element content measuring system in an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of a depth residual error network in an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of a fully-connected neural network in an embodiment of the present disclosure.

FIG. 7 is a flow chart of a training method in an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a target element content measuring apparatus in an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of an exercise device according to an embodiment of the present disclosure;

FIG. 10 is a schematic structural diagram of a target element content measuring system in an embodiment of the present disclosure;

fig. 11 is a schematic structural diagram of an electronic device in an embodiment of this specification.

Detailed Description

Wire rods, also called wire rods, are commonly referred to as small diameter round steel in coils. The wire rods are of various types, wherein the low-carbon steel wire rods in the carbon steel wire rods are commonly called flexible wires, and the medium-high carbon steel wire rods are commonly called hard wires. The high-carbon steel wire rod is widely applied to the engineering fields of high-rise buildings, bridges, petrochemical industry, railways and the like after being drawn. The sorbite is a matrix structure of the high-carbon steel wire rod, and the main factor influencing the drawing performance and the mechanical property of the high-carbon steel wire rod is the content of the sorbite (sorbite), wherein the higher the sorbite content is, the more uniform the sorbite content is, and the better the drawing performance is. With the great improvement of the high-carbon steel wire rod yield and the wide application of the high-carbon steel wire rod product, manufacturers and product industries pay more and more attention to the sorbite content index of the high-carbon steel wire rod product.

At present, according to the issued standard document of the ferrous metallurgy industry of the people's republic of China, "metallographic detection method of sorbite content of high-carbon steel wire rods", the sorbite measurement process can be divided into four steps: selecting a standard sample, polishing and corroding the standard sample and a sample to be measured, calibrating a detection condition and detecting the sample to be measured. In the detection of a sample to be detected, how to accurately identify the sorbite tissue from a microscope image directly influences the measurement precision of the sorbite content.

In the past, the detection is usually carried out manually, a metallographic microscope is used for observing through naked eyes, although the accuracy is high, the labor consumption is low, the efficiency is low, and in addition, the metallographic microscope is used for a long time to cause great damage to eyes of testers.

In recent years, with the development of image processing technology, people propose various image processing methods, design corresponding software systems, realize automatic identification of sorbite tissues and complete measurement of sorbite content on the basis of the automatic identification. For example, the metallographic picture is opened by adopting image processing software, the number of sorbite tissues is manually calculated by utilizing auxiliary lines of the image processing software, and the sorbite content value of the cross section of the wire rod in the whole metallographic image is obtained on the basis. For another example, a simple threshold segmentation method is adopted to identify a sorbite structure in the metallographic image, and then the sorbite content is calculated, wherein threshold parameters need to be manually given.

The conventional sorbite content measuring method depends on sorbite tissue identification precision, on one hand, the sorbite tissue is influenced by image noise, and is difficult to be completely and accurately identified for different images, so that the sorbite content measuring precision is poor in stability and low in accuracy; on the other hand, the conventional method needs to further calculate to obtain the sorbite content on the basis of identifying the sorbite tissues, and the tissue identification calculation is very time-consuming, so that the sorbite measurement speed is very slow.

Therefore, how to find a quick and effective method for measuring the content of the target element in the target object so that the measurement result is not influenced by the tissue identification effect of the target element has great significance.

Therefore, the target element content measuring method based on the deep regression network is provided, the target element content in the target object can be automatically measured without accurately identifying the target element organization, the noise resistance is high, and the stability of the target element content measuring precision can be ensured.

In order to make the embodiments of the present disclosure more clearly understood and implemented by those skilled in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some embodiments of the present disclosure, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present specification without any creative effort shall fall within the protection scope of the present specification.

Referring to a flow chart of a target element content measurement shown in fig. 1, in order to make a person skilled in the art better understand and implement the embodiment of the present specification, the sorbite content measurement in a high-carbon steel wire rod sample is exemplified, and the execution flow comprises the following steps:

and S11, acquiring the target object.

And obtaining a high-carbon steel wire rod sample as a target object.

And S12, preprocessing the target object.

For example, the high-carbon steel wire rod sample may be pretreated, for example, the sample may be inlaid, ground, polished to obtain a metallographic ground surface, and then may be etched with a nital solution and then dried.

And S13, observing by a metallographic microscope.

After pretreatment, the high carbon steel wire rod sample may be placed under a metallographic microscope.

And S14, automatically shooting a metallographic image.

In specific implementation, monitoring points can be determined on a coil cross section sample, and a plurality of fields with the magnification of 500 times are continuously selected on the monitoring points to shoot a metallographic image. In order to improve the testing efficiency, the metallographic images of the high-carbon steel wire rod sample collected by the metallographic microscope can be automatically acquired by the automatic shooting device.

And S15, measuring the content of the target element.

In step S15, a deep regression network model may be used for measurement during the measurement process of the target element content, and in order to achieve rapid and accurate measurement of the target element content, the deep regression network model may be trained in advance, and after the training is completed, the trained deep regression network model is used for measurement of the target element content.

For better understanding and implementation by those skilled in the art, the following detailed description is made from a target element content measurement process and a training process of a deep regression network model in a target object, respectively.

Firstly, a measuring process of the content of the target element in the target object is introduced, and the measuring process of the content of the target element can be understood as a using process of a deep regression network model, namely, a trained deep regression network model is adopted to calculate a metallographic image of the content of the target element to be measured, so that the content of the target element in the metallographic image is obtained.

Referring to the flowchart of the target element content measurement method shown in fig. 2, in an embodiment of this specification, the method may specifically include the following steps:

and S21, acquiring a metallographic image of the target object acquired by the metallographic microscope.

The high carbon steel wire rod sample is still exemplified as the target object. In specific implementation, the high-carbon steel wire rod sample can be placed under a metallographic microscope, and a metallographic image of the high-carbon steel wire rod sample under the metallographic microscope is shot. In some embodiments of the present disclosure, monitoring points may be determined on a cross-sectional sample of the wire rod, and several fields of view with 500-fold magnification are successively selected at the monitoring points to capture metallographic images. In order to improve the testing efficiency, the metallographic images of the high-carbon steel wire rod sample collected by the metallographic microscope can be automatically acquired by the automatic shooting device.

In addition, before the target object is placed under a metallographic microscope, for the intercepted target object, for example, a high-carbon steel wire rod sample, pretreatment may be performed, for example, the sample may be inlaid, ground and polished to obtain a metallographic ground surface, and then the metallographic ground surface may be etched by a nital solution and then dried.

In specific implementation, the metallographic microscope may be an optical metallographic microscope, and may also be another type of metallographic microscope.

And S22, cutting the metallographic image into a plurality of image blocks according to a preset cutting size.

In specific implementation, the cutting size of the metallographic image can be preset according to the size of the texture structure of the target element in the metallographic image. The preset cutting size and the distance between the target object and the metallographic microscope, the magnification of the metallographic microscope and the like are set.

In some embodiments of the present description, the cut size is preset to 384 × 384 pixels for the sorbitic metallographic image.

And S23, inputting the image blocks into the trained depth regression network model to extract the characteristics of the image blocks, and mapping the characteristics of the image blocks into a real number fixed stage to obtain real number measurement results of the image blocks.

In some embodiments of the present description, the deep regression network model comprises: the system comprises a preset convolution neural network and a full-connection neural network, wherein the output end of the convolution neural network is connected with the input end of the full-connection neural network.

In a specific implementation, the activation functions of the convolutional neural network and the fully-connected neural network may adopt nonlinear activation functions, in some embodiments of the present specification, the activation functions of the convolutional neural network and the fully-connected neural network are ReLU activation functions, i.e., Rectified Linear units (Rectified Linear units), which can activate some neurons to increase sparsity, and when x is less than 0, the output value is 0; when x > 0, the output value is x, which can be expressed as:

f(x)＝max(x,0)；

In step S23, the image blocks may be input to a preset convolutional neural network, the features of each image block are extracted, a real number vector with the features of a preset dimension is obtained, then, the real number vector may be output to the fully-connected neural network, the features of each image block are mapped to a real number constant level, and a real number measurement result of the image blocks is obtained.

In a specific implementation, the activation function of the fully-connected neural network may employ a non-linear activation function.

And S24, weighting the real number measurement results of the image blocks to obtain the target element content of the metallographic image.

By adopting the sorbite content measuring method of the embodiment, the sorbite structure in the metallographic image does not need to be identified in advance, the whole measuring process is automatically completed based on the trained deep regression network, manual measurement is not needed, and the measuring precision does not depend on the sorbite content identifying precision, so that the stability of the measuring result can be ensured, and the sorbite content can be measured efficiently, stably and accurately.

For ease of understanding and implementation, some specific convolutional neural networks suitable for the embodiments of the present specification are given below, for example, networks such as AlexNet, VGG, inclusion, Deep Residual Network (ResNet), and the like may be used.

The AlexNet deepens the structure of the network on the basis of LeNet, and can learn richer and higher-dimensional image features.

The VGG has two structures with different network depths, namely VGG16 and VGG19, and the VGGNet uses the convolution kernel size (3 x 3) and the maximum pooling size (2 x 2) of the same size in the whole network.

The entire inclusion structure, also known as google net, is formed by connecting multiple inclusion modules in series, which can use 1 × 1 convolution for the up-and-down dimension and can perform convolution re-aggregation on multiple dimensions simultaneously. Dimensionality is raised and lowered by 1 × 1 convolution, richer features can be extracted by being able to superimpose more convolutions in the same size of the receptive field, and computational complexity can be reduced by using 1 × 1 convolution for dimensionality reduction. And convolution is carried out on a plurality of sizes simultaneously, size characteristics of different scales can be extracted, the characteristics are richer, the real number obtained by subsequent mapping is judged to be more accurate, and the convergence speed can be accelerated by decomposing a sparse matrix into a dense matrix. In addition, the convergence acceleration function can also be achieved by gathering the features with strong correlation in advance.

The structure of ResNet is a residual structure, and the principle schematic diagram of the residual structure shown in fig. 3 is expressed by the formula:

y＝F(x)+x；

where x represents the input of ResNet and y represents the output of ResNet.

The Residual structure of ResNet includes two mappings, one is Identity Mapping (Identity Mapping) and the other is Residual Mapping (Residual Mapping), where Identity Mapping refers to itself, i.e. x in the formula, and Residual Mapping refers to "difference", i.e. y-x, i.e. part f (x) in the formula.

Aiming at the phenomenon that the accuracy of a training set is reduced along with the deepening of a network, ResNet provides two selection modes, namely identity mapping and residual mapping, if the network is already optimal, the network is deepened continuously, the residual mapping approaches to 0, only the identity mapping is left, and therefore the network is in an optimal state all the time theoretically, and the performance of the network cannot be reduced along with the increase of the depth.

It will be appreciated that the above are merely some specific examples of convolutional neural networks that may be employed, and are not intended to limit the type of convolutional neural networks to which embodiments of the present specification are applicable.

Two or more types of convolutional neural networks may be used in combination, and for example, an inclusion-ResNet structure may be formed by mixing an inclusion structure and a ResNet structure. In specific implementation, the inclusion-ResNet structure can be combined in various ways, and a specific structure finally used for sorbite content measurement can be selected according to an actual training effect.

In order to make those skilled in the art better understand and implement how to implement the sorbite content measurement independent of sorbite identification by using the deep regression network model, the implementation of a specific application scenario will be described in detail below by taking a deep residual error network and a fully-connected neural network as an example in the deep regression network model.

Referring to a specific application schematic diagram of the target element content measurement system shown in fig. 4, the sorbite content measurement can be specifically performed by the following steps:

and S41, cutting the input metallographic image into image blocks with preset cutting sizes.

As a specific example, for the obtained metallographic image of the high-carbon steel wire rod sample, the metallographic image may be cut into n image blocks with the cutting size of 384 × 384 pixels from top to bottom and from left to right.

And S42, inputting the image blocks with the pixel point size of 384 multiplied by 384 obtained by cutting into the trained depth regression network model, and obtaining real number measurement results of the image blocks.

Continuing with the above specific example, inputting each image block with the size of 384 × 384 pixels into the depth residual error network, extracting the features of the n image blocks, and obtaining a real number vector with the features as a preset dimension, where in a specific implementation, the preset dimension may be equal to the number of image blocks obtained by cutting one metallographic image. Then, the real number vector may be output to a fully-connected neural network, the features of each image block are mapped to a real number fixed stage, and real number measurement results of the n image blocks are obtained and may be recorded as:

p_i,i＝1,2,3,…,n。

in order to further improve the measurement accuracy, the artificial measurement result of the target element content corresponding to the image block to be trained can be preprocessed in the training stage, and the real number measurement result of the image block can be correspondingly post-processed in the target element content measurement stage. As an alternative example, the artificial measurement result of the content of the target element corresponding to the image block to be trained may be subjected to a logarithmic transformation, for example, a base-10 logarithmic transformation. Correspondingly, a depth regression network model obtained by training the artificial measurement result of the target element content corresponding to the preprocessed image block to be trained is adopted, and the real number measurement result of each output image block is in a logarithmic form of the real measurement result of each image block. For convenience of understanding, if the real number measurement result of the image block is p, the real number measurement result of the image block output here is in a logarithmic form of p, and as shown in fig. 4, after being processed by the depth regression network model, a real number measurement result lgp in a base-10 logarithmic form is obtained, so that here, the real number measurement result of each image block is subjected to corresponding exponential transformation, for example, correspondingly subjected to base-10 exponential transformation, to obtain a real number measurement result that can be expressed in a percentage form. It is understood that, in a specific implementation, the logarithmic transformation in the preprocessing process and the exponential transformation in the post-processing process may also take other bases, for example, base e, base 2, and the like.

And S43, calculating an average value of the real number measurement results of the image blocks, and converting the calculated average value into percentage to obtain the sorbite content percentage of the metallographic image.

Continuing with the specific example above, the percentage sorbite content G may be expressed as follows:

in one embodiment of the present description, a 50-layer deep residual network ResNet50 is employed.

Referring to the network structure diagram of the deep residual network ResNet50 shown in fig. 5, ResNet50 is a residual network with 50 layers, wherein no Layer requiring training parameters, such as a Pooling Layer (Pooling Layer), participates in counting. ResNet50 includes 5 parts in total, in order: stages 1-5 may also be referred to as convolution stages with 5 different parameters. Wherein:

stage1 includes: 64 convolution kernels of 7 × 7, step size 2, 0 padding added dimension 3(7 × 7conv, 64, stride 2, pad 3); and applying a Batch Normalization and activation function ReLU (Batch Normalization + ReLU) on the input channel axis; and a maximum pooling layer of 3 × 3, step size of 2(3 × 3max pond, stride 2).

Stage2 includes: three-layer bottleneckStructural unit (b):

which contains a Bottleneck Block (bottleeck Block) and two Identity blocks (Identity Block).

In stage3, 4, 5, also comprising a plurality of three-layer bottleneck structural units,

wherein each part can comprise a bottleneck block and m-1 identity blocks, and the specific structure of each bottleneck block and identity block is shown in fig. 5. Specific parameters in the deep residual network, e.g. d₁、d₂、d₃、n₁、n₂、n₃M, etc. in the specific implementation, can be specifically determined according to the simulation experiment effect.

As can be seen from fig. 5, in an embodiment of the present disclosure, after an image block with a cut size of 384 × 384 pixels is input into the depth residual network ResNet50, a feature map with a size of 96 × 96 × 64 may be extracted through Stage1, a feature map with a size of 96 × 96 × 256 may be output through Stage2, a feature map with a size of 48 × 48 × 512 may be output through Stage3, a feature map with a size of 24 × 24 × 1024 may be output through Stage4, a feature map with a size of 12 × 12 × 2048 may be output through Stage5, and finally, a feature map with a size of 1 × 1 × 2048 may be output through a global mean pooling layer (globalsageposing).

With continued reference to the schematic diagram of the fully-connected neural network structure shown in fig. 6, the 1 × 1 × 2048 feature map output by the depth residual error network may be converted into 2048 × 1 feature vectors and input to the fully-connected neural network, and the fully-connected neural network may output the network scaling result p of each image block_iI.e. a real number measurement of the respective image block.

As a specific example, the deep residual network ResNet50 and the activation function of the fully-connected neural network both select a ReLU function, and the ReLU function is used as the activation function, so that the extracted features are richer than a linear function, and for a non-linear function, because the gradient of the non-negative interval is a constant, the problem of gradient disappearance does not exist, so that the convergence rate of the deep regression network model is maintained in a stable state.

In particular implementations, other activation functions may also be used, such as ELU functions, Leaky ReLU, and the like.

While the specific method of applying the deep residual network ResNet50 to the sorbite content measurement is described in detail above, in a specific implementation, ResNet with other depths, for example, ResNet101, may also be used, and the embodiment of the present disclosure does not set any limit to the hierarchy of the deep residual network.

In order to make the skilled person better understand and implement the embodiments of the present specification, the following describes in detail the training process of the deep regression network model in the embodiments of the present specification.

Referring to the flowchart of the training method of the deep regression network model shown in fig. 7, the training process may specifically include the following steps:

and S71, acquiring a metallographic image of the target object to be trained.

The high-carbon steel wire rod is still exemplified as the target object. In specific implementation, the metallographic image of the high-carbon steel wire rod sample can be acquired through a metallographic microscope, and the specific method can be described in step S21 in the previous embodiment.

And S72, cutting the metallographic image according to the preset cutting size to obtain a plurality of image blocks serving as image blocks to be trained.

Firstly, in order to enable the deep regression network model to complete effective training through the learning of a small number of images, the metallographic images are subjected to blocking processing before training, and a specific cutting mode can be referred to as the step S22 in the previous embodiment, and is not described in detail here.

Further, in the case of limited training samples, training sample data can be constructed based on the samples, so that the deep regression network model can be effectively trained and learned even in the case of few samples. Specifically, training sample data may be constructed in various ways, and some implementation examples are given below, where an image block to be trained may be obtained specifically in one or more of the following ways:

example one: and extracting the image blocks with the size of the preset cutting size from top to bottom and from left to right according to the preset step length from the metallographic image to serve as the image blocks to be trained.

Example two: and processing the metallographic image in an up-sampling and down-sampling mode, and cutting the metallographic image obtained after sampling according to the preset cutting size to obtain the image block to be trained.

Example three: carrying out image enhancement treatment on the metallographic image, and cutting the metallographic image subjected to the enhancement treatment according to the preset cutting size to obtain the image block to be trained

The method has the advantages that the overall or local characteristics of the image are emphasized in a targeted manner, the original unclear image is changed into clear or some interesting characteristics are emphasized, the difference between different object characteristics in the image is enlarged, the uninteresting characteristics are inhibited, the image quality can be improved, the information content is enriched, and the image interpretation and identification effects are enhanced.

In a specific embodiment, the image enhancement may be performed by a frequency domain method such as low-pass filtering or high-pass filtering, or may be performed by a spatial domain method such as local averaging or median filtering.

The noise in the image can be removed or weakened by methods such as a low-pass filtering method, a local averaging method or a median filtering method, and the like, and high-frequency signals such as edges and the like can be enhanced by a high-pass filtering method, so that a blurred image becomes clearer.

And S73, acquiring the artificial measurement results of the target element content corresponding to the image blocks to be trained, inputting the artificial measurement results of the target element content corresponding to the image blocks to be trained into the deep regression network model, and inputting the image blocks to be trained into the deep regression network model by adopting a preset training method for training until the deep regression network model reaches a preset stability performance index.

In a specific implementation, the deep regression network model may include: the device comprises a preset convolution neural network and a fully-connected neural network, wherein the output end of the depth residual error network is connected with the input end of the fully-connected neural network.

The following takes the convolutional neural network as an example to adopt a deep residual error network to illustrate the whole training process:

the multiple image blocks to be trained may be input to the deep residual error network, and the features of the image blocks to be trained are extracted to obtain a real number vector with the features as preset dimensions. And then, outputting the real number vector to the fully-connected neural network, mapping the characteristics of each image block to be trained into a real number fixed stage, and obtaining real number measurement results of the plurality of image blocks to be trained.

For an image block with a size of 384 × 384, the average diameter of the grains in the image is between 1-384, and based on this, in some embodiments of the present specification, setting the loss function of the depth regression network model comprises:

wherein P represents the manually determined sorbite content measurement result of the image block with the preset cutting size,

and e is a minimum constant greater than 0, and represents the sorbite content of the metallographic image output in the training process by adopting the deep regression network model.

For the above loss function, if P is 0, lg (P + e) is lge, and if e is 0, the logarithmic function is meaningless, and to prevent this, e is a very small constant greater than 0, and in one embodiment of the present specification, e is 10^-6。

In specific implementation, real number measurement results of the image blocks to be trained and corresponding target element content artificial measurement results are respectively input into a preset network loss function until output values of the loss function converge to a preset threshold value, and training is determined to be completed.

In the implementation, in order to minimize the network loss function efficiently and complete the training as soon as possible, some training methods capable of automatically optimizing the learning rate may be used. For example, Batch Gradient (BGD), random gradient (SGD), Momentum method (Momentum), Adaptive gradient (Adaptive Moment Estimation) method, and the like can be used.

In an embodiment of the present specification, the Adam method is adopted, and the learning rate is set to 10^-3And after about 100 rounds of training, the loss function of the deep regression network gradually converges to a preset stable state index, and the training is determined to be completed and can be put into use.

The embodiment of the specification further provides a corresponding sorbite content measuring device, a measuring system and a training device, which are respectively described in the following through specific embodiments.

In some embodiments of the present disclosure, as shown in the schematic structural diagram of the target element content measuring apparatus shown in fig. 8, the target element content measuring apparatus 80 may include:

the metallographic image acquisition unit 81 is adapted to acquire a metallographic image of a target object acquired by a metallographic microscope;

the image cutting unit 82 is suitable for cutting the metallographic image into a plurality of image blocks according to a preset cutting size;

the measuring unit 83 is adapted to input the plurality of image blocks into a trained depth regression network model to extract features of the plurality of image blocks, map the features of the plurality of image blocks into a real number fixed stage to obtain real number measurement results of the plurality of image blocks, and perform weighting processing on the real number measurement results of the plurality of image blocks to obtain the target element content of the metallographic image.

By adopting the target element content measuring device, the target element structure in the metallographic image of the target object does not need to be identified in advance, the image characteristics are extracted through the depth regression network model, the target element content in the image is obtained based on the extracted characteristics, the whole measuring process is automatically completed based on the trained depth regression network, manual measurement is not needed, the measuring precision does not depend on the identification precision of the target element content, the stability of the measuring result can be ensured, and the target element content can be measured efficiently, stably and accurately.

In a specific implementation, the deep regression network model may include: the system comprises a preset convolution neural network and a full-connection neural network, wherein the output end of the convolution neural network is connected with the input end of the full-connection neural network.

In some embodiments of the present description, the convolutional neural network comprises: the activation functions of the depth residual network and the fully-connected neural network are nonlinear activation functions, such as ReLU functions. Correspondingly, the measuring unit 83 is adapted to input the plurality of image blocks into the depth residual error network, extract the features of each image block, obtain a real number vector with the features of a preset dimension, output the real number vector to the fully-connected neural network, map the features of each image block into a real number fixed stage, and obtain real number measurement results of the plurality of image blocks.

In a specific implementation, if the artificial measurement result of the target element content corresponding to the image block to be trained is preprocessed in the training stage, the real number measurement result of the image block can be correspondingly post-processed in the target element content measurement stage. For example, if the artificial measurement result of the target element content corresponding to the image block to be trained is subjected to logarithmic transformation (as an example, logarithmic transformation with 10 as a base is performed), then, with reference to fig. 8, the measurement unit 83 is adapted to perform exponential transformation on the real number measurement results of the plurality of image blocks (as a specific example, exponential transformation with 10 as a base is performed), calculate an average value of the real number measurement results after exponential transformation, and convert the calculated average value into a percentage to obtain the target element content of the metallographic image.

The embodiment of the present disclosure further provides a training apparatus for a deep regression network model suitable for sorbite content measurement, as shown in a schematic structural diagram of the training apparatus shown in fig. 9, the training apparatus 90 may include:

a metallographic image acquisition unit 91 adapted to acquire a metallographic image of a target object to be trained;

an image cutting unit 92, adapted to cut the metallographic image into a plurality of image blocks according to a preset cutting size, as image blocks to be trained;

the training unit 93 is adapted to obtain the target element content artificial measurement results corresponding to the multiple image blocks to be trained, input the target element content artificial measurement results corresponding to the multiple image blocks to be trained into the deep regression network model, and input the image blocks to be trained into the deep regression network model by using a preset training method for training until the deep regression network model reaches a preset stability performance index.

In some embodiments of the present description, the deep regression network model comprises: the device comprises a preset depth residual error network and a fully-connected neural network, wherein the output end of the depth residual error network is connected with the input end of the fully-connected neural network, and the activation functions of the depth residual error network and the fully-connected neural network are nonlinear activation functions.

For example, for the aforementioned depth regression network model composed of a preset depth residual network and a fully connected neural network, the training unit 93 is adapted to input the multiple image blocks to be trained into the depth residual network, extract the features of the respective image blocks to be trained, and obtain a real number vector with the features as preset dimensions; and outputting the real number vector to the fully-connected neural network, mapping the characteristics of each image block to be trained into a real number fixed stage, and obtaining real number measurement results of the plurality of image blocks to be trained.

In a specific implementation, the loss function of the deep regression network model includes:

In a specific implementation, the training unit 83 is adapted to obtain target element content manual measurement results corresponding to the image blocks to be trained, and input the real number measurement results of the image blocks to be trained and the corresponding target element content manual measurement results into a preset network loss function respectively until an output value of the loss function converges to a preset threshold.

As mentioned above, in a specific implementation, the training unit 83 may perform logarithmic transformation on the artificial measurement results of the target element content of the plurality of image blocks to be trained, and then input the result into the deep regression network model for training.

Embodiments of the present disclosure also provide a target element content measurement system, as shown in fig. 10, the target element content measurement system 100 may include:

a sample acquiring device 101 adapted to acquire a target object, such as a high carbon steel wire rod sample;

a preprocessing device 102 adapted to preprocess the target object;

the metallographic image shooting device 103 is suitable for automatically shooting a metallographic image of the target object collected by a metallographic microscope after the preprocessed target object is placed under the microscope;

and the target element content measuring device 104 is suitable for measuring the content of the target element.

In a specific implementation, the target element content measuring device 104 may be implemented by using the target element content measuring device described in any one of the foregoing embodiments, and a specific implementation manner may be described with reference to the foregoing embodiments, which is not described herein again.

The present specification further provides an electronic device, such as the structural schematic diagram of the electronic device shown in fig. 11, where the electronic device 110 may include a memory 111 and a processor 112, where the memory stores computer instructions executable on the processor, and the processor executes the computer instructions to perform the steps of the method for measuring a content of a target element according to any one of the foregoing embodiments or the method for training a deep regression network model suitable for measuring a content of a target element according to any one of the foregoing embodiments.

It should be noted that the processor 112 may specifically include a CPU chip 1121 formed by one or more CPU cores, or may include a GPU chip 1122, or a chip module composed of the CPU chip 1121 and the GPU chip 1122. The processor 112 and the memory 111 may communicate with each other via a bus or the like, and the chips may communicate with each other via corresponding communication interfaces.

The present specification further provides a computer readable storage medium, on which computer instructions are stored, and the computer instructions execute the steps of the method for measuring a target element content according to any one of the foregoing embodiments or the method for training a deep regression network model suitable for measuring a target element content according to any one of the foregoing embodiments.

In particular implementations, the computer-readable storage medium may include, for example, any suitable type of memory unit, memory device, memory article, memory medium, storage device, storage article, storage medium and/or storage unit, for example, memory, removable or non-removable media, erasable or non-erasable media, writeable or re-writeable media, digital or analog media, hard disk, floppy disk, compact disk read Only memory (CD-ROM), compact disk recordable (CD-R), compact disk Rewriteable (CD-RW), optical disk, magnetic media, magneto-optical media, removable memory cards or disks, various types of Digital Versatile Disk (DVD), a tape, a cassette, or the like.

The computer instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, and the like, implemented using any suitable high-level, low-level, object-oriented, visual, compiled and/or interpreted programming language.

Specific implementation manners, operation principles, specific actions and effects of each device, system, equipment or system in the embodiments of the present invention may be referred to in the detailed descriptions of the corresponding method embodiments.

In the above specific embodiment, the measurement of the sorbite content in the high-carbon steel wire rod sample is exemplified, and it can be understood by those skilled in the art that the measurement method adopted in the embodiments of the present specification is also applicable to other types of target elements in other target objects as long as the metallographic image of the target object can be obtained.

Although the embodiments of the present invention are disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected by one skilled in the art without departing from the spirit and scope of the embodiments of the invention as defined in the appended claims.

Claims

1. A method for measuring a content of a target element, comprising:

2. The method of measuring a target element content according to claim 1, wherein the deep regression network model includes: the system comprises a preset convolution neural network and a full-connection neural network, wherein the output end of the convolution neural network is connected with the input end of the full-connection neural network.

3. The target element content measurement method according to claim 2, wherein the convolutional neural network comprises: a depth residual error network;

4. The method according to claim 3, wherein the deep residual error network comprises any one of:

5. The method of measuring target element content according to claim 4, wherein the activation functions of the deep residual network and the fully-connected neural network comprise:

f(x)＝max(x,0)；

6. The method for measuring the content of the target element according to any one of claims 1 to 5, wherein the weighting the real number measurement results of the plurality of image blocks to obtain the content of the target element of the metallographic image comprises:

7. The method of claim 6, wherein the deep regression network model is trained by:

acquiring a metallographic image of a target object to be trained;

8. The method of measuring a target element content according to claim 7, wherein the loss function of the deep regression network model includes:

9. The method for measuring the content of the target element according to claim 8, wherein the step of inputting the image block to be trained into the deep regression network model by using a preset training method for training until the deep regression network model reaches a preset stability performance index comprises the steps of:

10. The target element content measurement method according to claim 7, characterized by further comprising:

11. The method for measuring the content of target elements according to claim 10, wherein the preprocessing the manual measurement result of the content of target elements corresponding to the image block to be trained comprises: carrying out logarithmic transformation on the artificial measurement result of the content of the target element corresponding to the image block to be trained;

12. The method for measuring the content of the target element according to claim 7, wherein the step of cutting the metallographic image according to the preset cutting size to obtain a plurality of image blocks comprises at least one of the following steps as an image block to be trained:

13. A training method of a deep regression network model suitable for target element content measurement is characterized by comprising the following steps:

acquiring a metallographic image of a target object to be trained;

14. The training method of claim 13, wherein the deep regression network model comprises:

the device comprises a preset depth residual error network and a fully-connected neural network, wherein the output end of the depth residual error network is connected with the input end of the fully-connected neural network.

15. The training method according to claim 14, wherein the inputting the image block to be trained into the deep regression network model by using a preset training method for training comprises:

16. A training method as defined in claim 15, wherein the loss function of the deep regression network model comprises:

17. The training method according to claim 16, wherein the inputting the image block to be trained into the deep regression network model by using a preset training method for training until the deep regression network model reaches a preset stability performance index comprises:

18. The training method according to claim 17, wherein the inputting the image block to be trained into the deep regression network model by using a preset training method for training until the deep regression network model reaches a preset stability performance index comprises:

the learning rate is set to 10 using an adaptive moment estimation method^-3And gradually converging the loss function of the deep regression network to a preset stable state index.

19. The training method according to claim 13, wherein before inputting the image block to be trained into the deep regression network model for training, the method further comprises:

and preprocessing the manual measurement result of the content of the target element corresponding to the image block to be trained.

20. The training method according to claim 19, wherein the preprocessing of the artificial measurement result of the content of the target element corresponding to the image block to be trained comprises:

and carrying out logarithmic transformation on the artificial measurement result of the target element content corresponding to the image block to be trained.

21. The training method according to claim 13, wherein the cutting the metallographic image into a plurality of image blocks according to a preset cutting size as the image blocks to be trained comprises at least one of the following:

22. A method for measuring a content of a target element, comprising:

acquiring a target object;

preprocessing the target object;

placing the pretreated target object under a metallographic microscope;

the method for measuring the content of the target element is used for measuring the content of the target element in the metallographic image of the target object by adopting the method for measuring the content of the target element in any one of claims 1 to 12.

23. A target element content measuring apparatus, characterized by comprising:

24. The target element content measurement device according to claim 23, wherein the deep regression network model includes: the system comprises a preset convolution neural network and a full-connection neural network, wherein the output end of the convolution neural network is connected with the input end of the full-connection neural network.

25. The target element content measurement device according to claim 24, wherein the convolutional neural network comprises: the activation functions of the depth residual error network and the fully-connected neural network are nonlinear activation functions;

26. The apparatus according to claim 25, wherein the measuring unit is adapted to perform an exponential transformation on the real number measurement results of the plurality of image blocks, obtain an average value of the real number measurement results after the exponential transformation, and convert the obtained average value into a percentage to obtain the target element content of the metallographic image.

27. A training device of a deep regression network model suitable for target element content measurement is characterized by comprising:

28. The training apparatus of claim 27, wherein the deep regression network model comprises:

29. The training apparatus according to claim 28, wherein the training unit is adapted to input the plurality of image blocks to be trained into the deep residual error network, extract the features of each image block to be trained, and obtain a real vector with the features of a preset dimension; and outputting the real number vector to the fully-connected neural network, mapping the characteristics of each image block to be trained into a real number fixed stage, and obtaining real number measurement results of the plurality of image blocks to be trained.

30. The training apparatus of claim 29, wherein the loss function of the deep regression network model comprises:

31. The training apparatus as claimed in claim 30, wherein the training unit is adapted to obtain the artificial measurement results of the target element contents corresponding to the image blocks to be trained, and input the real measurement results and the artificial measurement results of the target element contents of the image blocks to be trained into a predetermined network loss function respectively until the output value of the loss function converges to a predetermined threshold.

32. A training apparatus as claimed in any one of claims 27 to 31, wherein the training unit is adapted to perform logarithmic transformation on the manual measurement results of the target elements of the image blocks to be trained, and input the result into the deep regression network model for training.

33. A target element content measurement system, comprising:

a sample acquiring device adapted to acquire a target object;

the pretreatment device is suitable for pretreating the target object;

and the target element content measuring device of any one of claims 23 to 26.

34. An electronic device comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the method of any one of claims 1 to 12 or the method of any one of claims 13 to 21.

35. A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions when executed perform the steps of the method of any one of claims 1 to 12 or the method of any one of claims 13 to 21.