KR102178238B1 - Apparatus and method of defect classification using rotating kernel based on machine-learning - Google Patents

Abstract

The present invention relates to a defect classification technique, and more particularly, to a machine learning based defect classification device using a rotating kernel for detecting and classifying defects of a product from a product input image by using a plurality of rotating kernels obtained by rotating kernels determined through learning, and a method thereof. The machine learning based defect classification device using a rotating kernel comprises a feature extraction part and a classification part.

Description

Apparatus and method of defect classification using rotating kernel based on machine-learning}

The present invention relates to a defect classification technology, and in particular, based on machine learning using a rotating kernel that detects and classifies defects in a product from an input image of a product using a plurality of rotating kernels obtained by rotating a kernel determined through learning. It relates to a defect classification apparatus and method.

The recent confrontation between AlphaGo and Lee Se-dol has greatly increased interest in artificial intelligence. In particular, research by academics and industries on deep learning, known as AlphaGo's core technology, has exploded.

Deep learning improves the performance of known problems of artificial neural networks (i.e., vanishing problems, overfitting, etc.) by developing an activation function (ReLU) and improving algorithms such as drop-out. Also, thanks to the advancement of hardware such as GPU (Graphic Processing Units) and the power of big data to learn complex structures, it is showing excellent performance in various fields recently.

These deep learning technologies are rapidly being developed by many overseas companies (Google, Facebook, Apple, Microsoft, Alibaba, Baidu) and are applied to fields such as face recognition, voice recognition, natural language processing, search services, and medical care. It is urgent to secure the latest technology of deep learning, which is rapidly developing, and to preempt the application field and commercialize it quickly.

A defect classification method used in a conventional defect inspection equipment is shown in FIG. 1.

After the algorithm developer extracts features that are likely to be classified well in an image with an image processing algorithm, these features are learned with a classifier (SVM, Decision Tree).

An optical device is constructed using lighting and a camera, and the S/N ratio of the defective area is increased by imaging the change in the amount of light entering the camera as the light path changes in the defective area. In such an image, a defect detection algorithm detects a defect candidate, and a defect is detected and classified using a feature extraction algorithm and a classification algorithm for the defect candidate image.

In this method, the limit of performance is how well a person design a feature extraction algorithm to extract features. In addition, there is a problem in that the classification performance is limited according to the selected feature, and the classification performance is changed according to rotation, brightness change, size change, etc. Since the characteristics of the video are different for each product, there is a disadvantage that it takes a lot of time to analyze and develop it.

Among the latest technology, deep learning, the classification algorithm applying the CNN method is a method in which artificial intelligence extracts and learns features by itself from images. The above problem can be solved by constructing a defect classification algorithm by applying deep learning technology. Among various deep learning structures, a structure called CNN (Convolutional Neural Network) is used for deep learning in the image field.

Image classification using CNN has the feature that CNN itself extracts and learns features that can improve classification performance. If these features are applied to the field of image-based defect detection, a remarkable performance improvement can be expected.

The configuration of a classifier using general deep learning is shown in FIG. 2.

After applying a convolution layer and an activation function to the input image, the feature map is reduced in size through the pooling layer and then transferred to the next convolution layer. do.

These basic structures are repeatedly deeply stacked to effectively extract features for defect classification from images. Pooling has the advantage of reducing the size of the feature map and reducing the amount of computation, and has the advantage of being able to widely include features used for classification by increasing the receptive field.

In order to improve the performance of defect classification using deep learning in the related art, layers need to be deeply stacked, and the deeper the layer is, the more difficult it is to learn, so it is difficult to secure defect detection (surface defect inspection) performance.

Therefore, in order to deepen the layer, a sufficient amount of big data to train the model is essential. However, it is practically difficult to secure data in all cases due to the nature of the manufacturing industry.

In general, various methods (for example, flipping, rotation, shifting, noise addition, blurring, shaping, etc.) are used for data augmentation, but there are limitations when using these methods in the actual manufacturing industry.

In particular, in the case of a product with a texture in a specific direction, it is difficult to secure performance with data obtained by rotating the entire image. In addition, even if the entire image is rotated, data cannot be obtained if the entire defect is not rotated and only a part of the defect is rotated.

Therefore, it is necessary to develop a new deep learning-based defect classification method that can secure excellent defect classification performance even with a small amount of data.

U.S. Patent No. 985303

The present invention was invented to solve the above problems, and an object of the present invention is to secure excellent defect classification performance with only a small amount of data.

To this end, the machine learning-based defect classification apparatus according to the present invention includes a feature extraction unit that extracts a feature map from an input image of a product using a plurality of rotation kernels obtained by rotating a kernel determined through learning, and the extracted feature map. It includes a classification unit for classifying defects in the product by analyzing it.

In addition, the machine learning-based defect classification method according to the present invention includes the steps of extracting a feature map from an input image of a product using a plurality of rotation kernels obtained by rotating a kernel determined through learning, and analyzing the extracted feature map. Classifying product defects.

In addition, the program for performing the machine learning-based defect classification method according to the present invention is stored in a computer-readable recording medium, and generates a plurality of rotation kernels by rotating the kernel determined through learning, and using the plurality of rotation kernels. And extracting a multi-feature map from the input image of the product, generating a feature vector by performing a global average pooling on the extracted multi-feature map, and analyzing the feature vector using a neural network of a fully connected layer It characterized in that performing the step of classifying the defects.

As described above, the present invention has the effect of remarkably improving the performance of the classifier even if the learning data for defect classification is not sufficiently secured.

That is, the present invention can improve defect classification performance without using rotated learning data by using a deep learning classification algorithm using a rotating convolution filter instead of a classification method using a conventional convolutional filter (kernel) operation.

1 is a view for explaining a conventional defect classification method.
2 is a diagram for explaining a general deep learning-based classification method.
3 is a diagram showing a schematic configuration of a machine learning-based defect classification apparatus using a rotating kernel according to the present invention
4 is a detailed configuration diagram of a machine learning-based defect classification apparatus using a rotating kernel according to the present invention.
5 is a diagram for explaining convolution and polling in a machine learning-based defect classification method using a rotating kernel according to the present invention.
6 is a flow chart showing a machine learning-based defect classification method using a rotating kernel according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configuration of the present invention and its effect will be clearly understood through the detailed description below.

3 shows a schematic configuration of a machine learning-based defect classification apparatus using a rotating kernel according to the present invention.

Referring to FIG. 3, a machine learning-based defect classification apparatus using a rotating kernel (hereinafter, a defect classification apparatus) has a feature extraction unit 10 and a classification unit 20 as main components.

The feature extraction unit 10 and the classification unit 20 constituting the defect classification apparatus may be implemented in software or hardware. The defect classification device applies a model that performs defect detection and classification of products, and this model is a model learned based on machine learning.

The defect classification apparatus according to the present invention can be used not only for surface defect inspection of a product, but also for diagnostic fields such as non-destructive inspection. In addition, as an essential technology for implementing a smart factory, it can be used to determine the quality of a product according to production operating conditions.

The feature extractor 10 extracts a feature map from an input image of a product using a plurality of rotating kernels obtained by rotating a kernel determined through learning.

The feature extraction unit 10 generates a multi-feature map in which the size of an input image is reduced by repeating a convolution process of extracting a multi-feature map using a plurality of rotation kernels and a pooling process of reducing the size of the multi-feature map.

The classification unit 20 classifies defects of the product by analyzing the feature map output from the feature extraction unit 10.

The classification unit 20 performs global average pooling on the feature map output from the feature extraction unit 10, and then performs product defects through a neural network of a fully connected layer. Classify.

4 shows in detail the internal configuration of the defect classification apparatus according to the present invention.

Referring to FIG. 4, the feature extraction unit 10 includes a plurality of convolution layers and a pooling layer.

In the convolutional layer, a feature map is generated using a rotation kernel, and in the pooling layer, the size of the feature map is reduced. In the present invention, the size of the feature map can be reduced by using max pooling.

The feature extraction unit 10 generates multiple feature maps while performing the convolution and pooling process several times.

5 is a diagram illustrating a convolution and pooling process.

Referring to FIG. 5, in the convolutional layer, an input image (m × m × k) is first convolved with n kernels (h × h × k).

The n kernels are kernels obtained by rotating kernels determined by deep learning learning in various directions.

If such a rotation kernel is used, it is possible to obtain various rotated feature maps even with a small amount of training data by replacing a large amount of data obtained by rotating training data in a conventional model.

When the input image and n kernels are convolved, n multiple feature maps (m'×m'×k) are generated. That is, each rotation kernel selects the maximum value from the input image and the convolved value, and constructs a feature map with the maximum value.

The n multiple feature maps go through an activation function (eg, ReLU). Since the activation function is independently applied for each element of the feature map, the feature map maintains the size of m'×m'×k.

The feature map that has passed the activation function (τ) is pooled in the pooling layer, reducing the size of the feature map, and outputting it as a feature map of size m″×m″×k.

As such convolution and pooling are repeatedly performed several times, a final feature map is output.

The multi-feature map finally output from the feature extraction unit 10 is input to the classification unit 20 and analyzed. First, the global average pooling is performed in the global average pooling layer (GAP) 26 Multiple feature maps are output as feature vectors of a certain size.

The feature vector generated through the global average pooling layer is connected to a fully connected layer (FC) having the same size as the number of defect types to be classified, and finally the defect classification result through an activation function (softmax). Is displayed. That is, defect classification of the product is performed while the element value of the feature vector passes through the neural network of the fully connected layer.

6 shows a defect classification method according to the present invention.

Referring to FIG. 6, first, a learning step (S10) for learning a defect classification apparatus according to the present invention based on machine learning is performed.

In the learning step (S10), the deep learning model is trained using the product image tagged with the defect type as training data. That is, while training the deep learning model, the filter (kernel) value of the convolutional neural network (CNN) and the weight of the neural network (Deep Multi-Layer Perceptron) are determined. In this way, through the learning step (S10), a plurality of rotation kernels according to the present invention may be obtained.

In the feature map extraction step S20, a feature map is extracted from an input image using a plurality of rotation kernels obtained in the learning step S10. That is, multiple feature maps are extracted by repeating a convolution process for extracting a feature map using a plurality of rotation kernels and a pooling process for reducing a size of the feature map.

In the classification step S30, defects of the product are classified by analyzing multiple feature maps. That is, after performing global average pooling on the extracted multiple feature maps, defects of the product are classified through a neural network of a fully connected layer.

In order to verify the performance of the defect classification method according to the present invention, an experiment was performed using a computer (PC) having an Intel i7-7700k CPU, Titan XP GPU, and 32G memory. For learning, the batch size was 128, the learning rate was 0.01, and gradient descent was used as an optimization tool. The weight decay rate for normalization was set to 0.0001.

The data set used in this experiment is (MNIST, MNIST-rot36). This data set has been used to verify the performance verification of a model of rotation invariant properties proposed in the past.

(MNIST, MNIST-rot36) The data set consists of the basic MNIST, and the test set consists of the MNIST test set constantly rotated in the 360 direction.

Table 1 shows the defect classification error rate of the defect classification method according to the present invention and the conventional deep learning method.

Referring to Table 1, in the conventional deep learning method, the error rate is 57.15%, but it can be seen that the error rate is reduced to 16.67% when the method of the present invention is used.

system Error(%) Baseline (CNN) 57.15% Rotary kernel method 15.67%

That is, it can be seen that there is an effect of improving defect classification performance by using a deep learning classification algorithm using a rotating convolutional filter rather than a conventional method.

The above description is only illustrative of the present invention, and various modifications may be made without departing from the technical spirit of the present invention by those of ordinary skill in the technical field to which the present invention belongs.

Therefore, the embodiments disclosed in the specification of the present invention do not limit the present invention. The scope of the present invention should be interpreted by the following claims, and all technologies within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

10: feature extraction unit 20: classification unit

Claims (8)

Using a plurality of rotation kernels obtained by rotating the kernel determined through learning by 45 degrees, a feature map for a plurality of input images generated by rotating an input image for a product and a corresponding multi-feature map is extracted from a single input image for a product. A feature extraction unit that
Machine learning-based defect classification apparatus using a rotating kernel including a classification unit for classifying product defects by analyzing the extracted multiple feature maps. The method of claim 1,
The feature extracting unit repeats a convolution process for extracting a multi-feature map using a plurality of rotation kernels and a pulling process for reducing a size of the multi-feature map. The method of claim 1,
The classification unit classifies defects of a product through a neural network of a fully connected layer after performing global average pooling on the multiple feature maps output from the feature extraction unit. Using a plurality of rotation kernels obtained by rotating the kernel determined through learning by 45 degrees, a feature map for a plurality of input images generated by rotating an input image for a product and a corresponding multi-feature map is extracted from a single input image for a product. And the steps to do,
Machine learning-based defect classification method using a rotating kernel comprising the step of classifying defects of a product by analyzing the extracted multiple feature maps. The method of claim 4,
In the extracting of the multi-feature map, a convolution process of extracting a multi-feature map using a plurality of rotation kernels and a pooling process of reducing the size of the multi-feature map are repeated. Classification method. The method of claim 4,
In the step of classifying the defects, the defect classification method based on machine learning using a rotating kernel, characterized in that after performing global average pooling on the extracted multiple feature maps, defects of a product are classified through a neural network of a fully connected layer. Generating a plurality of rotation kernels by rotating the kernel determined through learning by 45 degrees,
Extracting a feature map for a plurality of input images generated by rotating an input image from one input image for a product using the plurality of rotation kernels and a multi-feature map corresponding to the plurality of input images;
Generating feature vectors by performing global average pooling on the extracted multiple feature maps; and
A computer-readable recording medium storing a program for performing the step of classifying defects of a product by analyzing the feature vector using a neural network of a fully connected layer. The method of claim 7,
In the step of extracting the multi-feature map, each rotation kernel (h × h × k) selects a maximum value from the input image (m × m × k) and the convolved value to generate a feature map. Stored computer-readable recording medium.

Images