KR102246085B1 - Anomaly detection - Google Patents

Abstract

A computer program stored in a computer-readable storage medium according to an embodiment of the present disclosure is disclosed. When the computer program is executed on one or more processors of a computing device, the following operations are performed for checking input data, and the operations include: deriving a first test result for input data; Classifying the input data using a classification model learned using training data labeled with cluster information; And outputting a final inspection result based at least in part on a classification result of the input data of the classification model.

Description

Anomaly detection {ANOMALY DETECTION}

The present disclosure relates to data processing using a computing device, and more specifically, to data processing using artificial intelligence implemented using a computing device.

In order to solve complex or unknown problems, research is being conducted to apply human recognition methods to devices. One of these studies is a neural network modeling human biological neurons. Neural networks use algorithms that mimic human learning capabilities. The neural network may perform mapping between input patterns and output patterns through learning. In addition, the neural network may have a generalization capability capable of generating a relatively correct output for an input pattern that has not been used for learning based on the learned result.

The present disclosure is intended to provide a method of processing data using a computing device.

A computer program stored in a computer-readable storage medium is disclosed according to an embodiment of the present disclosure for realizing the above-described problem. When the computer program is executed on one or more processors of a computing device, the following operations are performed for checking input data, and the operations include: deriving a first test result for input data; Classifying the input data using a classification model learned using training data labeled with cluster information; And outputting a final inspection result based at least in part on a classification result of the input data of the classification model.

Alternatively, the classification model is trained using a training data set including a plurality of training data subsets, classifying similar data into the same cluster from training data included in the training data set, and classifying dissimilar data. It can be learned to classify into different clusters.

Alternatively, the operation of deriving the first test result for the input data may include processing the input data using a pre-learned test model including one or more network functions, thereby providing an anomaly detection for the input data. Or performing an anomaly detection on the input data based on comparison of the input data and reference data.

Alternatively, the classification model may be trained using a training data set including a training data subset including training data labeled with different cluster information.

Alternatively, the classification model may be trained using a training data set including a subset of training data including target data, target-like data, and target dissimilar data.

Alternatively, the target data and the target similarity data may be data labeled with first cluster information, and the target dissimilar data may be data labeled with second cluster information.

Alternatively, when the target data is data including anomaly, the target-like data is data including an anomaly of a type similar to the anomaly included in the target data, and the target dissimilar data includes anomaly. It may be data that doesn't.

Alternatively, when the target data is an image including anomalies, the target-like data is an image that is cropped to include at least some of the anomalies included in the target data, and the target dissimilar data is annotated. It may be an image that does not contain Mali.

Alternatively, the target data may be an image including the anomaly in the center of the image.

Alternatively, the target dissimilar data may be an image in which at least a part of the target data except for an anomaly is overlapped.

Alternatively, the classification model may be trained to classify the target data and the target similarity data into the same cluster, and classify the target dissimilar data into a cluster different from the cluster to which the target data and the target similarity data belong. have.

Alternatively, the target data may include an image related to an abnormal situation that may occur separately from the target object, and the target similar data may include an image including at least a part of the target data.

Alternatively, the target data and the target-like data may be data including false positives.

Alternatively, the operation of outputting a final inspection result based at least in part on the classification result of the input data of the classification model is, when the classification model classifies the input data as belonging to a specific cluster, the specific cluster Generates the test result matched to the second test result, outputs the final test result based on the second test result, and outputs the first test result as the final test result when the classification of the input data fails. It may include an operation to do.

Alternatively, the secondary test result may include information related to at least one of an anomaly presence, an anomaly location, an anomaly type, and an erroneous sword type.

Alternatively, the operation of classifying the input data using a classification model learned using training data labeled with cluster information may include processing the input data using the pre-trained classification model, Mapping a feature to a solution space of the pre-learned classification model; And classifying the input data based on whether the input data belongs to one of one or more clusters in the solution space based on a position of the input data in the solution space.

Alternatively, the solution space is composed of a space of one or more dimensions, and includes one or more clusters, and each cluster is a feature based on each target data and a position on the solution space of the feature based on target-like data. It can be configured on the basis of.

An input data inspection method performed in a computing device including one or more processors according to another embodiment of the present disclosure for realizing the above-described problem is disclosed. The method includes the steps of deriving a first inspection result for input data; Classifying the input data using a classification model learned using training data labeled with cluster information; And outputting a final inspection result based at least in part on a classification result of the input data of the classification model.

A computing device is disclosed according to another embodiment of the present disclosure for realizing the above-described problem. The computing device may include one or more processors; And a memory for storing instructions executable in the processor, wherein the processor derives a result of a first check for input data, and uses the input data as a classification model learned using training data labeled with cluster information. The classification may be performed using the classification, and a final inspection result may be output based at least in part on the classification result of the input data of the classification model.

The present disclosure may provide a method of processing data using a computing device.

1 is a diagram illustrating a block diagram of a computing device that performs an input data inspection method according to an embodiment of the present disclosure.
2A is a schematic diagram illustrating a network function according to an embodiment of the present disclosure.
2B is a schematic diagram illustrating a convolutional neural network according to an embodiment of the present disclosure.
3a, 3b, 3c, 3d, 3e, and 3f are schematic diagrams showing training data according to an embodiment of the present disclosure.
4 is a schematic diagram illustrating a method of training a classification model according to an embodiment of the present disclosure.
5 is a schematic diagram showing a solution space of a classification model according to an embodiment of the present disclosure.
6 is a flowchart of a method for checking input data according to an embodiment of the present disclosure.
7 is a block diagram illustrating a means for implementing a method for checking input data according to an embodiment of the present disclosure.
8 is a block diagram illustrating a module for implementing a method for checking input data according to an embodiment of the present disclosure.
9 is a block diagram illustrating logic for implementing a method for checking input data according to an embodiment of the present disclosure.
10 is a block diagram illustrating a circuit for implementing a method for checking input data according to an embodiment of the present disclosure.
11 shows a simplified and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

Various embodiments are now described with reference to the drawings. In this specification, various descriptions are presented to provide an understanding of the present disclosure. However, it is clear that these embodiments may be implemented without this specific description.

As used herein, the terms "component", "module", "system" and the like refer to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or execution of software. For example, a component may be, but is not limited to, a process executed on a processor, a processor, an object, a thread of execution, a program, and/or a computer. For example, both an application running on a computing device and a computing device may be components. One or more components may reside within a processor and/or thread of execution. A component can be localized within a single computer. A component can be distributed between two or more computers. In addition, these components can execute from a variety of computer readable media having various data structures stored therein. Components can be, for example, through a signal with one or more data packets (e.g., data from one component interacting with another component in a local system, a distributed system, and/or a signal through another system and a network such as the Internet. Depending on the data being transmitted), it may communicate via local and/or remote processes.

In addition, the term “or” is intended to mean an inclusive “or” and not an exclusive “or”. That is, unless specified otherwise or is not clear from the context, "X employs A or B" is intended to mean one of the natural inclusive substitutions. That is, X uses A; X uses B; Or when X uses both A and B, "X uses A or B" can be applied to either of these cases. In addition, the term “and/or” as used herein should be understood to refer to and include all possible combinations of one or more of the listed related items.

In addition, the terms "comprises" and/or "comprising" should be understood to mean that the corresponding features and/or components are present. However, it is to be understood that the terms "comprising" and/or "comprising" do not exclude the presence or addition of one or more other features, elements, and/or groups thereof. In addition, unless otherwise specified or when the context is not clear to indicate a singular form, the singular in the specification and claims should be interpreted as meaning "one or more" in general.

Those of skill in the art would further describe the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein, including electronic hardware, computer software, or a combination of both. It should be recognized that it can be implemented as To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented in hardware or as software depends on the specific application and design restrictions imposed on the overall system. Skilled technicians can implement the described functionality in various ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The description of the presented embodiments is provided so that a person having ordinary knowledge in the technical field of the present disclosure can use or implement the present invention. Various modifications to these embodiments will be apparent to those of ordinary skill in the art. The general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present invention is not limited to the embodiments presented herein. The invention is to be accorded the widest scope consistent with the principles and novel features presented herein.

In the present disclosure, a network function, an artificial neural network, and a neural network may be used interchangeably.

1 is a diagram illustrating a block diagram of a computing device that performs an input data inspection method according to an embodiment of the present disclosure.

The configuration of the computing device 100 illustrated in FIG. 1 is only a simplified example, and in an embodiment of the present disclosure, the computing device 100 may include other components for performing the embodiments of the present disclosure.

The computing device 100 may include a processor 110, a memory 120, a communication module 130, and a camera module 140.

The processor 110 may be composed of one or more cores, a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. unit) may include a processor for data analysis and deep learning. The processor 110 may read a computer program stored in the memory 120 to perform an input data inspection method according to an exemplary embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 110 may perform calculations for learning a neural network. The processor 110 processes input data for learning in deep learning (DN), extracts features from the input data, calculates errors, and updates the weights of the neural network using backpropagation. You can perform calculations for learning. At least one of the CPU, GPGPU, and TPU of the processor 110 may process learning of the network function. For example, the CPU and GPGPU can work together to learn network functions and classify data using network functions. In addition, in an embodiment of the present disclosure, learning of a network function and data classification using a network function may be processed by using processors of a plurality of computing devices together. In addition, the computer program executed in the computing device according to an embodiment of the present disclosure may be a CPU, a GPGPU, or a TPU executable program.

In an embodiment of the present disclosure, the computing device 100 may distribute and process network functions using at least one of a CPU, a GPGPU, and a TPU. In addition, in an embodiment of the present disclosure, the computing device 100 may distribute and process network functions together with other computing devices.

In an embodiment of the present disclosure, an image processed using a network function is an image stored in a storage medium of the computing device 100, an image captured by the camera module 140 of the computing device 100, and/or a communication module ( 130) may be an image transmitted from another computing device such as an image database. In addition, in an embodiment of the present disclosure, an image processed using a network function may be an image stored in a computer-readable storage medium (eg, a flash memory may be included, but the present disclosure is not limited thereto). The computing device 100 may receive an image file stored in a computer-readable storage medium through an input/output interface (not shown).

In an embodiment of the present disclosure, the computing device 100 may receive image data and inspect the input data using a pre-learned network function. The input data inspection of the present disclosure may include an anomaly detection to determine whether an anomaly exists in the input data. Inspecting the input data of the present disclosure may include an anomaly detection and an erroneous determination of an anomaly detection.

The memory 120 may store a computer program for performing an input data inspection method according to an embodiment of the present disclosure, and the stored computer program may be read and driven by the processor 110.

The communication module 130 may transmit/receive data for performing an anomaly determination method of image data according to an embodiment of the present disclosure, such as other computing devices, servers, and the like. The communication module 130 may transmit/receive data, such as image data, necessary for an embodiment of the present disclosure, such as other computing devices, servers, and the like. For example, the communication module 130 may receive training image data from a training image database or the like. In addition, the communication module 130 may enable communication between a plurality of computing devices so that learning of a network function is distributedly performed in each of the plurality of computing devices. The communication module 130 may enable communication between a plurality of computing devices to perform distributed processing of data classification using a network function.

The camera module 140 may generate image data by photographing an object to be inspected in order to perform the method of determining an anomaly of image data according to an exemplary embodiment of the present disclosure. In an embodiment of the present disclosure, the computing device 100 may include one or more camera modules 140.

2A is a schematic diagram illustrating a network function 200 according to an embodiment of the present disclosure.

Throughout this specification, a computational model, a neural network, a network function, and a neural network may be used with the same meaning. A neural network can be made up of a set of interconnected computational units, which can generally be referred to as “nodes”. These “nodes” may also be referred to as “neurons”. The neural network is composed of at least one or more nodes. The nodes (or neurons) that make up neural networks can be interconnected by one or more “links”.

In a neural network, one or more nodes connected through a link may relatively form a relationship between an input node and an output node. The concepts of an input node and an output node are relative, and any node in an output node relationship with respect to one node may have an input node relationship in a relationship with another node, and vice versa. As described above, the input node to output node relationship can be created around the link. One or more output nodes may be connected to one input node through a link, and vice versa.

In the relationship between the input node and the output node connected through one link, the value of the output node may be determined based on data input to the input node. Here, a node interconnecting the input node and the output node may have a weight. The weight may be variable, and in order for the neural network to perform a desired function, it may be changed by a user or an algorithm. For example, when one or more input nodes are interconnected to one output node by respective links, the output node includes values input to input nodes connected to the output node and a link corresponding to each input node. The output node value may be determined based on the weight.

As described above, in a neural network, one or more nodes are interconnected through one or more links to form an input node and an output node relationship in the neural network. The characteristics of the neural network may be determined according to the number of nodes and links in the neural network, the association relationship between the nodes and the links, and a weight value assigned to each of the links. For example, when there are the same number of nodes and links, and two neural networks having different weight values between the links exist, the two neural networks may be recognized as being different from each other.

A neural network may be configured including one or more nodes. Some of the nodes constituting the neural network may configure one layer based on the distances from the initial input node. For example, a set of nodes with a distance n from the initial input node, n layers can be configured. The distance from the first input node may be defined by the minimum number of links that must be traversed to reach the node from the first input node. However, the definition of such a layer is arbitrary for explanation, and the order of the layer in the neural network may be defined in a different way than that described above. For example, the layer of nodes may be defined by the distance from the last output node.

The first input node may mean one or more nodes to which data is directly input without going through a link in a relationship with other nodes among nodes in the neural network. Alternatively, in a relationship between nodes based on a link in a neural network network, it may mean nodes that do not have other input nodes connected by a link. Similarly, the final output node may mean one or more nodes that do not have an output node in a relationship with other nodes among nodes in the neural network. In addition, the hidden node may refer to nodes constituting a neural network other than the first input node and the last output node. In the neural network according to an embodiment of the present disclosure, the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as the input layer proceeds to the hidden layer. I can. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes of an input layer is less than the number of nodes of an output layer, and the number of nodes decreases as the number of nodes progresses from the input layer to the hidden layer. have. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes of an input layer is greater than the number of nodes of an output layer, and the number of nodes increases as the number of nodes progresses from the input layer to the hidden layer. I can. The neural network according to another embodiment of the present disclosure may be a neural network in the form of a combination of the aforementioned neural networks.

A deep neural network (DNN) may mean a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to identify potential structures in data. In other words, it is possible to understand the potential structures of photos, texts, videos, voices, and music (for example, what objects are in photos, what are the content and emotions of the text, what are the contents and emotions of the voice, etc.). . Deep neural networks include convolutional neural networks (CNNs), recurrentneural networks (RNNs), auto encoders, Generative Adversarial Networks (GANs), and restricted boltzmann machines (RBMs). machine), deep belief network (DBN), Q network, U network, and Siam network. The description of the above-described deep neural network is only an example, and the present disclosure is not limited thereto.

In an embodiment of the present disclosure, the network function 200 may include an autoencoder. The auto encoder may be a kind of artificial neural network for outputting output data similar to input data. The auto encoder may include at least one hidden layer, and an odd number of hidden layers may be disposed between input/output layers. The number of nodes of each layer may be reduced from the number of nodes of the input layer to an intermediate layer called a bottleneck layer (encoding), and then reduced and expanded symmetrically from the bottleneck layer to an output layer (symmetric with the input layer). In this case, in the example of FIG. 2, the dimensional reduction layer and the dimensional restoration layer are illustrated as being symmetrical, but the present disclosure is not limited thereto, and nodes of the dimensional reduction layer and the dimensional restoration layer may or may not be symmetric. Auto encoders can perform nonlinear dimensionality reduction. The number of input layers and output layers may correspond to the number of sensors remaining after pre-processing of input data. In the auto-encoder structure, the number of nodes of the hidden layer included in the encoder may decrease as the distance from the input layer increases. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) is too small, a sufficient amount of information may not be delivered, so more than a certain number (e.g., more than half of the input layer, etc.) ) May be maintained.

The neural network may be learned in at least one of supervised learning, unsupervised learning, and semi supervised learning. Learning of neural networks is to minimize output errors. In learning of a neural network, iteratively inputs training data to the neural network, calculates the output of the neural network for the training data and the error of the target, and reduces the error from the output layer of the neural network to the input layer. This is a process of updating the weight of each node of the neural network by backpropagation in the direction. In the case of teacher learning, learning data in which the correct answer is labeled in each learning data is used (ie, labeled learning data), and in the case of non-satellite learning, the correct answer may not be labeled in each learning data. That is, for example, in the case of teacher learning related to data classification, the learning data may be data in which a category is labeled with each learning data. Labeled training data is input to a neural network, and an error may be calculated by comparing an output (category) of the neural network with a label of the training data. As another example, in the case of comparative history learning regarding data classification, an error may be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated in the neural network in the reverse direction (ie, from the output layer to the input layer), and the connection weight of each node of each layer of the neural network may be updated according to the backpropagation. The amount of change in the connection weight of each node to be updated may be determined according to a learning rate. The computation of the neural network for the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate may be applied differently according to the number of repetitions of the learning cycle of the neural network. For example, a high learning rate is used in the early stages of training of a neural network, so that the neural network quickly secures a certain level of performance to increase efficiency, and a low learning rate can be used in the later stages of training to increase accuracy.

In learning of a neural network, in general, training data may be a subset of actual data (that is, data to be processed using a trained neural network), and thus, errors in the training data decrease, but errors in the actual data are reduced. There may be increasing learning cycles. Overfitting is a phenomenon in which errors in actual data increase due to over-learning on learning data. For example, a neural network learning a cat by showing a yellow cat may not recognize that it is a cat when it sees a cat other than yellow, which may be a kind of overfitting. Overfitting can cause an increase in errors in machine learning algorithms. Various optimization methods can be used to prevent such overfitting. In order to prevent overfitting, methods such as increasing training data, regulation, or dropout in which some nodes of the network are omitted during the training process may be applied.

The convolutional neural network (CNN) shown in FIG. 2B is a type of deep neural network and includes a neural network including a convolutional layer. Convolutional neural networks are a type of multilayer perceptorns designed to use minimal preprocessing. The CNN may consist of one or several convolutional layers and artificial neural network layers combined therewith, and may additionally utilize weights and pooling layers. Thanks to this structure, the CNN can fully utilize the input data of the two-dimensional structure. A convolutional neural network can be used to recognize an object in an image. The convolutional neural network can process image data by representing it as a matrix having dimensions. For example, in the case of red-green-blue (RGB) encoded image data, each R, G, and B color may be represented as a two-dimensional (for example, a two-dimensional image) matrix. That is, the color value of each pixel of the image data may be a component of the matrix, and the size of the matrix may be the same as the size of the image. Therefore, image data can be represented by three two-dimensional matrices (three-dimensional data arrays).

The convolutional process (input/output of the convolutional layer) can be performed by multiplying the convolutional filter and matrix components at each position of the image while moving the convolutional filter in the convolutional neural network. The convolutional filter may be composed of an n*n type matrix, and may generally be composed of a fixed type filter that is smaller than the total number of pixels in the image. That is, in the case of inputting an m*m image into a convolutional layer (for example, a convolutional layer having a convolutional filter size of n*n), a matrix representing n*n pixels including each pixel of the image This convolutional filter can be multiplied by components (that is, the product of each component of the matrix). Components matching the convolutional filter may be extracted from the image by multiplying with the convolutional filter. For example, a 3*3 convolutional filter for extracting the upper and lower linear components from an image is composed of [[0,1,0],[0,1,0],[0,1,0]]. When such a convolutional filter is applied to an input image, upper and lower linear components matching the convolutional filter may be extracted and output from the image. The convolutional layer may apply a convolutional filter to each matrix for each channel representing an image (ie, R, G, and B colors in the case of R, G, and B coded images). The convolutional layer may extract features matching the convolutional filter from the input image by applying a convolutional filter to the input image. The filter value of the convolutional filter (that is, the value of each component of the matrix) may be updated by back propagation in the learning process of the convolutional neural network.

A subsampling layer is connected to the output of the convolutional layer to simplify the output of the convolutional layer, thereby reducing the amount of memory and computation. For example, if the output of the convolutional layer is input to a pooling layer having a 2*2 max pooling filter, the image is compressed by outputting the maximum value included in each patch for each 2*2 patch from each pixel of the image. I can. The above-described pooling may be a method of outputting a minimum value from a patch or an average value of a patch, and any pooling method may be included in the present disclosure.

The convolutional neural network may include one or more convolutional layers and subsampling layers. The convolutional neural network may extract a feature from an image by repeatedly performing a convolutional process and a subsampling process (eg, max pooling described above). Through the iterative convolutional process and subsampling process, the neural network can extract global features of the image.

The output of the convolutional layer or the subsampling layer may be input to a fully connected layer. The fully connected layer is a layer where all neurons in one layer and all neurons in neighboring layers are connected. The fully connected layer may mean a structure in which all nodes of each layer are connected to all nodes of another layer in a neural network.

In an embodiment of the present disclosure, in order to perform segmentation of image data, the neural network may include a deconvolutional neural network (DCNN). The deconvolutional neural network performs an operation similar to that of calculating the convolutional neural network in the reverse direction, and may output a feature extracted from the convolutional neural network as a feature map related to original data.

In an embodiment of the present disclosure, the processor 110 may derive a result of a first check for input data. The first test result may include anomaly information related to the input data, and the anomaly information may include any information related to an anomaly that may exist in the input data. For example, the first test result may include whether an anomaly exists, a position of an anomaly in input data, and a type of anomaly. If the input data is an image, the result of the first check is whether an anomaly exists in the input data, information on which pixel of the input data is an area where an anomaly exists, and what type of anomaly is included in the input data. It may include information related to whether it is anomaly (for example, in the case of an image related to a semiconductor, a soldering defect, a wire defect, a device defect, etc.).

The processor 110 may derive a result of a first check regarding whether an anomaly exists in the input data. The processor 110 processes the input data using a pre-learned test model including one or more network functions, thereby performing an anomaly detection on the input data to derive the first test result. The test model is trained only with normal data, and by determining whether a new pattern (i.e., anomaly) that has not been learned exists in the input data, it is possible to determine whether or not an anomaly exists in the input data. supervised learning) model. In addition, the test model may be a supervised learning model capable of determining whether an anomaly is included in the input data by learning from the labeled normal data and an anomaly data. The test model may include an arbitrary machine learning model capable of determining whether an anomaly exists in the input data.

Also, the processor 110 may derive a first inspection result by performing an anomaly detection on the input data based on the comparison between the input data and the reference data. When the input data is an image, the processor 110 may determine whether an anomaly exists in the input data based on a comparison between reference data including an anomaly or reference data not including an anomaly and input data. The processor 110 may determine an anomaly in the input data based on any machine vision solution.

The processor 110 may classify the input data using a classification model learned using training data labeled with cluster information. The classification model is trained using a training data set that includes a plurality of training data subsets, and is trained to classify similar data into the same cluster from the training data included in the training data set, and classify dissimilar data into different clusters. I can. The training data subset may include a plurality of training data. The training data subset may include a plurality of training data, and each training data may be labeled with cluster information. In the present disclosure, the training data subset may be data used in one iteration in learning of a network function. In the present disclosure, the training data set may mean all data used in one epoch in learning of a network function. The training data subset may include training data that are dissimilar to each other, and training data that are dissimilar to each other included in the training data subset may be classified into different clusters by a classification model.

That is, the training data subset may include a plurality of training data, each of the training data may be labeled with cluster information, and the training data subset may include training data labeled with different cluster information. More specifically, the training data subset may include target data, target-like data, and target dissimilar data, the target data and target-like data may be labeled with the same cluster information, and the target dissimilar data may be the target data and the target dissimilar data. Cluster information different from the target-like data may be labeled. The target data and target-like data included in the training data subset may be labeled with first cluster information, and the target dissimilar data may be labeled with second cluster information.

For example, when the target data is data including anomaly, the target similar data may be data including an anomaly, and the target dissimilar data may be data not including an anomaly. The target dissimilar data may be data in which at least a part of the target data except for an anomaly is duplicated. For example, when the target data is an image, the target dissimilar data may be an image including at least a part of a portion excluding an anomaly from an image including an anomaly. Also, for example, the target-like data may be data including an anomaly of a type similar to an anomaly included in the target data. For example, when the target data is an image including anomalies, the target-like data may be an image cropped to include at least some of the anomalies included in the target data. For example, an anomaly of a type similar to the anomaly included in the target data is an anomaly located at the same position as the anomaly included in the target data, and the defect type of the anomaly included in the target data. The same type of (for example, when the object to be inspected is a circuit board and an anomaly included in the target data is a soldering defect, an anomaly included in the target-like data is also a soldering defect, etc.) may be included.

The target-like data may include at least a part of the image of the target data, and the pixel portion corresponding to an anomaly among the images of the target data may be data that necessarily includes.

The target dissimilar data may be an image that does not include anomalies. Here, the target data may be an image including an anomaly in the center of the image. The target dissimilar data may be an image in which at least a part of the target data except for an anomaly is overlapped.

If the data is an image, an image containing an anomaly and an image not containing an anomaly may have different anomaly parts, but the other parts may be similar. Therefore, if the data is classified by a conventional method, it will be classified as a similar image. May be. For example, in an image of 100*100 pixels obtained in a semiconductor process, when an anomaly portion is 5*5 pixels, an anomaly portion is normally classified as 0.25% of the total, 99.75% of the two images are similar. Therefore, it can be classified as similar in the case of a conventional classification method. However, when the classification model is trained using target data, target-like data, and target dissimilar data, each of which cluster information is labeled in an embodiment of the present disclosure, even if only a part of the image is different, it can be classified as dissimilar data. have.

The processor 110 may train the classification model to classify the target data and target-like data into the same cluster, and classify the target dissimilar data into a cluster different from the cluster to which the target data and target-like data belong.

In the present disclosure, when the target data is an image, an anomaly that may be included in the target data may include at least one of an anomaly related to an object to be inspected included in the image and an anomaly separated from the object to be inspected. For example, if the target data is an image obtained in a semiconductor process, an anomaly that may be included in the target data is an anomaly (for example, defect) that may occur in a semiconductor product that is an object to be inspected and the object to be inspected. It may include at least one of irrelevant anomalies (eg, a problem with an image acquisition device, a lens foreign material, etc.). That is, from the viewpoint of anomaly detection, the target data and the target-like data may include false positives. More specifically, when an anomaly does not exist in the object to be inspected, and there is an abnormal situation that can occur separately from the object to be inspected, the target data and other similar data may include such a positive error. Therefore, the classification model can classify not only anomaly types but also positive errors into clusters.

When the input data includes an anomaly irrelevant to the object to be inspected, the first inspection result may include a positive error. In this case, the input data is secondarily classified using a classification model, and the processor 110 classifies the input data using the classification model and, as a result of classifying the input data, is input by the classification model learned using the training data including a positive error. In the second classification of the data, it is possible to determine whether it is an anomaly related to an actual object to be inspected or an anomaly that is not related to an object to be inspected (ie, a positive error).

Hereinafter, learning data in the present disclosure will be described in detail with reference to the drawings.

3A, 3B, 3C, 3D, 3E, and 3F are schematic diagrams illustrating a subset of training data according to an embodiment of the present disclosure. The images shown in FIGS. 3A to 3E are only examples of training data, and the present disclosure may include arbitrary training data and arbitrary images.

3A is a schematic diagram showing a target base image 300 that may become training data. The target base image 300 may itself be used as a target image used for learning a classification model, or may be used as a target image 310 by cropping the target base image 300. The target base image 300 may include, for example, an anomaly of a defect 301 of an object to be inspected. The target-like images 311 and 313 may be images that have been cropped to include the defective portion 301 of the target image 310. The cutout lines 310, 311, and 313 of FIG. 3A represent cropping of the target image 310 and cropping of target-like images 311 and 313.

3B is an exemplary view showing a target image 310 and target similar images 311 and 313 cropped by the cut line of FIG. 3A. Referring to FIG. 3B, both the target image 310 and the target-like images 311 and 313 include an anomaly 301 and may include other parts of an object to be inspected.

3C is an exemplary diagram showing a target dissimilar base image 320. The target dissimilar image 320 shown in FIG. 3C may be used as a target dissimilar image by itself, or a part of the target dissimilar image may be cropped to be used as a target dissimilar image. When the cropped target dissimilar image is used to learn the classification model of the present disclosure, the target dissimilar image 320 may have the same resolution as the target image 310 and the target similar image 311. The resolutions of the above-described target image, target-like image, and target dissimilar image are only examples, and their resolutions may be the same or different from each other. The target dissimilar image 320 may be an image that does not include an anomaly. The target dissimilar image 320 may be an image that does not include an anomaly in a portion 321 corresponding to the target image.

3D may be an exemplary diagram illustrating a subset of training data for training a classification model according to an embodiment of the present disclosure.

The training data subset may include a target image 310 that includes an anomaly 301, a target-like image 311, and a target dissimilar image 320 that does not include an anomaly 321. The same cluster information may be labeled on the target image 310 and the target-like image 311 so that they are classified into the same cluster. The target dissimilar image 320 may be labeled with cluster information different from the cluster information labeled on the target image 310 and the target similar image 311. For example, the target image 310 and the target-like image 311 may be labeled with first cluster information, and the target dissimilar image 320 may be labeled with second cluster information.

When the classification model is trained using the training data subset shown in FIG. 3D, the classification model may classify an image with an anomaly and an image without an anomaly into different clusters.

For example, in an image of 100*100 pixels obtained in a semiconductor process, when an anomaly portion is 5*5 pixels, an anomaly portion is normally classified as 0.25% of the total, 99.75% of the two images are similar. Therefore, it can be classified as similar in the case of a conventional classification method. However, when the classification model is trained using target data, target-like data, and target dissimilar data, each of which cluster information is labeled in an embodiment of the present disclosure, even if only a part of the image is different, it can be classified as dissimilar data. have.

3E is an exemplary diagram showing another subset of training data for training a classification model according to an embodiment of the present disclosure.

The target image 330 and the target-like image 333 of FIG. 3E may be images including a positive error. More specifically, the anomaly 331 included in the target image 330 and the target-like image 333 may be an abnormal situation (eg, lens foreign matter, etc.) that may occur after being separated from the object to be inspected. The target image 330 may be an image related to a normal product or an image including an anomaly generated according to an abnormal situation separated from the normal product. According to the conventional anomaly test method, in the case of an anomaly image including such a positive error, it is determined to be anomaly and an incorrect result is output. However, in an embodiment of the present disclosure, the classification model generates a cluster related to the positive error using an image containing a positive error, so that whether the test target object included in the input image to be tested actually contains an anomaly, the test target object Does not include anomaly, but for other reasons, it may be detected whether the input data is determined to be anomaly. In the example of FIG. 3E, the target dissimilar image 340 may be an image that does not include an anomaly.

When the classification model is trained by using the training data subset shown in FIG. 3E, the classification model may classify an image with a positive error and an image without a positive error into different clusters.

3F is an exemplary diagram showing another subset of training data for training a classification model according to an embodiment of the present disclosure.

The target image 330 and the target-like image 333 of FIG. 3F may be images including a positive error. More specifically, the anomaly 331 included in the target image 330 and the target-like image 333 may be an abnormal situation (eg, lens foreign matter, etc.) that may occur after being separated from the object to be inspected. The target image 330 may be an image related to a normal product or an image including an anomaly generated according to an abnormal situation separated from the normal product. According to the conventional anomaly test method, in the case of an anomaly image including such a positive error, it is determined to be anomaly and an incorrect result is output. However, in an embodiment of the present disclosure, the classification model generates a cluster related to the positive error using an image containing a positive error, so that whether the test target object included in the input image to be tested actually contains an anomaly, the test target object Does not include anomaly, but for other reasons, it may be detected whether the input data is determined to be anomaly. In the example of FIG. 3E, the target dissimilar image 310 may be an image including an anomaly 301 (ie, an anomaly with respect to an object to be inspected) that is not a positive error. When the classification model is trained using the training data subset shown in FIG. 3f, the classification model is an image in which a positive error exists and an image in which an anomaly exists in the actual object to be inspected (i.e. Images to be classified) can be classified into different clusters.

The target image 330 including the false positive may be added for each type of the false positive. In other words, when a new false test occurs in the process of performing an anomaly detection solution in an industrial site, the classification model may be additionally trained using the new false test as a target image including a new false positive error. The classification model of the present disclosure can cluster all known anomaly types and all known erroneous types, and determine which type the input data belongs to.

4 is a schematic diagram illustrating a method of training a classification model according to an embodiment of the present disclosure.

The classification model of the present disclosure is trained to form a cluster between similar data on the solution space 400. More specifically, in the classification model, the target data 401 and the target similar data 402 are included in one cluster 410, and the target dissimilar data 403 is different from the target data 401 and the target similar data 402. It is learned to be included in different clusters.

In the solution space of the learned classification model, each cluster may be positioned to have a predetermined distance margin 420.

The classification model receives a subset of training data including target data 401, target-like data 402, and target dissimilar data 403, maps each data to the solution space, and according to the cluster information labeled on the solution space. The weights of one or more network functions included in the classification model can be updated so that they can be clustered. That is, in the classification model, the distance between the target data 401 and the target-like data 402 in the solution space becomes close to each other, and the target data 401 and the target-like data 402 and the target dissimilar data 403 are It can be learned so that the distances in the solution space become distant from each other. The classification model can be trained using, for example, a triplet-based cost function. The triplet-based cost function aims to separate a pair of input data of the same classification from the third input data of another classification, and a first distance between the pair of input data of the same classification (that is, the size of the cluster 410), The difference value between the second distance (i.e., the distance between 401 or 402 and 403) between one of the pair of input data of the same classification and the third input data is at least as the distance margin 420, and the classification model is trained. The method includes reducing the first distance to less than a certain percentage of the distance margin. Here, the distance margin 420 may always be positive. In order to reach the distance margin 420, weights of one or more network functions included in the classification model may be updated, and the weight update may be performed per iteration or per epoch. Details of this distance margin are disclosed in “FaceNet: A Unified Embedding for Face Recognition and Clustering” by Schroff et al. and Korean Patent Application Publication No. 10-2018-0068292, which are incorporated herein by reference in their entirety.

In addition, the classification model can be trained as a magnet loss-based model that not only classifies clusters of dissimilar data, but also considers the semantic relationship between data in one cluster or between different clusters. have. The initial distance between the center points of each cluster in the solution space of the classification model can be modified during the learning process. After mapping the data on the solution space, the classification model may adjust the position of each data on the solution space based on similarities to the cluster to which each data belongs and data inside and outside the cluster. Details related to the learning of the classification model based on magnet loss are disclosed in “METRIC LEARNING WITH ADAPTIVE DENSITY DISCRIMINATION” by O. Rippel et al., the entirety of which is incorporated herein by reference.

5 is a schematic diagram showing a solution space of a classification model according to an embodiment of the present disclosure.

5 is a schematic diagram showing a solution space 400 of the learned classification model. The solution space 400 illustrated in FIG. 5 is only an example, and the classification model may include an arbitrary number of clusters and an arbitrary number of data for each cluster.

The shapes of the data 431, 433, 441, 443, etc. included in the cluster shown in FIG. 5 are only examples to indicate that they are similar data.

In the present disclosure, a solution space is composed of one or more dimensional spaces and includes one or more clusters, and each cluster may be configured based on a feature based on each target data and a position of a feature based on target-like data on the solution space. have.

In the solution space, the first cluster 430 and the second cluster 440 may be clusters for dissimilar data. Also, the third cluster 450 may be a cluster for data that is dissimilar to the first and second clusters. The distances 445 and 435 between clusters may be a measure representing a difference between data belonging to each cluster.

The twelfth distance 445 between the first cluster 430 and the second cluster 440 may be a measure representing a difference between data belonging to the first cluster 430 and data belonging to the second cluster 440. In addition, the thirteenth distance 435 between the first cluster 430 and the second cluster 440 may be a measure representing the difference between data belonging to the first cluster 430 and data belonging to the third cluster 450 have. In the example illustrated in FIG. 5, data belonging to the first cluster 430 may be more similar to data belonging to the second cluster 440 than to data belonging to the third cluster 450. That is, when the distance between clusters is far, data belonging to each cluster may be more dissimilar, and when the distance between clusters is close, data belonging to each cluster may be less dissimilar. The distances 435 and 445 between clusters may be greater than or equal to a predetermined ratio than the radius of the clusters. The processor 110 may classify the input data based on a position to which a feature of the input data is mapped to a solution space of the classification model by calculating the input data using the classification model.

The processor 110 may process the input data using a pre-learned classification model to map features of the input data to a solution space of the pre-trained classification model. The processor 110 may classify the input data based on which cluster the input data belongs to one or more clusters in the solution space based on a position of the input data in the solution space.

The processor 110 may determine a final inspection result based at least in part on the classification result of the input data of the classification model.

When the classification model classifies the input data as belonging to a specific cluster, the processor 110 may generate a test result matched to the specific cluster as a secondary test result, and determine a final test result based on the second test result. . In addition, when the classification model fails to classify the input data, the processor 110 may determine the first test result as the final test result. Here, the secondary test result may include information related to at least one of whether an anomaly is present, an anomaly location, an anomaly type, a false sword, and a false sword type.

Since the pre-trained classification model has clusters for each type of anomaly in the solution space, the processor 110 may generate a secondary test result for the input data using the classification model. The processor 110 may determine whether an anomaly exists in the input data based on which cluster a feature of the input data belongs to in the solution space of the classification model, and if an anomaly exists, the anomaly is checked. It is possible to determine whether it is an anomaly existing on a target object or an anomaly (ie, a false sword) that is not related to the object to be inspected. For example, when the position of the feature of the input data in the solution space belongs to a cluster related to the soldering defect of the substrate (that is, the input data is classified as the soldering defect of the substrate using the classification model), the input It is possible to generate a second inspection result that determines that the anomaly of a board soldering defect exists in the data, and a final inspection result can be determined based on the first inspection result and the second inspection result. In addition, for example, if the position of the feature of the input data in the solution space belongs to the cluster related to the target object photographing lens foreign object, the input data is a false test and determines that the corresponding input data is normal data without an anomaly. Secondary inspection results can be generated, and final inspection results can be determined based on the primary inspection and secondary inspection results.

When the input data of the above-described example is determined to be a board soldering defect in the first inspection, the first and second inspection results are the same, so that the processor 110 detects the board soldering defect as the final result of the anomaly detection of the above-described input data. Can be determined by In the above-described example, when the input data is classified into different anomalies in the primary examination (ie, when the primary and secondary classification results are different), the processor 110 may inform the operator of this. The processor 110 may mark the input data to inform the operator of the discrepancy between the first and second inspection results. In addition, when the first and second classification results are different, the processor 110 may generate a final test result by ensemble the first test result and the second test result. For example, the ensemble method may be a voting method or a method based on weights for each neural network, and may include any other ensemble method.

In addition, if the position of the feature of the input data in the solution space does not belong to any cluster, the processor 110 determines that the classification failure using the classification model of the input data and generates the first inspection result as the final inspection result. have. For example, if the input data contains a new anomaly related to an object to be inspected, which is a circuit lead defect, or an anomaly unrelated to an object to be inspected, such as a broken lens (that is, the learned classification model is trained). When there is a new pattern that has not been performed, etc.) Since input data cannot be classified using a classification model, a final inspection result for the corresponding input data can be generated based on the primary inspection result. In this case, the processor 110 may inform the operator of this. The processor 110 may mark the input data in order to inform the operator of this. In addition, when the corresponding input data that cannot be classified using the classification model includes anomalies, the processor 110 may generate the corresponding input data as new training data. In other words, when the data having an anomaly that cannot be classified using the classification model (that is, when the primary test is anomaly, etc.) is an image, the processor 110 uses the input data as a target image, and A new subset of training data can be created using another portion including an anomaly of the input data as target-like data. The processor 110 may additionally train a classification model to generate a cluster in a solution space of a classification model for a new anomaly pattern by using the newly created training data subset.

That is, in one embodiment of the present disclosure, by determining the anomaly of the input data through the primary and secondary inspection, the anomaly related to an abnormal situation in which the input data is caused by factors other than the object to be inspected (i.e., positive In the case of including an error), it is possible to derive the test result rather than a false test. In addition, when a new false gum type or an anomaly type occurs, the classification model is retrained using this, so that a certain test performance can be maintained in response to the newly generated false test and an anomaly.

In addition, by separating the first and second tests, it is possible to add a new type of anomaly or a new type of false test to the anomaly detection solution without affecting the classification performance of existing data. That is, in the case of a repeated false check occurs in the existing solution, the problem of the corresponding false check can be solved by modifying the solution itself, but this may cause other malfunctions. You can block the possibility.

6 is a flowchart of a method for checking input data according to an embodiment of the present disclosure.

The computing device 100 may derive a result of the first inspection for the input data (610). In the present disclosure, the input data may include image data, and the computing device 100 may determine a primary inspection result for the captured image data or image data transmitted from an external device. The first inspection result may include anomaly information related to the input data.

The computing device 100 may classify the input data using a classification model learned using training data labeled with cluster information (630 ). The classification model is trained using a training data set including a plurality of training data subsets, and can be trained to classify similar data into the same cluster from training data included in the training data set and classify dissimilar data into different clusters. .

The computing device 100 may output a final inspection result based at least in part on the classification result of the input data of the classification model (650 ). When the classification model classifies the input data as belonging to a specific cluster, the computing device 100 generates the test result matched to the specific cluster as a secondary test result, and generates a final test result based at least in part on the second test result. You can decide. Also, when the classification model fails to classify the input data, the computing device 100 may determine the first test result as the final test result.

7 is a block diagram illustrating a means for implementing a method for checking input data according to an embodiment of the present disclosure.

A method for checking input data according to an embodiment of the present disclosure may be implemented by the following means.

According to an embodiment of the present disclosure, a method for inspecting input data includes means 710 for deriving a first inspection result for the input data; Means (730) for classifying the input data using a classification model learned using training data labeled with cluster information; And means 750 for outputting a final inspection result based at least in part on the classification result of the input data of the classification model.

In an alternative embodiment of the method for inspecting input data of the present disclosure, the means for deriving a first-order inspection result for the input data (710) comprises a pre-trained inspection model comprising one or more network functions of the input data. And a means for performing an anomaly detection on the input data, or performing an anomaly detection on the input data based on a comparison of the input data and reference data by processing by using.

In an alternative embodiment of the method for inspecting input data of the present disclosure, means for outputting a final inspection result (750) based at least in part on a classification result for the input data of the classification model, wherein the classification model is When the input data is classified as belonging to a specific cluster, a test result matched to the specific cluster is generated as a secondary test result, a final test result is output based on the second test result, and the input data is When the classification fails, a means for outputting the first inspection result as a final inspection result may be included.

In an alternative embodiment of the method for inspecting input data of the present disclosure, the means 730 for classifying the input data using a classification model learned using training data labeled with cluster information comprises: Means for mapping a feature of the input data to a solution space of the pre-learned classification model by processing using the pre-learned classification model; And means for classifying the input data based on whether the input data belongs to one of one or more clusters in the solution space based on a position of the input data in the solution space.

8 is a block diagram illustrating a module for implementing a method for checking input data according to an embodiment of the present disclosure.

A method for checking input data according to an embodiment of the present disclosure may be implemented by the following modules.

According to an embodiment of the present disclosure, a method for inspecting input data includes a module 810 for deriving a first inspection result for input data; A module 830 for classifying the input data using a classification model learned using training data labeled with cluster information; And a module 850 for outputting a final inspection result based at least in part on the classification result of the input data of the classification model.

9 is a block diagram illustrating logic for implementing a method for checking input data according to an embodiment of the present disclosure.

A method for checking input data according to an embodiment of the present disclosure may be implemented by the following logic.

According to an embodiment of the present disclosure, a method for inspecting input data includes a logic 910 for deriving a first inspection result for input data; Logic (930) for classifying the input data by using a classification model learned using training data labeled with cluster information; And logic 950 for outputting a final inspection result based at least in part on the classification result of the input data of the classification model.

10 is a block diagram illustrating a circuit for implementing a method for checking input data according to an embodiment of the present disclosure.

A method for checking input data according to an embodiment of the present disclosure may be implemented by the following means.

According to an embodiment of the present disclosure, a method for inspecting input data includes: a circuit 1010 for deriving a first inspection result for input data; A circuit 1030 for classifying the input data using a classification model learned using training data labeled with cluster information; And a circuit 1050 for outputting a final inspection result based at least in part on the classification result of the input data of the classification model.

Those of skill in the art would further describe the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein, including electronic hardware, computer software, or a combination of both. It should be recognized that it can be implemented as To clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented in hardware or as software depends on the specific application and design restrictions imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

11 shows a simplified and general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

While the present disclosure has generally been described above with respect to computer-executable instructions that can be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in combination with other program modules and/or as a combination of hardware and software. will be.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, to those skilled in the art, the method of the present disclosure is not limited to single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable household appliances, etc. (each of which is It will be appreciated that other computer system configurations may be implemented, including one that may operate in connection with one or more associated devices.

The described embodiments of the present disclosure may also be practiced in a distributed computing environment where certain tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. Computer-readable media can be any computer-readable media, including volatile and nonvolatile media, transitory and non-transitory media, removable and non-transitory media. Includes removable media. By way of example and not limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and nonvolatile media, temporary and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Includes the medium. Computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device, or other magnetic storage device, Or any other medium that can be accessed by a computer and used to store the desired information.

Computer-readable transmission media typically implement computer-readable instructions, data structures, program modules or other data on a modulated data signal such as a carrier wave or other transport mechanism. Includes all information delivery media. The term modulated data signal refers to a signal in which one or more of the characteristics of the signal is set or changed to encode information in the signal. By way of example and not limitation, computer-readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above-described media are also intended to be included within the scope of computer-readable transmission media.

An exemplary environment 1100 is shown that implements various aspects of the present disclosure, including a computer 1102, which includes a processing device 1104, a system memory 1106, and a system bus 1108. do. System bus 1108 couples system components, including but not limited to, system memory 1106 to processing device 1104. The processing unit 1104 may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be used as processing unit 1104.

The system bus 1108 may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read-only memory (ROM) 2110 and random access memory (RAM) 2112. The basic input/output system (BIOS) is stored in non-volatile memory 2110 such as ROM, EPROM, EEPROM, etc. This BIOS is a basic input/output system that helps transfer information between components in the computer 1102, such as during startup. Includes routines. RAM 2112 may also include high speed RAM such as static RAM for caching data.

The computer 1102 also has an internal hard disk drive (HDD) 2114 (e.g., EIDE, SATA), and the internal hard disk drive 2114 can also be configured for external use within a suitable chassis (not shown). ), a magnetic floppy disk drive (FDD) 2116 (e.g., for reading from or writing to a removable diskette 2118), and an optical disk drive 1120 (e.g., a CD-ROM disk (For reading 1122) or reading from or writing to other high-capacity optical media such as DVD). The hard disk drive 2114, magnetic disk drive 2116, and optical disk drive 1120 are each connected to the system bus 1108 by a hard disk drive interface 1124, a magnetic disk drive interface 1126, and an optical drive interface 1128. ) Can be connected. The interface 1124 for implementing an external drive includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide non-volatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of the computer-readable medium above refers to a removable optical medium such as a HDD, a removable magnetic disk, and a CD or DVD, those skilled in the art may use a zip drive, a magnetic cassette, a flash memory card, a cartridge, etc. It will be appreciated that other types of computer-readable media, such as the ones, may also be used in the exemplary operating environment and that any such media may contain computer-executable instructions for performing the methods of the present disclosure.

A number of program modules, including the operating system 2130, one or more application programs 2132, other program modules 2134, and program data 2136, may be stored in the drive and RAM 2112. All or part of the operating system, applications, modules and/or data may also be cached in RAM 2112. It will be appreciated that the present disclosure may be implemented in several commercially available operating systems or combinations of operating systems.

A user may input commands and information into the computer 1102 through one or more wired/wireless input devices, for example, a pointing device such as a keyboard 2138 and a mouse 1140. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. These and other input devices are often connected to the processing unit 1104 through the input device interface 1142, which is connected to the system bus 1108, but the parallel port, IEEE 1394 serial port, game port, USB port, IR interface, It can be connected by other interfaces such as etc.

A monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146. In addition to the monitor 1144, the computer generally includes other peripheral output devices (not shown) such as speakers, printers, and the like.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148, via wired and/or wireless communication. The remote computer(s) 1148 may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and is generally connected to the computer 1102. Although it includes many or all of the components described for simplicity, only memory storage device 1150 is shown. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or to a larger network, eg, a wide area network (WAN) 1154. Such LAN and WAN networking environments are common in offices and companies, and facilitate enterprise-wide computer networks such as intranets, all of which can be connected to worldwide computer networks, for example the Internet.

When used in a LAN networking environment, the computer 1102 is connected to the local network 1152 via a wired and/or wireless communication network interface or adapter 1156. Adapter 1156 may facilitate wired or wireless communication to LAN 1152, which also includes a wireless access point installed therein to communicate with wireless adapter 1156. When used in a WAN networking environment, the computer 1102 may include a modem 1158, connected to a communication computing device on the WAN 1154, or through the Internet, to establish communication over the WAN 1154. Have other means. The modem 1158, which may be an internal or external and a wired or wireless device, is connected to the system bus 1108 through a serial port interface 1142. In a networked environment, program modules described for the computer 1102 or portions thereof may be stored in the remote memory/storage device 1150. It will be appreciated that the network connections shown are exemplary and other means of establishing communication links between computers may be used.

Computer 1102 is associated with any wireless device or entity deployed and operated in wireless communication, e.g., a printer, scanner, desktop and/or portable computer, portable data assistant (PDA), communication satellite, wireless detectable tag. It operates to communicate with any device or place, and a phone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network or may simply be ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) allows you to connect to the Internet or the like without wires. Wi-Fi is a wireless technology such as a cell phone that allows such devices, for example computers, to transmit and receive data indoors and outdoors, that is, anywhere within the coverage area of a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide a secure, reliable and high-speed wireless connection. Wi-Fi can be used to connect computers to each other, to the Internet, and to a wired network (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz radio bands, for example, at a data rate of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band). have.

Those of ordinary skill in the art of this disclosure will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols and chips that may be referenced in the above description are voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields. Or particles, or any combination thereof.

A person of ordinary skill in the art of the present disclosure includes various exemplary logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein, electronic hardware, (convenience). For the sake of clarity, it will be appreciated that it may be implemented by various forms of program or design code or a combination of both (referred to herein as "software"). To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. A person of ordinary skill in the art of the present disclosure may implement the described functions in various ways for each specific application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program, carrier, or media accessible from any computer-readable device. For example, computer-readable media include magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CD, DVD, etc.), smart cards, and flash memory. Devices (eg, EEPROM, card, stick, key drive, etc.). In addition, the various storage media presented herein include one or more devices and/or other machine-readable media for storing information.

It is to be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on the design priorities, it is to be understood that within the scope of this disclosure a specific order or hierarchy of steps in processes may be rearranged. The appended method claims provide elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

A description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those of ordinary skill in the art, and general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (19)

A computer program stored in a computer-readable storage medium, wherein the computer program performs the following operations for checking input data when executed on one or more processors of a computing device, the operations comprising:
Deriving a result of the first inspection for the input data;
Classifying the input data using a classification model including one or more network functions learned based on at least two subsets of training data labeled with different cluster information; And
Outputting a final inspection result based on the primary inspection result and the classification result of the input data of the classification model;
Including,
The classification model,
The same cluster information is learned based on a loss function that minimizes the distance in the solution space between labeled data and maximizes the distance in the solution space between the labeled data with different cluster information,
The cluster information,
Clusters according to the types of anomalies related to the object to be inspected; And
Cluster according to the type of anomaly that is not related to the object to be inspected;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The classification model,
Learning using a training data set including a plurality of training data subsets, learning to classify similar data into the same cluster from training data included in the training data set, and classify dissimilar data into different clusters,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The operation of deriving the first inspection result for the input data,
By processing the input data using a learned test model including one or more network functions, an anomaly detection is performed on the input data, or the input data is compared based on the comparison of the input data and the reference data. Performing an anomaly detection;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The classification model,
Different cluster information is learned using a training data set comprising a subset of training data comprising labeled training data,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The classification model,
Trained using a training data set comprising a subset of training data including target data, target-like data, and target dissimilar data,
A computer program stored on a computer readable storage medium.
The method of claim 5,
The target data and the target similarity data are data labeled with first cluster information, and the target dissimilar data is data labeled with second cluster information,
A computer program stored on a computer readable storage medium.
The method of claim 5,
When the target data is data including anomaly, the target-like data is data including an anomaly of a type similar to the anomaly included in the target data, and the target dissimilar data is data not including anomaly. ,
A computer program stored on a computer readable storage medium.
The method of claim 5,
When the target data is an image including anomalies, the target similarity data is an image that is cropped to include at least some of the anomalies included in the target data, and the target dissimilar data does not include anomalies. Which is not an image,
A computer program stored on a computer readable storage medium.
The method of claim 8,
The target data is an image including the anomaly in the center of the image,
A computer program stored on a computer readable storage medium.
The method of claim 8,
The target dissimilar data,
An image in which at least a part of a portion of the target data excluding anomalies overlaps,
A computer program stored on a computer readable storage medium.
delete The method of claim 5,
The target data includes an image related to an abnormal situation that may occur separately from the target object, and the target-like data includes an image including at least a part of the target data,
A computer program stored on a computer readable storage medium.
The method of claim 5,
The target data and the target similar data,
Data containing false positives,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The operation of outputting a final inspection result based at least in part on the classification result of the input data of the classification model,
When the classification model classifies the input data as belonging to a specific cluster, the test result matched to the specific cluster is generated as a secondary test result, and a final test result is output based on the second test result, and the Outputting the first inspection result as a final inspection result when the classification of the input data fails;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 14,
The second test result is,
Including information related to at least one of the presence or absence of an anomaly, a position of an anomaly, a type of anomaly, and a type of anomaly,
A computer program stored on a computer readable storage medium.
The method of claim 1,
The operation of classifying the input data using a classification model learned by using training data labeled with cluster information,
Mapping a feature of the input data to a solution space of the learned classification model by processing the input data using the learned classification model; And
Classifying the input data based on whether the input data belongs to one of one or more clusters in the sea space based on a position of the input data in the sea space;
Containing,
A computer program stored on a computer readable storage medium.
The method of claim 16,
The sea space,
Consisting of one or more dimensional spaces, including one or more clusters, and each cluster is configured based on a feature based on each target data and a position in the solution space of the feature based on target-like data,
A computer program stored on a computer readable storage medium.
An input data inspection method performed in a computing device including one or more processors,
Deriving a first inspection result for the input data;
Classifying the input data using a classification model including one or more network functions learned based on at least two or more subsets of training data labeled with different cluster information; And
Outputting a final inspection result based on the primary inspection result and the classification result of the input data of the classification model;
Including,
The classification model,
The same cluster information is learned based on a loss function that minimizes the distance in the solution space between labeled data and maximizes the distance in the solution space between the labeled data with different cluster information,
The cluster information,
Clusters according to the types of anomalies related to the object to be inspected; And
Cluster according to the type of anomaly that is not related to the object to be inspected;
Containing,
How to check input data.
As a computing device,
One or more processors; And
A memory storing instructions executable in the processor;
Including,
The processor,
Derive the results of the first inspection on the input data,
Classifying the input data using a classification model including one or more network functions learned based on at least two or more subsets of training data labeled with different cluster information, and
Outputting a final test result based on the first test result and the classification result for the input data of the classification model,
The classification model,
The same cluster information is learned based on a loss function that minimizes the distance in the solution space between labeled data and maximizes the distance in the solution space between the labeled data with different cluster information,
The cluster information,
Clusters according to the types of anomalies related to the object to be inspected; And
Cluster according to the type of anomaly that is not related to the object to be inspected;
Containing,
Computing device.

Images