JP7533263B2 - Image inspection device, image inspection method, and trained model generation device - Google Patents

Description

The present invention relates to an image inspection device, an image inspection method, and a trained model generation device.

Conventionally, image inspection devices are known that inspect an object based on an image of the object.

For example, Patent Document 1 describes an anomaly determination device that performs anomaly determination based on input image data to be determined, which has a process execution means that generates reconstructed image data from the features of the image data to be determined using reconstruction parameters for reconstructing normal image data from features extracted from a group of normal image data, and executes anomaly determination processing to perform anomaly determination based on difference information between the generated reconstructed image data and the image data to be determined. When the image data to be determined includes image data of multiple channels, the anomaly determination device of Patent Document 1 generates reconstructed image data for each channel from the features of the image data of each channel using reconstruction parameters, and performs anomaly determination based on difference information between each generated reconstructed image data and the image data of each channel of the image data to be determined.

JP 2018-5773 A

Here, in the past, there was a method in which a good-quality image of a good target object was divided into smaller pieces and input, and a trained model was used that was trained to output the feature quantities of the good-quality image, and defects in the inspection image were detected in the feature space based on the feature vector extracted from the inspection image and the feature vector of the good-quality image.

However, when a special pattern is present locally in a good product image, the feature vector of the good product image containing this special pattern is plotted at a position in the feature space away from the feature vectors of other good product images, which can result in the object containing the special pattern being detected as a defective product.

In addition, if another good product image contains a pattern that is good in one location but defective in another location, the feature vector of the inspection image that contains such a pattern in the other location will be plotted close to the feature vector of the good product image in feature space, which can cause the defective product to be overlooked.

The present invention has been made in consideration of these circumstances, and one of its objectives is to provide an image inspection device, an image inspection method, and a trained model generation device that can plot the feature vectors of good product images that include special patterns close to the feature vectors of good product images, and plot the feature vectors of defective product inspection images farther away.

An image inspection device according to one aspect of the present disclosure includes an extraction unit that inputs an inspection split image, which is a split image of an inspection object of a good product, and an inspection surroundings-containing image, which is based on an image surrounding the inspection split image, to a trained model that has been trained to output features using as input a good product split image, which is a split image of an inspection object of a good product, and a good product surroundings-containing image, which is based on an image surrounding the inspection split image, and extracts feature vectors whose components are the feature amounts for the inspection split image, an acquisition unit that acquires a defect degree indicating the degree of defect in the inspection split image corresponding to the feature vector, based on a point indicated by the feature vector and a set formed by multiple feature vectors for multiple good product split images, in a feature space represented by the extracted feature vector, and an inspection unit that inspects the inspection object based on the defect degree.

According to this aspect, the inspection split image and the inspection surroundings-containing image are input to a trained model that has been trained to output features using a good-product split image and a good-product surroundings-containing image as inputs, and a feature vector for the inspection split image is extracted. As a result, compared to a case where a trained model is trained using only good-product split images and a point indicated by a feature vector of a good-product split image containing a special pattern is plotted at a position far from a point indicated by a feature vector of a good-product split image containing another pattern, the trained model trains using a good-product surroundings-containing image in which the feature amount changes greatly depending on the position, so that the distance between points indicated by feature vectors for good-product split images at the same position can be plotted close to each other, and the distance between points indicated by feature vectors for good-product split images at different positions can be plotted far from each other, so that a pattern specific to a good-product split image can be accurately learned, and in the feature space, a point indicated by a feature vector for an inspection split image is plotted close to a set formed by the feature vector for the corresponding good-product split image. Therefore, in the feature space, a point indicated by a feature vector for an inspection split image including a special pattern can be plotted near a set formed by feature vectors for multiple good split images corresponding to the inspection split image, and a defect degree with a small degree of defect can be obtained to determine that the product is good. In addition, the trained model can learn different judgment criteria according to the position and part in the good product image of the inspection target of a good product by using the image containing the good product surroundings, and in the feature space, the points indicated by the feature vector for each inspection split image are collected in different ranges and regions according to the position of the inspection split image in the inspection image. Therefore, in the feature space, a point indicated by a feature vector for an inspection split image at another position including a pattern that is good at one position but defective at another position can be plotted far away from the set formed by multiple feature vectors for multiple good split images at the other position, and a defect degree with a large degree of defect can be obtained to suppress overlooking of defective products. Therefore, the degree of defect in the inspection split image is obtained based on the point indicated by the feature vector for the inspection split image in the feature space and the set formed by multiple feature vectors for multiple good product split images, and the inspection object is inspected based on this degree of defect, thereby improving the inspection accuracy of the inspection object.

In the above-mentioned aspect, the inspection surroundings-containing image may include an image that is a reduced image of the surroundings of the inspection division image.

According to this aspect, the inspection surroundings-containing image includes an image that is a reduced image of the surroundings of the inspection divided image. This makes it possible to suppress the influence of the inspection surroundings-containing image from becoming too large in the output of features by the trained model, and makes it possible to extract feature vectors for the inspection divided image with high accuracy.

In the above-mentioned aspect, the acquisition unit may acquire the degree of defect of the inspection segmentation image corresponding to the extracted feature vector based on the distance between a point indicated by the feature vector and a set in a feature space represented by the feature vector.

According to this aspect, the degree of defect in the inspection split image corresponding to the feature vector is obtained based on the distance in the feature space between the point indicated by the feature vector and a set formed by multiple feature vectors of multiple good product split images. This makes it easy to indicate the degree of defect in the inspection split image.

In the above-mentioned aspect, the acquisition unit may acquire the degree of defect of the inspection segmented image corresponding to the extracted feature vector based on the distance between a point indicated by the feature vector and a point indicated by one of the multiple feature vectors of the multiple good segmented images in a feature space represented by the feature vector.

According to this aspect, the degree of defect of the inspection split image corresponding to the feature vector is obtained based on the distance in the feature space between a point indicated by the feature vector and a point indicated by one of the multiple feature vectors of the multiple good product split images. This makes it easy to indicate the degree of defect in the inspection split image.

In the above-mentioned aspect, the degree of defect may be a value indicating the degree of defect in the inspection division image.

According to this embodiment, the degree of defects in the inspection division image can be quantitatively indicated.

In the above-mentioned aspect, the inspection unit may generate a defect degree image based on multiple defect degrees and inspect the inspection object based on the defect degree image.

According to this aspect, it is easy to determine whether the object being inspected is a good or bad item, and inspection with high inspection accuracy can be easily achieved.

In the above-mentioned aspect, a division unit may be further provided that divides the inspection image of the inspection object into a plurality of inspection divided images.

According to this aspect, a division unit that divides the inspection image of the inspection object into a plurality of inspection divided images is further provided. This makes it possible to easily obtain the inspection divided images.

In the above-mentioned aspect, an imaging unit may be further provided for acquiring an inspection image of the object to be inspected.

According to this aspect, an imaging unit is further provided for acquiring an inspection image of the object to be inspected. This makes it easy to obtain the inspection image.

An image inspection method according to another aspect of the present disclosure includes the steps of: inputting an inspection split image, which is a split image of an inspection object, and an inspection surroundings-containing image, which is based on an image surrounding the inspection split image, into a trained model that has been trained to output features using as input a good-item split image, which is a split image of an inspection object that is a good item, and a good-item surroundings-containing image, which is based on an image surrounding the good-item split image, and extracting feature vectors whose components are the features for the inspection split image; acquiring a defect degree indicating the degree of defect in the inspection split image corresponding to the feature vector, based on a point indicated by the feature vector and a set formed by multiple feature vectors for multiple good-item split images, in a feature space represented by the extracted feature vector; and inspecting the inspection object based on the defect degree.

According to this aspect, the inspection split image and the inspection surroundings-containing image are input to a trained model that has been trained to output features using a good-product split image and a good-product surroundings-containing image as inputs, and a feature vector for the inspection split image is extracted. As a result, compared to a case where a trained model is trained using only good-product split images and a point indicated by a feature vector of a good-product split image containing a special pattern is plotted at a position far from a point indicated by a feature vector of a good-product split image containing another pattern, the trained model trains using a good-product surroundings-containing image in which the feature amount changes greatly depending on the position, so that the distance between points indicated by feature vectors for good-product split images at the same position can be plotted close to each other, and the distance between points indicated by feature vectors for good-product split images at different positions can be plotted far from each other, so that a pattern specific to a good-product split image can be accurately learned, and in the feature space, a point indicated by a feature vector for an inspection split image is plotted close to a set formed by the feature vector for the corresponding good-product split image. Therefore, in the feature space, a point indicated by a feature vector for an inspection split image including a special pattern can be plotted near a set formed by feature vectors for multiple good split images corresponding to the inspection split image, and a defect degree with a small degree of defect can be obtained to determine that the product is good. In addition, the trained model can learn different judgment criteria according to the position and part in the good product image of the inspection target of a good product by using the image containing the good product surroundings, and in the feature space, the points indicated by the feature vector for each inspection split image are collected in different ranges and regions according to the position of the inspection split image in the inspection image. Therefore, in the feature space, a point indicated by a feature vector for an inspection split image at another position including a pattern that is good at one position but defective at another position can be plotted far away from the set formed by multiple feature vectors for multiple good split images at the other position, and a defect degree with a large degree of defect can be obtained to suppress overlooking of defective products. Therefore, the degree of defect in the inspection split image is obtained based on the point indicated by the feature vector for the inspection split image in the feature space and the set formed by multiple feature vectors for multiple good product split images, and the inspection object is inspected based on this degree of defect, thereby improving the inspection accuracy of the inspection object.

A trained model generating device according to another aspect of the present disclosure includes a model generating unit that generates a trained model trained to output features using as input a non-defective divided image, which is a divided image of a non-defective inspection object, and a non-defective surroundings-containing image based on an image surrounding the non-defective divided image.

According to this aspect, a trained model is generated that is trained to output features using a good-product divided image and an image containing the surroundings of a good product as input. As a result, compared to a case where a trained model is trained using only good-product divided images and a point indicated by a feature vector of a good-product divided image containing a special pattern is plotted at a position far from a point indicated by a feature vector of a good-product divided image containing another pattern, the trained model trains using images containing the surroundings of a good product, in which the feature amount varies greatly depending on the position, so that the distance between points indicated by feature vectors for good-product divided images at the same position can be plotted close to each other and the distance between points indicated by feature vectors for good-product divided images at different positions can be plotted far from each other. This makes it possible to accurately learn patterns specific to good-product divided images, and in the feature space, a point indicated by a feature vector for an inspection divided image containing a special pattern can be plotted close to a set formed by feature vectors for multiple good-product divided images corresponding to the inspection divided image. In addition, the trained model can learn different judgment criteria depending on the position and part in the good product image of the inspection target object of good product by using the image containing the surroundings of the good product, and in the feature space, the points indicated by the feature vector for each inspection split image are collected in different ranges and regions depending on the position of the inspection split image in the inspection image. Therefore, in the feature space, a point indicated by the feature vector for an inspection split image at another position, which includes a pattern that is good at one position but defective at another position, can be plotted far away from the set formed by the multiple feature vectors for the multiple good product split images at the other position.

According to the present invention, the feature vectors of good product images that contain special patterns can be plotted close to the feature vectors of good product images, and the feature vectors of defective product inspection images can be plotted farther away.

FIG. 1 is a configuration diagram illustrating a schematic configuration of an image inspection system according to an embodiment. FIG. 2 is a configuration diagram showing the physical configuration of a trained model generation device and an image inspection device in one embodiment. FIG. 3 is a configuration diagram showing a functional block configuration of a trained model generation device in one embodiment. FIG. 4 is a conceptual diagram for explaining generation of a non-defective divided image and a non-defective surroundings-containing image by the learning image data set generation unit shown in FIG. FIG. 5 is a conceptual diagram for explaining a first example of a learning model used by the model generating unit shown in FIG. FIG. 6 is a conceptual diagram for explaining a second example of a learning model used by the model generating unit shown in FIG. FIG. 7 is a configuration diagram showing the configuration of functional blocks of an image inspection device according to an embodiment. FIG. 8 is a conceptual diagram for explaining an example of a method for acquiring the defect degree. FIG. 9 is a conceptual diagram for explaining another example of a method for acquiring the defect degree. FIG. 10 is a conceptual diagram for explaining the processes performed by the division unit, the learning image dataset generation unit, the extraction unit, and the acquisition unit shown in FIG. FIG. 11 is a diagram showing an example of an inspection image. FIG. 12 is a conceptual diagram for explaining the process performed by the inspection unit shown in FIG. FIG. 13 is a diagram showing an example of a defect degree image. FIG. 14 is a flowchart illustrating an example of a trained model generation process performed by a trained model generation device in one embodiment. FIG. 15 is a flowchart for explaining an example of an image inspection process performed by an image inspection device in one embodiment.

The following describes an embodiment of the present invention. In the following description of the drawings, identical or similar parts are denoted by identical or similar reference symbols. However, the drawings are schematic. Therefore, specific dimensions, etc. should be determined in light of the following description. Furthermore, the drawings include parts with different dimensional relationships and ratios. Furthermore, the technical scope of the present invention should not be interpreted as being limited to the embodiment.

First, the configuration of an image inspection system according to one embodiment will be described with reference to FIG. 1. FIG. 1 is a configuration diagram illustrating the schematic configuration of an image inspection system 1 according to one embodiment.

As shown in FIG. 1, the image inspection system 1 includes an image inspection device 20 and illumination IL. The image inspection device 20 is connected to a trained model generation device 10 via a communication network NW. The illumination IL irradiates light L onto an inspection object TA. The image inspection device 20 captures reflected light R and inspects the inspection object TA based on an image of the inspection object TA (hereinafter also referred to as an "inspection image"). The trained model generation device 10 generates a trained model that the image inspection device 20 uses to perform the inspection.

Next, the physical configuration of a trained model generation device and an image inspection device according to one embodiment will be described with reference to FIG. 2. FIG. 2 is a configuration diagram showing the physical configuration of a trained model generation device 10 and an image inspection device 20 in one embodiment.

As shown in FIG. 2, the trained model generating device 10 and the image inspection device 20 each include a processor 31, a memory 32, a storage device 33, a communication device 34, an input device 35, and an output device 36. These components are connected to each other via a bus so that they can transmit and receive data.

In the example shown in FIG. 2, the trained model generating device 10 and the image inspection device 20 are each configured with a single computer, but this is not limited to the above. The trained model generating device 10 and the image inspection device 20 may each be realized by combining multiple computers. Furthermore, the configuration shown in FIG. 2 is an example, and the trained model generating device 10 and the image inspection device 20 may each have other configurations or may not have some of these configurations.

The processor 31 is configured to control the operation of each part of the trained model generating device 10 and the image inspection device 20. The processor 31 is configured to include integrated circuits such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), and a SoC (System-on-a-Chip).

The memory 32 and the storage device 33 are each configured to store programs, data, etc. The memory 32 is composed of, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and/or a RAM (Random Access Memory), etc. The storage device 33 is composed of, for example, a storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), and/or an eMMC (embedded Multi Media Card).

The communication device 34 is configured to communicate via at least one of a wired and wireless network. The communication device 34 includes, for example, a network card, a communication module, an interface for connecting to other devices, etc.

The input device 35 is configured to allow information to be input by a user's operation. The input device 35 includes, for example, a keyboard, a touch panel, a mouse, a pointing device, and/or a microphone.

The output device 36 is configured to output information. The output device 36 includes a display device such as a liquid crystal display, an EL (Electro Luminescence) display, a plasma display, or an LCD (Liquid Crystal Display), and/or a speaker, etc.

Next, functional blocks of a trained model generating device according to one embodiment will be described with reference to Figures 3 to 6. Figure 3 is a configuration diagram showing the configuration of functional blocks of a trained model generating device 10 in one embodiment. Figure 4 is a conceptual diagram for explaining the generation of a good-product segmented image and a good-product surroundings-containing image by the training image dataset generating unit 131 shown in Figure 3. Figure 5 is a conceptual diagram for explaining a first example of a learning model 51 used by the model generating unit 135 shown in Figure 3. Figure 6 is a conceptual diagram for explaining a second example of a learning model 52 used by the model generating unit 135 shown in Figure 3.

As shown in FIG. 3, the trained model generation device 10 includes a communication unit 110, a memory unit 120, and a learning unit 130.

The communication unit 110 is configured to be able to transmit and receive various types of information. For example, the communication unit 110 transmits a trained model 55 (described later) to the image inspection device 20 via the communication network NW. The communication unit 110 also receives good-quality images from other devices via the communication network NW, for example. That is, the communication unit 110 acquires good-quality images that are images of the inspection object TA that has been determined to be good-quality. The multiple good-quality images acquired by the communication unit 110 (hereinafter, when there is no need to distinguish between them, one of them is also referred to as a "good-quality image 40") are written and stored in the memory unit 120.

The storage unit 120 is configured to store various types of information. For example, the storage unit 120 stores a non-defective product image 40, a plurality of learning image datasets (hereinafter, when there is no need to distinguish between them, one of them is also referred to as a "learning image dataset 124"), a trained model 55, and a plurality of feature vectors (hereinafter, when there is no need to distinguish between them, one of them is also referred to as a "feature vector 129").

The learning unit 130 is for training a learning model to generate a trained model. The learning unit 130 includes, for example, a learning image dataset generation unit 131 and a model generation unit 135.

The training image dataset generation unit 131 is configured to generate the training image dataset 124. The training image dataset 124 is a set of image data used to generate the trained model 55.

More specifically, the learning image dataset generation unit 131 reads out a plurality of good-quality images 40 stored in the storage unit 120, and for each of the good-quality images 40, generates a plurality of images (hereinafter referred to as "good-quality divided images") by dividing the good-quality images 40 into a plurality of parts.

As shown in FIG. 4, the learning image dataset generation unit 131 generates a total of 25 good-quality divided images by dividing one good-quality image 40, for example, vertically and horizontally into five parts. Note that the good-quality image 40 may be divided into 2 to 24 good-quality divided images, or into 26 or more good-quality divided images. In addition, the shape of the good-quality divided images is not limited to a rectangle, and may be any shape.

The learning image dataset generation unit 131 also generates a good product surroundings-containing image for a good product divided image based on images arranged around the good product divided image. For example, the learning image dataset generation unit 131 generates a good product surroundings-containing image for each of the nine good product divided images, excluding the 16 good product divided images arranged on the edge, out of the 25 good product divided images included in the good product image 40. Note that a good product surroundings-containing image may be generated by generating pixel values of the area protruding from the inspection image for each of the 16 good product divided images arranged on the edge. In this case, the pixel values of the protruding area may be generated using a specific value determined by the user, or may be generated by copying the pixel values of the image edge closest to the good product divided image.

Then, the learning image dataset generation unit 131 generates a learning image dataset 124 that includes a non-defective divided image and an image containing the non-defective surroundings of the non-defective divided image.

For example, the learning image dataset generating unit 131 generates a first good-quality surroundings-containing image 404 including the surrounding good-quality divided images for the first good-quality divided image 400 located second from the left and second from the top in the good-quality image 40. The first good-quality divided image 400 and the first good-quality surroundings-containing image 404 constitute one learning image dataset 124. Similarly, the learning image dataset generating unit 131 generates a second good-quality surroundings-containing image 406 for the second good-quality divided image 402 located third from the left and second from the top in the good-quality image 40, and generates a learning image dataset 124 including the second good-quality divided image 402 and the second good-quality surroundings-containing image 406. In this way, a learning image dataset 124 is generated for each of the nine good-quality divided images. The generated multiple learning image datasets 124 are written and stored in the storage unit 120.

In the example shown in FIG. 4, the first good product surroundings containing image 404 includes eight good product divided images located around the first good product divided image 400 as good product surroundings images, but the good product surroundings containing image is not limited to this example. The good product surroundings containing image may not include, for example, some of the eight good product divided images around the first good product divided image 400. In addition, the good product surroundings containing image is not limited to being composed of units of good product divided images, and may be composed of any units. In other words, the good product surroundings containing image may be composed of units smaller than the good product divided images, or may be composed of units larger than the good product divided images.

When the resolution of the good product divided image and the good product surroundings containing image are the same, if the number of pixels in the good product surroundings containing image is greater than the number of pixels in the good product divided image, the good product surroundings containing image will contribute more to the learning process than the good product divided image. For this reason, if the expressive ability of the learning model used is low, it may not be possible to obtain the feature quantities of the good product divided image described below with sufficient accuracy.

For the above reasons, it is preferable that the contribution of the good product surroundings-containing image to the learning process is smaller than the contribution of the good product divided image to the learning process. Therefore, it is preferable that the image around the good product divided image included in the good product surroundings-containing image is reduced. More specifically, it is preferable that the image around the good product divided image is reduced so that the size of the reduced good product surroundings image, i.e., the number of pixels, is smaller than the size of the good product divided image, i.e., the number of pixels. This reduction is performed, for example, by the learning image dataset generation unit 131. At this time, the learning image dataset generation unit 131 may reduce only the image around the good product divided image, or may reduce the remaining image included in the good product surroundings-containing image, for example the good product divided image, together with the image around the good product divided image.

The model generation unit 135 is configured to perform a learning process in which the learning model learns a plurality of learning image datasets 124, and generate a trained model 55. The trained model 55 is a model that receives a non-defective product segmentation image and a non-defective product surroundings-containing image as input, and outputs feature quantities.

As shown in FIG. 5, the learning model 51 used by the model generation unit 135 to generate the trained model 55 is, for example, an autoencoder, which is a type of neural network and also a method of unsupervised machine learning, and the autoencoder includes an input layer 511, an output layer 517, and an intermediate layer 515 arranged between the input layer 511 and the output layer 517.

The learning model 51 is not limited to using an autoencoder. For example, the learning model 51 may use PCA (Principal Component Analysis). In addition, the intermediate layer 515 is not limited to one layer, and may be two or more layers.

The input layer 511 and the output layer 517 each have two channels. Specifically, the input layer 511 has a first input channel 512 and a second input channel 513, and the output layer 517 has a first output channel 518 and a second output channel 519. The non-defective divided image and the corresponding non-defective surroundings-containing image are input to different channels. For example, the non-defective divided image is input to the first input channel 512, and the non-defective surroundings-containing image is input to the second input channel 513. The non-defective divided image and the non-defective surroundings-containing image input to the input layer 511 have their image dimensions reduced in the intermediate layer 515, and the dimensions are restored in the output layer 517 and output. Specifically, an output non-defective divided image corresponding to the non-defective divided image is output from the first output channel 518, and an output non-defective surroundings-containing image corresponding to the non-defective surroundings-containing image is output from the second output channel 519. The autoencoder learns weights (also called coefficients) so that the sum of the difference between the non-defective segmented image and the output non-defective segmented image (hereinafter also called the "first difference") and the difference between the non-defective surroundings-containing image and the output non-defective surroundings-containing image (hereinafter also called the "second difference") is minimized. More specifically, each node represented by a circle is weighted uniquely at each edge represented by a line, and the weighted value is input to the node in the next layer. By learning the weights in this weighting, the sum of the first difference and the second difference is minimized. This learning makes it possible to output features from the intermediate layer 515.

Here, the feature is a variable that quantitatively represents the feature of the object to be found. The feature output from the intermediate layer 515 characterizes the image as a good-quality divided image, and specifically includes the arrangement of shading in the good-quality divided image, the brightness in the good-quality divided image, red alone, green alone, blue alone, or a histogram of red, green, and blue (RGB). In general, it is rare that a good-quality divided image can be recognized as a good-quality divided image using only one type of feature, and it is often possible to recognize that a good-quality divided image is a good-quality divided image using multiple types of feature. As described above, by reducing the contribution of the good-quality surroundings-containing image to the learning process, in other words, by increasing the contribution of the good-quality divided image to the learning process, the trained model 55 outputs a feature for the good-quality divided image that takes the good-quality surroundings-containing image into consideration.

A feature vector is a vector representation of one or more features as components. The number of features represents the dimension of the feature vector. The space represented by the feature vectors, or in other words, the space spanned by the feature vectors, is called the feature space, and the feature vector is represented as a point in the feature space. For example, if the output layer 505 is trained to output d features (d is a positive integer), the trained model 55 can extract a d-dimensional vector feature vector 129 from the input image, and the extracted feature vector 129 can be represented as a point in the d-dimensional feature space.

As shown in FIG. 6, the learning model 52 used by the model generation unit 135 to generate the trained model 55 is, for example, an autoencoder, similar to the learning model 51 shown in FIG. 5, and the autoencoder includes an input layer 521, an output layer 527, and an intermediate layer 525 arranged between the input layer 521 and the output layer 527. Note that the learning model 52 may use, for example, PCA, and the intermediate layer 525 may have two or more layers.

The input layer 521 and the output layer 527 each have one channel. The good-product segmented image and the corresponding good-product surrounding-containing image are combined to generate a combined image. When the combined image is input to the input layer 521, the dimension of the combined image is reduced in the intermediate layer 525. Then, the dimension is returned in the output layer 527 to output an output combined image corresponding to the combined image. The autoencoder learns weights so that the difference between the combined image and the output combined image is minimized. More specifically, each node represented by a circle is uniquely weighted at each edge represented by a line, and the weighted value is input to the node of the next layer. By learning the weights in this weighting, the difference between the combined image and the output combined image is minimized. This learning makes it possible to output features from the intermediate layer 525.

In addition, the model generation unit 135 extracts feature vectors 129 for each good-quality divided image in the process of generating the trained model 55. The extracted feature vectors 129 are written and stored in the storage unit 120. For example, when each of j good-quality images is divided into 25 good-quality divided images and a learning image dataset 124 is generated for each of the k good-quality divided images, the number of feature vectors 129 extracted and stored is, for example, j×k. Note that the dimension d of each of the multiple feature vectors 129 may be the same or different from each other.

In this way, by generating a trained model 55 that is trained to output features using a good-product split image and an image containing good-product surroundings as input, as described in detail below, compared to a case in which the trained model is trained using only good-product split images and the points indicated by the feature vectors of the good-product split image containing a special pattern are plotted at a position far from the points indicated by the feature vectors of the good-product split images containing other patterns, the trained model 55 also trains using images containing good-product surroundings in which the feature values change significantly depending on the position, making it possible to plot the distance between points indicated by the feature vectors for good-product split images at the same position close to each other and the distance between points indicated by the feature vectors for good-product split images at different positions far from each other, thereby making it possible to accurately learn patterns unique to good-product split images, and in feature space, the points indicated by the feature vectors for an inspection split image containing a special pattern can be plotted near the set formed by the feature vectors for multiple good-product split images corresponding to the inspection split image. Furthermore, the trained model 55 can learn different judgment criteria depending on the position and part in the good product image of the inspection target object TA by using the good product surroundings-containing image, and in the feature space, the points indicated by the feature vector for each inspection split image are collected in different ranges and regions depending on the position of the inspection split image in the inspection image. Therefore, in the feature space, a point indicated by the feature vector for an inspection split image at another position, including a pattern that is good at one position but defective at another position, can be plotted far away from the set formed by the multiple feature vectors for the multiple good product split images at the other positions.

At least one of the learning image dataset generation unit 131 and the model generation unit 135 may be realized by the processor 31 executing a program stored in the storage device 33. When executing a program, the program may be stored in a storage medium. The storage medium storing the program may be a non-transitory computer readable medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a USB memory or a CD-ROM (Compact Disc ROM).

Next, functional blocks of an image inspection device according to one embodiment will be described with reference to Figs. 7 to 13. Fig. 7 is a configuration diagram showing the configuration of functional blocks of an image inspection device 20 in one embodiment. Fig. 8 is a conceptual diagram for explaining an example of a method for acquiring the degree of defect. Fig. 9 is a conceptual diagram for explaining another example of a method for acquiring the degree of defect. Fig. 10 is a conceptual diagram for explaining processing by the division unit 240, the inspection image data set generation unit 250, the extraction unit 260, and the acquisition unit 270 shown in Fig. 7. Fig. 11 is a diagram showing an example of an inspection image 60. Fig. 12 is a conceptual diagram for explaining processing by the inspection unit 280 shown in Fig. 7. Fig. 13 is a diagram showing an example of a defect degree image 70.

As shown in FIG. 7, the image inspection device 20 includes a communication unit 210, a memory unit 220, an imaging unit 230, a division unit 240, an inspection image dataset generation unit 250, an extraction unit 260, an acquisition unit 270, and an inspection unit 280.

The communication unit 210 is configured to be able to transmit and receive various types of information. The communication unit 210 receives the trained model 55 and multiple feature vectors 129 from the trained model generation device 10, for example, via the communication network NW. The received trained model 55 and multiple feature vectors 129 are written and stored in the memory unit 220.

The memory unit 220 is configured to store various types of information. For example, the memory unit 220 stores a trained model 55 and a plurality of feature vectors 129. In this way, by providing the memory unit 220 that stores a plurality of trained models 55, the trained model can be easily read out.

The imaging section 230 is for acquiring an inspection image of the inspection object TA. The imaging section 230 is configured to include an imaging device such as a camera. In this embodiment, the imaging section 230 receives reflected light R from the inspection object TA and acquires the inspection image. The imaging section 230 then outputs the acquired inspection image to the division section 240. In this manner, by acquiring the inspection image of the inspection object TA, the inspection image can be easily obtained.

The division unit 240 is configured to divide the inspection image of the inspection object TA into a plurality of inspection split images. More specifically, the division unit 240 divides the inspection image using a method similar to that used to divide a non-defective product image in the trained model generation device 10. Specifically, the division unit 240 divides the inspection image of the inspection object TA into five parts in the vertical and horizontal directions, generating 25 inspection split images. The division unit 240 then outputs the generated inspection split images to the inspection image dataset generation unit 250. In this manner, by dividing the inspection image of the inspection object TA into a plurality of inspection split images, the inspection split images can be easily obtained.

The inspection image dataset generating unit 250 is configured to generate an inspection periphery-containing image for an inspection split image based on images arranged around the inspection split image. The inspection periphery-containing image is an image based on at least a portion of the images around the inspection split image. The inspection image dataset generating unit 250 may generate the inspection periphery-containing image based on a pre-specified algorithm, or may generate the inspection periphery-containing image based on a user operation.

The inspection image dataset generation unit 250 is also configured to generate an inspection image dataset including the inspection division image and the generated inspection surroundings-containing image.

More specifically, the inspection image dataset generating unit 250 generates an inspection periphery-containing image for each of the multiple inspection split images to generate multiple inspection image datasets. In this embodiment, the inspection image dataset generating unit 250 generates an inspection periphery-containing image for 9 inspection split images, excluding the edge inspection split images, of the 25 inspection split images generated by the splitting unit 240. Here, as described above, if the good product periphery-containing image is reduced during the learning process, it is preferable that the inspection image dataset generating unit 250 reduces the image surrounding the inspection split image in accordance with the reduction of the good product periphery-containing image, and the inspection periphery-containing image includes this reduced image. The inspection image dataset generating unit 250 outputs the generated inspection image dataset to the extraction unit 260.

The extraction unit 260 is configured to input the inspection image data set to the trained model 55 and extract a feature vector having the feature amount for the inspection split image included in the inspection image data set as a component. Specifically, the extraction unit 260 reads out the trained model 55 stored in the storage unit 220 and sequentially inputs each of the multiple inspection image data sets to the trained model 55. As described above, since the trained model 55 is trained to output the feature amount of the input image, the extraction unit 260 can obtain one or more feature amounts for the input inspection split image from the trained model 55. This extracts a feature vector having one or more feature amounts as components. Then, the extraction unit 260 outputs the extracted multiple feature vectors to the acquisition unit 270. Note that the feature vector of the inspection split image extracted by the extraction unit 260 is a d-dimensional vector, the same as the feature vector 129 for the non-defective split image.

The acquisition unit 270 is configured to acquire, for each of the extracted feature vectors, a defect degree indicating the degree of defect in the inspection split image corresponding to the feature vector, based on a point indicated by the feature vector in a feature space represented by the feature vector and a set formed by the feature vectors for the non-defective split images. The defect degree may be expressed in a discrete manner, such as a level, step, class, grade, layer, etc., or in a continuous manner, such as a number.

Specifically, the acquisition unit 270 reads out the multiple feature vectors 129 stored in the storage unit 220, and plots the points indicated by each feature vector 129 in the feature space, which is a d-dimensional space. As a result, as shown in FIG. 8, for example, in the two-dimensional feature space, a set S1 indicated by a dashed line is formed by the black dots indicated by each feature vector 129. This set is formed, for example, according to the position in the non-defective product image 40. Note that the acquisition unit 270 may form the set S1 in the feature space in advance based on the multiple feature vectors 129, and write and store information about the set S1 in the storage unit 220.

Next, the acquisition unit 270 plots in the feature space the point indicated by the feature vector for the inspection split image at the position corresponding to the set S1, among the multiple feature vectors input from the extraction unit 260. In FIG. 10, the feature vector for the inspection split image at the corresponding position is indicated by a white circle point P1. Then, the acquisition unit 270 acquires the degree of defect of the inspection split image based on the point P1 indicated by the feature vector for this inspection split image and the set S1.

More specifically, the acquisition unit 270 is configured to acquire the degree of defect of the inspection split image corresponding to each of the extracted feature vectors based on the distance in the feature space between the point indicated by the feature vector and a set formed by the feature vectors for the non-defective split images.

In the example shown in FIG. 8, in a feature space represented by a feature vector whose components are the first feature amount and the second feature amount, the acquisition unit 270 acquires the degree of defect of an inspection split image based on the distance between point P1 indicated by the feature vector for a certain inspection split image and a set S1 of multiple feature vectors for multiple good product split images corresponding to the inspection split image. This degree of defect is, for example, a value according to the distance. In this way, the degree of defect of the inspection split image corresponding to the feature vector is acquired based on the distance in the feature space between point P1 indicated by the feature vector and a set S1 formed by multiple feature vectors for multiple good product split images, so that the degree of defect of the inspection split image can be easily indicated.

Alternatively, the acquisition unit 270 may be configured to acquire the degree of defect of the inspection split image corresponding to each of the extracted feature vectors based on the distance in the feature space between a point indicated by the feature vector and a point indicated by one of the feature vectors for the non-defective split images.

For example, as shown in FIG. 9, in a feature space represented by a feature vector having the first feature amount and the second feature amount as components, the acquisition unit 270 acquires the degree of defect of an inspection split image based on the distance between point P2 indicated by the feature vector of a certain inspection split image and a point included in a set S2 of multiple feature vectors for multiple good product split images corresponding to the inspection split image. This degree of defect is, for example, a value according to the distance. In addition, the point included in the set S2 is, for example, the point closest to point P2 among the multiple points included in the set S2. The acquisition unit 270 can calculate the distance between each of the multiple points included in the set S2 and point P2 and determine the point closest to point P2. In this way, the degree of defect of the inspection split image corresponding to the feature vector is acquired based on the distance in the feature space between point P2 indicated by the feature vector and one of the multiple feature vectors for the multiple good product split images, so that the degree of defect of the inspection split image can be easily indicated.

The acquisition unit 270 repeats the above procedure for each of the multiple feature vectors extracted by the extraction unit 260 to acquire the defect degree of each of the multiple inspection split images. The acquisition unit 270 then outputs the multiple acquired defect degrees to the inspection unit 280.

As shown in FIG. 10, the inspection image 42 of the inspection object TA acquired by the imaging unit 230 is divided into five parts vertically and horizontally by the division unit 240 to generate 25 inspection split images. For each of the nine inspection split images excluding the edge inspection split images out of the 25 inspection split images generated, an inspection periphery-containing image is generated by the inspection image dataset generation unit 250. For example, a first inspection periphery-containing image 424 is generated for the first inspection split image 420, and a second inspection periphery-containing image 426 is generated for the second inspection split image 422. The pair of the inspection split image and the inspection periphery-containing image becomes the inspection image dataset.

The extraction unit 260 inputs each of the nine inspection image data sets generated into the trained model. The feature amounts for each of the nine inspection split images are output from the trained model, and the extraction unit 260 extracts nine feature vectors whose components are the feature amounts output for each of the nine inspection image data sets. For example, the first feature vector 440 is extracted based on the first inspection split image 420 and the first inspection surroundings-containing image 424, and the second feature vector 442 is extracted based on the second inspection split image 422 and the second inspection surroundings-containing image 426.

For each of the nine extracted feature vectors, the acquisition unit 270 acquires the defect degree of the inspection split image corresponding to that feature vector based on the point indicated by that feature vector and a set of multiple feature vectors for multiple good product split images. This allows the defect degree of each of the nine inspection split images to be acquired. For example, the first defect degree 460 is acquired based on the first feature vector 440, and the second defect degree 462 is acquired based on the second feature vector 442.

As shown in FIG. 11, an inspection image 60 of an inspection object TA, which is known in advance to be a non-defective product, is divided, for example, into six inspection split images 600, 602, 604, 606, 608, and 610. Of these six inspection split images, inspection split images 602, 606, and 608 contain similar patterns. Furthermore, inspection split image 604 contains a special pattern that is different from the other inspection split images.

Unlike the trained model 55 of this embodiment, if a trained model is generated using these six inspection split images without using a good product surroundings-containing image, the trained model can extract and output feature quantities of the patterns of the inspection split images 602, 604, or 608, but depending on the expressive ability of the trained model, the feature quantities of the special pattern contained in the inspection split image 604 may not be sufficiently extracted. In this case, in the feature space, the point indicated by the feature vector of the inspection split image 604 containing the special pattern may be plotted at a position away from the set formed by the feature vectors of the other inspection split images 600, 602, 606, 608, and 610. As a result, the degree of defect of the inspection split image 604 becomes large, and the inspection target TA, which is a good product, may be determined to be a defective product.

In contrast, the image inspection device 20 of this embodiment inputs the inspection split images 600, 602, 604, 606, 608, and 610 and the inspection periphery-containing image into a trained model 55 that has been trained to output features using images containing the good product periphery as input in addition to the good product split images, and extracts feature vectors for the inspection split images 600, 602, 604, 606, 608, and 610. As a result, compared to a case where only good-product divided images are used for learning and points indicated by feature vectors of good-product divided images containing special patterns are plotted far from points indicated by feature vectors of good-product divided images containing other patterns, the learned model 55 learns using images containing good-product surroundings in which the feature amount changes greatly depending on the position, making it possible to plot points indicated by feature vectors for good-product divided images at the same position close to each other and points indicated by feature vectors for good-product divided images at different positions far from each other, so that patterns specific to good-product divided images can be accurately learned, and in the feature space, the points indicated by the feature vectors for each of the inspection divided images 600, 602, 604, 606, 608, and 610 are plotted near the sets formed by the feature vectors for the corresponding good-product divided images. Therefore, in the feature space, the points indicated by the feature vector for the inspection divided image 604 containing a special pattern can be plotted near the sets formed by the feature vectors for the multiple good-product divided images corresponding to the inspection divided image 604, and a defect degree with a small degree of defect can be obtained to determine that the product is good.

Furthermore, when generating a trained model using an inspection split image that includes a pattern that is good in one position but defective in another position in an inspection image of another inspection object TA that is known in advance to be a good product, the point indicated by the feature vector of the inspection split image that includes such a pattern may be plotted in the feature space near the set formed by the feature vector of the other inspection split image. As a result, the degree of defect of the inspection split image that includes such a pattern in the other position becomes smaller, and the inspection object TA of the inspection image that includes the inspection split image that is actually defective may be judged as a good product, and the defective product may be overlooked.

Even in such a case, in the image inspection device 20 of this embodiment, the trained model 55 can learn different judgment criteria depending on the position and part in the good product image of the inspection target object TA by using the image containing the good product surroundings, and in the feature space, the points indicated by the feature vector for each inspection split image are collected in different ranges and regions depending on the position of the inspection split image in the inspection image. Therefore, in the feature space, a point indicated by the feature vector for the inspection split image at another position, including a pattern that is good at one position but defective at another position, can be plotted far away from the set formed by the multiple feature vectors for the multiple good product split images at the other positions, and the degree of defect of a large degree of defect can be obtained to prevent overlooking defective products.

In addition, the inspection periphery-containing image in this embodiment includes a reduced image of the image surrounding the inspection split image. This makes it possible to suppress the influence of the inspection periphery-containing image from becoming too large in the output of features by the trained model 55, and makes it possible to extract feature vectors for the inspection split image with high accuracy.

Returning to the explanation of FIG. 7, the inspection unit 280 is configured to inspect the inspection object TA based on the degree of defect. In this way, the degree of defect of the inspection split image is obtained based on the point indicated by the feature vector for the inspection split image in the feature space and a set formed by multiple feature vectors for multiple good product split images, and the inspection object TA is inspected based on this degree of defect, thereby improving the inspection accuracy of the inspection object TA.

The aforementioned degree of defect is preferably a value indicating the degree of defect in the inspection split image. This makes it possible to quantitatively indicate the degree of defect in the inspection split image.

In this case, the inspection unit 280 is configured to generate a defect degree image based on the multiple defect degrees acquired by the acquisition unit 270, and to inspect the inspection object TA based on the defect degree image. This makes it easy to determine whether the inspection object is a good or defective item, and makes it easy to achieve inspection with high inspection accuracy.

Specifically, as shown in FIG. 12, the inspection unit 280 generates a plurality of partial images 480, 482, 484, ... by imaging the plurality of defect degrees 460, 462, 464, ... acquired by the acquisition unit 270 based on their respective values. Each of the plurality of partial images 480, 482, 484, ... is, for example, a grayscale image in which the defect degree value is converted into black and white gradation, or a color image in which the defect degree value is converted into RGB gradation. The inspection unit 280 generates a defect degree image 48 by integrating the generated plurality of partial images 480, 482, 484, .... The vertical and horizontal sizes (number of pixels) of the defect degree image 48 may be the same as or different from the inspection image 42. Then, the inspection unit 280 judges whether the inspection target TA is a non-defective product based on the defect degree image 48.

As shown in FIG. 13, the defect degree image 70 includes, for example, two defect portion images 700 and 702. The defect portion image 700 is a partial image with a relatively low degree of defect, and the defect portion image 702 is a partial image with a relatively high degree of defect.

For example, the inspection unit 280 determines that the inspection object TA is a good product when the proportion of the defective part images 700, 702 in the defect degree image 70 is equal to or less than a predetermined threshold, and determines that the inspection object TA is not a good product, i.e., a defective product, when the proportion exceeds the predetermined threshold.

Alternatively, the inspection unit 280 may detect defects in the inspection object TA based on the presence or absence of defective portion images 700, 702 contained in the defect degree image 70.

At least one of the division unit 240, the inspection image data set generation unit 250, the extraction unit 260, the acquisition unit 270, and the inspection unit 280 may be realized by the processor 31 executing a program stored in the storage device 33. When executing a program, the program may be stored in a storage medium. The storage medium storing the program may be a computer-readable non-transient storage medium. The non-transient storage medium is not particularly limited, and may be, for example, a storage medium such as a USB memory or a CD-ROM.

Next, a processing procedure performed by a trained model generation device according to one embodiment will be described with reference to FIG. 14. FIG. 14 is a flowchart for explaining an example of a trained model generation process S100 performed by the trained model generation device 10 in one embodiment.

As shown in FIG. 14, first, the communication unit 110 acquires a plurality of non-defective product images 40 via the communication network NW (step S101). The acquired non-defective product images 40 are stored in the storage unit 120.

Next, the learning image dataset generation unit 131 reads out a plurality of good-quality images 40 from the storage unit 120, and generates a plurality of learning image datasets 124 based on the plurality of good-quality images 40 (step S102). As described above, the learning image dataset 124 includes a good-quality divided image and an oil-containing image of the surrounding area of the good-quality product.

Next, the model generation unit 135 generates a trained model 55 that is trained to output features using the multiple training image datasets 124 generated in step S102 as input (step S103). The generated trained model 55 and multiple feature vectors 129 extracted during the generation process of the trained model 55 are stored in the storage unit 120.

Next, the communication unit 110 transmits the trained model 55 generated in step S103 and the multiple feature vectors 129 extracted during the generation process to the image inspection device 20 via the communication network NW (step S104). This allows the image inspection device 20 to use the trained model generated by the trained model generation device 10.

After step S104, the trained model generation device 10 terminates the trained model generation process S100.

Next, the processing procedure performed by the image inspection device according to one embodiment will be described with reference to FIG. 15. FIG. 15 is a flowchart for explaining an example of the image inspection process S200 performed by the image inspection device 20 in one embodiment.

In the following example, it is assumed that the communication unit 210 receives the trained model 55 and multiple feature vectors 129 from the trained model generation device 10, and that the trained model 55 and multiple feature vectors 129 are stored in the storage unit 220.

As shown in FIG. 15, first, the imaging unit 230 acquires an inspection image of the inspection object TA (step S201). The acquired inspection image is output to the division unit 240.

Next, the division unit 240 divides the inspection image acquired in step S201, and the inspection image dataset generation unit 250 generates an inspection periphery-containing image for each of the divided inspection divided images to generate multiple inspection image datasets (step S202). As described above, the inspection image dataset includes the inspection divided image and the inspection periphery-containing image. The generated multiple inspection image datasets are output to the extraction unit 260.

Next, the extraction unit 260 reads out the trained model 55 previously stored in the storage unit 220, inputs each of the multiple test image data sets generated in step S207 into the trained model 55, and extracts multiple feature vectors (step S203). The generated multiple feature vectors are output to the acquisition unit 270.

Next, the acquisition unit 270 reads out the multiple feature vectors 129 previously stored in the storage unit 220, and for each of the multiple feature vectors extracted in step S207, acquires the defect degree of the multiple inspection split images based on the point indicated by the feature vector in the feature space and the set formed by the multiple feature vectors 129 for the corresponding multiple non-defective split images (step S204). The multiple defect degrees acquired are output to the inspection unit 280.

Next, the inspection unit 280 generates multiple partial images from each of the multiple defect degree values obtained in step S204, and integrates each of the generated partial images to generate a defect degree image (step S205).

Next, the inspection unit 280 inspects the inspection object TA based on the defect degree image generated in step S205 (step S206).

After step S206, the image inspection device 20 terminates the image inspection process S200.

The sequence and flow chart described in this embodiment may be rearranged as long as no inconsistencies occur in the processing.

The above describes an exemplary embodiment of the present invention. According to the image inspection device 20 and image inspection method of this embodiment, the inspection divided images 600, 602, 604, 606, 608, and 610 and the inspection surroundings-containing image are input to the trained model 55, which has been trained to output features using a good-product divided image and a good-product surroundings-containing image as input, and feature vectors for the inspection divided images 600, 602, 604, 606, 608, and 610 are extracted. As a result, compared to a case where only good-product divided images are used for learning and the points indicated by the feature vectors of the good-product divided images including the special pattern are plotted far from the points indicated by the feature vectors of the good-product divided images including other patterns, the learned model 55 learns using images including the good-product surroundings, in which the feature amount changes greatly depending on the position, so that the distance between the points indicated by the feature vectors for the good-product divided images at the same position can be plotted close to each other and the distance between the points indicated by the feature vectors for the good-product divided images at different positions can be plotted far from each other, so that the patterns specific to the good-product divided images can be accurately learned, and in the feature space, the points indicated by the feature vectors for each of the inspection divided images 600, 602, 604, 606, 608, and 610 are plotted close to the sets formed by the feature vectors for the corresponding good-product divided images. Therefore, in the feature space, the points indicated by the feature vectors for the inspection divided image 604 including the special pattern can be plotted close to the sets formed by the feature vectors for the multiple good-product divided images corresponding to the inspection divided image 604, and a defect degree with a small degree of defect can be obtained to determine the product as a good product. In addition, the trained model 55 can learn different judgment criteria depending on the position and part in the good product image of the inspection target TA by using the good product surroundings-containing image, and in the feature space, the points indicated by the feature vectors for each inspection split image are collected in different ranges and regions depending on the position of the inspection split image in the inspection image. Therefore, in the feature space, the points indicated by the feature vectors for the inspection split image at another position, including a pattern that is good at one position but defective at another position, can be plotted far away from the set formed by the multiple feature vectors for the multiple good product split images at the other positions, and the defect degree of the large defect can be obtained to suppress overlooking of defective products. Therefore, the defect degree of the inspection split image can be obtained based on the points indicated by the feature vectors for the inspection split image and the set formed by the multiple feature vectors for the multiple good product split images in the feature space, and the inspection target TA can be inspected based on this defect degree, thereby improving the inspection accuracy of the inspection target TA.

In addition, according to the trained model generating device 10 of this embodiment, a trained model 55 is generated that is trained to output features using a good-product divided image and a good-product surroundings-containing image as inputs. As a result, compared to a case where only good-product divided images are used for training and a point indicated by a feature vector of a good-product divided image including a special pattern is plotted at a position far from a point indicated by a feature vector of a good-product divided image including other patterns, the trained model 55 also uses images including good-product surroundings in which the feature amount changes greatly depending on the position, so that the distance between the points indicated by the feature vectors for good-product divided images at the same position can be plotted close to each other and the distance between the points indicated by the feature vectors for good-product divided images at different positions can be plotted far away, so that a pattern specific to a good-product divided image can be accurately learned, and in the feature space, a point indicated by a feature vector for an inspection divided image including a special pattern can be plotted close to a set formed by the feature vectors for multiple good-product divided images corresponding to the inspection divided image. Furthermore, the trained model 55 can learn different judgment criteria depending on the position and part in the good product image of the inspection target object TA by using the good product surroundings-containing image, and in the feature space, the points indicated by the feature vector for each inspection split image are collected in different ranges and regions depending on the position of the inspection split image in the inspection image. Therefore, in the feature space, a point indicated by the feature vector for an inspection split image at another position, including a pattern that is good at one position but defective at another position, can be plotted far away from the set formed by the multiple feature vectors for the multiple good product split images at the other positions.

The above-described embodiments are intended to facilitate understanding of the present invention, and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents are also included in the present invention. In other words, designs modified by a person skilled in the art as appropriate are also included within the scope of the present invention as long as they include the characteristics of the present invention. For example, the elements of the embodiments and their arrangements, materials, conditions, shapes, sizes, etc. are not limited to those exemplified, and can be modified as appropriate. Furthermore, the embodiments are merely examples, and it goes without saying that partial substitution or combination of the configurations shown in different embodiments is possible, and these are also included within the scope of the present invention as long as they include the characteristics of the present invention.

[Appendix 1]
an extraction unit (260) that inputs an inspection split image, which is a split image of an inspection object (TA) that is a non-defective item, and an inspection periphery-containing image, which is based on an image surrounding the inspection split image, into a trained model (55) that has been trained to output features using as input a non-defective item split image, which is a split image of an inspection object (TA) that is a non-defective item, and a non-defective item surroundings-containing image, which is based on an image surrounding the inspection split image, and extracts feature vectors each having features for the inspection split image as components;
an acquisition unit (270) that acquires a defect degree indicating the degree of defect of the inspection division image corresponding to the extracted feature vector based on a point indicated by the feature vector and a set formed by a plurality of feature vectors (129) for a plurality of non-defective division images in a feature space represented by the extracted feature vector;
An inspection unit (280) that inspects an inspection target (TA) based on a defect degree.
An image inspection device (20).
[Appendix 9]
A trained model (55) is trained to output features by inputting a good-quality divided image, which is a divided image of a good-quality inspection object (TA), and a good-quality surroundings-containing image, which is based on an image surrounding the good-quality divided image, into the trained model (55), and a step of inputting an inspection divided image, which is a divided image of the inspection object (TA), and an inspection surroundings-containing image, which is based on an image surrounding the inspection divided image, into the trained model (55), and extracting feature vectors each having features for the inspection divided image as components;
A step of acquiring a defect degree indicating the degree of defect of the inspection division image corresponding to the extracted feature vector based on a point indicated by the feature vector and a set formed by a plurality of feature vectors (129) for a plurality of non-defective division images in a feature space represented by the extracted feature vector;
Inspecting the inspection object (TA) based on the defect degree.
Imaging methods.
[Appendix 10]
A model generating unit (135) is provided for generating a trained model (55) trained to output features using a non-defective divided image, which is a divided image of a non-defective inspection target (TA), and a non-defective surroundings-containing image based on an image surrounding the non-defective divided image as input.
A trained model generating device (10).

1...Image inspection system, 10...Model generation device, 20...Image inspection device, 31...Processor, 32...Memory, 33...Storage device, 34...Communication device, 35...Input device, 36...Output device, 40...Good product image, 42...Inspection image, 48...Defect degree image, 51...Learning model, 52...Learning model, 55...Learning model, 60...Inspection image, 70...Defect degree image, 110...Communication unit, 120...Storage unit, 124...Learning image data sensor , 129... feature vector, 130... learning unit, 131... learning image data set generation unit, 135... model generation unit, 210... communication unit, 220... storage unit, 230... imaging unit, 240... division unit, 250... inspection image data set generation unit, 260... extraction unit, 270... acquisition unit, 280... inspection unit, 400... first non-defective divided image, 402... second non-defective divided image, 404... first non-defective surroundings-containing image, 406... second non-defective surroundings-containing image, 42 0...first inspection divided image, 422...second inspection divided image, 424...first inspection periphery-containing image, 426...second inspection periphery-containing image, 440...first feature vector, 442...second feature vector, 480...partial image, 482...partial image, 484...partial image, 511...input layer, 512...first input channel, 513...second input channel, 515...intermediate layer, 517...output layer, 518...first output channel, 519...second output channel, 5 21...input layer, 525...intermediate layer, 527...output layer, 600...inspection split image, 602...inspection split image, 604...inspection split image, 606...inspection split image, 608...inspection split image, 700...defective part image, 702...defective part image, IL...illumination, L...light, NW...communication network, P1...point, P2...point, R...reflected light, S1...set, S2...set, S100...model generation process, S200...image inspection process, TA...inspection object.

Claims (10)

an extraction unit that inputs an inspection split image, which is a split image of an inspection object, and an inspection periphery-containing image, which is based on an image surrounding the inspection split image, to a trained model that has been trained to output features using a good-item split image, which is a split image of an inspection object of a good item, and a good-item surroundings-containing image, which is based on an image surrounding the inspection split image, and extracts feature vectors each having the feature amount for the inspection split image as components;
an acquisition unit that acquires a defect degree indicating the degree of defect of the inspection divided image corresponding to the feature vector based on a point indicated by the extracted feature vector in a feature space represented by the feature vector and a set formed by a plurality of feature vectors for a plurality of the non-defective divided images;
and an inspection unit that inspects the inspection object based on the defect degree.
Image inspection equipment. The inspection surroundings-containing image includes an image obtained by reducing an image surrounding the inspection divided image.
The image inspection device according to claim 1 . the acquiring unit acquires the defect degree of the inspection divided image corresponding to the extracted feature vector based on a distance between a point indicated by the feature vector and the set in a feature space represented by the extracted feature vector.
3. An image inspection device according to claim 1 or 2. the acquiring unit acquires the defect degree of the inspection divided image corresponding to the extracted feature vector based on a distance between a point indicated by the feature vector and a point indicated by one of the plurality of feature vectors for the plurality of non-defective divided images in a feature space represented by the extracted feature vector;
3. An image inspection device according to claim 1 or 2. The defect degree is a value indicating the degree of a defect in the inspection divided image.
5. An image inspection device according to claim 1. the inspection unit generates a defect degree image based on the plurality of defect degrees, and inspects the inspection object based on the defect degree image.
6. An image inspection device according to claim 5. A division unit that divides an inspection image of an inspection object into a plurality of the inspection divided images.
7. An image inspection device according to claim 1. Further comprising an imaging unit for acquiring an inspection image of the inspection object.
An image inspection device according to any one of claims 1 to 7. a step of inputting an inspection split image, which is a split image of an inspection object, and an inspection periphery-containing image, which is based on an image surrounding the inspection split image, into a trained model that has been trained to output features using a good-item split image, which is a split image of an inspection object of a good item, and a good-item surroundings-containing image, which is based on an image surrounding the inspection split image, and extracting feature vectors each having the feature amounts for the inspection split image as components;
acquiring a defect degree indicating the degree of defect of the inspection divided image corresponding to the extracted feature vector based on a point indicated by the feature vector and a set formed by a plurality of feature vectors for a plurality of the non-defective divided images in a feature space represented by the extracted feature vector;
and inspecting the inspection object based on the defect degree.
Imaging methods. A model generating unit is provided which generates a trained model which is trained to output a feature amount by inputting a non-defective divided image which is a divided image of an inspection target of a non-defective product and a non-defective surroundings-containing image based on an image surrounding the non-defective divided image.
A trained model generator.

Images