JP6650986B2 - Image inspection apparatus, image inspection method, image inspection program, computer-readable recording medium, and recorded device - Google Patents

Description

  The present invention relates to an image inspection apparatus using a photometric stereo method, an image inspection method, an image inspection program, and a computer-readable recording medium or a recorded apparatus.

  2. Description of the Related Art An image inspection apparatus that performs an inspection such as the presence or absence of a scratch on a surface of a work (inspection object or subject), an external shape, and reading of printed characters is used. Such an image inspection apparatus illuminates a workpiece with necessary illumination, captures an image, performs image processing such as edge detection on the obtained image data, and determines quality based on the result. Had gone.

  However, in such an image inspection apparatus, the visibility sometimes changes depending on the type of illumination and the direction of application depending on the type of work. Therefore, sufficient knowledge and experience were required to perform an appropriate inspection for such a work.

  Further, in the conventional image inspection apparatus, there is also a problem that erroneous detection is apt to occur even if the illumination condition and the installation condition are slightly changed, and it is difficult to perform stable detection. Furthermore, in the visual inspection of a workpiece, information on the shape of the workpiece such as a scratch or an edge, in other words, both three-dimensional information and two-dimensional information such as printing and spots are to be inspected. As a result, there has been a problem that detection may not be performed well.

  As a technique for solving such a problem, an image processing apparatus that acquires height information by using a photometric stereo method is known (for example, see Patent Document 1). Here, the photometric stereo method (photometric stereo method) is also called a photometric stereo method, and is a method of three-dimensional measurement in which images illuminated from a plurality of different illumination directions are captured, and a normal vector of the workpiece is obtained from the shadow information. This is one of the methods. An image processing apparatus using such a photometric stereo method is an image in which a component of the X-axis or the Y-axis is replaced with a luminance value from a normal vector (corresponding to a tilt image), or a reflectance image (corresponding to an albedo image). Has been created and applied to image inspection. Here, in order to perform three-dimensional measurement with high accuracy by the photometric stereo method, mainly, a method of installing a lighting device and a method of irradiating illumination light are studied.

JP 2007-206797 A

  However, in image inspection based on the photometric stereo method, since accurate positional relationship information between the light source and the work is required to calculate an accurate normal vector, for example, the surface on which the work is placed is inclined from a horizontal plane. In addition, if the lighting is tilted, a large number of large changes occur, and there is a problem that accuracy is reduced. Further, there is a problem that a large amount of images under different illumination directions are required to calculate an accurate normal vector, and the processing time becomes enormous. On the other hand, even if the height dimension of the work cannot always be detected accurately, it is often sufficient for inspection purposes.

  The present invention has been made in view of such a problem, and a main object of the present invention is to use a photometric stereo method to maintain a sufficient accuracy for practical use, and to more easily damage a work piece. It is an object of the present invention to provide an image inspection apparatus, an image inspection method, an image inspection program, and a computer-readable recording medium or a device on which printing can be inspected.

Means for Solving the Problems and Effects of the Invention

According to the image inspection apparatus according to the first aspect of the present invention, an image inspection apparatus for visual inspection of the work, the three or more lighting block is arranged annularly, the three or more lighting blocks work and an annular illumination unit which is illuminable constructed from different illumination directions, are connected via the annular illumination unit each lighting block and a single cable or each different cables, given each of the respective lighting blocks The illumination branch unit to be turned on at the timing of, and the annular lighting unit are provided separately, and at the timing at which the three or more lighting blocks are turned on, by imaging the work from a certain direction, the lighting direction a camera for imaging a plurality of different partial illumination images, is connected via the lighting branch unit and the camera and cable, An image processing unit that generates a contour extracted image showing the inclination of the contour of the workpiece surface based on serial partial illumination image, the image processing unit or provided in the illumination branch unit integral, one by one said three or more lighting block Lighting control means for controlling lighting in the lighting order; display means capable of displaying the contour extraction image generated by the image processing unit ; and setting of an inspection area on the contour extraction image displayed on the display means and a test area setting means capable, the image processing unit, a plurality of partial illumination between images captured by the camera, using the pixel value of each pixel in the corresponding relationship, based on photometric stereo method , changes according to the installation position of the camera in the optical axis direction of the camera and the installation position of the annular lighting unit changes relatively, word of each pixel Of calculating a normal vector to the surface, to generate an X-coordinate component image and the Y coordinate component image corresponding to the x and y components of the normal vector, a differential processing in the X direction and Y direction of the normal vector subjected, to generate the contour extracted image showing the inclination of the contour of the surface of the workpiece, the inspection of the inspection region set in the inspection area setting means capable intends row. With the above configuration, the normal vector is differentiated, a contour image indicating the contour of the inclination of the surface of the workpiece is generated, and the contour image is applied as an image to be inspected, thereby reducing the influence of the inclination of the workpiece and illumination. An image inspection that is hard to receive and that is more robust to the environment than conventional height measurement using a photometric stereo can be realized.

According to the image inspection apparatus of the second aspect, the image processing unit further performs photometric measurement on the plurality of partial illumination images captured by the camera using pixel values of corresponding pixels. A normal vector calculation means for calculating a normal vector to the surface of the work of each pixel, which changes according to a change in the relative position between the camera and the illumination block in the optical axis direction of the camera, based on the stereo method; A contour image for performing a differentiation process in the X direction and the Y direction with respect to a normal vector of each pixel calculated by the normal vector calculating means to generate a contour image indicating a contour of a tilt of a surface of a work; a generation unit, wherein the normal vector of each pixel calculated by the normal vector calculating means, said calculating the albedo for each pixel, from the albedo, the surface of the workpiece slope shape Includes a texture extraction image generating means for generating a texture extraction image showing a pattern was removed, it can be switched and the contour image generating means and the texture extraction image generating means.

Further , according to the image inspection apparatus according to the third aspect, the texture extraction image generation means exists from the normal vector of each pixel calculated by the normal vector calculation means as many times as the number of times of imaging by the camera. Calculating the albedo for each pixel of the plurality of partial illumination images, sorting the albedo values of the corresponding pixels for each pixel of the plurality of partial illumination images, and adopting pixel values in a specific order from above Can be used as a texture extraction image.

Still further, according to the image inspection apparatus of the fourth aspect, the camera and the illumination unit are independent members, and can be arranged at any position. With the above configuration, the lighting means and the camera can be arranged according to the actual work and the installation environment, so that a highly flexible installation with an increased degree of freedom in arrangement can be realized.

Furthermore, according to the image inspection device of the fifth aspect, four illumination blocks can be provided.

Still further, according to the image inspection apparatus of the sixth aspect, the three or more lighting blocks are configured by a plurality of light emitting elements arranged in a ring, and the lighting control unit is configured to control the adjacent light emitting elements. As a first lighting block, the light emitting elements of the first lighting block are simultaneously turned on and the other light emitting elements are turned off, so that the first lighting means functions as the first lighting from the first lighting direction. Controlling the second illumination block, which is adjacent to the first illumination block and includes a predetermined number of light-emitting elements, to illuminate from a second illumination direction different from the first illumination direction. A first lighting direction and a second lighting direction by controlling a third lighting block formed of a predetermined number of light emitting elements adjacent to the second lighting block to be turned on. You can configure the third illumination block for illuminating the third illumination direction that. With the above configuration, one illumination unit in which a plurality of light emitting elements are arranged in a ring can be divided into three or more illumination blocks functioning as three or more illumination units, and the configuration of the illumination unit can be further simplified.

Still further, according to the image inspection apparatus of the seventh aspect, the image processing unit creates a plurality of reduced tilt images having different reduction rates from the calculated normal vectors of the respective pixels, and generates each reduced tilt image. In the image, differentiation processing is performed in the X and Y directions, and the obtained reduced outline image is weighted so that the reduced outline image at a predetermined reduction ratio is reflected, and is enlarged to the original size. Therefore, the sum of all the enlarged reduced outline images can be used as the outline extraction image.

Furthermore, according to the image inspection apparatus according to the eighth aspect, the weighting is performed by preparing a set of predetermined weights and multiplying the reduced contour image by the set of weights. An image inspection apparatus characterized in that the inspection is performed by apportioning the adoption ratio of reduced outline images at each reduction ratio.

Still further, according to the image inspection apparatus of the ninth aspect, the set of weights may include a set in which the adoption ratio of a reduced contour image that can obtain a contour extraction image in which unevenness on the surface of a workpiece is clear is increased. it can.

Furthermore, according to the image inspection apparatus of the tenth aspect, the set of weights can include a set in which the adoption ratio of reduced contour images that can obtain contour extracted images suitable for OCR increases.

Further, according to the image inspection method according to the eleventh aspect, the inspection method is to take an image of the work and inspect the appearance, wherein the work is formed by annularly arranging three or more lighting blocks. The illumination block is configured to be able to illuminate the work from different illumination directions with the annular illumination unit, and from three or more different illumination directions, with the illumination control means provided integrally with the image processing unit or the illumination branch unit, The three or more lighting blocks are lit and lit one by one in the lighting order, and the work is provided in a fixed direction at a timing at which the three or more lighting blocks are provided separately from the annular lighting unit. by imaging from for imaging a plurality of partial lighting images of different illuminating directions to adjust the relative position and imaging direction of advance the illumination block Using a common camera, and images the one partial illumination image for each illumination direction, a step of acquiring a plurality illumination direction different partial illumination images, different in a plurality of partial illumination between images of illumination direction, using the pixel value of each pixel in the corresponding relationship, based on photometric stereo method, the installation position of the camera in the optical axis direction of the camera and the installation position of the annular lighting unit according to relative changes The normal vector to the surface of the work of each pixel, which varies with the lighting block, is connected to each lighting block of the annular lighting unit via a single cable or a plurality of different cables, and each of the lighting blocks is connected to a predetermined cable. the lighting branch unit and through the camera and cable are turned on with the timing calculated in the connected the image processing section, and the x component of the normal vector Generates X-coordinate component image and the Y coordinate component image corresponding to the component is subjected to differential processing in the X direction and Y direction of the normal vector, contour extraction indicating the inclination of the contour of the workpiece surface based on the partial illumination image Generating an image by the image processing unit , displaying the contour extracted image on a display unit, setting an inspection region on the displayed contour extracted image, and examining the inspection in the set inspection region. Performing the steps.

Still further, according to the image inspection program according to the twelfth aspect, the inspection program is for inspecting the appearance by capturing an image of a work, wherein the work is arranged in a computer in which three or more lighting blocks are arranged in a ring. The three or more lighting blocks are provided integrally with the image processing unit or the lighting branching unit from the three or more different lighting directions , with the annular lighting unit configured to be able to illuminate the work from different lighting directions . In the lighting control means, the three or more lighting blocks are lit and illuminated one by one in the lighting order, and the three or more lighting blocks, which are provided separately from the annular lighting unit, are respectively lit, by imaging the work from a certain direction, for imaging a plurality of partial lighting images of different illuminating directions, advance the illumination Bro Using a common camera to adjust the relative position and imaging direction of the click, by photographing one of the partial illumination image for each illumination direction, a function of acquiring a plurality illumination direction different partial illumination images, the illumination A plurality of partial illumination images in different directions, using a pixel value of each pixel in a corresponding relationship, based on a photometric stereo method, based on an installation position of the camera in an optical axis direction of the camera and an installation of the annular illumination unit. A normal vector with respect to the surface of the work of each pixel, which changes according to a relative change in the position, is connected to each lighting block of the annular lighting unit via a single cable or a plurality of different cables. in is, the said image processing unit which respectively through the illumination branch unit and the camera and cable to be turned at a predetermined timing are connected to each lighting block Out to generate the X-coordinate component image and the Y coordinate component image corresponding to the x and y components of the normal vector, a differential processing on the X and Y directions of the normal vectors, the partial illumination image A function of generating, by the image processing unit , a contour extraction image indicating a contour of the inclination of the work surface based on the image; a function of displaying the contour extraction image on a display unit; and an inspection area on the displayed contour extraction image. And a function of performing an inspection in the set inspection region.

  Further, a computer-readable recording medium or a stored device of the present invention stores the image inspection program. Recording media include CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray (products) , HD DVD (AOD), and other magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and other media capable of storing programs. The programs include those distributed by being downloaded via a network line such as the Internet, in addition to those stored and distributed on the recording medium. Further, the stored devices include general-purpose or special-purpose devices in which the above-mentioned programs are mounted in a state where the programs can be executed in the form of software or firmware. Furthermore, each process and function included in the program may be executed by program software executable by a computer, or the process of each unit may be performed by hardware such as a predetermined gate array (FPGA, ASIC) or program software. And a partial hardware module that realizes some elements of the hardware.

1 is a block diagram illustrating a configuration of an image inspection device according to a first embodiment of the present invention. FIG. 2 is a schematic plan view illustrating a positional relationship between an imaging unit and a lighting unit of the image inspection device in FIG. 1. FIG. 2 is a schematic side view illustrating a positional relationship between an imaging unit and a lighting unit of the image inspection apparatus in FIG. 1. It is a schematic diagram which shows the illumination means which implement | achieves four illumination blocks concerning Embodiment 2. FIG. 5 is a schematic plan view illustrating how five lighting blocks are realized by the lighting unit of FIG. 4. FIG. 14 is a schematic plan view illustrating a lighting unit according to a third embodiment. FIG. 14 is a schematic plan view illustrating a lighting unit according to a fourth embodiment. It is a schematic plan view which shows the illumination means which concerns on a modification. It is a schematic plan view which shows the illumination means which concerns on another modification. FIG. 14 is a schematic diagram illustrating a configuration of an image inspection device according to a fifth embodiment. FIG. 14 is a schematic diagram illustrating a configuration of an image inspection device according to a sixth embodiment. FIG. 3 is a block diagram for describing a signal processing system of an image processing unit. FIG. 13A is a diagram illustrating the positional relationship between the diffuse reflection surface S and the illumination for explaining the basic principle of the photometric stereo method, FIG. 13B is a diagram illustrating a state where light is irradiated from L1, and FIG. FIG. 13D is a diagram showing a state where light is irradiated, and FIG. 13D is a diagram for explaining estimating the surface direction from a combination of the irradiation direction and the brightness of the reflected light. FIG. 14A is a diagram illustrating an example of a tilt image differentiated in the vertical direction for explaining a method of generating a contour extracted image, FIG. 14B is a diagram illustrating an example of a tilt image differentiated in a horizontal direction, and FIG. 14C is a contour extracted image It is a figure showing an example of. FIG. 15A is a diagram illustrating surface information for explaining a method of calculating δ ² s / δx ² and δ ² s / δy ² , FIG. 15B is a diagram illustrating a tilt image, FIG. 15C is a diagram illustrating a forward difference, FIG. 15D is a diagram illustrating the central difference. 16A is a diagram illustrating a source image for explaining a method of generating a texture extraction image, FIG. 16B is a diagram illustrating an image that has been processed by an averaging method, and FIG. 16C is an image that has been processed by a halation removal method. FIG. It is a figure for explaining angle noise reduction. It is a schematic cross section which shows a mode that it interferes with an obstacle when the illumination and the camera are integrated. It is a schematic diagram which shows a mode that a camera blocks a part of illumination light. FIG. 3 is a schematic cross-sectional view showing a configuration in which lighting and a camera are separated. FIG. 21A is a diagram illustrating a case where LWD is set to be small, and FIG. 21B is a diagram illustrating a case where LWD is set to be large, for explaining an advantage that the illumination and the camera are separated types. It is a schematic diagram which shows a mode that arranges illumination of a ring shape. It is a schematic diagram which shows a mode that the rotation position of a ring-shaped illumination and a camera is matched. It is a schematic diagram which shows the installation assistance means which concerns on a 1st Example. It is a schematic diagram which shows the example which applied the installation assistance means to the connection of a lighting cable and a lighting means. It is a schematic diagram which shows the installation assistance means which concerns on 2nd Example. It is a schematic diagram which shows the installation assistance means which concerns on 3rd Example. It is a schematic diagram which shows the installation assistance means which concerns on a modification. It is a schematic diagram which shows the installation assistance means which concerns on another modification. It is a schematic diagram which shows the installation assistance means which concerns on another modification. It is a schematic diagram which shows the installation assistance means which concerns on another modification. It is a schematic diagram which shows the installation assistance means which concerns on another modification. It is a schematic diagram which shows the installation assistance means which concerns on another modification. It is a schematic diagram which shows the installation assistance means which concerns on another modification. 4 is a flowchart illustrating an image inspection method according to the embodiment. 9 is a flowchart illustrating an image inspection method according to a modification.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiment described below is an example for embodying the technical idea of the present invention, and the present invention is not limited to the following. Further, the present specification does not specify the members described in the claims as members of the embodiments. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the components described in the embodiments are not intended to limit the scope of the present invention thereto, unless otherwise specified. It is only an example. In addition, the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of description. Further, in the following description, the same names and reference numerals denote the same or similar members, and a detailed description thereof will be appropriately omitted. Further, each element constituting the present invention may be configured such that a plurality of elements are formed of the same member and one member also serves as the plurality of elements, or conversely, the function of one member may be performed by a plurality of members. It can be realized by sharing.
(1. Configuration of image inspection apparatus 1)

  FIG. 1 shows a block diagram of the image inspection apparatus according to Embodiment 1 of the present invention. The image inspection apparatus 1 shown in FIG. 1 includes an imaging unit 11 that images an inspection target (hereinafter, also referred to as a “work WK”) from a certain direction, and an illumination unit that illuminates the work W from three or more different illumination directions. An illumination unit, an illumination control unit 31 for lighting each illumination unit one by one in the lighting order, and an image processing unit 41 connected to the illumination control unit and the imaging unit and controlling them are provided. The image processing unit 41 and the imaging unit are connected via the imaging cable 12, and the image processing unit 41 and the illumination control unit 31 are connected via the illumination cable 32, respectively. The image processing unit connects the display unit 51 and the operation unit 61. Further, the image processing unit can be connected to an external device such as a PLC (Programmable Logic Controller) or a computer as needed.

  The imaging unit captures a plurality of partial illumination images having different illumination directions by imaging the work W from a fixed direction at an illumination timing at which each illumination unit is turned on by the illumination control unit.

  The image processing unit implements the functions of a normal vector calculation unit 41a, a contour image generation unit 41b, a texture extraction image generation unit 41c, an inspection area identification unit 41d, an image processing unit 41e, and a determination unit 41f. The normal vector calculation unit 41a calculates a normal vector n of each pixel with respect to the surface of the workpiece W using a pixel value of each pixel in a correspondence relationship between the plurality of partial illumination images captured by the imaging unit. . The contour image generation unit 41b performs a differentiation process on the calculated normal vector n of each pixel in the X direction and the Y direction, and generates a contour image indicating the contour of the inclination of the surface of the work W. The texture extraction image generation unit 41c calculates the albedo of each pixel of the same number as the normal vector n from the calculated normal vector n of each pixel, which is the number of times of illumination by the illumination unit, and calculates the work W from the albedo. A texture extraction image showing a pattern from which the surface inclination state has been removed is generated. The inspection area specifying means 41d specifies the position of the inspection area to be inspected with respect to the generated contour image. The image processing means 41e performs image processing for detecting a flaw in the specified inspection area. The determining unit 41f determines the presence or absence of a flaw on the surface of the work W based on the processing result.

  The imaging means and the illumination means can be arranged as separate members. This enables a layout with a high degree of freedom. As an example, as shown in the schematic plan view of FIG. 2 and the schematic side view of FIG. 3, the imaging unit 11 with the optical axis directed in the vertical direction is disposed right above the work W placed on the stage SG. . Also, in the east-west-north-south direction around the imaging means 11, the four lighting means, that is, the first lighting means 21, the second lighting means 22, the third lighting means 23, and the fourth lighting means 24 are set at the same height. Deploy. The positional relationship between the arranged imaging means and each illumination means is recorded on the image inspection device side. Each lighting unit is sequentially turned on at a predetermined lighting timing by the lighting control unit, and the workpiece is imaged from a fixed direction by a common imaging unit to acquire a partial illumination image.

Note that the imaging unit and the illumination unit are not limited to being configured as separate members, and may be integrally configured via an arm or the like. In this case, since the positional relationship between the imaging unit and the illumination unit is fixed in advance, it is not necessary to perform an adjustment operation such as matching the optical axes. However, the degree of freedom is lost.
(Imaging means)

As the imaging unit 11, for example, an imaging element such as a charge coupled device (CCD) camera or a complementary metal oxide semiconductor (CMOS) imager can be used. The image of the subject is photoelectrically converted by the image sensor to output an image signal, and the output image signal is converted into a luminance signal and a color difference signal by the signal processing block, and the converted signal is transmitted to the image processing unit 41 connected by the imaging cable 12. Output.
(Lighting means)

  The illumination means 21, 22, 23, and 24 are arranged so as to surround the periphery of the work WK as shown in a schematic plan view of FIG. 2 so that the work WK can be irradiated with illumination light from different illumination directions. . Further, as shown in the schematic side view of FIG. 3, each lighting unit is installed with the optical axis directed obliquely downward. It is preferable that the optical axis of the imaging unit and the center axis of a plane (virtual rotation plane) provided with the illumination unit coincide with each other so that the work illuminated by each illumination unit can be imaged by the common imaging unit. In addition, it is preferable that the intervals (azimuths from the central axis) between the illumination units are arranged such that 360 ° is equally divided by the number of illumination units. Further, it is preferable that the zenith angle is constant for all the lighting means. Furthermore, it is preferable that the distance between each lighting means and the work is also constant. By doing so, it is possible to simplify the input of the azimuth and zenith angle information required for the calculation of the photometric stereo processing. As will be described later, when capturing the full-light image MC, all the illuminating means are imaged in the full-light state in which all the lighting means are turned on. Images can be taken in the lighting state.

In the example of FIG. 1, the first lighting unit 21, the second lighting unit 22, the third lighting unit 23, and the fourth lighting unit 24 are configured. As each lighting unit, an incandescent bulb, a fluorescent lamp, or the like can be used. In particular, a semiconductor light-emitting element such as a light-emitting diode (Light Emitting Diode: LED) is preferable because it consumes less power, has a long life, and has excellent responsiveness. As shown in FIG. 1, each lighting unit is connected to a lighting branching unit 75 via a corresponding cable 71, 72, 73, 74, and further connected to a lighting control unit 31 via a cable 76. ing.
(Lighting branch unit)

  The lighting branch unit is an interface for connecting each lighting means to the lighting control means. Specifically, a lighting connection connector for connecting a lighting cable extended from the lighting means is provided. In the example of FIG. 1, four lighting connection connectors are provided so as to connect four lighting means. At this time, a mark or the like may be provided as an installation assisting means so that the lighting cable is correctly connected to the lighting connection connector (details will be described later). By correctly connecting the lighting table of each lighting unit to the lighting connection connector of the lighting branch unit, the lighting unit can be turned on in the correct order by the lighting control unit, and the imaging unit can be synchronized with each lighting timing to set the imaging unit. By operating, a partial illumination image can be captured. Although the lighting branch unit is provided separately from the lighting control means in the example of FIG. 1, the present invention is not limited to this configuration. For example, the lighting branch unit may be incorporated in the lighting control means.

  The illumination colors of the illumination means 21, 22, 23, 24 can be changed according to the type of the work WK. For example, when it is desired to inspect a fine scratch, blue having a short wavelength is preferable. When inspecting a colored work, it is preferable to use white illumination so that the color of the illumination does not interfere. Further, when the work has oil, it is preferable to adopt red illumination to prevent the influence.

  In addition, although the number of illumination means is four in the examples of FIGS. 1 to 3 described above, at least three illumination means are sufficient so that the work WK can be illuminated from three or more different illumination directions. When the number of illumination means is increased, partial illumination images can be obtained from more illumination directions, so that the accuracy of image inspection can be improved. For example, a total of eight directions including the northeast, northwest, southeast, and southwest directions may be arranged. In addition, it is preferable that the intervals (azimuths from the central axis) between the illumination units are arranged such that 360 ° is equally divided by the number of illumination units. Further, it is preferable that the zenith angle is constant for all the lighting means. It should be noted that an increase in the number of images to be processed increases the amount of processing, thereby reducing the processing time. In the present embodiment, the number of illumination means is set to four as described above in consideration of the balance between the processing speed, the easiness of the arithmetic processing, the accuracy, and the like.

  Further, the lighting means may be constituted by a plurality of light emitting elements arranged in a ring. For example, in the ring illumination according to the second embodiment shown in FIG. 4, the light emitting elements arranged in a ring are divided into four illumination blocks. Then, the lighting control unit sets the first lighting block as the first lighting unit, the second lighting block as the second lighting unit, the third lighting block as the third lighting unit, and the fourth lighting block as the fourth lighting unit. By making the illumination timing of the blocks different, it is possible to control as if there are four separate illumination means.

  Further, with this configuration, there is an advantage that the number of illumination means can be arbitrarily changed using the same ring illumination. That is, when lighting of each light emitting element can be arbitrarily controlled by the lighting control means, as shown in FIG. 5, the lighting block is so divided as to divide the circumference of the annularly arranged light emitting elements into four to five sections. By changing the illumination timing to turn on the five illumination blocks and controlling the illumination control unit to capture each partial illumination image, partial illumination images from five illumination directions can be obtained. Similarly, by changing the annular circumference into six and seven divisions, the number of illumination directions can be further increased. Further, the present invention is not limited to a configuration in which a partial illumination image is always acquired by a constant illumination block, and the illumination block may be changed in each cycle. As described above, by adjusting the lighting pattern of each light emitting element, it is possible to obtain the same effect as virtually changing the number of illumination blocks and adjusting the number of illumination means using the same ring illumination. Can be. In other words, different accuracy can be handled by common hardware.

  Further, in addition to arranging the illuminating means in an annular shape, as shown in the schematic plan view of FIG. 6 as the third embodiment, illuminating means configured in a bar shape may be arranged in a rectangular shape, and as the fourth embodiment, the illuminating means of FIG. As shown in the schematic plan view, they can be arranged in a polygonal shape.

  Alternatively, the illuminating means may be arranged in a circle, a polygonal ring, or a plane. For example, different illumination directions can be realized by arranging a large number of light emitting elements in a plane and changing illumination blocks to be lit. Specifically, as in the illumination unit according to the modification shown in FIG. 8, an illumination unit 20 ′ in which a plurality of concentric rings are stacked is configured, and each of the rings having different radii is used as an illumination block, and the illumination unit is used as the illumination unit. Constitute. Alternatively, as in the illumination unit according to another modification shown in FIG. 9, the illumination unit 20 ″ in which light-emitting elements are arranged in a dot matrix form, and an illumination unit divided by a plurality of line segments passing through the center. As described above, in the present invention, the lighting means and the lighting direction are not necessarily limited to physically separated lighting, and lighting is performed by a lighting block obtained by dividing one lighting means into a plurality. It is used in the sense including the configuration.

In this embodiment, the processing is performed on the assumption that the partial illumination light from each illumination unit is parallel light within the imaging range. As long as the partial illumination light is parallel light, only the direction of the illumination light (for example, any of north, south, east and west) becomes a problem, and it is not necessary to consider its detailed position, for example, the coordinate position of the light source of the illumination means.
(Lighting control means)

  The lighting control means lights up the three or more lighting means one by one in the lighting order, and sets each lighting means and the imaging means so that the work is imaged from a certain direction by the imaging means at the lighting timing for lighting each lighting means. Control to synchronize. In other words, the illumination control unit synchronizes the illumination timing of the illumination unit with the imaging timing of the imaging unit. The lighting order in which the lighting control means turns on each lighting means is such that the lighting means arranged to surround the periphery of the work is turned on in a clockwise, counterclockwise, or other order, and in a discrete order, for example, one. It is good also as the order which crosses and intersects. In any order, if it is possible to grasp which illumination direction the partial illumination image captured at each illumination timing is captured in which illumination direction, a normal vector image based on the photometric stereo method Can be constructed.

In the first embodiment of FIG. 1, the illumination control unit 31 is provided separately from the image processing unit 41, but is not limited to this configuration. For example, it may be integrated with the image processing unit 41 as in a fifth embodiment shown in FIG. 10, or may be built in the lighting means 25 as in a sixth embodiment shown in FIG.
(Image processing unit)

  The image processing unit 41 controls the operations of the imaging unit 11 and the illumination units 21, 22, 23, and 24, and uses the image signals Q 1 to Q 4 of the four partial illumination images input from the imaging unit 11 to set each pixel. A normal vector image (hereinafter, referred to as an “inclination image”) of the surface at is generated, and a secondary inclination image (hereinafter, referred to as an “outline extraction image”) in the X direction and the Y direction of the inclination image and an albedo are generated. (Albedo means reflectivity.) An image (hereinafter, also referred to as a “texture extraction image”) is created, and a process for inspecting a flaw, detecting a character, and the like is performed using the image. It should be noted that the processing for performing a flaw inspection, character detection, and the like is not limited to the processing performed by the image processing unit 41, and may be executed by an external device such as the PLC 81, for example.

  FIG. 12 shows a signal processing system 42 of the image processing unit 41. The signal processing system 42 includes a CPU 43, a memory 44, a ROM 45, a display unit 51, an operation unit 61 such as a pointing device, an imaging unit 11, an illumination control unit 31, and a PLC (programmable logic controller) 81 for executing a result output process. These are connected to each other via a bus 46 and cables 12, 32, 52, 62, 82. Note that the ROM 45 may be a portable medium. The display means 51 may also be used as the operation means 61, for example, as a touch panel.

  The CPU 43 controls transmission and reception of data among the memory 44, the ROM 45, the display unit 51, the operation unit 61, the imaging unit 11, the illumination control unit 31, and the PLC 81 based on a program stored in the ROM 45, The display unit 51, the imaging unit 11, and the illumination control unit 31 are controlled.

The image processing unit 41 is, for example, assumed to be one computer storing a program, but each unit may be configured by a combination of a plurality of computers, or some of the units may be dedicated. It may be constituted by a circuit. Alternatively, a specially designed member such as an ASIC may be used.
(Judgment means)

The image processing unit 41 realizes the function of the determination unit as described above. The judging unit inspects the presence or absence and size of a flaw based on the obtained texture extraction image. For example, when it is equal to or more than a predetermined threshold value, it is determined that the scratch is present. Further, the determination means can perform OCR based on the contour extraction image as needed, and can output the recognized character string.
(Basic principle)

  Next, the basic principle of inspecting the appearance of a workpiece using the above-described image inspection apparatus will be described in comparison with the conventional technique of Patent Document 1. First, the basic principle of the technology disclosed in Patent Document 1 uses the principle of the photometric stereo method, irradiates an unknown surface with light from various directions, and utilizes the difference in the reflected light of the work. It is intended to estimate the shape of the work. The reflected light of the work reflects the incident angle of the illumination, the distance of the illumination, and the like, and has the property of becoming brightest when the incident angle is 90 °, and becomes darker as the distance from the illumination increases.

  By utilizing this property, a plurality of illuminations whose brightness and position are known are prepared, and the lighting is switched in order, and at each point, the brightness of the reflected light when emitted from the illumination in each direction. Estimate which surface is facing using the difference in height. More specifically, an X component image in which the X component and Y component of the tilt image are replaced with the X component luminance, and a Y component image in which the Y component luminance is replaced are created to be applied to inspection.

  However, this method is robust in that the obtained inspection image is greatly changed due to a slight inclination of the installation surface of the lighting or the work or an error in the input information such as the position of the originally input lighting. There was a problem that it was inferior. For example, a work may have an uneven image without any unevenness, or the work may look like an image where the vicinity of the work normally looks bright for illumination, or the center of the work is raised due to a change in brightness. However, there is a drawback that an inspection image conforming to the actual shape cannot be always obtained.

  On the other hand, the basic principle of the image inspection technique according to the present embodiment is that the first-order tilt image is generated using the photometric stereo method as a starting point, but the obtained first-order tilt image is obtained. Is differentiated in the X direction and the Y direction to create a second-order tilt image, that is, a contour extraction image, and inspects the scratch using the image. Even in the case where the above-described defect occurs, the second-order tilt image has a small effect, and a portion where the surface tilt change is large is darkened, while a portion where the surface tilt change is small is bright. By setting the gradation, an image suitable for extracting a flaw, an outline, or the like in which the inclination of the surface of the work greatly changes is obtained.

  In the technique disclosed in Patent Document 1, a reflectance image (corresponding to a texture extraction image) generated by using the photometric stereo method may cause halation, and it may be difficult to detect characters. there were. On the other hand, the image inspection technology according to the present embodiment basically utilizes the basic principle that only two illuminations cause halation at the same location, and for each pixel, for example, out of four pixel values. , The effect of halation is removed by adopting the third pixel value from the top.

In addition, in the image processing apparatus disclosed in Patent Document 1, the camera and the light source for illumination are integrally configured. However, in this configuration, the size of the camera and the light source is increased, and the size of the camera and the light source is restricted during installation. There was also a problem. On the other hand, in the image inspection apparatus according to the present embodiment, since the imaging unit and the illumination unit can be separate members, more flexible installation can be performed in consideration of the arrangement space, which is advantageous in terms of usability. .
(Basic principle of photometric stereo method)

  Here, the basic principle of the photometric stereo method will be described with reference to FIGS. 13A to 13D. First, as shown in FIG. 13A, it is assumed that there is an unknown diffuse reflection surface S and a plurality of illuminations (first illumination means L1 and second illumination means L2 in this example) whose brightness and position are known. I do. For example, as shown in FIG. 13B, when light is emitted from the first illumination unit L1, the diffuse reflection light on the surface of the diffuse reflection surface S is (1) the brightness of the illumination (known), and (2) the direction of the illumination (known). ), (3) the orientation of the surface of the work WK (normal vector n), and (4) the albedo parameter of the surface of the work WK alone.

Therefore, as shown in FIGS. 13B and 13C, the partial illumination images formed by the diffuse reflection light when the illumination light is projected from a plurality of different illumination directions, specifically, three or more illumination directions, are respectively taken by the imaging unit. To shoot. Then, as shown in FIG. 13D, by using three or more partial illumination images as input images, the unknown (3) the orientation (normal vector n) of the surface of the work WK, and (4) the albedo of the surface of the work WK, It can be calculated based on the following relational expression.
I = ρLSn

In the above formula,
ρ: albedo L: illumination brightness S: illumination direction matrix n: surface normal vector I: image tone value

  From the above equation, when there are three illumination means, it can be expressed by the following equation.

  In the case of four illumination means, it can be expressed by the following equation.

(Normal vector n)

From the above equation, the normal vector n can be expressed by the following equation.
n = 1 / ρL · S ⁺ I

In the above formula,
S ⁺ : If it is a square matrix, ordinary inverse matrix S ⁺ : The inverse matrix of the vertical matrix is a pseudo-inverse matrix of Moore-Penrose S ⁺ = (S ^t S) ⁻¹ S ^t represented by the following equation.
Ask for.
(Albedo)

Further, the albedo ρ can be expressed by the following equation.
ρ = | I | / | LSn |
(2-2. Contour extraction image)

Next, a method of generating a tilt image by the photometric stereo method and obtaining surface information of a work such as a scratch or a contour from the obtained tilt image will be described.
(Tilted image)

First, a method of generating a tilt image will be described. When the curved surface of the work is S, the tilt image is given by the following equation.
x direction: δs / δx, y direction: δs / δy

  Here, as an example of the tilt image, an example in which a single coin is used as a workpiece is shown in FIGS. 14A and 14B. FIG. 14A is a Y-coordinate component image in the normal direction, and FIG. 14B is an X-coordinate component image in the normal direction. Here, the tilt image shown in FIG. 14A is differentiated in the X direction (horizontal direction in the figure) by differentiating in the Y direction (vertical direction in the figure) using partial illumination images captured from four illumination directions. Thus, the tilt image shown in FIG. 14B is obtained in FIG. 14B.

Here, scratches, contours, and the like are portions where the inclination of the work surface changes, so that the inclination image is differentiated in each direction. The second tilt image is given by the following equation.
x direction: δ ² s / δx ² , y direction: δ ² s / δy ²
(Contour extraction image)

From the above, the parts δ ² s / δx ² and δ ² s / δy ² of the tilt image in the x direction and the y direction are combined to generate a contour extraction image including the contour of the work and the flaw information. The contour extraction image E is given by the following equation.
E = δ ² s / δx ² + δ ² s / δy ²

In the above equation, E indicates contour information, and S indicates a curved surface of the work. FIG. 14C shows an example of the contour extraction image calculated from FIGS. 14A and 14B. In the contour extraction image, the height is expressed by shading (luminance) of the image such that a high portion is white and a low portion is black.
(Differential synthesis method)

Differential synthesis methods performed at the time of generating the contour extraction image as described above include (1) simple addition, (2) multiple resolution, and (3) sum of squares.
(1: Simple addition)

Here, the simple addition in (1) is the sum of the derivatives of the X and Y tilt images at each pixel.
(2: multiple resolution)

In the multiple resolution of (2), a plurality of reduced tilt images obtained by reducing the tilt image at different reduction rates are created, and the strength of the contour is obtained for each reduced tilt image by the method (1). The reduction ratio is, for example, 1/1, 1/2, 1/4, 1/8, 1/16, 1/32, or the like. A predetermined weight is applied to the plurality of reduced outline images obtained in this way, an enlargement process is performed, and a sum of all the enlarged reduced outline images is set as an outline extracted image. Here, if the weight is changed, it becomes possible to extract a scratch, an outline, or the like having an arbitrary thickness.
(3: Sum of squares)

  In the sum of squares of (3), a contour extraction image is created in which the sum of the squares of the derivatives of the X and Y tilt images is the strength of the contour. In this embodiment, the multi-resolution (2) is adopted.

  The size to be determined as a scratch differs depending on the use of the user. For example, there is a case where a scratch over 10 pixels is determined as a scratch, and a recess over 100 pixels is determined as a scratch. In some cases, only a steep edge is desired to be extracted as an edge.

  If the number of pixels in the tilted image is large, the processing will cause a large flaw. Therefore, if a large flaw is to be extracted, the tilted image is reduced, the strength of the contour is obtained by the method (1), and then enlarged. On the other hand, if it is desired to extract a small flaw, differential synthesis may be performed by the method (1) without weighting.

  That is, for the weighting, a set of predetermined weights is prepared at the time of synthesis, all kinds of reduced tilt images are created, and if a large flaw is to be seen, the result from the more reduced image is weighted. If you want to see small scratches, make the results from the image with reduced reduction heavier.

Here, the sum of all of the enlarged and reduced outline images is used as the outline extraction image, because the flaw is usually detected over a plurality of frequencies. This is because only the flaw detected at the limited frequency is extracted, and the entire image is blurred.
(Feature size)

  The above-mentioned weight set is provided with, for example, a parameter called a feature size. When this value is 1, the finest flaw can be detected, and when this value is increased, a large flaw can be detected. When the feature size is increased and larger flaws are more easily detected, irregularities on the work surface become clearer. Therefore, a predetermined threshold value may be provided for the feature size, and a case where the threshold value or more is exceeded is set as the concavo-convex mode, and the contour extraction mode may be selectively used depending on the feature size of the contour extraction image.

Next, a method of calculating δ ² s / δx ² and δ ² s / δy ² will be described. This calculation method includes (1) forward difference, (2) central difference, and the like.
(1: forward difference)

  In the forward difference, the pixel G (x, y) at the coordinates (x, y) of the contour image E is calculated by the following equation, using the horizontal tilt image Gh and the vertical tilt image Gv as inputs.

  E (x, y) = Gh (x-1, y) -Gh (x, y) + Gv (x, y-1) -Gv (x, y)

Here, the information of the flaw appearing in the contour image is shown in FIGS. 15A to 15D as a schematic profile. In these figures, FIG. 15A shows the surface information of the work, FIG. 15B shows the tilt image, FIG. 15C shows the profile image based on the predecessor difference, and FIG. 15D shows the profile of the profile image based on the central difference. The forward difference of (1) has an advantage that a flaw in units of one pixel can be clearly seen as shown in FIGS. 15A and 15C. On the other hand, there is a demerit that an image shifted by 0.5 pixel from the original image is obtained.
(2: Central difference)

Next, a method of calculating δ ² s / δx ² and δ ² s / δy ^{2 using} the central difference will be described. With the horizontal tilt image Gh and the vertical tilt image Gv as inputs, the pixel G (x, y) at the coordinates (x, y) of the contour image E is calculated by the following equation.

  E (x, y) = Gh (x-1, y) -Gh (x + 1, y) + Gv (x, y-1) -Gv (x, y + 1)

As shown in FIGS. 15A and 15D, the central difference in (2) has the advantage that the coordinates do not deviate from the original image, but also has the disadvantage that the result is slightly blurred.
(2-3. Texture extraction image)

  Next, a method of removing a surface state of a work from a tilt image obtained by the photometric stereo method to obtain a texture extraction image suitable for character detection or the like will be described. First, the texture information is calculated from the albedo ρ on the surface of the work. The albedo ρ is given by the following equation.

  ρ = | I | / | LSn |

In the above formula,
ρ: albedo L: illumination brightness S: illumination direction matrix n: surface normal vector I: image tone value

  Note that one texture extraction image (albedo) can be obtained by the above expression ρ = | I | / | LSn |, but from the normal vector obtained by this expression and N input images (partial illumination images) It is also possible to obtain N texture extraction images (albedo), synthesize them, and obtain one texture extraction image (albedo). Specific examples of the synthesis method include an averaging method and a halation removal method.

16A to 16C show examples of a texture extraction image. In these figures, FIG. 16A shows four texture extraction images as input images, FIG. 16B shows a texture extraction image obtained by applying an averaging method thereto, and FIG. 16C shows a texture extraction image obtained by applying a halation removal method. Each image is shown.
(1: average method)

The averaging method uses an average value of N ρs for each pixel as a pixel value at that pixel. As shown in FIG. 16B, although the entire shadow has disappeared, the portion where the halation has occurred in the input image cannot be eliminated by the photometric stereo method, so that the image is still affected by the halation. That is, in the four input images (partial illumination images) of FIG. 16A, white portions are portions where halation occurs, and when averaged by the averaging method, as shown in FIG. However, although it is easy to read, some irregularities remain on the base.
(2: Halation removal method)

  Due to the limitation of the dynamic range of the camera as the imaging means and the diversity of the reflection of the work surface, the expression of ρ = | I | / | LSn | itself exceeds the applicable range, so that ρ contains an error. It is. To correct this error, a halation elimination method can be used.

  Since the location where halation occurs is determined by the position of the illumination, it is basically considered that no halation occurs at the same location in the four partial illumination images. Strictly speaking, although halation may straddle two places between two directions, basically, it can be said that halation does not occur at the same place with only two illuminations.

  In the halation elimination method, when synthesizing a texture extraction image for each illumination direction calculated from N partial illumination images, the partial illumination image having the highest pixel value or the first to Nth highest pixel values in each pixel The partial illumination image is considered to have a lot of halation, and is synthesized after removing them.

  Specifically, in the present embodiment, when each pixel of the texture extraction image for each of the four illumination directions is synthesized by adopting the third largest pixel value (for example, the albedo value or luminance), as shown in the image of FIG. 16C. Thus, the effects of halation can be eliminated. When the fourth largest pixel value is used, the image is slightly dark because of the influence of shadow. Conversely, if the second largest pixel value is used, the effect of halation will remain slightly.

  When the illumination means is in eight directions, the fifth pixel value from the top is adopted assuming that the influence of halation does not occur on the fifth and subsequent pixel values. In fact, according to a test performed by the inventor, it has been confirmed that the image is best when the fifth pixel value is adopted. It has also been found that the use of the sixth and subsequent pixel values has the effect of shadows.

The method of synthesis and the method of obtaining the average are not limited to these, and various methods can be used. For example, the values of the albedo may be sorted by combining the above-described halation elimination method and the averaging method, and values in a specific order may be adopted from the top, for example, the third and fourth values may be averaged.
(Feature size)

Next, the details of the setting will be described. As described above, the feature size can be set when creating the contour extraction image. The contour extraction image can be an image suitable for OCR by setting the feature size to a predetermined value or more.
(3-2. Gain)

  When creating a contour extraction image or a texture extraction image, a gain can be multiplied by the pixel value of the original image in the process of generating these images.

  The gain at the time of creating the contour extraction image refers to a constant when the pixel values calculated by the calculation process are dispersed in a density of 0 to 255. For example, when the scratches and contours are too shallow and it is difficult to grasp the scratches and contours, by increasing the gain value, the shading change of the pixel value becomes large, so that the scratches and contours can be easily grasped.

  If the pixel value calculated by the calculation process is outside the range of 0 to 255, the pixel value is adjusted to fall within the range. Is easy to grasp.

  In the above-described halation removal method, the value of the albedo is sorted and, for example, the third value is adopted from the top, so that the brightness of the generated image cannot be predicted. Therefore, as a result of the removal of the specular reflection, the image may become darker than expected. Therefore, in order to adjust the brightness, a predetermined gain is multiplied by the pixel value when creating a texture extraction image.

In calculating the tilt image, the gain can be similarly adjusted so that the pixel value falls within the range of 0 to 255.
(3-3. Noise removal filter)

When a tilt image is created or the like, calculation is performed using simultaneous equations using a plurality of images. In practice, however, a difference calculation is performed. At this time, since the image data picked up by the image pickup unit has noise at the time of the raw data, a noise component is emphasized when creating a tilted image, and the outline may be rough. In order to reduce such noise, a noise removal filter such as a guided filter is used. With a general low-pass filter, not only the noise but also the information on the scratches may be blurred or lost.However, with a guided filter, the edge can be reduced by applying a guided filter when obtaining the tilt image. This is suitable because noise can be removed while maintaining it.
(3-4. Angle noise reduction)

Next, the principle of angle noise reduction will be described with reference to the schematic diagram of FIG. As shown in this figure, assuming that there are two illuminations having incident angles α and β, and that the work WK is formed of a diffuse reflection surface, the inclination of the work WK with respect to the reference plane is γ, When the angle is θ, the brightness of the reflected light from the illumination α is I _α , and the brightness of the reflected light from the illumination β is I _β , γ is given by the following equation.

γ = arctan (A · | I β -I α | / | I β + I α |), A = cotθ

Angle noise reduction is to forcibly set the slope γ to 0 when | I _β + I _α | is somewhat small.

Even if I _beta I _alpha is extremely dark, for _{example, I β = 2, I α} = 1 _{Datosuruto, | I β -I α | /} | I β + I α | = a large value of 1/3. On the other hand, bright nor I _beta I _alpha, for _{example, I β = 300, I α} = 200 _{Datosuruto, | I β -I α | /} | I β + I α | = a small value of 1/5. Since I _β = 2 and I _α = 1 have a large influence on the slope despite the possibility of being simply noise, there is angle noise reduction to reduce the influence of the noise. It is possible to set a threshold value of | _Iβ + _Iα | for which the slope is forcibly set to 0.
(Separable type structure of illumination means and imaging means)

  In the image inspection using the photometric stereo method, it is necessary to strictly define in advance the corresponding positions of the illumination means and the imaging means. For this reason, in the conventional image inspection apparatus, the illumination unit and the imaging unit are integrally configured. In other words, since the photometric stereo method is a measurement method that performs accurate three-dimensional measurement after strictly positioning the relative position of the imaging means and the illumination means, it is originally installed when installing the illumination means and the imaging means. Giving a degree of freedom of position has not been conventionally performed. However, in a configuration in which the illumination unit and the imaging unit are fixed in advance, the imaging illumination unit in which these members are integrated is inevitably increased in size, so that the handling becomes poor.

  For example, if an obstacle is present at the inspection position, it may interfere with the imaging illumination unit and cause a situation in which it cannot be installed. In particular, there are large lenses such as a line camera, a zoom lens, and a large macro lens as a lens to be mounted on a camera which is one form of the imaging means. As the size of the imaging means becomes longer, such as when such a large lens is attached to the imaging means, the obstacle OS existing around the work WK and the camera e1 or the periphery thereof are arranged as shown by arrows in FIG. The risk of interference with the illuminating light sources e21 to e24 increases. Further, if the camera e1 as the imaging means becomes longer, the illumination light is partially blocked by the camera e1 and a part of the lens as shown in FIG. It is also possible. In order to avoid such a situation, if the imaging illumination unit is to be arranged at a position where the imaging illumination unit does not interfere with an obstacle, the distance (light working distance (LWD)) between the illumination means and the work is long. Therefore, the amount of illumination light decreases, leading to a decrease in inspection accuracy.

  On the other hand, if the imaging means and the illumination means can be separated, it is easy to arrange them at positions where they do not interfere with obstacles. For example, even when the obstacle OS shown in FIG. 18 exists, the illumination units 21 to 24 are separated from the imaging unit 11 so that the illumination units 21 to 24 are installed at positions that do not interfere with each other as shown in FIG. It becomes possible. Similarly, even when a large lens is attached to the imaging unit, the adjustment can be made so as to avoid physical interference between the lens 12 and the illumination units 21 to 24. As described above, the illumination units 21 to 24 and the imaging unit 11 are of a separate type, so that they can be adjusted to a position where they do not interfere with obstacles, thereby increasing the degree of freedom of installation and inspecting the work WK in various environments. Becomes possible.

  It is also possible to adjust the mounting position of the illumination means and the like so that the imaging means does not block the illumination light. Similarly, it is possible to adjust the arrangement position of the illumination means so as to suppress the influence of halation and shadow. In particular, since the photometric stereo method assumes that the surface of the work is a diffuse reflection surface, the normal vector of the actual work surface and the surface of the surface in the tilt image obtained by the photometric stereo method generally deviate. Would. Therefore, measures such as performing second-order differential processing have been taken. In addition to such measures, when installing the lighting means, by adjusting the posture so that halation can be avoided, such a deviation is reduced. be able to. Also, like a mirror surface, halation has an adverse effect on large irregularities where the brightness changes. For example, when the work is cylindrical, specular reflection occurs when illuminating light is applied. However, if there is unevenness, the influence of the specular reflection will be considerable. Therefore, giving the degree of freedom to the installation position and the angle of the illumination light makes it possible to perform adjustment in advance so as to reduce such specular reflection, thereby obtaining an advantage that leads to improvement in inspection accuracy.

  Further, in the photometric stereo method, it is necessary to consider the influence of shadow. In the photometric stereo method, since a normal vector is calculated from reflected light of illumination, detection of reflected light is essential. However, for example, when the work has a complicated shape and a shadow is likely to be formed, if an illuminating unit is installed in the vicinity of the work, an appropriate reflected light cannot be obtained from a place where light does not reach, and a normal vector is calculated. May cause problems. Even in such a case, the illumination units 21 to 24 and the imaging unit 11 are separated from each other, so that the illumination units 21 to 24 can be installed at optimal positions and angles to suppress such an influence. it can.

In addition, the height of the illumination light can be changed by separating the illumination unit from the imaging unit. As a result, it is possible to perform an inspection in which the distance LWD (Light Working Distance) between the work WK and the illumination is reduced or increased, and an appropriate arrangement according to the inspection application can be selected.
(1: When LWD is small)

  If the LWD is made small, as shown in FIG. 21A, a large amount of light from the illumination units 21 to 24 impinges from the lateral direction. In general, halation occurs when the incident angle and the reflection angle are the same. Therefore, by making the illumination means 21 to 24 close to the work WK and reducing the LWD, the position where the halation occurs can be changed to the position of the work WK. It can be near the outside. In other words, halation is easily generated intentionally outside the work WK, and halation near the surface of the work WK can be suppressed. Thereby, the effective visual field of the work WK can be increased.

  When the LWD is small, illumination is performed at a low angle. For example, even when unevenness on the work surface is difficult to recognize due to the diffusion of light by direct illumination from above, irradiating from the oblique side surface greatly changes the contrast even with a small change in inclination, making it easy to catch changes in shallow irregularities. it can. By enhancing the contrast at a low angle in this manner, an advantage that the tilt image and the contour extraction image can be made clearer in the photometric stereo method can be obtained.

On the other hand, as a disadvantage when the LWD is reduced, shadows are likely to be generated by irradiating the illumination light at a shallow angle, so that the effective visual field is reduced and photometric stereo processing may be difficult. Further, since the illumination means 21 to 24 easily interfere with the work WK, use of the work WK having a large height may be restricted.
(2: When LWD is large)

  Conversely, if the LWD is set to be large, as shown in FIG. 21B, the illumination light from the illumination units 21 to 24 is emitted more from above. As a result, shadows are less likely to occur, so that invisible areas are reduced, and the accuracy of the photometric stereo processing is improved. Further, since the illumination means 21 to 24 and the work WK hardly interfere with each other, the illumination means 21 to 24 can be suitably used for a work WK having a large height.

  On the other hand, the position where the halation occurs is located inside the work WK, so that the effective visual field may be reduced. Further, when the LWD is large, it corresponds to a case where the LWD is far in multi-angle illumination, and it is difficult to obtain a contrast, so that it may be difficult to catch a change in shallow irregularities on the work surface. On the other hand, if the LWD is distant due to the multi-angle illumination, a uniform imaging state with less reflection of the illumination means 21 to 24 and uneven illuminance is created, and an image that clearly captures the surface state of the work WK itself is obtained. Obtainable.

  As described above, by separating the imaging means and the illumination means, there are obtained advantages that the inspection can be performed for various purposes and uses, and that the degree of freedom of arrangement can be increased. In particular, the normal vector calculated by the normal vector calculation means has a dependency that the intensity of the normal vector changes in accordance with a change in the relative distance between the imaging means and the illumination means in the optical axis direction of the imaging means. Have. That is, when the light amounts of the illumination means are equal, the intensity of the normal vector tends to increase as the relative distance decreases, and the intensity of the normal vector tends to decrease as the distance increases. The higher the intensity of the normal vector, the more clearly the inclination (concavo-convex shape) of the work surface appears, so that the accuracy of the obtained inspection image is also improved. For this reason, in the conventional photometric stereo method, the relative position between the imaging means and the illumination means has been strictly defined in advance so that the intensity of the normal vector can be increased to improve the accuracy. That is, the intensity of the obtained normal vector is increased by fixing the imaging unit and the illumination unit in advance.

  On the other hand, in appearance inspections such as scratches on the work surface and OCR, high accuracy is not always required. For example, it is often sufficient to be able to detect scratches or detect OCRs of numbers and characters. In such applications, it is rather advantageous for the user to increase the degree of freedom in installing cameras and lights rather than the accuracy. Therefore, in the present embodiment, rather than increasing the strength of the normal vector, the strength of the normal vector is allowed to be reduced, and the level at which the necessary accuracy of the visual inspection is sufficiently maintained is maintained. At the same time, it has succeeded in improving user convenience by enabling the installation of more flexible lights and cameras, freed from the restriction of fixing the lighting means and the imaging means. It is.

  On the other hand, when the illumination unit and the imaging unit are configured separately to perform image inspection by the photometric stereo method, it is necessary to clearly set the relative positional relationship between the illumination unit and the imaging unit. If the set illumination direction is significantly different from the actual illumination direction, there is a possibility that the inspection accuracy is reduced or an incorrect result is output. For example, when the imaging means and the lighting means are installed, if the user mistakenly connects a large number of lighting means or installs them with the azimuthal angle shifted as a whole, accurate inspection may not be possible.

  For example, as shown in FIG. 22, a case is considered in which imaging means is arranged on a vertical line where the work is arranged above the work, and illumination means are arranged on the north, south, east and west so as to surround the periphery of the work. In such an arrangement, when the height shape of the work is acquired by using the photometric stereo method, if the illumination means and the illumination cable have the same shape, connection and arrangement may be confused. Specifically, when a predetermined lighting cable is connected to a lighting connection connector provided in the lighting branch unit, each lighting cable generally has a common shape. Since the connectors also have a common shape, it can happen that the lighting cable is connected to the wrong lighting connector. In particular, if the lighting cable is drawn for a long time, it is easy to get confused. Also, when installing the illuminating means above the periphery of the work, it is conceivable to fix the illuminating means having the same shape by mistake.

  Alternatively, as shown in FIG. 23, also in an example in which a ring-shaped illumination unit is arranged, it is necessary to make the direction of the illumination light coincide with the rotation angle of the imaging unit. In this case, when the imaging means and the illuminating means have a point-symmetrical shape such as a cylindrical shape or an annular shape, the rotation angle is difficult to recognize and the alignment is not easy. As a result, the rotational position may deviate from the expected position. In particular, the characteristic amount such as the shape of the surface of the work is detected to some extent even if the setting of the rotation angle, the arrangement position, and the like is slightly incorrect. Was. Furthermore, if the lighting unit is completely mistaken, it is relatively easy to notice the setting error, but if the setting error is such that the rotation angle is slightly shifted, it is difficult for the user to notice, and the accuracy is low. There was a problem that led to a decline.

Therefore, in the present embodiment, when acquiring partial illumination images illuminated from a plurality of different illumination directions by the photometric stereo method, when installing, so that the illumination direction and lighting order of the illumination means are as intended settings. An installation assisting means is provided for assisting in setting a correct setting. Specifically, in FIG. 1, when connecting each lighting unit to the lighting control unit, an appropriate connection destination is indicated to the user according to a predetermined installation setting so that the lighting direction and the lighting order match. Alternatively, a correct posture user is instructed when the lighting unit is installed so that the circumferential posture (rotational position or rotation angle) of the lighting unit matches. Thereby, mistakes at the time of installation can be prevented, and the appearance inspection of the work using the photometric stereo method can be accurately performed.
(Installation setting)

Here, the installation setting refers to an installation in which the workpiece can be irradiated with illumination light from a predetermined illumination direction according to the number of each illumination means so that the normal vector is correctly calculated according to the photometric stereo method. This is a setting that defines the position. For example, as shown in the plan view of FIG. 2, in an example in which four illumination units are used, the first illumination unit 21, the second illumination unit 22, the third illumination unit 23, and the fourth illumination unit 24 , The first lighting means 21 is located north, the second lighting means 22 is located east, the third lighting means 23 is located south, and the fourth lighting means 24 is located west. It becomes the installation position information of the lighting that each is arranged. The installation setting may include the lighting order of the lighting unit by the lighting control unit and the imaging timing of the imaging unit.
(Installation assistance means)

Hereinafter, specific examples of the installation assisting means will be described. In the example illustrated in FIG. 1, the image pickup unit 11, the illumination units 21 to 24, and the illumination branching unit 75 are in an intended connection configuration that can appropriately perform photometric stereo processing, in other words, are not erroneously connected or arranged. Thus, the connection destination is displayed by the installation assisting means. For example, installation notation indicating the specific direction or the rotation angle of the lighting arrangement is written on the connection part, or the characters, symbols, other marks, shapes, colors, etc. of the installation notation are attached to the connectors or terminals to be connected. By providing commonality and correspondence, the arrangement position, direction, rotation angle, connection destination, and the like are visually indicated. In this way, by assisting the mutual connection work with the installation assisting means, the user can visually grasp the correspondence between the mutual connection destinations by referring to the installation notation. Hereinafter, FIGS. 24 to 34 show specific examples of the installation assisting means.
(First embodiment)

  FIG. 24 shows a first lighting cable 71, a second lighting cable 72, a second lighting cable 72 extending from the first lighting means 21, the second lighting means 22, the third lighting means 23, and the fourth lighting means 24 shown in FIG. The state where the three lighting cable 73 and the fourth lighting cable 74 are connected to the lighting branch unit is shown. The lighting branch unit is provided with a lighting connection connector for connecting each lighting cable. By connecting a lighting cable to each lighting connector, power is supplied from the lighting control means 31 at a predetermined lighting timing, and the lighting means can be turned on. In this case, if the lighting cable is erroneously connected to a lighting connector different from the original, a partial lighting image may be taken at a different lighting timing or direction, and accurate inspection may not be performed. Therefore, by providing the lighting connection connector with the first installation notation indicating the lighting connector of the connection destination as the installation assisting means, the lighting connector of the connection destination can be indicated to the user, and incorrect connection can be avoided. Here, the user can visually confirm the connection destination of the first lighting means by writing “1” to the lighting connection connector to which the first lighting cable 71 is connected. Also preferably, a second installation notation indicating information on a connection destination is provided at a connection end of a lighting cable extended from each lighting means. In the example of FIG. 24, “1” is described as the second installation notation at the connection end of the first lighting cable extended from the first lighting means. This allows the user to connect the “1” of the lighting connection connector and the “1” of the first lighting cable in a corresponding manner, thereby further ensuring the connection accuracy. In this way, by matching the first installation notation provided on the lighting connector side with the second installation notation provided on the connection cable side, connection work errors can be further reduced, and the reliability of inspection can be improved. . Similarly, the second lighting cable 72, the third lighting cable 73, the fourth lighting cable 74, and the lighting connection connector to which the second lighting cable 72, the third lighting cable 73, and the like are connected can also be provided with the second installation notation and the first installation notation, respectively. . As described above, the lighting branch unit and the lighting control means may be integrated.

Further, the installation assisting means can be applied to the connection between the lighting cable and the lighting means. That is, as shown in FIG. 25, the interface portion for connecting the first lighting cable 71 to the first lighting means may be similarly provided with an installation notation indicating a wiring correspondence as an installation auxiliary means. Here, "1" is described as the third installation notation on the connector portion of the first lighting means for connecting the first lighting cable. Also, “1” is written on the edge of the first lighting cable. Thereby, even when wiring the first lighting means and the first lighting cable, it is possible to avoid a mistake in wiring. In particular, when a plurality of illuminating units have the same shape or a long illuminating cable, even in an environment where a wiring error is likely to occur in other words, the wiring is reliably performed, and the reliability of the inspection accuracy is improved. In the example of FIG. 25, only the connection between the first lighting means and the first lighting cable is shown for simplicity of illustration, but it goes without saying that the same applies to the second lighting means to the fourth lighting means. Absent. In the above example, each lighting cable is of a pluggable type with the lighting means and the lighting branch unit. However, the present embodiment can be applied to a configuration in which the lighting cable is directly attached to the lighting means or the lighting branch unit. Further, as the installation notation, a numeral is used in the above example, but a visual notation such as a character, a symbol, a mark, and a color can be appropriately used. Also, a plurality of notations may be combined. For example, if the installation notation is configured by combining colors and numbers, such as 1, red, 2 blue, 3 green, and 4 purple, the wiring relationship becomes clearer. Furthermore, the present invention is not limited to visual observation, and an installation notation that can be recognized by touch, such as Braille, may be applied in addition to or instead of this. In particular, the present invention can be suitably used when the connection portion is fine, printing or engraving is physically difficult, or when it is difficult to visually check such as installation in a narrow or dark place.
(Second embodiment)

  In the above example, the example in which the installation assisting unit assists the work of wiring each lighting unit to the lighting control unit has been described. In this case, the installation assisting means is provided on the lighting means or the lighting cable. However, the installation assisting means is not limited to this, and can be applied to assist the work of installing the lighting means at a correct position. In this case, the installation assisting means can be provided in the imaging means and the lighting means. Such an example is shown in the schematic plan view of FIG. In this example, the first illuminating means 21, the second illuminating means 22, the third illuminating means 23, and the fourth illuminating means 24 formed in an arc shape are arranged around the imaging means (here, east, west, north and south) in plan view. are doing. As described above, in the case where the shape of each lighting unit is the same, a mistake may occur when installing the lighting unit. Therefore, by arranging the illumination unit as an installation assisting unit on the imaging unit, the illuminating unit can be arranged in accordance with this notation, thereby avoiding a mistake. Here, the top surface of each lighting unit is colored with a different color, and the top surface of the imaging unit is colored with the same color on the top surface of the imaging unit as the fourth installation notation. In this example, the first lighting means 21 is colored red, the second lighting means 22 is colored blue, the third lighting means 23 is colored green, and the fourth lighting means 24 is colored purple. This allows the user to distinguish the lighting means in accordance with the fourth installation notation provided on the imaging means during the installation work of the lighting means, and to easily visually confirm whether or not the lighting means has been correctly installed even after the installation. Therefore, there is an advantage that the mistake of the lighting means can be prevented.

  Further, this example is merely an example, and the color for coloring each lighting unit can be arbitrarily changed. In addition, not only colors but also patterns, patterns, symbols, numbers, characters, and the like can be used for distinction. Further, the position where such a notation is provided may be on the side surface or the bottom surface in addition to the top surface of the illumination unit or the imaging unit. Furthermore, the installation assisting means is not limited to the configuration in which the installation assisting means is provided in the imaging means. For example, the installation assisting means may be provided in another part such as a stage on which a work is mounted or a housing of an image inspection apparatus.

In the wiring work between the lighting cable drawn from each lighting means and the lighting branch unit, the correct combination of the wiring destinations is displayed by the installation assisting means in the same manner as described above to prevent mistakes in the wiring work, and Confirmation work after wiring can be easily performed.
(Third embodiment)

  In the second embodiment described above, an example has been described in which, when three or more illumination units are annularly arranged around a work, the relative installation positions are confirmed by the installation auxiliary unit. On the other hand, the illuminating means can be integrally formed in advance as an annular illuminating unit as described above. Even in such a case, as shown in FIG. 23, unless the annular illumination unit 20 formed in an annular shape is fixed by adjusting the relative rotation position, that is, the rotation angle with respect to the imaging means, accurate photometric stereo processing is performed. No results can be obtained. In particular, when the ring-shaped annular illumination unit 20 has a point-symmetrical shape, the rotational position is not determined, so that the adjustment work is also difficult. Even in such a case, the relative rotation positions of the annular illumination unit and the imaging unit can be determined by the installation assisting unit. For example, as shown in FIG. 27, as an installation auxiliary unit, a line or a line Show seams. This clarifies the area occupied by each lighting means. In addition, a line is displayed on the upper surface of the image pickup means corresponding to this. By providing such an installation assisting means, it is easy to define the rotational positions of the imaging means and the annular illumination unit 20 and to position them.

  Furthermore, since the determination cannot be made even if the annular illumination unit 20 is rotated by 45 ° in this state, the reference position is clarified. For example, by coloring the first illumination means, the posture of the annular illumination unit 20 becomes clear, and a situation in which the annular illumination unit 20 shifts by 45 ° can be avoided. Corresponding to this, it is preferable to similarly apply the same coloring to the upper surface of the imaging means corresponding to the colored first lighting means. This makes it possible to easily determine the rotation angle of the annular illumination unit so that the colored regions match each other.

  Further, the above is an example, and the installation assisting unit can appropriately adopt another notation that can define the rotational position of the annular illumination unit. For example, as shown in a modification shown in FIG. 28, a line is displayed at a reference position of the annular illumination unit 20 and the line is defined so as to be located at the zenith. Accurate positioning can be achieved. The notation indicating the reference position is not limited to a line, and any pattern such as another mark, character, or pattern can be used. For example, in the example of FIG. 29, the position at the zenith is displayed by an arrow.

  Also, the reference position is not limited to one example, and a plurality of reference positions may be provided. For example, as shown in FIG. 30, different marks may be displayed in regions corresponding to the respective lighting units. In addition, by displaying the same mark in the lighting connector of the lighting branch unit in correspondence with each mark in correspondence with each mark, it can also be used to distinguish the wiring of the lighting cable drawn out from each lighting means. it can.

  Notation is not limited to marks, and any pattern such as characters, colors, and patterns can be used. For example, in the example shown in the plan view of FIG. 31, characters N, E, S, and W indicating the direction are displayed. Similarly, characters indicating the azimuth are added to the image pickup means 11 and the illumination branch unit 75. Thus, positioning and wiring according to the settings of the photometric stereo processing can be easily performed.

  In the example shown in the plan view of FIG. 32, a colored mark is displayed at a portion corresponding to each lighting unit. In this example, the first lighting means 21 is provided with a red mark, the second lighting means 22 is provided with a blue mark, the third lighting means 23 is provided with a green mark, and the fourth lighting means 24 is provided with a purple mark. In addition, by attaching the same mark to the image pickup means 11 and the illumination branch unit 75, positioning and wiring work and confirmation can be easily performed.

  Further, as a method of determining the reference position, in addition to the configuration of providing a reference indicating the reference, a method of clarifying the posture from the outer shape by eliminating the physical shape in point symmetry can be adopted. For example, in the example shown in FIG. 33, positioning can be performed by providing a triangular projection on a part of the annular illumination unit 20 and adjusting the projection to the position of the zenith based on the projection. Also, by forming similar projections on the image pickup means 11 and the branch unit 75, these postures and wiring can be matched. Further, the shapes of the illumination means 21 to 24 and the illumination branching unit 75 are not limited to a circular shape in a plan view, and any shape such as a rectangular shape or a polygonal shape can be used. For example, in the example shown in FIG. 34, the shape is rectangular. In addition, any form such as a concave part or a concave part is applicable as long as the protrusion part can serve as a mark indicating a reference position or direction.

In the example of FIG. 27 described above, an illumination cable is connected to each illumination unit from the annular illumination unit 20 in which the first illumination unit 21, the second illumination unit 22, the third illumination unit 23, and the fourth illumination unit 24 are integrated. Although they are drawn out correspondingly, in the integrated annular lighting unit 20, the lighting cables may be combined. As a result, the wiring work can be reduced in labor, avoiding mistakes, and the wiring itself can be simplified. FIG. 28 and FIG. 29 show examples of such wiring.
(Image inspection method)

  Here, a procedure of an image inspection method of performing an appearance inspection of a work using the image inspection apparatus 1 will be described with reference to a flowchart of FIG.

  In step ST1, the image processing unit 41 issues a trigger signal to the lighting units 21, 22, 23, 24 via the lighting control unit 31, and the lighting units 21, 22, 23, 24 respond to the trigger signal. Lights one by one in order.

  In step ST2, the imaging unit 11 is activated each time each of the illumination units 21, 22, 23, and 24 is turned on to photograph the work WK.

  In step ST <b> 3, the imaging unit 11 transfers the four image signals Q <b> 1 to Q <b> 4 of the image of the work WK to the image processing unit 41.

  In step ST4, the image processing unit 41 calculates the normal vector of the surface of each pixel for each of the image signals Q1 to Q4 using the four image signals Q1 to Q4 input from the imaging unit 11.

  In step ST5, the image processing unit 41 converts each of the image signals Q1 to Q4 into 1/1, 1/2, 1/4, 1/8, 1/16, and 1/32, which are required in the subsequent processing. A reduced tilt image that is reduced in each case is created. Further, the image processing unit 41 creates a texture extraction image required for subsequent processing for each of the image signals Q1 to Q4.

  In step ST6, the feature size is adjusted as needed. By changing the feature size, the size of the unevenness extracted in the unevenness extracted image changes. Specifically, when the feature size is increased, a concavo-convex extracted image in which large-sized concavities and convexities are extracted is obtained. Conversely, when the feature size is reduced, small-sized irregularities are extracted. Therefore, the user adjusts the feature size according to the size of the flaw to be extracted. Alternatively, in the case of an OCR application, a feature extraction image suitable for OCR can be obtained by increasing the feature size.

  In step ST7, an unevenness extraction image calculation is performed. In this example, the image processing unit 41 generates an unevenness extraction image in which unevenness is extracted according to the feature size set in step ST6, and causes the display unit 51 to display the image.

  In steps ST10 to ST12, the image processing unit 41 or the PLC 81 performs a flaw detection process for detecting a flaw using the flaw inspection tool on the contour extracted image, and determines whether or not the flaw is a flaw. do.

  First, in step ST10, the image processing unit 41 or the PLC 81 specifies the position of the inspection area to be inspected with respect to the generated contour image. When setting the inspection area, an image of the work WK is extracted by, for example, extracting an edge. When the position of the work WK does not significantly deviate, the set position of the inspection area may be registered in advance.

  In step ST11, the image processing unit 41 or the PLC 81 performs image processing for detecting a flaw in the specified inspection area. The image processing unit 41 or the PLC 81 calculates the reference density value by the stored calculation method, and calculates the difference between the density value of each pixel in the inspection area and the reference density value for each pixel. Then, the image processing unit 41 or the PLC 81 performs a labeling process (a set of white pixels of the binary image to “0, 1, 2,. ..) is performed, and the feature amount is calculated for each of the specified scratches. The feature amount to be calculated is, for example, positive / negative information on the sign of the difference, the sum of the differences, the maximum value of the differences, the average value of the differences, or the standard deviation of the differences.

In step ST12, the image processing unit 41 or the PLC 81 executes a flaw determination process on the flaw specified in step ST11 based on a determination standard used for flaw determination. If it is determined to be a flaw, marking is made on the flaw on the display means 51, and the process is terminated.
(Modification)

  In the above example, an example has been described in which the feature size is adjustable by the user, and a concavo-convex extraction image in which concavities and convexities of a size desired by the user are extracted using the parameter of the feature size. However, the present invention is not limited to this configuration, and a plurality of observation modes are prepared in advance according to a user's observation purpose and purpose, and a desired image is generated by allowing the user to select an observation mode. You can also. Such an example will be described with reference to the flowchart in FIG. Steps ST "1 to ST" 5 are the same as those in FIG. 35 described above, and a detailed description thereof will be omitted.

  In step ST "6, the user is caused to select an observation mode. In this example, any one of the contour extraction mode, the texture extraction mode, and the concavo-convex mode is selectable. Each observation mode has a feature size suitable for each observation purpose. It should be noted that after the observation mode is selected, the user may manually configure the feature size so as to be able to finely adjust it.The next step differs according to the selected observation mode, that is, contour extraction. When the mode is selected, the process proceeds to step ST "7, when the texture extraction mode is selected, the process proceeds to step ST" 8, and when the unevenness mode is selected, the process proceeds to step ST "9. After the processing in each step, the process proceeds to step ST ″ 10. In this manner, the unevenness extraction image generation unit, the outline image generation unit 41b, and the texture extraction image generation unit 41c can be switched.

  In step ST "7, the image processing unit 41 performs an unevenness extraction image calculation. That is, the image processing unit 41 displays, on the display unit 51, a contour extraction image of a feature size in which the unevenness extraction image is displayed.

  In step ST "8, the image processing unit 41 executes a process when the user has selected the contour extraction mode. That is, the image processing unit 41 performs processing based on the reduced inclination image created in step ST" 5. Then, the contour extraction image calculation is performed, and the contour extraction image is displayed on the display means 51.

  In step ST "9, the image processing unit 41 executes a process when the user selects the texture extraction mode. That is, the image processing unit 41 performs processing based on the texture extraction image created in step ST" 5. Then, a texture extraction image calculation is performed, and the texture extraction image is displayed on the display means 51.

  In the subsequent steps ST "10 to ST12, the same procedure as in FIG. 35 described above can be used, and detailed description will be omitted.

  According to the image inspection apparatus described above, the first-order skew image obtained by using the photometric stereo method is used as a starting point, and the obtained first-order skew image is converted in the X direction and the Y direction. Differential processing is performed to create a secondary inclination image, that is, an outline extraction image. Due to this processing, the obtained inspection image greatly changes with respect to a slight inclination of the illumination or the work installation surface or an error in the input information such as the originally input illumination position. Inspections based on the real thing, such as the phenomenon that an image becomes uneven even though it is not performed, and the phenomenon that the center of the work WK rises due to the tendency to become brighter near the illumination in general. The disadvantage of the conventional photometric stereo method that an image cannot be obtained can be reduced. By setting the gradation where the change in inclination in the concave direction of the surface is large to be dark, and setting the change in the direction of small change in the convex direction of the surface to be bright, scratches and contours where the inclination of the work surface greatly changes are set. An image suitable for extraction is obtained.

  Further, the contour image generating means creates a plurality of reduced inclination images having different reduction rates from the calculated normal vectors of the respective pixels, and performs a differentiation process on each of the reduced inclination images in the X direction and the Y direction. The reduced outline image is weighted so that the reduced outline image at the predetermined reduction rate is largely reflected, enlarged to the original size, and the sum of the enlarged reduced outline images is added. It can be a contour extraction image. According to this configuration, the size to be determined as a flaw differs depending on the user's application, but weighting can be performed so that a reduced contour image at a predetermined reduction ratio is largely reflected in the contour extraction image. According to the use of the user, a contour extraction image in which the reduced contour image at the reduction ratio desired by the user is enhanced can be obtained. As the user's needs, for example, there is a case where it is desired to determine a scratch extending over 10 pixels as a scratch, and a case where it is desired to determine a dent that extends over 100 pixels as a scratch. In some cases, only a steep edge is desired to be extracted as an edge.

  In addition, when a flaw is usually detected over a plurality of frequencies, a sum of all enlarged reduced contour images is used as a contour extracted image. As compared with the above, a flaw, an outline, and the like can be detected more clearly without blurring as a whole.

  Further, the weighting is performed by preparing a predetermined set of weights and multiplying the reduced outline image by the set of weights, thereby apportioning the adoption ratio of the reduced outline image at each reduction ratio. Can be. According to this, a predetermined set of weights (feature size) is preset when combining with the contour extraction image, so that the user can instantly and easily switch to a desired contour extraction image.

  Furthermore, the set of weights can include a set in which the adoption ratio of the reduced outline image that can obtain the outline extraction image in which the unevenness of the surface of the work is clear is large. By setting this usage method to, for example, the concavo-convex mode, it is possible to selectively use the contour extraction mode.

  Furthermore, since the set of weights can include a set in which the adoption ratio of the reduced outline image that can obtain the outline extraction image suitable for OCR is increased, the OCR is performed on, for example, an inscription on a casting. It is possible to create an image suitable for the above.

  Furthermore, the albedo of each pixel of the same number as that of the input image is calculated from the calculated normal vector of each pixel, which is the number of times of illumination by the illumination means, and the pattern in which the inclination state of the work surface is removed from the albedo. Is generated, and the contour image generating means and the texture extracted image generating means are configured to be switchable. Therefore, it is usually difficult to distinguish the original scratch from the original contour. Although it is required to search for a part where there should not be an area as an inspection area, a flaw inspection can be performed after a search is performed with a texture extraction image to determine an inspection area. Also, when performing OCR, OCR can be performed after a search is performed on the texture extraction image to determine a target region for OCR.

  The texture extraction image generation means can sort the albedo values of the same number of pixels as the input image and adopt pixel values in a specific order from the top as a texture extraction image. As a result, a pixel value with high luminance causing halation is not adopted, and a texture extraction image from which the influence of halation has been removed is obtained.

  In addition, by providing a flaw inspection tool necessary for flaw inspection after the generation of an outline extraction image, it is generally accepted that photometric stereo technology is one method of three-dimensional measurement. An image inspection apparatus which is positioned as a practical product in which the image inspection apparatus is applied to a flaw inspection can be provided.

  The image inspection apparatus further includes an imaging unit that images the workpiece from a certain direction, an illumination unit that illuminates the workpiece at least three times from different directions, an illumination control unit that sequentially lights the illumination units one by one, Calculating a normal vector of each pixel with respect to the surface of the workpiece by using the image generation unit that generates a plurality of images by driving the imaging unit and pixel values of pixels corresponding to each other in the plurality of images. From the calculated normal vector of each pixel, which is the number of times of illumination by the normal vector calculating means and the illuminating means, calculate the albedo of each pixel as many as the normal vector, and from the albedo, calculate the inclination of the surface of the workpiece. Texture extraction image generation means for generating a texture extraction image showing a pattern from which a state has been removed, wherein the number of texture extraction image generation means is equal to the number of normal vectors. It can be a material obtained by employing the pixel values of a specific order from the top to sort the values of albedo of each pixel of the texture extracted image. According to this, the texture extraction image generating means sorts the albedo values of the same number of pixels as the normal vector and adopts a pixel value in a specific order from the top as the texture extraction image. Therefore, a pixel value with high luminance causing halation is not adopted, and a texture extraction image from which the influence of halation has been removed is obtained.

  As described above, according to the image inspection apparatus according to the embodiment, it is possible to more easily and robustly inspect the work for scratches and prints using the photometric stereo method.

  The image inspection apparatus, the image inspection method, the image inspection program, and the computer-readable recording medium or the recorded apparatus of the present invention can be suitably used for an inspection apparatus or a digitizer using a photometric stereo.

DESCRIPTION OF SYMBOLS 1 ... Image inspection apparatus 11 ... Imaging means 12, 32, 52, 62, 82 ... Cable 71, 72, 73, 74, 76 ... Cable 20, 20 ', 20 "... Ring illumination unit 21, 22, 23, 24, 25 illumination unit 31 illumination control unit 41 image processing unit 41a normal vector calculation unit 41b outline image generation unit 41c texture extraction image generation unit 41d inspection area identification unit 41e image processing unit 41f ... Determination means 42. Signal processing system 43. CPU
44 ... memory 45 ... ROM
46 bus 51 display means 61 operating means 75 lighting branch units 75a, b, c, d lighting connector 81 PLC
e1, cameras e21 to e24, light sources WK for illumination, work SG, stage L1, first illumination means; L2, second illumination means n, normal vector S, diffuse reflection surface OS, obstacles

Claims (13)

An image inspection device for performing a visual inspection of a work,
An annular illumination unit in which three or more illumination blocks are arranged in a ring, and the three or more illumination blocks are configured to be able to illuminate the workpiece from different illumination directions.
An illumination branch unit connected to each illumination block of the annular illumination unit via a single cable or a plurality of different cables, and lighting each of the illumination blocks at a predetermined timing,
In order to capture a plurality of partial illumination images having different illumination directions, provided separately from the annular illumination unit, and by imaging the work from a certain direction at a timing at which the three or more illumination blocks are respectively turned on. Camera and
An image processing unit that is connected to the illumination branch unit and the camera via a cable and generates a contour extraction image indicating a contour of a tilt of a work surface based on the partial illumination image;
An illumination control unit that is provided integrally with the image processing unit or the illumination branch unit and controls the three or more illumination blocks to be lit one by one in a lighting order.
Display means capable of displaying the contour extraction image generated by the image processing unit ,
Inspection area setting means capable of setting an inspection area on the contour extraction image displayed on the display means,
Equipped with a,
The image processing unit uses a plurality of partial illumination images captured by the camera , using a pixel value of each pixel in a corresponding relationship, based on a photometric stereo method, and based on the photometric stereo method, the camera in the optical axis direction of the camera. varies according to the installation position and the installation position of the annular illumination unit is relatively changed, it calculates a normal vector to the surface of the workpiece for each pixel, corresponding to the x and y components of the normal vector Generating the X-coordinate component image and the Y-coordinate component image, performing differential processing on the normal vector in the X direction and the Y direction, generating the contour extraction image showing the contour of the inclination of the surface of the work, line cormorant image inspection apparatus inspecting the set inspection area by setting means. The image inspection apparatus according to claim 1 ,
The image processing unit is further,
The relative positions of the camera and the illumination block in the optical axis direction of the camera, based on the photometric stereo method, using a plurality of partial illumination images captured by the camera, using a pixel value for each pixel in a corresponding relationship. Normal vector calculation means for calculating a normal vector to the surface of the work of each pixel, which changes in accordance with the change of
Contour image generation for performing a differentiation process on the normal vector of each pixel calculated by the normal vector calculation means in the X direction and the Y direction to generate a contour image indicating the contour of the inclination of the surface of the work. Means,
From the normal vector of each pixel calculated by the normal vector calculating means calculates the albedo for each pixel, from the albedo, to generate a texture extraction image showing a pattern obtained by removing the tilt state of the surface of the workpiece Texture extraction image generation means,
An image inspection apparatus, wherein the contour image generating means and the texture extraction image generating means are switchable. The image inspection apparatus according to claim 2 ,
The texture extraction image generation unit calculates an albedo for each pixel of the plurality of partial illumination images, which is present by the number of times of imaging by the camera, from the normal vector of each pixel calculated by the normal vector calculation unit. An image inspection apparatus characterized in that, for each pixel of the plurality of partial illumination images, the albedo value of the corresponding pixel is sorted, and a pixel extracted in a specific order from the top is used as a texture extraction image. The image inspection apparatus according to any one of claims 1 to 3 ,
The image inspection apparatus, wherein the camera and the illuminating unit are independent members, and can be arranged at an arbitrary position. The image inspection apparatus according to any one of claims 1 to 4 ,
An image inspection apparatus, comprising: four illumination blocks . The image inspection apparatus according to any one of claims 1 to 5 ,
The three or more lighting blocks are constituted by a plurality of light emitting elements arranged in a ring,
The lighting control means,
By setting a predetermined number of adjacent light emitting elements as a first lighting block, simultaneously turning on the light emitting elements of the first lighting block and extinguishing the other light emitting elements, the first lighting means sets the first light source from the first lighting direction. Function as lighting,
The second lighting block, which is adjacent to the first lighting block and is controlled to light a second lighting block including a predetermined number of light emitting elements, to illuminate from a second lighting direction different from the first lighting direction. Constitute
Further, by controlling to turn on a third lighting block composed of a predetermined number of light emitting elements adjacent to the second lighting block, from the third lighting direction different from the first lighting direction and the second lighting direction. An image inspection apparatus comprising a third lighting block for lighting. The image inspection apparatus according to any one of claims 1 to 6 ,
The image processing unit generates a plurality of reduced tilt images having different reduction rates from the calculated normal vectors of the respective pixels, and performs a differentiation process in the X direction and the Y direction on each reduced tilt image, thereby obtaining The reduced outline image is weighted so that the reduced outline image at the predetermined reduction ratio is reflected, enlarged to the original size, and the sum of all the enlarged reduced outline images is added to the outline. An image inspection device characterized by using an extracted image. The image inspection apparatus according to claim 7 ,
The weighting is performed by preparing a predetermined set of weights and multiplying the reduced outline image by the set of weights to proportionate the adoption ratio of the reduced outline image at each reduction ratio. An image inspection apparatus characterized by performing. The image inspection apparatus according to claim 7 ,
The image inspection apparatus according to claim 1, wherein the set of weights includes a set in which the adoption ratio of a reduced contour image that can obtain a contour extracted image in which the contour of the surface of the workpiece is clear is large. The image inspection apparatus according to claim 8 or 9 ,
The image inspection apparatus according to claim 1, wherein the set of weights includes a set in which the adoption ratio of a reduced contour image that can obtain a contour extracted image suitable for OCR increases. An inspection method for capturing an image of a work and inspecting the appearance,
The work, the three or more lighting blocks are arranged in a ring, the annular lighting unit configured to be able to illuminate the work from different lighting directions from each other, the three or more lighting blocks, from three or more different lighting directions different from each other, the image in the processing unit or the illumination branch unit and the lighting control means provided integrally, the with three or more lighting block is turned to the lighting order one by one to illumination, provided separately from the said annular illumination unit, the three By imaging the workpiece from a certain direction at the timing at which the above illumination blocks are respectively turned on, in order to capture a plurality of partial illumination images having different illumination directions, a relative position with respect to the illumination block and an imaging direction are previously determined. using common camera adjusted, by imaging one of the partial illumination image for each illumination direction, preparative plurality illumination direction different partial illumination image A step of,
A plurality of partial illumination images having different illumination directions, using a pixel value for each pixel in a corresponding relationship, based on a photometric stereo method, based on the installation position of the camera in the optical axis direction of the camera and the annular illumination unit. The normal vector to the surface of the work of each pixel, which changes in accordance with the relative change in the installation position, is obtained through a single cable or a plurality of different cables with each of the illumination blocks of the annular illumination unit. The image processing unit connected via a cable to the lighting branch unit and the camera connected to the lighting branch unit and the camera to connect each of the lighting blocks at a predetermined timing, and calculates the x component and the y component of the normal vector. generates X-coordinate component image and the Y coordinate component image the corresponding, subjected to differential processing in the X direction and Y direction of the normal vector, the partial irradiation A step of the contour extracted image showing the inclination of the contour of the workpiece surface, to produce with by the image processing unit based on the image,
Displaying the contour extracted image on a display unit, setting an inspection area on the displayed contour extracted image, and performing an inspection in the set inspection area. An inspection program for inspecting the appearance by capturing an image of the work, the computer,
The work, the three or more lighting blocks are arranged in a ring, the annular lighting unit configured to be able to illuminate the work from different lighting directions from each other, the three or more lighting blocks, from three or more different lighting directions different from each other, the image in the processing unit or the illumination branch unit and the lighting control means provided integrally, the with three or more lighting block is turned to the lighting order one by one to illumination, provided separately from the said annular illumination unit, the three By imaging the workpiece from a certain direction at the timing at which the above illumination blocks are respectively turned on, in order to capture a plurality of partial illumination images having different illumination directions, a relative position with respect to the illumination block and an imaging direction are previously determined. using common camera adjusted, by imaging one of the partial illumination image for each illumination direction, preparative plurality illumination direction different partial illumination image And a function of,
A plurality of partial illumination images having different illumination directions, using a pixel value for each pixel in a corresponding relationship, based on a photometric stereo method, based on the installation position of the camera in the optical axis direction of the camera and the annular illumination unit. The normal vector to the surface of the work of each pixel, which changes in accordance with the relative change in the installation position, is obtained through a single cable or a plurality of different cables with each of the illumination blocks of the annular illumination unit. The image processing unit connected via a cable to the lighting branch unit and the camera connected to the lighting branch unit and the camera to connect each of the lighting blocks at a predetermined timing, and calculates the x component and the y component of the normal vector. generates X-coordinate component image and the Y coordinate component image the corresponding, subjected to differential processing in the X direction and Y direction of the normal vector, the partial irradiation The contour extraction image showing the outline of the inclination of the workpiece surface based on the image, and generating with by the image processing unit,
A function of displaying the contour extraction image on a display means,
A function of setting an inspection area on the displayed contour extraction image,
A function of performing an inspection in the set inspection area. A computer-readable recording medium or a device on which the image inspection program according to claim 12 is recorded.