JP2020012816A - Detection system and detection method - Google Patents

Abstract

To estimate the type of defects in the time correlation image inspection.SOLUTION: An inspection system of an embodiment includes: a planar illumination part which gives periodic temporal and spatial changes in light intensity; a time correlation image generation part which generates a time correlation image by an imaging system for multiplying a light intensity signal from the inspection object surface and a reference signal which changes periodically in response to the temporal change in the light intensity; an abnormality detection part which performs detection of a plurality of types of abnormalities with respect to features obtained from the time correlation image; and an abnormality type estimation part which estimates the type of abnormality based on the combination of the detection results of the plurality of types of abnormalities detected by the abnormality detection part.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to an inspection system and an inspection method.

  Conventionally, there has been proposed a technique of irradiating a test object with light, capturing reflected light from the surface of the test object as image data, and detecting an abnormality of the test object based on a change in luminance of the image data. Have been.

  At that time, there has been proposed a technique of periodically changing the intensity of light applied to an object to be inspected and detecting an abnormality based on a change in luminance of captured image data.

JP 2014-002125 A

  However, in the related art, the light intensity is changed, but the captured image data does not include information on the time transition when the light intensity is changed. For this reason, when detecting an abnormality of the object to be inspected from the captured image data, the detection accuracy may be reduced. Further, in this type of technology, it is significant if the type of defect can be grasped.

  The inspection system according to the embodiment includes a planar illumination unit that provides a periodic time change and a spatial change in light intensity, a light intensity signal from the inspection target surface, and a periodic signal corresponding to the time change in light intensity. A time correlation image generation unit that generates a time correlation image by an imaging system that multiplies the reference signal that changes to a reference signal, an abnormality detection unit that detects a plurality of types of abnormalities for features obtained from the time correlation image, and an abnormality detection An abnormality type estimating unit that estimates an abnormality type based on a combination of detection results of a plurality of types of abnormalities detected by the unit.

FIG. 1 is a diagram illustrating a configuration example of the inspection system according to the first embodiment. FIG. 2 is a block diagram illustrating a configuration of the time correlation camera according to the first embodiment. FIG. 3 is a conceptual diagram illustrating frames accumulated in chronological order by the time correlation camera of the first embodiment. FIG. 4 is a diagram illustrating an example of a stripe pattern emitted by the illumination device according to the first embodiment. FIG. 5 is a diagram illustrating a first detection example of the abnormality of the inspection object by the time correlation camera of the first embodiment. FIG. 6 is a diagram illustrating an example of light amplitude that changes according to the abnormality when the abnormality illustrated in FIG. 5 is present in the inspection object. FIG. 7 is a diagram illustrating a second detection example of the abnormality of the inspection object by the time correlation camera according to the first embodiment. FIG. 8 is a diagram illustrating a third detection example of the abnormality of the inspection object by the time correlation camera of the first embodiment. FIG. 9 is a diagram illustrating an example of a stripe pattern output to the lighting device by the lighting control unit according to the first embodiment. FIG. 10 is a diagram illustrating an example of a wave shape representing a stripe pattern after passing through the screen according to the first embodiment. FIG. 11 is a block diagram illustrating a configuration of the abnormality detection unit according to the first embodiment. FIG. 12 is a flowchart illustrating a procedure of the abnormality detection processing based on the amplitude in the abnormality detection unit according to the first embodiment. FIG. 13 is a flowchart illustrating a procedure of a phase-based abnormality detection process in the abnormality detection unit according to the first embodiment. FIG. 14 is a flowchart illustrating a procedure of an abnormality detection process based on the amplitude and the intensity in the abnormality detection unit according to the first embodiment. FIG. 15 is a flowchart illustrating a procedure of the inspection processing of the inspection object in the inspection system of the first embodiment. FIG. 16 is a diagram illustrating an example of switching a stripe pattern output by the illumination control unit according to the second modification. FIG. 17 is a diagram illustrating an example in which the illumination control unit of the second modification irradiates a stripe pattern on a surface including an abnormality (defect). FIG. 18 is a diagram showing a relationship between an abnormality (defect) and a stripe pattern on the screen when the stripe pattern is changed in the y direction. FIG. 19 is a diagram illustrating an example of a stripe pattern output by the illumination control unit of the third modification to the illumination device. FIG. 20 is a diagram illustrating an example of a stripe pattern output by the illumination control unit according to the second embodiment to the illumination device.

(First embodiment)
An inspection system according to the present embodiment will be described. The inspection system according to the first embodiment has various configurations for inspecting an object to be inspected. FIG. 1 is a diagram illustrating a configuration example of an inspection system according to the present embodiment. As shown in FIG. 1, the inspection system according to the present embodiment includes a PC 100, a time correlation camera 110, a lighting device 120, a screen 130, and an arm 140.

  The arm 140 is used to fix the test object 150, and changes the position and orientation of the surface of the test object 150, which can be photographed by the time correlation camera 110, under the control of the PC 100.

  The illuminating device 120 is a device that irradiates the inspection object 150 with light, and can control the intensity of the irradiating light on a region basis according to a stripe pattern from the PC 100. Further, the lighting device 120 can control the intensity of light in the area unit according to a periodic transition of time. In other words, the lighting device 120 can provide a periodic time change and a spatial change in light intensity. A specific light intensity control method will be described later.

  The screen 130 diffuses the light output from the illumination device 120 and then irradiates the inspection object 150 with light in a planar manner. The screen 130 of the present embodiment irradiates the object 150 with the light to which the periodic time change and spatial change given from the illumination device 120 are applied. Note that an optical system component (not shown) such as a Fresnel lens for focusing may be provided between the illumination device 120 and the screen 130.

  In the present embodiment, an example will be described in which the illumination device 120 and the screen 130 are combined to form a planar irradiation unit that provides a periodic time change and a spatial change in light intensity. There is no limitation. For example, the lighting unit may be configured by arranging LEDs in a plane.

  The time correlation camera 110 includes an optical system 210, an image sensor 220, a data buffer 230, a control unit 240, and a reference signal output unit 250. FIG. 2 is a block diagram illustrating a configuration of the time correlation camera 110 according to the present embodiment.

  The optical system 210 includes a photographic lens and the like, transmits a light beam from a subject (including an object to be inspected) outside the time correlation camera 110, and forms an optical image of the subject formed by the light beam.

  The image sensor 220 is a sensor that can output the intensity of light incident via the optical system 210 at a high speed for each pixel as a light intensity signal.

  The light intensity signal of the present embodiment is obtained by the illumination device 120 of the inspection system irradiating a subject (including the subject) with light, and the image sensor 220 receiving reflected light from the subject.

  The image sensor 220 is, for example, a sensor that can read at a higher speed than a conventional sensor, and has a two-dimensional planar shape in which pixels are arranged in two types of directions, a row direction (x direction) and a column direction (y direction). It shall be configured. Each pixel of the image sensor 220 is defined as a pixel P (1,1),..., P (i, j),..., P (X, Y). × Y). Note that the reading speed of the image sensor 220 is not limited, and may be the same as the conventional one.

  The image sensor 220 receives the light flux from the subject (including the subject to be inspected) transmitted by the optical system 210 and performs photoelectric conversion on the received light flux, thereby generating a light intensity signal (imaging) indicating the intensity of light reflected from the subject. A two-dimensional frame composed of the signals is generated and output to the control unit 240. The image sensor 220 of the present embodiment outputs the frame for each readable unit time.

  The control unit 240 according to the present embodiment is configured by, for example, a CPU, a ROM, a RAM, and the like, and executes an inspection program stored in the ROM to execute a transfer unit 241, a reading unit 242, and an intensity image superimposing unit 243. , A first multiplier 244, a first correlation image superposition unit 245, a second multiplier 246, a second correlation image superposition unit 247, and an image output unit 248. Note that the present invention is not limited to being realized by a CPU or the like, and may be realized by an FPGA or an ASIC.

  The transfer unit 241 stores frames composed of light intensity signals output from the image sensor 220 in the data buffer 230 in chronological order.

  The data buffer 230 accumulates frames composed of light intensity signals output from the image sensor 220 in chronological order.

  FIG. 3 is a conceptual diagram showing frames stored in chronological order by the time correlation camera 110 of the present embodiment. As shown in FIG. 3, the data buffer 230 of the present embodiment includes a plurality of light intensity signals G (1,1, t) at each time t (t = t0, t1, t2,..., Tn). .., G (i, j, t),..., G (X, Y, t) are combined into a plurality of frames Fk (k = 1, 2,..., N) in chronological order. Stored. One frame created at time t is a light intensity signal G (1,1, t),..., G (i, j, t),..., G (X, Y, t). Be composed.

  The light intensity signal (imaging signal) G (1,1, t),..., G (i, j, t),..., G (X, Y, t) of this embodiment includes a frame image Fk ( Each pixel P (1,1),..., P (i, j),..., P (X, Y) constituting k = 1, 2,.

  The frame output from the image sensor 220 is composed of only the light intensity signal, in other words, it can be considered as monochrome image data. In this embodiment, an example in which the image sensor 220 generates monochrome image data in consideration of resolution, sensitivity, cost, and the like will be described. However, the image sensor 220 is not limited to a monochrome image sensor. Instead, a color image sensor may be used.

  Returning to FIG. 2, the reading unit 242 of the present embodiment outputs the light intensity signals G (1,1, t),..., G (i, j, t),. Y, t) are read out in chronological order on a frame basis and output to the first multiplier 244, the second multiplier 246, and the intensity image superimposing unit 243.

  The time correlation camera 110 of the present embodiment generates image data for each output destination of the reading unit 242. In other words, the time correlation camera 110 creates three types of image data.

  The time correlation camera 110 of the present embodiment generates intensity image data and two types of time correlation image data as three types of image data. Note that the present embodiment is not limited to the generation of three types of image data, and may include cases where intensity image data is not generated and cases where one or three or more types of time-correlated image data are generated. .

  As described above, the image sensor 220 of the present embodiment outputs a frame composed of a light intensity signal for each unit time. However, in order to generate normal image data, a light intensity signal for an exposure time required for photographing is required. Thus, in the present embodiment, the intensity image superimposing unit 243 generates intensity image data by superimposing a plurality of frames for an exposure time necessary for photographing. Note that each pixel value (a value representing light intensity) G (x, y) of the intensity image data can be derived from the following equation (1). Note that the exposure time is a time difference between t0 and tn.

  As a result, intensity image data of a subject (including an object to be inspected) is generated in the same manner as the conventional camera. Then, the intensity image superimposing unit 243 outputs the generated intensity image data to the image output unit 248.

  The time correlation image data is image data indicating a change in light intensity according to a time transition. That is, in the present embodiment, for each frame in chronological order, the light intensity signal included in the frame is multiplied by the reference signal indicating the time transition, and the reference signal is multiplied by the light intensity signal, and A time correlation value frame composed of correlation values is generated, and time correlation image data is generated by superimposing a plurality of time correlation value frames.

  By the way, in order to detect an abnormality of the test object using the time correlation image data, it is necessary to change the light intensity signal input to the image sensor 220 in synchronization with the reference signal. For this reason, as described above, the illumination device 120 performs planar light irradiation that periodically changes the time and spatial movement of the stripes via the screen 130, as described above.

In the present embodiment, two types of time-correlated image data are generated. The reference signal may be a signal representing a time transition, but in this embodiment, a complex sine wave e ^-jωt is used. It is assumed that the angular frequency ω and the time t. The angular frequency ω is set so that the complex sine wave e ^−jωt representing the reference signal is correlated with one cycle of the above-described exposure time (in other words, the time required to generate the intensity image data and the time correlation image). Shall be set. In other words, the two-dimensional and dynamic light formed by the illumination unit 120 and the illumination unit such as the screen 130 is generated at each position on the surface (reflection surface) of the inspection object 150 at the first period (time period). And a change in spatial irradiation intensity in a second cycle (spatial cycle) along at least one direction along the surface. When this planar light is reflected by a surface, it is subjected to complex modulation according to the specifications of the surface (such as the distribution of normal vectors). The time-correlation camera 110 receives time-modulated image data as a complex signal by receiving light that is complex-modulated on the surface and performing quadrature detection (quadrature demodulation) using the reference signal of the first cycle. By performing modulation and demodulation based on such complex time correlation image data, it is possible to detect a feature corresponding to the distribution of the surface normal vector.

The complex sine wave e ^-jωt can also be expressed as e ^-jωt = cos (ωt) -j · sin (ωt). Therefore, each pixel value C (x, y) of the time correlation image data can be derived from the following equation (2).

  In the present embodiment, in equation (2), two types of time-correlated image data are divided into a pixel value C1 (x, y) representing a real part and a pixel value C2 (x, y) representing an imaginary part. Generate.

For this reason, the reference signal output unit 250 generates and outputs different reference signals to the first multiplier 244 and the second multiplier 246, respectively. Reference signal output section 250 of this embodiment outputs the first reference signal cosωt corresponding to the real part of the complex sine wave e ^-Jeiomegati the first multiplier 244, the imaginary part of the complex sine wave e ^-Jeiomegati The corresponding second reference signal sinωt is output to the second multiplier 246. As described above, the reference signal output unit 250 of the present embodiment describes an example of outputting two types of reference signals represented as time functions of a sine wave and a cosine wave that form a Hilbert transform pair with each other. Any reference signal that changes according to time transition, such as a function, may be used.

Then, the first multiplier 244 ^converts the real part cosωt of the complex sine wave e ^−jωt input from the reference signal output unit 250 for each frame of the light intensity signal of the frame input from the reading unit 242. Multiply.

  The first correlation image superimposing unit 245 performs a process of superimposing the multiplication result of the first multiplier 244 for each pixel on a plurality of frames corresponding to the exposure time required for photographing. Thus, each pixel value C1 (x, y) of the first time-correlated image data is derived from the following equation (3).

Then, the second multiplier 246 multiplies the light intensity signal of the frame input from the reading unit 242 by the imaginary part sinωt of the complex sine wave e ^−jωt input from the reference signal output unit 250.

  The second correlation image superimposing unit 247 performs a process of superimposing the multiplication result of the second multiplier 246 for each pixel on a plurality of frames corresponding to the exposure time required for photographing. Thereby, each pixel value C2 (x, y) of the second time-correlated image data is derived from the following equation (4).

  By performing the above-described processing, two types of time-correlated image data, in other words, time-correlated image data having two degrees of freedom can be generated.

Further, the present embodiment does not limit the type of the reference signal. For example, in the present embodiment, two types of time-correlated image data of the real part and the imaginary part of the complex sine wave e ^-jωt are created, but two types of image data are generated based on the amplitude of light and the phase of light. May be.

  Note that the time correlation camera 110 of the present embodiment can create a plurality of systems as time correlation image data. Thus, for example, when light in which stripes having a plurality of types of widths are combined is irradiated, two types of time-correlated image data based on the real part and the imaginary part described above can be created for each width of the stripes. For this purpose, the time correlation camera 110 includes a plurality of combinations of two multipliers and two correlation image superimposing units for a plurality of systems, and the reference signal output unit 250 provides an angular frequency suitable for each system. A reference signal based on ω can be output.

  Then, the image output unit 248 outputs the two types of time correlation image data and the intensity image data to the PC 100. Thereby, the PC 100 detects an abnormality of the test object using the two types of time correlation image data and the intensity image data. For that purpose, it is necessary to irradiate the subject with light.

  The illumination device 120 according to the present embodiment irradiates a stripe pattern that moves at a high speed. FIG. 4 is a diagram illustrating an example of a stripe pattern emitted by the illumination device 120 of the present embodiment. In the example shown in FIG. 4, the stripe pattern is scrolled (moved) in the x direction. A white area is a bright area corresponding to a stripe, and a black area is an interval area (dark area) corresponding to a stripe.

  In the present embodiment, the stripe pattern illuminated by the illumination device 120 is moved by one cycle by the exposure time at which the time correlation camera 110 captures the intensity image data and the time correlation image data. Thereby, the lighting device 120 gives a periodic time change of the light intensity due to the spatial movement of the stripe pattern of the light intensity. In this embodiment, by associating the time during which the stripe pattern of FIG. 4 moves by one cycle with the exposure time, each pixel of the time-correlated image data has at least the light intensity signal of one cycle of the stripe pattern. Information is embedded.

  As illustrated in FIG. 4, in the present embodiment, an example will be described in which the illumination device 120 irradiates a stripe pattern based on a rectangular wave, but an illumination device other than a rectangular wave may be used. In the present embodiment, by irradiating the illumination device 120 via the screen 130, it is possible to blur the bright and dark boundary region of the rectangular wave.

  In the present embodiment, the stripe pattern irradiated by the illumination device 120 is represented as A (1 + cos (ωt + kx)). That is, the stripe pattern includes a plurality of stripes repeatedly (periodically). Note that the intensity of the light applied to the object to be inspected can be adjusted between 0 and 2 A, and is set to the light phase kx. k is the wave number of the stripe. x is the direction in which the phase changes.

  Then, the fundamental frequency component of the light intensity signal f (x, y, t) of each pixel of the frame can be expressed as the following equation (5). As shown in Expression (5), the brightness of the stripe changes in the x direction.

f (x, y, t) = A (1 + cos (ωt + kx))
= A + A / 2 ｛e ^{j (ωt + kx)} + e− ^{j (ωt + kx)} ｝ (5)

  As shown in Expression (5), the intensity signal of the stripe pattern illuminated by the illumination device 120 can be considered as a complex number.

  Then, the light from the illumination device 120 is input to the image sensor 220 after being reflected from the subject (including the object to be inspected).

  Therefore, the light intensity signal G (x, y, t) input to the image sensor 220 can be used as the light intensity signal f (x, y, t) of each pixel of the frame when the illumination device 120 is irradiated. Therefore, by substituting Expression (5) into Expression (1) for deriving the intensity image data, Expression (6) can be derived. Note that the phase is kx.

  From Expression (6), it can be confirmed that a value obtained by multiplying the exposure time T by the intermediate value A of the light intensity output from the illumination device 120 is input to each pixel of the intensity image data. Further, by substituting equation (5) into equation (2) for deriving the time correlation image data, equation (7) can be derived. Note that AT / 2 is an amplitude and kx is a phase.

  Accordingly, the time correlation image data represented by the complex number represented by the equation (7) can be replaced with the above-described two types of time correlation image data. That is, the above-described time correlation image data including the real part and the imaginary part includes the phase change and the amplitude change in the light intensity change applied to the test object. In other words, the PC 100 of the present embodiment can detect the phase change of the light emitted from the lighting device 120 and the amplitude change of the light based on the two types of time-correlated image data. Therefore, the PC 100 of the present embodiment uses the time-correlation image data and the intensity image data to generate amplitude image data representing the amplitude of light entering each pixel and phase image data representing the phase change of light entering each pixel. And generate

  Further, the PC 100 detects an abnormality of the inspection object based on the generated amplitude image data and phase image data.

  By the way, when an abnormality based on irregularities occurs in the surface shape of the inspection object, a distribution corresponding to the abnormality occurs in the distribution of the normal vectors on the surface of the inspection object. Further, when an abnormality such as light absorption occurs on the surface of the inspection object, the intensity of the reflected light changes. The change in the distribution of the normal vector is detected as at least one of a phase change and an amplitude change of light. Thus, in the present embodiment, at least one of a phase change and an amplitude change of light corresponding to a change in the distribution of the normal vector is detected using the time correlation image data and the intensity image data. This makes it possible to detect an abnormality in the surface shape. Next, the relationship between the abnormality of the test object, the normal vector, and the phase change or amplitude change of light will be described.

  FIG. 5 is a diagram illustrating a first detection example of the abnormality of the inspection object by the time correlation camera 110 according to the first embodiment. In the example illustrated in FIG. 5, it is assumed that the inspection object 500 has a protruding abnormality 501. In this situation, it can be confirmed that the normal vectors 521, 522, and 523 face different directions in the area near the point 502 of the abnormality 501. Since the normal vectors 521, 522, and 523 are oriented in different directions, the light reflected from the abnormality 501 diffuses (for example, light 511, 512, and 513), and the image sensor 220 of the time correlation camera 110 The width 503 of the stripe pattern entering an arbitrary pixel 531 becomes wider.

  FIG. 6 is a diagram illustrating an example of the light amplitude that changes according to the abnormality when the abnormality 501 illustrated in FIG. In the example shown in FIG. 6, the amplitude of light is divided into a real part (Re) and an imaginary part (Im) and is represented on a two-dimensional plane. In FIG. 6, light amplitudes 611, 612, and 613 corresponding to the light 511, 512, and 513 in FIG. Then, the light amplitudes 611, 612, and 613 cancel each other, and the light having the amplitude 621 enters the arbitrary pixel 531 of the image sensor 220.

  Therefore, in the situation shown in FIG. 6, it can be confirmed that the amplitude is small in a region where the abnormality 501 of the test object 500 is imaged. In other words, in the amplitude image data showing the amplitude change, if there is a region that is darker than the surroundings, it can be estimated that the cancellation of the amplitudes of the light in the region has occurred. It can be determined that the abnormality 501 has occurred at the corresponding position of the test object 500.

  The inspection system according to the present embodiment is not limited to the one in which the inclination changes steeply like the abnormality 501 in FIG. FIG. 7 is a diagram illustrating a second detection example of the abnormality of the inspection object by the time correlation camera 110 according to the first embodiment. In the example shown in FIG. 7, in a normal case, the surface of the test object is a flat surface (in other words, the normals are parallel), but the test object 700 has a gentle gradient 701. In such a situation, the normal vectors 721, 722, and 723 on the gradient 701 also change gradually. Therefore, the lights 711, 712, and 713 input to the image sensor 220 also slightly shift. In the example shown in FIG. 7, since the light amplitudes do not cancel each other due to the gentle gradient 701, the light amplitudes shown in FIGS. 5 and 6 hardly change. However, light originally projected from the screen 130 should enter the image sensor in parallel as it is. However, because of the gentle gradient 701, light projected from the screen 130 does not enter the image sensor in a parallel state. A phase change occurs in the light. Therefore, by detecting a difference between the light phase change and the surroundings, an abnormality due to a gentle gradient 701 as shown in FIG. 7 can be detected.

  In addition, abnormalities may occur in addition to the surface shape of the test object (in other words, the distribution of normal vectors of the test object). FIG. 8 is a diagram illustrating a third detection example of the abnormality of the inspection object by the time correlation camera 110 according to the first embodiment. In the example shown in FIG. 8, since the dirt 801 is attached to the inspection object 800, the light emitted from the illumination device 120 is absorbed or diffusely reflected, and the dirt 801 of the time correlation camera 110 is imaged. In this example, light intensity hardly changes in an arbitrary pixel region. In other words, in an arbitrary pixel area where the dirt 801 is photographed, the phase of the light intensity cancels out, the vibration component is canceled, and the brightness becomes almost DC.

  In such a case, since the amplitude of light hardly exists in the pixel area where the dirt 801 is photographed, an area that is darker than the surroundings when displaying the amplitude image data occurs. Therefore, it can be estimated that there is an abnormality such as dirt 801 at the position of the test object 800 corresponding to the area.

  As described above, in the present embodiment, by detecting a change in the amplitude of light and a change in the phase of light based on the time-correlation image data, it is possible to estimate that the test object has an abnormality.

  Returning to FIG. 1, the PC 100 will be described. The PC 100 controls the entire detection system. The PC 100 includes an arm control unit 101, a lighting control unit 102, and a control unit 103.

  The arm control unit 101 controls the arm 140 to change the surface of the inspection object 150 to be imaged by the time correlation camera 110. In the present embodiment, in the PC 100, a plurality of surfaces to be imaged of the object to be inspected are set. Each time the time-correlation camera 110 finishes capturing an image of the object 150, the arm control unit 101 moves the arm 140 so that the time-correlation camera 110 can capture an image of the set surface in accordance with the setting. Move 150. In the present embodiment, the arm 140 is not limited to repeatedly moving the arm every time the photographing is completed and stopping it before the photographing is started. The arm 140 may be continuously driven. Note that the arm 140 may be referred to as a transport unit, a moving unit, a position changing unit, a posture changing unit, or the like.

  The illumination control unit 102 outputs a stripe pattern emitted by the illumination device 120 to inspect the inspection object 150. The illumination control unit 102 of the present embodiment transfers at least three or more stripe patterns to the illumination device 120, and instructs the illumination device 120 to switch and display the stripe patterns during the exposure time.

  FIG. 9 is a diagram illustrating an example of a stripe pattern output from the illumination control unit 102 to the illumination device 120. The illumination control unit 102 controls such that a stripe pattern in which a black area and a white area shown in FIG. 9A are set is output according to the rectangular wave shown in FIG. 9B.

  In the present embodiment, the interval between stripes for each stripe pattern to be irradiated is set in accordance with the size of an abnormality (defect) to be detected, and a detailed description thereof is omitted here.

  Further, the angular frequency ω of the rectangular wave for outputting the stripe pattern has the same value as the angular frequency ω of the reference signal.

  As shown in FIG. 9, the stripe pattern output by the illumination control unit 102 can be shown as a rectangular wave. However, the boundary area of the stripe pattern is blurred through the screen 130, that is, a bright area in the stripe pattern. By making the intensity change of light at the boundary between the (striped area) and the dark area (spaced area) gentle (dulling), it is possible to approximate a sine wave. FIG. 10 is a diagram showing an example of a wave shape representing a stripe pattern after passing through the screen 130. As shown in FIG. 10, when the wave shape approaches a sine wave, the measurement accuracy can be improved. Further, a gray area in which the brightness changes in multiple steps may be added to the stripes, or a gradation may be given. Further, a stripe pattern including color stripes may be used.

  Returning to FIG. 1, the control unit 103 includes an amplitude-phase image generation unit 104, an abnormality detection unit 105, and an abnormality type estimation unit 106, and controls the intensity image data input from the time correlation camera 110 Based on the image data, a process for calculating a feature corresponding to the distribution of the normal vector of the inspection target surface of the inspection object 150 and detecting an abnormality based on a difference from the surroundings is performed. In the present embodiment, in order to perform an inspection, instead of time-correlation image data represented by complex numbers (referred to as complex time-correlation image data), two types of real-number and imaginary parts of complex-number correlation image data are used. Is received from the time correlation camera 110.

  The amplitude-phase image generation unit 104 generates amplitude image data and phase image data based on the intensity image data input from the time correlation camera 110 and the time correlation image data.

  The amplitude image data is image data representing the amplitude of light entering each pixel. The phase image data is image data representing the phase of light entering each pixel.

  Although the present embodiment does not limit the calculation method of the amplitude image data, for example, the amplitude-phase image generation unit 104 includes the pixel values C1 (x, y) and C2 (x, From y), each pixel value F (x, y) of the amplitude image data can be derived using Expression (8).

  Then, in the present embodiment, it is possible to determine whether or not there is a region where an abnormality has occurred, based on the pixel value (amplitude) of the amplitude image data and the pixel value of the intensity image data. For example, a region where the value obtained by dividing the pixel value (AT) of the intensity image data by 2 and the amplitude of the amplitude image data (AT / 2 when there is no cancellation) coincides to some extent causes an abnormality. I can guess that it is not. On the other hand, it can be inferred that the amplitude is canceled in the non-matching areas. The specific method will be described later.

  Similarly, the amplitude-phase image generation unit 104 calculates each pixel value P (x, y) of the phase image data from the pixel values C1 (x, y) and C2 (x, y) using Expression (9). Can be derived.

  The abnormality detection unit 105 uses the amplitude image data, the phase image data, and the intensity image data to determine a feature corresponding to the distribution of the normal vector of the inspection symmetry plane. Detect related features.

  FIG. 11 is a diagram illustrating a configuration of the abnormality detection unit 105. As illustrated in FIG. 11, the abnormality detection unit 105 includes an amplitude abnormality detection unit 105a, a phase abnormality detection unit 105b, an intensity abnormality detection unit 105c, and a WF abnormality detection unit 105d.

  First, an abnormality detection process based on amplitude in the amplitude abnormality detection unit 105a of the present embodiment will be described. FIG. 12 is a flowchart illustrating a procedure of the processing in the amplitude abnormality detection unit 105a according to the present embodiment. Note that the amplitude distribution of the complex time correlation image is data indicating the amplitude distribution of each pixel of the complex time correlation image, and corresponds to amplitude image data.

  First, the amplitude abnormality detection unit 105a calculates the average amplitude of the N × N region with the pixel as a reference (for example, the center) from the light amplitude value (pixel value representing) stored in each pixel of the amplitude image data. The value is subtracted (step S1101) to generate average difference image data of the amplitude. The average difference image data of the amplitude corresponds to the gradient of the amplitude. The integer N is set to an appropriate value according to the embodiment.

  Next, the amplitude abnormality detection unit 105a performs a mask process using a predetermined amplitude threshold value on the average difference image data of the amplitude generated by the subtraction (step S1102).

  Further, the amplitude abnormality detection unit 105a calculates a standard deviation for each pixel within the mask area of the average difference image data (step S1103). In the present embodiment, a method based on the standard deviation will be described. However, the present invention is not limited to the case where the standard deviation is used. For example, an average value may be used.

  Then, the amplitude abnormality detection unit 105a detects a pixel having a value whose amplitude pixel value obtained by subtracting the average is smaller than -4.5σ (σ: standard deviation) as a region having an abnormality (defect) (step S1104).

  According to the above-described processing procedure, it is possible to detect the abnormality of the inspection object from the amplitude value of each pixel (in other words, the amplitude distribution). However, the present embodiment is not limited to detecting an abnormality from the amplitude distribution of the complex time correlation image. As a feature corresponding to the distribution of the normal vector of the inspection symmetry plane, the gradient of the phase distribution may be used. Therefore, an example using the gradient of the phase distribution will be described next.

  Next, an abnormality detection process based on the phase in the phase abnormality detection unit 105b of the present embodiment will be described. FIG. 13 is a flowchart illustrating a procedure of the process in the phase abnormality detection unit 105b (the abnormality detection unit 105) of the present embodiment.

  First, the phase abnormality detection unit 105b subtracts the average phase value of the N × N region with the pixel as a reference (for example, the center) from the phase value (pixel value representing light) of each pixel of the phase image data. (Step S1201), phase average difference image data is generated. The phase average difference image data corresponds to the phase gradient.

  Next, the phase abnormality detection unit 105b compares the size (absolute value) of the average difference image data of the phase generated by the subtraction with a threshold value, and determines a pixel whose average difference image data size is equal to or larger than the threshold value. Is detected as a pixel having an abnormality (defect) (step S1202).

  Based on the detection result in step S1202, the phase abnormality detection unit 105b can determine the unevenness based on the sign of the average difference image data, that is, the magnitude relationship between the pixel phase value and the average phase value (step S1203). Which one of the pixel phase value and the average phase value becomes convex when it is large depends on the setting of each unit, but if the magnitude relation is different, the irregularities are different.

  Note that an abnormality can be detected from the gradient of the phase distribution obtained by another method. For example, as another method, the phase abnormality detection unit 105b determines that the magnitude of the difference between the average vector of the N × N region of the normalized time-correlated image data and the vector of each normalized pixel is equal to a threshold. If it is larger than this, it can be detected as a pixel having an abnormality (defect). Further, the abnormality of the test object may be detected based on information corresponding to the phase distribution, without being limited to the gradient of the phase distribution.

  Next, an abnormality detection process based on amplitude and intensity by the amplitude abnormality detection unit 105a and the intensity abnormality detection unit 105c of the present embodiment will be described. FIG. 14 is a flowchart illustrating a procedure of the processing in the abnormality detection unit 105 of the present embodiment.

  First, the amplitude abnormality detection unit 105a and the intensity abnormality detection unit 105c (the abnormality detection unit 105) express the amplitude () of each pixel from the time correlation image data and the intensity image data using the following equation (100). The ratio R (x, y) between the pixel value) C (x, y) (see equation (7)) and the intensity (pixel value representing) G (x, y) (see equation (6)) is calculated ( Step S1301).

R (x, y) = C (x, y) / G (x, y) (100)

  Next, the amplitude abnormality detection unit 105a and the intensity abnormality detection unit 105c compare the ratio R (x, y) with the threshold, and determine a pixel whose value of the ratio R (x, y) is equal to or less than the corresponding threshold. It is detected as a (defective) pixel (step S1302). Further, the amplitude abnormality detection unit 105a and the intensity abnormality detection unit 105c compare the ratio R (x, y) with a threshold, and determine a pixel whose ratio R (x, y) is equal to or greater than another corresponding threshold. It is detected as a pixel having unevenness (dirt, etc.) (step S1303). When the cancellation (decay) of the amplitude becomes remarkable due to the abnormal distribution of the normal vector, the amplitude is much lower than the intensity. On the other hand, when the distribution of the normal vectors does not have much abnormality, but the light absorption becomes remarkable due to the contamination of the surface of the test object 150 or the like, the intensity is much lower than the amplitude. Therefore, the amplitude abnormality detection unit 105a and the intensity abnormality detection unit 105c can detect the abnormality type in steps S1302 and S1303.

  The WF abnormality detection unit 105d detects an abnormality based on a complex difference image between the complex restored image obtained by applying the Wiener filter to the complex time correlation image and the complex time correlation image.

  The WF abnormality detection unit 105d first performs a two-dimensional Fourier transform (discrete Fourier transform, fast Fourier transform) on the complex time correlation image data in the x direction and the y direction to generate a spectrum image.

  The Wiener filter K (m, n) is a kind of mathematical filter for removing a noise component from a signal. The WF abnormality detection unit 105d is included in the spectrum image by applying the Wiener filter K (m, n) set for each pixel to the (spatial frequency) spectrum image obtained by the two-dimensional Fourier transform. The defect part of the inspection target is removed as a noise component. That is, the WF abnormality detection unit 105d generates a spectrum image of the complex restored image in which the defect part has been removed from the inspection target. Here, the complex restored image is an image that is included in the complex time correlation image data and includes an amplitude and a phase from which a noise component representing a defect on the inspection target surface of the inspection object 150 is removed. Note that m indicates the coordinates in the x-axis direction in the spectrum image, and n indicates the coordinates in the y-axis direction in the spectrum image.

  The Wiener filter K (m, n) can be generated in advance based on the complex time correlation image data of the inspection target surface of the inspection object, which is a nondefective non-defective product. The Wiener filter K (m, n) is set for each stripe pattern.

  Specifically, when calculating the Wiener filter K, a complex time correlation representing a non-defective product is calculated by calculating the following equation (101) using a plurality of (spatial frequency) spectrum images of a non-defective product under test 150. The power spectrum ΦFF of the average spectrum image of the image is calculated. Note that the numerical value nsum is the total number of the spectrum images s.

ΦFF = | (1 / nsum) * Σs | ^ 2 (101)

  The Wiener filter K is generated from the calculation of the following equation (102) using the power spectrum ΦFF of the average spectrum image. The power spectrum ΦNN of the noise component is a parameter determined in advance based on the noise component to be removed.

  K = 1 / (1 + ΦNN / ΦFF) (102)

  By reducing the power spectrum ΦNN of the noise component, a higher frequency can be passed, and the restoration rate at the edge can be improved. However, when the power spectrum ΦNN of the noise component is reduced, the defective portion is restored together with the high frequency at the edge.

  Thus, as an example, a value that does not completely restore a defective portion such as sagging or dents in which the surface to be inspected of the object to be inspected changes gradually can be set in the power spectrum ΦNN of the noise component. However, this is only an example, and the defect to be removed is not limited to the sauce and the dent.

  Conventionally, the Wiener filter has been used to generate an original image free from deterioration by removing a noise component indicating deterioration from a deteriorated image when the deterioration component is recognized. However, in the present embodiment, instead of using the Wiener filter to generate an original image in which no deterioration has occurred, the WF abnormality detection unit 105d uses a complex time correlation image based on a defect or the like of a surface to be inspected. By removing the defect or the like as the noise component from the spectral image obtained, a spectrum image of a complex restored image that does not include a defect or the like on the inspection target surface is generated.

  Then, the WF abnormality detection unit 105d generates a difference spectrum image indicating a difference between pixel values of each pixel between the spectrum image based on the complex restored image and the spectrum image based on the complex time correlation image data. Here, the difference spectrum image represents a spectrum image based on a difference between the complex restored image and the complex time correlation image, in other words, a defect or the like included in the complex time correlation image of the inspection target surface of the inspection object. It is a spectrum image of a noise component.

  The WF abnormality detection unit 105d performs a Fourier inverse transform on the difference spectrum image to obtain a complex difference image.

  For example, the WF abnormality detection unit 105d can detect, as an abnormal region, a region where the value of the phase (含 ま phase difference) included in the complex difference image is equal to or larger than a predetermined threshold. The threshold value is not limited to a fixed value, and may be variably set according to the amplitude, for example. Note that the WF abnormality detection unit 105d may detect an abnormal region based on a feature different from the phase obtained from the complex difference image. Further, when detecting a defect, the WF abnormality detection unit 105d may execute a filter process for removing a noise component. Examples of the filter processing include a high-frequency removal filter. An abnormal area or an abnormal state is a set of abnormal pixels adjacent to each other, and is grouped and labeled.

  The abnormality type estimation unit 106 estimates the type of the abnormality based on a combination of a plurality of abnormality detection results obtained by the abnormality detection unit 105. Therefore, according to the present embodiment, there is an advantage that a separate detailed inspection is not required, and preparation for a separate detailed inspection can be performed more easily or more quickly. Hereinafter, the estimation of the abnormality type will be exemplified.

[Abnormality type estimation 1: protrusion]
The inventors' earnest study has revealed that both the amplitude and the phase often show abnormal values with respect to the inclination of the inspection target surface corresponding to the “projection”. Therefore, the abnormality type estimating unit 106 estimates the type of abnormality in the area where the abnormality is detected in the amplitude and the phase is detected as “protrusion”.

[Abnormality type estimation 2: high protrusion]
As described above, both the amplitude and the phase show abnormal values with respect to the inclination of the inspection target surface corresponding to the “projection”. In addition, as a result of intensive studies by the inventors, if an abnormality is detected in the area where the abnormality is detected by the inspection by the intensity abnormality detection unit 105c, the height of the “projection” is often high. It is known. Therefore, the abnormality type estimation unit 106 determines the type of abnormality in the region where the abnormality is detected in the amplitude, the phase is detected, and the abnormality is detected in the intensity, and the abnormality is detected in the amplitude and the phase is detected. In addition, it is estimated that the “protrusion” is higher than the “protrusion” of the abnormality type estimation 1 estimated in a region where no abnormality is detected in the intensity.

[Estimation of abnormal type 3: Defect damage]
The “defect seed wound” is, for example, an elongated scratch formed by hitting an object to be inspected 150 with an object. As an example, the width is about several μm to several tens of μm, and the length is about several mm. is there. According to the inventor's intensive studies, no abnormality is detected in the inspection by the phase abnormality detection unit 105b and the inspection by the WF abnormality detection unit 105d, and when an abnormality is detected in the inspection by the amplitude abnormality detection unit 105a, the abnormality is detected. Has often been found to be a "defective seed wound". Therefore, the abnormality type estimating unit 106 estimates the type of abnormality in a region where no abnormality is detected in the amplitude, no abnormality is detected in the phase, and no abnormality is detected in the difference image as “defect seed wound”. .

[Abnormality type estimation 4: gentle irregularities]
As a result of the inventor's intensive studies, no abnormality is detected in the inspection by the amplitude abnormality detection unit 105a for a relatively gentle inclination on the inspection target surface, and the inspection by the phase abnormality detection unit 105b and the WF abnormality detection unit It has been found that an abnormality is often detected in the inspection by 105d. Therefore, the abnormality type estimating unit 106 estimates the type of abnormality in the area where the abnormality is not detected in the amplitude, the abnormality is detected in the phase, and the abnormality is detected in the difference image as “slow irregularities”.

[Estimation of abnormal type 5-1: spot system defect # 1]
The “stain system defect” is, for example, a defect in which the glossiness is reduced, for example, the surface to be inspected becomes partially white or cloudy. The inventor's diligent studies have revealed that when an abnormality is detected in the inspection by the amplitude abnormality detection unit 105a and the inspection by the WF abnormality detection unit 105d, the abnormality is often a “stain system defect”. ing. Therefore, the abnormality type estimating unit 106 estimates the type of abnormality in the region where the abnormality is detected in the amplitude and the abnormality is detected from the difference image as “stain system defect”.

[Abnormality type estimation 5-2: stain system defect # 2]
According to the inventor's earnest studies, when the region where an abnormality is detected in the inspection by the WF abnormality detection unit 105d is the first region, and the region where the abnormality is detected in the inspection of the phase abnormality detection unit 105b is the second region, When the one area and the second area are close to each other or at least partially overlap with each other, the stain system defect can be detected as an area of a union (OR) of the first area and the second area. found.

  Therefore, the abnormality type estimating unit 106 performs, for example, a predetermined number of times (for example, one time or a plurality of times) of expansion processing of expanding the periphery of the first area and the second area outward by one pixel, and performs the first area and the second area. The first region where the two regions are combined is referred to as an abnormal candidate region. Here, the dilation processing is performed when at least one abnormal pixel is included in eight (3 × 3-1) pixels adjacent to and surrounding the non-abnormal operation target pixel. This is a process of changing a non-abnormal calculation target pixel to an abnormal pixel. Further, the process of combining the second region with the first region, when at least one pixel of the expanded first region and at least one pixel of the expanded second region overlap or are adjacent to each other, The process of including the expanded second region in the first region as an abnormal candidate region, in other words, when the expanded first region and the expanded second region partially overlap or are adjacent to each other, This is a process of setting those OR sets as abnormal candidate regions. Further, the abnormality type estimating unit 106 may determine whether the distance between the center of gravity of the first region and the center of gravity of the second region is within a predetermined value, or that the center of gravity of the second region is centered on the center of gravity of the first region. If it is located within the range, or if the center of gravity of the second region is located within a predetermined range centered on the center of gravity of the first region and the predetermined range is elongated in a predetermined direction, Based on the condition of the distance between the first region and the second region, and the condition of the distance and the arrangement (existing direction), such as determining an abnormal candidate region including the first region and the second region, Candidate regions can be determined.

Next, the abnormality type estimating unit 106 determines whether or not the abnormal candidate region is a stain system defect based on the geometrical characteristics of the abnormal candidate region. The abnormality type estimation unit 106, for example,
(1) When the size (size) of the abnormal candidate area, that is, the number of pixels included in the abnormal candidate area is equal to or larger than a predetermined threshold,
(2) When at least one of the lengths (widths) of the abnormal candidate area in two directions (for example, the i direction and the j direction) of the pixel arrangement direction is equal to or larger than a predetermined threshold value,
(3) When the value obtained by dividing the diagonal length of the abnormal candidate region by the approximate circle diameter of the abnormality type estimating unit is equal to or smaller than a predetermined threshold (here, the diagonal length is, for example, the length of the length in the i direction). The square root of the sum of the square and the square of the length in the j direction, and the approximate circle diameter is the square root of the value obtained by dividing the number of pixels by π.)
(4) When at least one, a plurality, or all of a plurality of conditions such as when the circularity of the abnormal candidate area is within a predetermined range, the abnormal candidate area is replaced with a stain system defect. Can be determined. This is based on the finding that, for example, an object having a too small size or an elongated object is often not a stain system defect. However, the satisfaction condition or the exclusion condition of the geometric feature may be another condition, for example, when the similarity with the registered predetermined shape is equal to or more than a predetermined threshold.

  Further, even when the geometric condition is not satisfied, for example, the abnormality type estimation unit 106 determines that the number of abnormality candidate regions detected in the inspection target region of the stain system defect is equal to or larger than a predetermined threshold. In this case, the abnormal candidate area can be determined as a stain system defect. This is based on the finding that spot system defects rarely appear alone (only one). Note that, when the above-mentioned geometric condition is satisfied and the number of abnormal candidate areas detected in the inspection target area of the stain system defect is equal to or greater than a predetermined threshold, the abnormal type estimation unit 106 The region may be determined as a stain system defect. In addition, the inspection target area of the stain system defect is an area where the stain system defect is likely to occur on the inspection surface, and the inspection object having the same specification is set to the same area.

  Various thresholds and condition settings in the abnormality type estimation 5-2 can be variously changed.

[Estimation of abnormal type 6: dust]
According to the inventor's diligent studies, when a minute object such as dust adheres to the inspection target surface, an abnormality is detected in the inspection by the amplitude abnormality detecting unit 105a and the inspection by the intensity abnormality detecting unit 105c. In the inspection by the phase abnormality detection unit 105b and the inspection by the WF abnormality detection unit 105d, it has been found that an abnormality is not often detected. Therefore, the abnormality type estimating unit 106 sets the type of abnormality in the region where the abnormality is detected in the amplitude, the abnormality is not detected in the phase, the abnormality is detected in the intensity, and the abnormality is not detected in the difference image as “dust”. presume.

  Next, the inspection processing of the inspection object in the inspection system of the present embodiment will be described. FIG. 15 is a flowchart illustrating a procedure of the above-described processing in the inspection system of the present embodiment. It is assumed that the test object 150 is already fixed to the arm 140 and is arranged at the initial position of the test.

  The PC 100 according to the present embodiment outputs a stripe pattern for inspecting an object to be inspected to the illumination device 120 (step S1401).

  The lighting device 120 stores the stripe pattern input from the PC 100 (Step S1421). Then, the lighting device 120 displays the stored stripe pattern so as to change according to the time transition (step S1422). Note that the condition under which the lighting device 120 starts displaying is not limited to when the stripe pattern is stored, and may be, for example, when the inspector performs a start operation on the lighting device 120.

  Then, the control unit 103 of the PC 100 transmits a shooting start instruction to the time correlation camera 110 (Step S1402).

  Next, the time-correlation camera 110 starts capturing an image of the area including the inspection object 150 according to the transmitted capturing instruction (step S1411). Next, the control unit 240 of the time correlation camera 110 generates intensity image data and time correlation image data (step S1412). Then, the control unit 240 of the time correlation camera 110 outputs the intensity image data and the time correlation image data to the PC 100 (Step S1413).

  The control unit 103 of the PC 100 receives the intensity image data and the time correlation image data (Step S1403). Then, the amplitude-phase image generation unit 104 generates amplitude image data and phase image data from the received intensity image data and time correlation image data (step S1404).

  Then, the abnormality detection unit 105 performs various types of abnormality detection of the inspection object based on the amplitude image data, the phase image data, the intensity image data, and the like, and the abnormality type estimation unit 106 estimates the defect type ( Step S1405). Then, the abnormality detection unit 105 outputs the results of the abnormality detection and the defect type estimation to a display device (not shown) included in the PC 100 (Step S1406).

  As an output example of the abnormality detection result, the intensity image data is displayed, and the area of the intensity image data corresponding to the area where the abnormality is detected based on the amplitude image data and the phase image data is displayed by the inspector. For example, decoration display may be performed so as to be recognized. In addition, as a result of the defect type estimation, a character or image indicating the type may be attached to the image of the abnormal area, or a color that can identify the defect type may be attached to the image indicating the abnormal area. Or a pattern or the like may be provided. It should be noted that the output is not limited to the visual output, and the fact that the abnormality is detected may be output by voice or the like.

  The control unit 103 determines whether or not the inspection of the subject has been completed (step S1407). If it is determined that the inspection has not been completed (step S1407: No), the arm control unit 101 can take an image of the surface of the object to be inspected next using the time correlation camera 110 in accordance with a predetermined setting. Thus, the movement of the arm is controlled (step S1408). After the end of the arm movement control, the control unit 103 transmits a shooting start instruction to the time correlation camera 110 again (step S1402).

  On the other hand, when the control unit 103 determines that the inspection of the object to be inspected has been completed (step S1407: Yes), the control unit 103 outputs an end instruction to the time correlation camera 110 (step S1409), and ends the processing.

  Then, the time correlation camera 110 determines whether an end instruction has been received (step S1414). If the end instruction has not been received (step S1414: No), the processing is performed again from step S1411. On the other hand, when the end instruction has been received (step S1414: Yes), the process ends.

  Note that the termination processing of the lighting device 120 may be performed by the inspector, or may be terminated according to an instruction from another configuration.

  Further, in the present embodiment, an example has been described in which the intensity image data generated by using the time correlation camera 110 and the time correlation image data are generated. However, the present invention is not limited to the use of the time correlation camera 110 to generate the intensity image data and the time correlation image data, and a time correlation camera that can be realized by analog processing and an operation equivalent thereto are used. An imaging system that performs the operations may be used. For example, image data generated by a normal digital still camera is output, and the information processing apparatus superimposes the time-correlated image data by superimposing a reference signal using the image data generated by the digital still camera as frame image data. The time-correlated image data may be generated using a digital camera that superimposes a reference signal on a light intensity signal in the image sensor.

(Modification 1)
In the present embodiment, an example has been described in which a feature related to an abnormality is detected based on a difference from the surroundings. However, the embodiment is not limited to detecting the feature based on a difference from the surroundings. The feature may be detected based on a difference from data (for example, reference data such as time correlation data, amplitude image data, and phase image data). In this case, it is necessary to align and synchronize the spatial phase modulation illumination (stripe pattern) with the reference data.

  In the present modified example, the abnormality detection unit 105 stores the amplitude image data and the phase image data obtained from the reference surface and the amplitude image data and the phase image data of the test object 150 stored in a storage unit (not shown) in advance. To determine whether there is a difference between the surface of the test object 150 and the reference surface with respect to any one or more of the light amplitude and the light phase that is greater than or equal to a predetermined reference. .

  This modified example is an example in which an inspection system having the same configuration as that of the first embodiment is used, and a normal surface of the object to be inspected is used as a reference surface.

  While the illuminating device 120 irradiates the fringe pattern via the screen 130, the time-correlation camera 110 captures an image of a normal surface of the test object and generates time-correlation image data. Then, the PC 100 inputs the time-correlated image data generated by the time-correlated camera 110, generates amplitude image data and phase image data, and stores the amplitude image data and the phase image data in a storage unit (not shown) of the PC 100. deep. Then, the time-correlation camera 110 captures an image of the subject to be determined whether or not an abnormality has occurred, and generates time-correlation image data. After the PC 100 generates the amplitude image data and the phase image data from the time correlation image data, the PC 100 compares the amplitude image data and the phase image data with the amplitude image data and the phase image data of the normal subject stored in the storage unit. At that time, a comparison result between the amplitude image data and the phase image data of the normal object to be inspected and the amplitude image data and the phase image data of the object to be inspected is used as data indicating a feature of detecting an abnormality. Output. Then, when the feature of detecting the abnormality is equal to or higher than the predetermined reference, it can be estimated that the inspection object 150 has an abnormality.

  Thus, in the present modification, it is possible to determine whether or not there is a difference from the normal surface of the test object, in other words, whether or not the surface of the test object is abnormal. Note that any method may be used for comparing the amplitude image data and the phase image data, and a description thereof will be omitted.

  Further, in the present modification, an example has been described in which data indicating a feature of detecting an abnormality is output based on a difference between reference surfaces, but the difference between the reference surface and the surroundings described in the first embodiment is different. May be combined to calculate a feature for detecting an abnormality. Since any method may be used for the combination, the description is omitted.

(Modification 2)
In the first embodiment, an example has been described in which the stripe pattern is moved in the x direction to detect an abnormality (defect) of the inspection object. However, when an abnormality (defect) in which the distribution of the normal changes steeply in the y direction perpendicular to the x direction occurs in the inspection object, the stripe pattern is moved in the y direction rather than the x direction. It may be easier to detect defects. Thus, in a modified example, an example will be described in which a stripe pattern that moves in the x direction and a stripe pattern that moves in the y direction are alternately switched.

  The illumination control unit 102 of the present modification switches the stripe pattern output to the illumination device 120 at predetermined time intervals. As a result, the illumination device 120 outputs a plurality of stripe patterns extending in different directions on one inspection target surface.

  FIG. 16 is a diagram illustrating an example of switching the stripe pattern output by the illumination control unit 102 according to the present modification. In FIG. 16A, the illumination control unit 102 causes the stripe pattern displayed by the illumination device 120 to transition in the x direction. Thereafter, as shown in (B), the illumination control unit 102 causes the stripe pattern displayed by the illumination device 120 to transition in the y direction.

  Then, the control unit 103 of the PC 100 performs abnormality detection based on the time-correlated image data obtained from the stripe pattern irradiation in FIG. 16A, and obtains the abnormality from the stripe pattern irradiation in FIG. 16B. Abnormality detection is performed based on the time correlation image data.

  FIG. 17 is a diagram illustrating an example in which the illumination control unit 102 according to the present modification irradiates a surface including an abnormality (defect) 1601 with a stripe pattern. In the example shown in FIG. 17, the abnormality (defect) 1601 extends in the x direction. In this case, the illumination control unit 102 sets the stripe pattern to move in the y direction crossing the x direction, in other words, in the direction crossing the longitudinal direction of the abnormality (defect) 1601. With this setting, detection accuracy can be improved.

  FIG. 18 is a diagram showing the relationship between the abnormal (defect) 1701 and the stripe pattern on the screen 130 when the stripe pattern is changed in the y direction, in other words, the direction orthogonal to the longitudinal direction of the abnormal (defect) 1601. It is. As shown in FIG. 18, when an abnormality (defect) 1701 whose width is narrow in the y direction and whose longitudinal direction is the x direction crossing the y direction occurs, the light emitted from the lighting device 120 is The cancellation of the amplitude of the light increases in the y direction crossing the x direction. Therefore, the PC 100 can detect the abnormality (defect) from the amplitude image data corresponding to the stripe pattern moved in the y direction.

  In the inspection system of the present modified example, when the longitudinal direction of the defect generated in the inspection object is random, the stripe pattern is displayed in a plurality of directions (for example, the x direction and the y direction intersecting the x direction). Thus, the defect can be detected irrespective of the shape of the defect, and the abnormality (defect) detection accuracy can be improved. Further, by projecting a stripe pattern according to the shape of the abnormality, it is possible to improve the detection accuracy of the abnormality.

(Modification 3)
Further, the second modification described above does not limit the method of switching the stripe pattern when performing the abnormality detection in the x direction and the abnormality detection in the y direction. Therefore, in a third modification, an example will be described in which the illumination control unit 102 simultaneously moves the stripe pattern output to the illumination device 120 in the x direction and the y direction.

  FIG. 19 is a diagram illustrating an example of a stripe pattern output from the illumination control unit 102 of the present modification to the illumination device 120. In the example shown in FIG. 19, the illumination control unit 102 moves the stripe pattern in the direction 1801.

  The stripe pattern shown in FIG. 19 includes a stripe pattern of one cycle 1802 in the x direction, and a stripe pattern of one cycle 1803 in the y direction. That is, the stripe pattern shown in FIG. 19 has a plurality of stripes extending in the intersecting directions having different widths. It is necessary to make the width of the stripe pattern in the x direction different from the width of the stripe pattern in the y direction. Thereby, when generating the time correlation image data corresponding to the x direction and the time correlation image data corresponding to the y direction, it is possible to make the corresponding reference signals different. Since the period (frequency) of the change in light intensity due to the stripe pattern may be changed, the moving speed of the stripe pattern (strip) may be changed instead of changing the width of the stripe.

  Then, the time correlation camera 110 generates time correlation image data corresponding to the stripe pattern in the x direction based on the reference signal corresponding to the stripe pattern in the x direction, and based on the reference signal corresponding to the stripe pattern in the y direction. Then, time-correlated image data corresponding to the stripe pattern in the y direction is generated. After that, the control unit 103 of the PC 100 performs the abnormality detection based on the time correlation image data corresponding to the stripe pattern in the x direction, and then detects the abnormality based on the time correlation image data corresponding to the stripe pattern in the y direction. I do. As a result, in this modification, detection can be performed regardless of the direction in which the defect has occurred, and the accuracy of detecting an abnormality (defect) can be improved.

(Second embodiment)
In the first embodiment, an example has been described in which the illumination device 120 displays a stripe pattern having one type of stripe width in the x direction. However, in the above-described embodiment, the width of the stripe in the x direction is not limited to one type, and the width of the stripe in the x direction may be plural types. Therefore, in this embodiment, an example in which the width of the stripe in the x direction is set to a plurality of types will be described.

  The present embodiment uses a plurality of types of stripe widths in the x direction for one inspection target surface of the inspection object, in other words, uses a plurality of stripe patterns having different stripes, and detects an abnormality caused in the inspection object. This is an example in which it is determined whether or not the shape is a protrusion.

  The inspection system according to the second embodiment has the same configuration as the inspection system according to the first embodiment, and a description thereof will be omitted. Next, a stripe pattern output by the illumination control unit 102 according to the second embodiment will be described.

  FIG. 20 is a diagram illustrating an example of a stripe pattern output from the illumination control unit 102 of the second embodiment to the illumination device 120. As shown in FIG. 20, the illumination control unit 102 controls to output a stripe pattern having a plurality of stripes having different widths and extending in the same direction.

  In the example illustrated in FIG. 20, the illumination control unit 102 outputs a stripe pattern (D) corresponding to a rectangular wave (C) obtained by superimposing the rectangular wave (A) and the rectangular wave (B) in FIG. . In the present embodiment, while the rectangular wave (A) makes one round during the distance 1901, the rectangular wave (B) makes eight rounds.

  In the present embodiment, the number of stripes represented by the rectangular wave (B) is set to be an even multiple of the number of stripes represented by the rectangular wave (A). This is based on the fact that an odd-numbered harmonic component is superimposed on a rectangular wave, and interference with the harmonic component can be suppressed by using an even-numbered harmonic component.

  When the analysis process is performed using a plurality of types of stripe patterns, it is preferable that there is an appropriate difference in stripe width between the plurality of types of stripe patterns. For example, if the width of one stripe is 6 times or more the width of the other stripe, a favorable analysis result can be obtained. Therefore, in the present embodiment, as an example, the width of one stripe is set to be eight times the width of the other stripe.

  In the present embodiment, an example has been described in which the rectangular wave (A) makes one round while the rectangular wave (B) makes eight rounds. However, the present invention is not limited to such an example. The frequency (period) of A) and the rectangular wave (B) need only be different.

  Then, the illumination device 120 irradiates the inspection object 150 with the stripe pattern shown in FIG. 20 output from the illumination control unit 102 via the screen 130. Along with this, the time correlation camera 110 captures the reflected light of the stripe pattern shown in FIG. The time correlation camera 110 according to the present embodiment generates two systems of time correlation image data as imaging results.

  Specifically, the time correlation camera 110 generates time correlation image data corresponding to the rectangular wave (A) stripe pattern based on the reference signal corresponding to the rectangular wave (A) stripe pattern, and generates the rectangular wave ( Based on the reference signal corresponding to the stripe pattern B), time-correlated image data corresponding to the rectangular wave (B) stripe pattern is generated. The output two types (two systems) of time correlation image data and intensity image data are output to the PC 100. Note that, as described in the first embodiment, each type of time-correlated image data includes time-correlated image data of a real part and an imaginary part.

  Thereafter, the control unit 103 of the PC 100 determines the inspection object based on the time correlation image data corresponding to the rectangular wave (A) stripe pattern and the time correlation image data corresponding to the rectangular wave (B) stripe pattern. Is detected.

  The PC 100 of the present embodiment receives time-correlated image data corresponding to the rectangular wave (A) stripe pattern and time-correlated image data corresponding to the rectangular wave (B) stripe pattern, thereby performing the first embodiment. In addition to the abnormalities based on the difference in phase and the abnormalities based on the difference in amplitude as shown in the embodiments, it becomes easy to confirm the shape of the surface of the inspection object.

  In the present embodiment, an example in which a plurality of types of stripe widths are superimposed on one stripe pattern will be described. However, a method using a plurality of types of stripe widths is not limited to the method of superimposing, and one inspection target may be used. Any method may be used as long as it can output a plurality of stripe patterns having different widths on the surface. For example, control may be performed such that a plurality of stripe patterns having different widths are switched and displayed.

  The inspection program executed by the PC 100 according to the above-described embodiment is a file in an installable format or an executable format in a computer such as a CD-ROM, a flexible disk (FD), a CD-R, and a DVD (Digital Versatile Disk). It is provided by being recorded on a readable recording medium.

  Further, the inspection program executed by the PC 100 of the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the inspection program executed by the PC 100 of the above-described embodiment may be provided or distributed via a network such as the Internet.

  While some embodiments and modifications of the present invention have been described, these embodiments and modifications are provided as examples and are not intended to limit the scope of the invention. These new embodiments and modifications can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and equivalents thereof.

  100: PC, 101: arm control unit, 102: illumination control unit, 103: control unit, 104: amplitude-phase image generation unit (time correlation image generation unit), 105: abnormality detection unit, 106: abnormality type estimation unit 110: time correlation camera, 120: illumination device (illumination unit), 130: screen (illumination unit).

[Estimation of abnormal type 3: Defect damage]
The “defect seed wound” is, for example, an elongated scratch made by hitting an object to be inspected 150 with an object. For example, the width is about several μm to several tens of μm, and the length is about several mm. is there. According to the inventor's intensive studies, no abnormality is detected in the inspection by the phase abnormality detection unit 105b and the inspection by the WF abnormality detection unit 105d, and when an abnormality is detected in the inspection by the amplitude abnormality detection unit 105a, the abnormality is detected. Has often been found to be a "defective seed wound". Therefore, the abnormal category estimation unit 106, abnormality is detected in the amplitude abnormality is not detected in phase, and the region abnormality from the difference image is not detected the type of abnormality, estimated as "defect type flaw" I do.

Claims (11)

A two-dimensional lighting unit that provides a periodic time change and a spatial change in light intensity;
A light intensity signal from the surface to be inspected, and a reference signal that periodically changes in response to a time change in light intensity, a time correlation image generation unit that generates a time correlation image by an imaging system that multiplies the signal by:
An abnormality detection unit that performs detection of a plurality of types of abnormalities on features obtained from the time correlation image,
An abnormality type estimation unit that estimates the type of abnormality based on a combination of detection results of a plurality of types of abnormality detected by the abnormality detection unit,
Inspection system with The abnormality detection unit detects an abnormality for each of the amplitude of the time correlation image, and the phase of the time correlation image,
The abnormality type estimating unit is configured to estimate an abnormality type in a region where an abnormality is detected in the amplitude, and an abnormality is detected in the phase as a protrusion,
The inspection system according to claim 1. The abnormality detection unit further detects an abnormality with respect to the intensity of the time correlation image,
The abnormality type estimating unit is configured to detect an abnormality in the amplitude, detect an abnormality in the phase, and estimate a type of abnormality in a region where the abnormality is detected in the intensity as a high protrusion,
The inspection system according to claim 2. The abnormality type estimating unit is configured to calculate an abnormality in the amplitude of the time correlation image, an abnormality in the phase of the time correlation image, and a difference image between the restored image restored from the time correlation image using a Wiener filter and the time correlation image. Each of the obtained abnormalities is detected,
The abnormality type estimation unit estimates an abnormality type in a region where no abnormality is detected in the amplitude, an abnormality is detected in the phase, and no abnormality is detected from the difference image, as a defect seed defect.
The inspection system according to claim 1. The abnormality detection unit obtains an abnormality in the amplitude of the time correlation image, an abnormality in the phase of the time correlation image, and a difference image between the restored image restored from the time correlation image using a Wiener filter and the time correlation image. Detected abnormalities, respectively,
The abnormality type estimating unit, the abnormality is not detected in the amplitude, the abnormality is detected in the phase, and the type of abnormality in the region where the abnormality is detected from the difference image is estimated as gentle irregularities,
The inspection system according to claim 1. The abnormality detection unit detects an abnormality in the amplitude of the time correlation image and an abnormality obtained from a difference image between the restored image restored using the Wiener filter from the time correlation image and the time correlation image, respectively.
The abnormality type estimation unit is configured to estimate an abnormality type in a region where an abnormality is detected in the amplitude, and an abnormality is detected from the difference image as a stain system defect,
The inspection system according to claim 1. The abnormality detection unit detects an abnormality of the phase of the time correlation image and an abnormality obtained from a difference image between the restored image restored using the Wiener filter from the time correlation image and the time correlation image, respectively.
The abnormal type estimating unit, when the geometric feature of the abnormal candidate region determined based on the phase abnormality and the abnormality obtained from the difference image satisfies a predetermined condition, the abnormal candidate region The inspection system according to claim 1, wherein: The abnormality detection unit detects an abnormality of the phase of the time correlation image and an abnormality obtained from a difference image between the restored image restored using the Wiener filter from the time correlation image and the time correlation image, respectively.
The abnormality type estimating unit, when a predetermined number or more of abnormal candidate regions determined based on the abnormality of the phase and the abnormality obtained from the difference image are present in the inspection target region, identifies the abnormal candidate region with a stain system defect. The inspection system according to claim 1, wherein:   9. The inspection system according to claim 7, wherein the abnormality candidate area is a union of an area in which the phase abnormality is detected and an area in which the abnormality is detected from the difference image. The abnormality detection unit detects an abnormality for each of the amplitude of the time correlation image, the phase of the time correlation image, and the intensity of the time correlation image, and restores the restored image using the Wiener filter from the time correlation image. And detecting an abnormality obtained from a difference image of the time correlation image,
The abnormality type estimation unit detects an abnormality in the amplitude, detects no abnormality in the phase, detects an abnormality in the intensity, and sets the type of abnormality in the region in which no abnormality is detected from the difference image as dust. presume,
The inspection system according to claim 1. For a test object illuminated by a two-dimensional illumination that gives a periodic temporal change in light intensity and a spatial change, the light intensity signal from the inspection target surface and the light intensity A time-correlated image is generated by an imaging system that multiplies the changing reference signal by:
From the time correlation image, a feature corresponding to the distribution of the normal vector of the inspection target surface, and calculating a feature for detecting an abnormality by at least one of a difference from the surroundings and a difference from the reference surface,
Estimating the type of abnormality based on a combination of the detection results of the plurality of types of abnormality detected,
Inspection methods.

Images