JP2007285880A - Sample image registration method in board inspection, and sample image producing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To register a sample image suitable for readily checking of inspection results by using images that are used at actual inspection. <P>SOLUTION: The board inspection system comprises an inspection system 1A for inspection after component mounting, an inspection system 1B for inspection after soldering, and an information processing system 2. Images, used in inspection and inspection results, are transmitted from the inspection systems 1A, 1B to the information processing system 2 for each part to be inspected. A plurality of indicator data, indicating the contents represented by sample images of non-defective items and defective items are registered in advance in the information processing system. For each indicator data, an image which best matches the indicator data is extracted from among the images transmitted from the inspection systems 1A, 1B and is registered as a sample image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a field of inspecting a part to be inspected, such as a part or a soldering part, using an image obtained by imaging the board as a subject to be inspected using a board that has undergone one process related to the manufacturing process of the component mounting process. Belonging to. In particular, the present invention relates to a method for registering a sample image for referring to a region to be inspected which is a target of good or bad determination, and a sample image creating apparatus using this method.

  A production line for a component mounting board (hereinafter sometimes simply referred to as “substrate”) is generally composed of a cream solder printing process using a solder printer, a component mounting process using a mounter, and a soldering process using a reflow furnace. . Of these processes, an inspection apparatus is generally installed at least after the final soldering process to inspect the component mounting state and soldering state.

A typical example of the substrate inspection apparatus (hereinafter simply referred to as “inspection apparatus”) is an appearance inspection apparatus using a camera or a line sensor. In general, in this type of apparatus, a region to be inspected is extracted and measured by image processing, and the quality of the region to be inspected is determined by comparing the measured value with a pre-registered determination reference value.
Further, the determination result and the image used for the inspection can be displayed on a monitor attached to the inspection apparatus or the external information processing apparatus. Therefore, the user must use the displayed image to recognize the type and content of the defect, analyze the cause of the defect, and correct the defective part. Can do.

For example, in the following Patent Document 1, when it is determined that there is a defect from the measured component displacement amount in the component displacement inspection, the image of the component determined to be defective is displayed, and the user registers the display image. It is possible to judge whether the quality is good or bad compared to the template (see paragraph [0005]).
Although the detailed description of the template is not found in this Patent Document 1, the paragraph [0022] states that “a library in which the shape of the part to be inspected is registered as a template and the image capturing unit 201 captures a partial image. Since the pattern matching with the shape of the registered part is performed, the template is considered to be an image of a non-defective model of the part.

JP 2005-1228949 A

  As described above, when a defect is determined by the inspection apparatus, the image of the determination target part is displayed and the user performs a confirmation operation or the like. However, as in Patent Document 1, refer to a non-defective model image. Even so, it is not easy to confirm the detailed contents of the defect. Especially when the user is a beginner, it is very difficult to perform this kind of recognition work.

  Here, for various types of defects, sample images representing the contents of the defects are created in advance and registered in the memory, and these can be displayed on a monitor as appropriate so that they can be compared with actual images. It seems that the kind of defect can be easily recognized. However, in order to register a sample image in advance, it is necessary to prepare various defective models and perform imaging, or edit a sample image to create another sample image (see Patent Document 2). There is a problem that a great deal of labor is required.

JP-A-5-149728

  The present invention has been made paying attention to the above problems, and an object of the present invention is to make it possible to easily register a sample image suitable for confirmation of an inspection result by using an image used for an actual inspection. And

  A first method according to the present invention sequentially receives a plurality of substrates that have undergone one process relating to the manufacture of a component mounting substrate, and performs imaging and inspection of a region to be inspected on the substrate using an image generated by the imaging. During the execution of the test, or after the inspection of a plurality of substrates is completed, an image representing an example of an image that has been judged good or bad from the images used for the inspection is extracted and registered as a sample image To do. In this method, on the premise that a plurality of sample images representing different contents are registered for at least one region to be inspected, index data indicating the contents to be represented by the sample image is set in advance for each sample image. Then, for the inspected part for which the index data is set, the image having the highest degree of conformity to the index data for each registered index data among the images of the inspected part generated in the inspection of the plurality of substrates. And the extracted image is registered in the memory as a sample image.

  In the above method, for example, when the substrate after the solder printing process is inspected, the part to be inspected is cream solder printed on the land. In addition, for example, when inspecting a substrate after a component mounting process, the inspected part is a part where the parts or solder may be scattered, and when inspecting a substrate after the soldering process, the inspected part Are parts and soldering parts.

  The sample image to be registered preferably includes a plurality of images indicating defects. However, although a good determination is made without registering a sample image representing a defect, an image representing a state close to a defect may be registered as a sample image.

  As the index data, for example, data (for example, color, density, etc.) representing features that should appear at the site to be inspected in the sample image can be set. However, the index data is not limited to this and is obtained by further measuring the above features. It is also possible to obtain an ideal value for the parameter for the discriminating process to be performed and use this as index data. For example, when a sample image is registered for a component displacement inspection, data representing the component displacement amount can be set as index data.

  The sample image extraction and registration processing can be performed in parallel with the inspection of a plurality of substrates as in the mode described later, or may be executed after the inspection of all of the plurality of substrates is completed. In either case, the image with the highest degree of fitness for the index data for each index data is registered in the memory as a sample image from the images of the inspected part used for the inspection. A sample image accurately representing the determined various states can be obtained. Therefore, once the sample image registration process is completed, the user can check the inspection result while comparing the image used for the inspection with the sample image.

  In a preferred aspect of the above method, each time an inspection is performed on the inspected site in which the index data is set, an image of the inspected site used for the inspection is set as a processing target, and the processing target is selected from the registered index data. Index data having the highest degree of fitness for the image is extracted. If the sample image corresponding to the index data extracted here is not registered, the processing target image is registered as a sample image. On the other hand, if a sample image corresponding to the extracted index data has already been registered, the processing target image is processed on the condition that the matching degree with respect to the index data is higher than the registered sample image. Register the target image as a sample image.

  According to the above aspect, the sample image is extracted and registered while a plurality of substrates are being inspected, so that the sample image registration process can be performed efficiently. Even if a sample image corresponding to a certain index data is registered, if an image with a higher degree of fitness is obtained after that, the sample image is replaced by the subsequent image. The degree of appropriateness can be increased.

  Next, a second method according to the present invention sequentially accepts a plurality of substrates that have undergone a component mounting process, and performs imaging and misalignment inspection of components on the substrate using an image generated by the imaging. During or after the inspection of a plurality of substrates is completed, a sample image representing an example of an image that has been judged good or bad is extracted from the images used for the inspection and registered. . In this method, first, for at least one part on a substrate or at least one kind of part, a plurality of sample images having different positional deviation amounts with respect to the ideal position of the part are registered for each sample image in advance. Index data indicating the amount of positional deviation to be represented by the sample image is set. Then, for the parts for which the index data is set, from the misregistration amounts measured in the inspection of a plurality of substrates, the one having the highest fitness for the index data is extracted for each registered index data, and extracted. A sample image is registered as a sample image when the measured positional deviation amount is measured.

  In the above method, the positional deviation amount indicated by the index data may be the value of the positional deviation amount itself, but is not limited to this. For example, when it is assumed that each sample image is displaced by a predetermined pitch interval from the ideal position, the pitch interval can be used as index data.

  According to the above method, it is possible to register a plurality of sample images representing different misregistration amounts regardless of the determination result, but only when a good determination or a defective determination is made. A plurality of sample images may be registered.

  In the third and fourth methods according to the present invention, a plurality of substrates that have undergone a soldering process are sequentially received, and an inspection for a soldered portion on the substrate is performed using an image and an image obtained by the imaging. During or after the inspection of the plurality of substrates is completed, a sample image representing an example of an image for which good or bad determination is obtained is extracted and registered from images used for the inspection. is there.

  In the third method, imaging on the substrate is performed under illumination (so-called “color highlight illumination”) by a method of irradiating a plurality of types of color light from different directions. According to such illumination, the color of the specularly reflected light incident on the imaging means from the soldering site varies depending on the gradient of the soldering site, so the color distribution state that appears in the image is the solder It is different depending on whether the attachment site is normal or defective. Therefore, in the inspection, for example, the quality of the soldering part can be determined by comparing the color distribution state of the soldering part in the image with the reference distribution state.

  In the third method, for each sample image, a plurality of sample images having different color distribution states in the soldered portion image are registered for at least one component or at least one type of component on the substrate. Information indicating the color distribution state to be represented by the sample image is set as index data. Then, for the parts for which the index data is set, the distribution state of the color of the soldered portion most closely matches the index data for each index data from the images of the parts generated in the inspection of a plurality of substrates. An image is extracted, and an image of at least a soldering portion in the extracted image is registered as a sample image.

  In the fourth method, imaging on the substrate is performed under illumination with light from a predetermined direction. For example, by performing illumination from a direction in which specularly reflected light from a flat part on the substrate can be incident on the imaging means, it is possible to generate an image that becomes darker as the gradient becomes steep at the soldered part. . Therefore, the brightness change pattern in the image differs between when the soldered part is normal and when it is abnormal. Therefore, in the inspection, for example, the quality of the soldering site can be determined by comparing the brightness change pattern in the soldering site in the image with the reference pattern.

  In the fourth method, a sample image is registered in advance on the premise that a plurality of sample images having different brightness change patterns in the image of the soldering portion are registered for at least one component or at least one type of component on the board. Each time, information indicating a brightness change pattern to be represented by the sample image is set as index data. Furthermore, the brightness change pattern of the soldered part for each index data is the best match for the index data for the parts for which index data is set, from the images of the parts generated in the inspection of multiple boards. An image to be extracted is extracted, and at least an image of a soldering portion in the extracted image is registered as a sample image.

  In the third and fourth methods, it is desirable to register a sample image for each type of defect found on the soldered board. Further, if each sample image is stored in association with a defect name or the like represented by the image, even a beginner can easily recognize the content of the defect with reference to the sample image.

A sample image creating apparatus according to the present invention includes a board inspection apparatus that inspects a part to be inspected on the board using an image obtained by imaging a board that has undergone one step relating to the manufacture of a component mounting board by an imaging unit. , Which receives the distribution of the inspection result, creates a sample image representing the state of the inspected part according to the inspection result, and accepts the distribution of the inspection result and the image used for the inspection from the board inspection apparatus Means: For at least one examination site, index data indicating the contents to be represented by a sample image of the examination site and a memory capable of storing a plurality of each of the sample images; About the examination site in which the index data is saved , Adapted image extraction that extracts the image with the highest degree of fitness for the index data for each registered index data from the images distributed from the board inspection device Comprising registration means for registering in a memory the image extracted by and adapted image extraction means as a sample image; stage.
By storing a plurality of index data in the memory of this apparatus, the first sample image registration method can be executed.

  The sample image creation apparatus may be configured as a separate apparatus from the board inspection apparatus, but is not limited thereto, and may be configured integrally with the board inspection apparatus.

  According to the present invention, by inspecting a predetermined number of substrates, an image most suitable for the sample image is extracted from the images used for the inspection and registered. Therefore, even if there are a large number of sample images to be registered, it takes no effort to register the sample images, and an appropriate sample image can be registered.

FIG. 1 shows a configuration example of a substrate inspection system to which the present invention is applied.
This board inspection system has a configuration in which two board inspection apparatuses 1A and 1B and an information processing apparatus 2 are connected via a communication line 3. Among the two board inspection apparatuses 1A and 1B, the inspection apparatus 1A inspects a board that has undergone a component mounting process by a mounter (not shown) for component misalignment and component mounting. The other inspection device 1B inspects the fillets formed on the lands of the respective parts, with respect to the substrate that has undergone a soldering process by a reflow furnace (not shown). In each of the inspection apparatuses 1A and 1B, a liquid crystal panel 11, a setting switch 12 and the like are provided on the front surface of the machine body.

The information processing apparatus 2 is mainly a personal computer 20 and includes peripheral devices such as a monitor 21 and a keyboard 22.
The information processing apparatus 2 receives distribution of inspection results and images used for inspection from each of the inspection apparatuses 1A and 1B, and accumulates them in a memory (not shown). Further, in response to a user's calling operation, an image used for the inspection can be called and displayed on the monitor 21, or input such as user evaluation on the image can be received.

  Further, the information processing apparatus 2 extracts a plurality of samples that are suitable as good or defective samples from the images distributed from the inspection apparatuses 1A and 1B, and registers these images in the memory as sample images. Once the sample image is registered, it becomes possible to display the registered sample image on the image display screen of the region to be inspected in response to the user's calling operation.

The board inspection system shown in FIG. 1 includes an inspection apparatus 1A for a component-mounted board and an inspection apparatus 1B for a board after soldering. However, the invention is not limited to this, and the board after the solder printing process is inspected. An inspection device may also be installed and included in the system. Conversely, only the final soldering process inspection device 1B may be included in the system.
Further, when the accumulated data of the images from the respective inspection devices 1A and 1B has a large capacity, a server computer for image storage may be connected to the communication line 3 in addition to the information processing apparatus 2.

FIG. 2 shows a main configuration of the inspection apparatuses 1A and 1B.
The inspection apparatus 1A, 1B of this embodiment incorporates a camera 4, an illumination unit 5, a substrate stage 6, a control processing unit 7, and the like. The liquid crystal panel 11 and the switch 12 illustrated in FIG. 1 are connected to the control unit 10 in the control processing unit 7.

The substrate stage 6 includes a table unit 61 for supporting the substrate 8 and a moving mechanism 62 including an X-axis stage and a Y-axis stage (both not shown).
The camera 4 generates a color still image, and is arranged above the substrate stage 61 with the imaging surface facing downward and the optical axis aligned in the vertical direction.

  The illumination unit 5 is provided between the substrate stage 6 and the camera 4. The illumination unit 5 of this embodiment is provided with three annular light sources 5R, 5G, and 5B that emit red, green, and blue colored lights, respectively, with each central portion aligned with the optical axis of the camera 4. ing. The light sources 5R, 5G, and 5B are set to have different diameters so that the substrate can be irradiated with light from different directions.

The control processing unit 7 is provided with an image input unit 13, an imaging control unit 14, an illumination control unit 15, an XY stage control unit 16, a memory 17, an inspection result output unit 18, and the like, in addition to a computer control unit 10.
The image input unit 13 includes an interface circuit for the camera 4 and the like. The imaging control unit 14 is for outputting a timing signal instructing imaging to the camera 4.

  The illumination control unit 15 controls the lighting / extinguishing operation of each of the light sources 5R, 5G, and 5B of the illumination unit 5 and adjusts the amount of light. The XY stage control unit 16 controls the movement timing and movement amount of the substrate stage 6.

  The memory 17 stores a program in which a series of processing procedures for inspection is described, an inspection data file, and the like. In the inspection data file, for each region to be inspected, the setting data of the inspection region, the binary threshold value for detecting the region to be inspected in the inspection region, and the suitability of the measurement value for the detected region to be inspected are determined. Stores the criterion value for the determination.

  The control unit 10 controls the movement of the substrate stage 62 via the XY stage control unit 16, thereby aligning the camera 4 with each part of the substrate 8 in order and taking an image. The color image generated by this imaging is input to the control unit 10 via the image input unit 13 and stored in an internal memory (such as a RAM). The control unit 10 sets an inspection region for each region to be inspected in the color image stored in the RAM using the inspection data file, and then detects, measures, and determines the region to be inspected in the region. Each process is executed in order.

Further, the control unit 10 transmits the measurement result and determination result for each site to be inspected and the image used for the inspection to the information processing apparatus 2 via the inspection result output unit 18.
The information processing apparatus 2 includes a large-capacity memory (such as a hard disk) not shown. In this memory, areas corresponding to the inspection apparatuses 1A and 1B are provided, and information transmitted from inspection apparatuses corresponding to these areas is collectively stored for each substrate. In the inspection apparatuses 1A and 1B of this embodiment, every time inspection of one component is completed, an image of the component is cut out from the entire image and transmitted to the information processing apparatus 2 together with the measurement result and the determination result. The information processing apparatus 2 executes processing related to registration of sample images every time information is transmitted from the inspection apparatuses 1A and 1B.

  FIG. 3 shows an example of an image generated in the inspection apparatus 1A that performs inspection after component mounting. The five parts included in the image of this example are assigned identification numbers 1 to 5 (hereinafter referred to as “part numbers”), respectively. In this example, the parts 1, 4 and 5 belong to the same part number A, but the parts 2 and 3 belong to part numbers B and C different from the other parts, respectively.

  The information processing apparatus 2 extracts a plurality of sample images for each of these components and registers them in the memory. In the memory, an individual folder is set for each part, and various sample images extracted for the part are stored in the folder.

Also, a component information table TB1 as shown in FIG. 4A is registered in the memory of the information processing apparatus 2 for each component on the board.
The part information table TB1 in this example stores information on the five parts shown in FIG. 3, and a folder in which the mounting position, rotation angle, part number, and sample image are stored in the part number of each part. (Hereinafter referred to as “image folder”) are associated with each other. Although not shown here, when registering sample images for a plurality of examinations, an image folder is set for each examination, and therefore, in the parts information table TB1, a plurality of images are provided for each part. The folder number is stored.

The sample image is registered for each part as in the above example, even if the part has the same part number, the size of the land and the surrounding state of the part may differ depending on the mounting position and the relationship with other parts. Because there is.
However, in this embodiment, the sample image is extracted from the image used for the actual inspection. Therefore, if registration is performed for each part, it may be difficult to collect particularly defective sample images. In such a case, for each product number, a sample image may be extracted using images of all parts belonging to the product number.

  When creating a sample image for each product number, as shown in FIG. 4B, the information processing apparatus 2 includes a component information table TB1 that individually stores only the mounting position, rotation angle, and product number of each component. The product number information table TB2 in which each product number is associated with the folder number of the image folder in which the sample image is stored is registered.

  Hereinafter, (1) component displacement inspection performed in the inspection apparatus 1A, (2) fillet inspection performed in the inspection apparatus 1B, and (3) component presence inspection performed in the inspection apparatus 1A, respectively. Detailed processing related to image registration will be described.

(1) Processing for registering sample image for component misregistration In component misregistration inspection, after the component mounted board image is binarized to detect the component, the coordinates of the predetermined position of the component are further extracted, and x , Y for each axis, the amount of deviation between the extracted coordinates and the reference position registered in advance is obtained. Furthermore, the quality of the part is determined by checking the deviation amount with a predetermined reference value.

In the above inspection, a good judgment may be made even if the amount of misalignment of the part becomes considerably large. This is because when the solder melts in the soldering process, the part is pulled back to the correct position by its surface tension. This is because there is a phenomenon (so-called “self-alignment”). However, since the effect of this self-alignment varies depending on the direction of misalignment or the like, it is difficult for a person who is not an expert to confirm the difference between a state where a good judgment is made and a state where a bad judgment is made.
Therefore, in this embodiment, a plurality of sample images are registered so that the relationship between the degree of misalignment and the quality determination can be confirmed.

  FIG. 5 shows an example of registration of sample images for positional deviation. Here, for the sake of convenience, for chip components, the direction along the direction in which the pair of lands are arranged is defined as the x direction, and the direction orthogonal to the x direction is defined as the y direction. The relationship between this part and the x and y directions is maintained even when the part rotates.

  In this embodiment, assuming that Δx is an upper limit value for the positional deviation amount in the x-axis direction and Δy is an upper limit value for the positional deviation amount in the y-axis direction, the position in each direction is determined by 20 sample images. The deviation is shown increasing at a constant pitch. Specifically, when 20 sample images are arranged in a matrix with 4 pieces in the x-axis direction and 5 pieces in the y-axis direction in order from the one with the smallest amount of displacement, the displacement in the x-axis direction is Δx. / 3, and the positional deviation in the y-axis direction is increased by Δy / 4.

In the figure, the numerical values 0, 1, 2, 3, and 4 arranged along the x and y directions represent the level of the amount of misalignment in each direction (hereinafter referred to as “position misalignment level”). Is. Of these, x, image M ₀₀ of positional shift level in the y direction are 0 represents the ideal implementation state without completely positional deviation. Other sample images can also be expressed as M _ij , where i is the displacement level in the x-axis direction and j is the displacement level in the y-axis direction.

Note that the sample image included in the broken line frame 100 in the figure is extracted from an image that has been determined to be good, and the sample image outside the frame 100 is extracted from an image that has been determined to be defective. .
If each sample image is displayed in the form shown in FIG. 5, it is possible for a non-expert to easily determine a misalignment state in which a good judgment is made and a misalignment state in which a bad judgment is made. The sample image in FIG. 5 shows a state in which the positional deviation increases toward the right and down. However, the positional deviation increases toward the left and upward by rotating or inverting each sample image. Can also be expressed.

FIGS. 6A and 6B show how to obtain the maximum positional deviation amounts Δx and Δy represented by the sample image.
In the figure, 31 indicates the body of the chip component, 32a and 32b indicate component-side electrodes, 33a and 33b indicate lands, and 34a and 34b indicate cream solder. Also, chip components 6 (1) (2), like the sample image M ₀₀ in FIG. 5, is assumed to be implemented in an ideal position.

On the other hand, broken-line frames A1 to A3 in FIGS. 6 (1) and 6 (2) indicate the contour lines of the chip components displaced in the left direction, the right direction, and the downward direction, respectively.
Among these, A1 indicates a chip part at the time when the edge of the right electrode 32a comes out of the land 33a due to the displacement in the left direction, and A2 indicates the end of the left electrode 32b due to the displacement in the right direction. The chip component at the time when the edge goes out of the land 33a is shown. A3 shows the chip component at the time when the upper edge of the component comes out of the lands 33a and 33b due to the downward displacement.

  In this embodiment, for the components at the positions of the frames A1 and A2, the displacement amounts Δx1 and Δx2 with respect to the chip components at the ideal positions are obtained, and the smaller one of them is the displacement amount represented by the sample image. The maximum value is Δx. On the other hand, in the y-axis direction, the positional deviation amount of the component at the position of the frame A3 with respect to the ideal component is set to the maximum value Δy of the positional deviation amount represented by the sample image. In this embodiment, regarding the y direction, even if the component is displaced in any direction in the vertical direction, the amount of positional deviation is considered to be substantially the same, and only the downward positional displacement amount is obtained. Not only this but the amount of position shift in each up and down direction may be calculated like the x direction.

  In this embodiment, in order to register the 20 sample images shown in FIG. 5, the pitches xp, yp (xp = Δx / 3, yp) between the sample images in the x and y directions based on the above Δx and Δy in advance. = Δy / 4) and the pitches xp and yp are registered in the information processing apparatus. The process shown in FIG. 7 is performed each time an image of a part to be inspected and a measurement value of the amount of displacement are received from the inspection apparatus 1A.

In ST101, which is the first step in FIG. 7, the image I of the predetermined part and the measurement values zx and zy of the deviation amount are acquired from the inspection apparatus 1A. In the next ST102, values (zx / xp) and (zy / yp) obtained by dividing the shift amounts zx and zy by the pitches xp and yp between the sample images are applied to the function f, so that the shift amounts zx and zy are the most. Find the near misregistration levels i and j. The function f is a function that rounds off the decimal part of the argument.
For example, in the case of xp = 0.3, if zx = 0.65, i = f (0.65 / 0.3) = 2, and if zx = 0.75, i = f (0.75 /0.3)=3.

In ST103, based on i and j obtained in ST102, it is checked whether or not the sample image _Mij has been registered. If the sample image M _ij is not registered, ST103 is “NO”, the process proceeds to ST108, and the image I acquired in ST101 is registered as the sample image M _ij . In the next ST109, the shift amounts zx and zy are registered as shift amounts mx _i and my _j corresponding to the sample image M _ij .

On the other hand, if the sample image M _ij has already been registered, the process proceeds from ST103 to ST104, and the registered image shift amounts mx _i and my _j are read out. In ST105 of Matatsugi, by multiplying the obtained position deviation level i, pitch j respectively xp, and yp in ST 102, the ideal value Ox shift amount in the sample image _{M ij,} seeking Oy.

In ST106, similarities zd and Md with respect to ideal values Ox and Oy are obtained based on the Euclidean distance for the deviations zx and zy acquired in ST101 and the registered deviations mx _i and my _j . If the similarity zd is greater than the similarity Md, ST107 is “YES” and ST108 and 109 are executed. As a result, the sample image M _ij is replaced with the newly acquired image I from the registered image.
When the similarity zd is not greater than the similarity Md, ST107 is “NO”, and the processing of ST108 and 109 is skipped.

According to the above processing, while the inspection apparatus 1A is in progress, the 20 combinations of misalignment levels shown in FIG. 5 are based on the measurement values of misalignment amounts obtained by the hourly inspection. The most suitable images are extracted and registered as sample images M _ij .
This type of sample image can also be registered by collecting or artificially creating a plurality of models corresponding to each combination of misregistration levels and imaging each model. However, it is not easy for a user to collect or create a model suitable for creating a sample image. On the other hand, according to the above method, by executing an inspection for a certain period, an appropriate image is automatically extracted and registered as a sample image, so that the sample image is registered without imposing a burden on the user. It becomes possible to do. In addition, even after the sample images are registered, if an image that is more suitable for the misalignment state indicated by the ideal values Ox and Oy is obtained, the image is replaced as a sample image, so a considerable number of substrates must be inspected. Thus, the accuracy of the sample image can be increased.

(2) Fillet Sample Image Registration Processing Since the fillet of the substrate after soldering becomes a highly specular reflective part, when imaging is performed under illumination by the illumination unit 5 having the configuration shown in FIG. It is possible to generate an image in which the color regions of red, green, and blue are distributed reflecting the inclination state of the fillet.
By using this phenomenon, the post-soldering board inspection apparatus 1B extracts red, green, and blue color areas from the land area in the image, and determines the position and size of these areas as a predetermined criterion. The suitability of the fillet is determined by comparing with the value.

FIG. 8 is a schematic diagram showing a normal color distribution and a color distribution corresponding to various defects for a fillet on a land on one side of a chip component.
Of the various defects, “missing” means a state where no fillet is formed due to missing parts, and “floating” means a state where no fillet is formed due to lifting of parts. “Non-wetting” refers to a state in which the fillet has not been correctly formed due to insufficient joining of the solder to the component. “Low solder” refers to a state where the size of the fillet is insufficient due to a small amount of solder. “Fillet abnormality” refers to a state in which the fillet is inclined in the opposite direction to that in the normal state.

  In normal images, blue, green, and red colors line up from the side closer to the part, while each image representing a defective state is normal except for the image of “low solder”. The distribution state is different from the time. However, if it is not an expert, it can only recognize that it is defective from the color distribution of these images, and it is difficult to specify the kind of defect.

Therefore, in the information processing apparatus 2 of this embodiment, a sample image is registered for each type of defect. For this registration process, in this embodiment, after preparing the schematic diagram of the color distribution shown in FIG. 8 for each defect in advance, a model pattern for matching with the pattern PT described later from this schematic diagram is added. It is created and registered in the memory of the information processing apparatus 2.
The information processing apparatus 2 executes the process of FIG. 9 every time an image or inspection result for a predetermined fillet is received from the inspection apparatus 1B. By repeating this process, one sample image is registered for each type of defect.

In the first ST201 of FIG. 9, it is checked whether or not the determination result acquired from the inspection apparatus 1B is a defect determination. If the determination result is “good determination”, ST201 becomes “NO”, and the process ends.
On the other hand, if the determination result is “defect determination”, ST201 becomes “YES” and the process proceeds to the following.

  In ST202, the image F transmitted from the inspection apparatus 1B together with the defect determination is acquired. Furthermore, in ST203, a transmission request is issued to the inspection apparatuses 1A and 1B, and the results of the land and component extraction processing performed during the inspection are acquired.

  In ST204, the land image in the image F is divided into a total of 64 areas, 8 in both length and width. In ST205, for each divided area, text data (any one of R, G, and B) representing the most dominant color in the area is obtained, and the combination of text in each area is data indicating a color distribution pattern on the land. (Hereinafter, this color distribution pattern is referred to as “color pattern PT”). Further, in ST206, based on the information acquired in ST203, the data corresponding to the lands and parts in the pattern PT are replaced with texts L and P, respectively.

FIG. 10 specifically shows the processing of ST205 and 206.
FIG. 10A is an image F (this example is an “unwetting” image) acquired in ST202. In ST205, this image is divided into 8 × 8 areas, and average values r _AV , g _AV , and b _AV of r, g, and b values are obtained for each area. Further, the following arithmetic expressions (a), (b), and (c) are executed using each average value to obtain hue values R _OP, G _OP , and B _OP for each of r, g, and b, and the highest value among them is obtained. A value having a high value is set as a representative hue value.
R _OP = (r _AV / (r _AV + g _AV + b _AV )) × 100 (a)
G _OP = (g _AV / (r _AV + g _AV + b _AV )) × 100 (b)
B _OP = (b _AV / (r _AV + g _AV + b _AV )) × 100 (c)

Furthermore, the text “R” is displayed in the region where the representative hue value is R _OP , the text “G” is displayed in the region where the representative hue value is G _OP, and the region where the representative hue value is B _OP. The text “B” is set respectively.

FIG. 10B is a color pattern PT created by performing the above processing on the image F. In the color pattern PT at this stage, any text of “R”, “G”, and “B” is associated with each region.
FIG. 10 (3) shows the color pattern PT after the replacement process of ST206 is performed. In the color pattern PT at this stage, the texts in the areas corresponding to the lands and parts in the image F are respectively replaced with “L” and “P”.

Returning to FIG. 9, when the color pattern PT is created as described above, the variable k and the maximum value R _max are each reset to zero in ST207. Note that k is a counter for specifying a color pattern created from the schematic diagram. Hereinafter, the kth registered pattern is referred to as “model pattern MPT (k)”.
R _max is for storing the maximum value of the similarity R (k) described below.

In ST208, the similarity R (k) between the model pattern MPT (k) and the color pattern PT created by the processing in ST204 to 206 is calculated. Specifically, the 64 areas described above are collated in order, the number of areas where the text matches between both patterns is counted, and the ratio of the count value CT to the total number of areas (CT / 64) Is the similarity R (k).
In the next ST209, the similarity R (k) is compared with the maximum value _Rmax . Where R (k) to execute ST210 only larger than _{R max.}

In ST210, the similarity R (k) is stores the value of k when it becomes larger than _{R max} to the variable kt, the similarity R (k) is an R (kt).
In ST211, the value of k is updated.

Hereinafter, until the value of k reaches the registration number _{K max} of the model patterns, repeated ST208～212. Thereby, finally, a numerical value representing the model pattern having the highest similarity with the pattern PT is stored in kt. Also, the similarity between the model pattern MPT (kt) and the color pattern PT is stored in R (kt).

Next, in ST213, it is checked whether or not the kt-th sample image M (kt) has been registered. If the sample image M (kt) is not registered, ST213 becomes “NO” and the process proceeds to ST215.
In ST215, the image F acquired in ST202 is registered as the kt-th sample image M (kt). Also, the similarity R (kt) obtained in the previous loop of ST208 to 212 is registered as the similarity MR (kt) with respect to the model pattern MPT (kt) of the sample image M1.

Even when the sample image M (kt) is registered, if the similarity MR (kt) of the sample image M (kt) is lower than the similarity R (kt), ST213 and 214 are “YES”. The process proceeds to ST215. As a result, the previously registered sample image M (kt) is replaced with the image F, and the similarity MR (kt) is also rewritten with R (kt).
On the other hand, when similarity R (kt) is lower than MR (kt), ST215 is skipped.

  In the registration process shown in FIG. 9 as well, an appropriate image can be registered as each sample image by executing inspection of a predetermined number of substrates, similar to the registration process of the sample image for positional deviation. become. Further, even after the sample image is registered, when an image having a higher degree of matching with the model pattern corresponding to the sample image is acquired, the image can be replaced with the registered sample image. Therefore, as the inspection proceeds, the accuracy indicated by the sample image can be increased.

In the fillet inspection of this embodiment, illumination by a method of irradiating the substrate with three kinds of colored light from different directions (so-called color highlight illumination) is performed, and red, green, and blue appearing in the image are performed. However, instead of this, the quality of the fillet is determined, but instead of this, imaging is performed under illumination with a single color light source, and an inspection based on the pattern of change in brightness of the image is performed. Also good. For example, when the camera 4 is placed in the same arrangement state as in FIG. 2 and coaxial epi-illumination with monochromatic light is performed on the substrate, specularly reflected light from a portion close to a flat surface is likely to be incident on the camera 4, but tilted. When the angle becomes steep, the amount of specularly reflected light entering the camera 4 decreases. Therefore, since it is possible to generate an image representing the inclination state of the fillet as a pattern of change in brightness, the quality of the fillet is determined by comparing the pattern with a reference pattern registered in advance. can do.
In this way, even when the inspection is performed with the illumination light of a single color, by applying the registration algorithm shown in FIG. 9, an image in which a brightness change pattern representing the content of the defect appears for each defect type is displayed. , Can be registered as a sample image.

(3) Registration of part sample image FIG. 11 shows an image created in the part presence inspection carried out by the inspection apparatus 1A in association with the inspection result.
In the component presence / absence inspection, an inspection window w is set at a position corresponding to the component in the image, and the color in the window w is extracted. Then, by comparing the extracted color with the color registered in the memory, it is determined whether there is a missing part or a different part.

  In the above inspection, since a defect is determined when a color different from the color of the correct part is extracted, it is not difficult to confirm each inspection result after the fact. However, after analyzing a large number of substrates, when analyzing what kind of failure has occurred, it is necessary to call and check the images at the time of determination as defective, It takes a lot of work.

  Therefore, in this embodiment, an image that has been determined to be defective in the presence / absence inspection of the part is taken into the information processing apparatus 2 to extract a representative color in an area corresponding to the part, and an image corresponding to the extracted representative color is extracted. Each sample image is registered as a sample image.

FIG. 12 shows a flow of a series of processing for the representative color extraction processing. Note that the loop of ST 401 to 408 at the beginning targets the image in the window w shown in FIG. 11 and uses x and y as variables for specifying one pixel in the window w. ing. X _max and y _{max in} ST406 and 408 correspond to the position data of the pixel at the lower right vertex in the window w. T and TS are two-dimensional arrays for determining the frequency of hue data contained in the window w.

In ST 401 to 408, processing for obtaining red and green hue values R _OP and G _OP for each pixel (x, y) using the above-described equations (a) and (b) while moving the pixel of interest one by one. and (ST 403), the _{(R OP, G} OP) combining _T (R _{OP, G op)} of and a process (ST 404) to vote for. However, in this embodiment, since only image data of one pixel is used, the r, g, b values themselves are substituted into the equations (a) and (b) instead of the average values r _AV , g _AV , b _AV .

When the processing on the image in the window w is completed, in ST409 and 410, initial values “1” are set in the arguments “rop” and “gop” of the two-dimensional array T. Subsequently, ST409 to 415 are repeated while changing rop and gop until the maximum values r _max and g _max calculated in the loop of ST 401 to 408 are changed.

Substantial processing in the above loop is performed in ST411. In this ST411, a two-dimensional coordinate system with the above R _op and G _op as axes is assumed, and in this coordinate system, as shown in FIG. 13, one point specified by rop and gop of interest is centered. Set a circle with radius m. Then, the sum of the vote values of each point included in this circle is obtained, and this is voted for one element TS (rop, gop) in the two-dimensional array TS.

When the processes of ST409 to 415 are completed, initial values “1” and “0” are set in the number N of representative colors and the variable S in the next ST416.
In ST417, as with the array T, the two-dimensional array TS is assumed to be a two-dimensional coordinate system based on R _OP and G _op, and the coordinates (r _c , g _c ) of the point having the largest value in the coordinate system are extracted. To do. In ST418, the combination of (r _c , g _c ) is data C _N representing the representative color (hereinafter referred to as “representative color data C _N ”).

In ST419, the variable S is updated to a value obtained by adding TS (r _c , g _c ) to the current value. Here, if the S after the update is equal to or less than a predetermined threshold value _{S th,} the process proceeds to ST421, in the two-dimensional coordinate system represented by the sequence TS, the point _{(r c, g} c) radius in m around the Reset the vote value of to zero. Thereafter, in ST422, after increasing the value of N by one, the process returns to ST417, and the point where the value is maximum is extracted again in the two-dimensional coordinate system represented by the array TS. At this time, since the previously extracted representative color and the surrounding voting values are cleared in ST421, a point close to the previous representative color is not extracted.

After ST417~422 the loop has been executed a predetermined number of times, when S exceeds a threshold _{S th,} the process proceeds to ST423 ST 420 becomes "YES". In ST423, save the combinations of the representative color data C _N set so far as C, and the process ends. The threshold value S _th is set to a value of about 75% of the total number of pixels in the window w, for example. Therefore, the loop of ST417 to 422 is executed until the hue data corresponding to the majority of pixels is processed. However, in many cases, since the color of the component is one color, the loop of ST417 to 422 is executed. It is practically performed once. However, when a plurality of colors appear in the window w as in the rightmost image in FIG. 11, the loop may be executed as many times as the number of colors.

FIG. 14 shows the result of executing the above representative color extraction process for each image shown in FIG. In this figure, in the two-dimensional coordinate system by rop and gop, points corresponding to the representative colors obtained for each image are indicated by ●.
In this way, the representative colors in each image are represented as a two-dimensional array of hue data “rop” and “gop”, so that the algorithm of the sample image registration process can be simplified.

FIG. 15 shows the flow of registration processing of sample images for parts.
This process is also executed every time the inspection apparatus 1A for a board after component mounting inspects one component. Unlike the above two sample image registration processes, an image for which a defect determination is made. When a color that has not been extracted before appears, the image is registered as a sample image. Note that UN in the figure is the number of registered sample images, and u is a variable for specifying one of the registered sample images.

  First, in ST301, it is checked whether or not the inspection apparatus 1A has made a defect determination. If this ST301 is “YES”, the process proceeds to ST302, and the determination target image G is acquired. Further, in ST303, a window w is set for the acquired image G, and the above-described representative point extraction process is executed for the image in the window w.

Next, in ST304, the value of the number of registered sample images UN is checked. If UN = 0, that is, if no sample image is registered, the process proceeds from ST304 to ST310, and the registration number UN is updated to 1. Further, in ST311, the image G acquired in ST302 is registered as the UN-th sample image. In ST312, the number N and the combination C of representative colors extracted in the process of ST303 are registered as N _UN and C _UN , respectively.

  On the other hand, when one or more sample images are registered, in ST305 to 309, the variable u is moved from the initial value “1” to the registered number UN of the sample images, and the number of representative colors N and the representative colors are changed. The combination C is compared with the number Nu of representative colors of the u-th sample image and the combination Cu of representative colors. If both the number of representative colors and the combination match for any of the sample images, ST306 and 307 are “YES”, the following collation process is terminated, and the process ends.

  If none of the sample images match the number of representative colors and the combination, ST309 is “YES” and ST310 to 312 are executed. As a result, the image G acquired in ST302 is additionally registered as a new sample image.

  According to the above-described processing, it is possible to easily confirm defects that have occurred so far by calling and displaying the sample images registered when the inspection of the predetermined number of substrates is completed. Therefore, it becomes possible to promptly analyze the cause of the defect.

Note that the above-described three examples of sample image registration processing are all performed by the information processing apparatus 2. However, the present invention is not limited to this, and each inspection apparatus 1A, 1B may be performed in parallel with the inspection. it can.
It is also possible to transfer the sample image registered in the information processing apparatus 2 to the inspection apparatuses 1A and 1B and other computers for use.

  In any of the three examples of the sample image registration process described above, the user may be able to confirm the registered sample image. In this case, if all the sample images are not registered, the confirmation work cannot be performed, but if the sample images can be properly checked in the registration process, the registration of the sample images will be difficult. The user can confirm the processing state.

  In addition, when the user's confirmation work is possible, an image that the user determines to be inappropriate as a sample image may be deleted. Alternatively, an image that has not been registered as a sample image using an image server or the like is stored, and if an inappropriate sample image is registered, an appropriate sample image is selected from the stored images. A thing may be extracted and the image may be replaced with a sample image. In this way, it is possible to cope with a case where an incorrect image is registered as a sample image due to a lighting condition defect or the like.

When storing images, each image may be stored while being classified based on the degree of suitability for the sample image. For example, regarding the registration process of the sample image for positional deviation of the first part, the image to be processed in FIG. 7 is stored in association with (i, j) obtained in ST102. When the predetermined sample image M _{i1, j1} is determined to be inappropriate when the sample image registration process is completed, the image associated with (i1, j1) is displayed on the monitor, Let the user choose the most appropriate one. Alternatively, for the image associated with (i1, j1), the similarity Md with respect to the ideal values Ox and Oy described above may be obtained, and the image with the highest similarity Md may be used as a sample image.

It is explanatory drawing which shows the structure of the board | substrate system to which this invention was applied. It is a block diagram which shows the structure of a board | substrate inspection apparatus. It is explanatory drawing which shows the image produced | generated with the inspection apparatus of the board | substrate after component mounting. It is explanatory drawing which shows the example of registration of the information regarding components. It is explanatory drawing which shows the example of registration of the sample image for position shift. It is explanatory drawing which shows how to obtain | require the maximum amount of positional deviation which a sample image shows. It is a flowchart which shows the flow of the registration process of the sample image for position shift. It is a schematic diagram of the image of the quality goods recognized by the inspection of a fillet, and inferior goods. It is a flowchart which shows the flow of the registration process of the sample image for fillets. It is explanatory drawing which shows the specific example of the creation method of a color pattern. It is explanatory drawing which shows the example of the image of the quality goods and defect which are recognized by the presence-and-absence inspection of components. It is a flowchart which shows the flow of the extraction process of a representative color. It is explanatory drawing which shows the processing content of ST411 of FIG. FIG. 11 is an explanatory diagram illustrating a result of executing representative color extraction processing on the image of FIG. 10. It is a flowchart which shows the flow of the registration process of the sample image for components.

Explanation of symbols

1A, 1B Inspection device 2 Information processing device M _ij , M (kt) Sample image

Claims (6)

While sequentially receiving a plurality of substrates that have undergone one process related to the manufacture of a component mounting substrate, and performing an inspection on a region to be inspected on the substrate using an image and an image generated by the imaging, or the plurality of substrates A method for extracting and registering as an example image an image representing an example of an image that has been judged good or bad from the images used for the inspection after the inspection of a single substrate is completed,
On the premise that a plurality of sample images representing different contents are registered for at least one site to be inspected, index data indicating the contents that the sample image should represent is set for each sample image in advance.
For the site to be inspected for which the index data is set, from the images of the site to be inspected generated in the inspection of a plurality of substrates, an image having the highest degree of fitness for the index data is extracted for each index data, Register the extracted image as a sample image in the memory.
A method for registering a sample image in board inspection, characterized in that: The method of claim 1, wherein
Each time an inspection is performed on the inspected part in which the index data is set, the image of the inspected part used for the inspection is a processing target, and the degree of fitness for the processing target image is the highest among the registered index data Extract index data,
If the sample image corresponding to the extracted index data is not registered, the processing target image is registered as a sample image,
When a sample image corresponding to the extracted index data has already been registered, the processing image is processed on the condition that the processing target image has a higher degree of matching with the index data than the registered sample image. A method for registering a sample image in substrate inspection, wherein the target image is registered as a sample image. A plurality of boards that have undergone the component mounting process are received in order, and an image and an image generated by the imaging are used to perform an inspection of component misalignment on the board, or an inspection of the plurality of boards. Is a method of extracting and registering a sample image representing an example of an image that has been judged good or bad from the images used for the inspection,
With respect to at least one part or at least one kind of part on the board, a sample image is represented in advance for each sample image on the assumption that a plurality of sample images having different positional deviation amounts with respect to the ideal position of the part are registered. Set index data to indicate the amount of misalignment,
For the parts for which the index data is set, from the positional deviation amounts measured in the inspection of the plurality of substrates, the one having the highest degree of fitness for the index data is extracted for each index data, and the extracted position Register the image when the amount of deviation is measured as a sample image,
A method for registering a sample image in board inspection, characterized in that: The plurality of substrates that have undergone the soldering process are sequentially received, and while the imaging and the inspection for the soldering portion on the substrate are performed using the image obtained by the imaging, the inspection for the plurality of substrates is performed. A method of extracting and registering a sample image representing an example of an image that has been judged good or bad from the images used for the inspection after the completion,
Imaging for the substrate is performed under illumination by a method of irradiating a plurality of types of color light from different directions,
For at least one component on the board or at least one type of component, the sample image represents each sample image in advance on the premise that a plurality of sample images having different color distribution states in the image of the soldering part are registered. Information indicating the distribution state of power color is set as index data,
For the component for which the index data is set, an image in which the color distribution state of the soldering portion for each index data is most suitable for the index data among the images of the component generated in the inspection of a plurality of substrates. Extract and register at least a soldering part image in the extracted image as a sample image,
A method for registering a sample image in board inspection, characterized in that: The plurality of substrates that have undergone the soldering process are sequentially received, and while the imaging and the inspection for the soldering portion on the substrate are performed using the image obtained by the imaging, the inspection for the plurality of substrates is performed. A method of extracting and registering a sample image representing an example of an image that has been judged good or bad from the images used for the inspection after the completion,
Imaging for the substrate is performed under illumination with light from a predetermined direction,
For at least one part on the board or at least one kind of part, on the premise that a plurality of sample images having different brightness change patterns in the image of the soldering part are registered, the sample image is previously stored for each sample image. Information that represents the pattern of change in brightness that should be expressed as index data,
For the parts for which the index data is set, the pattern of change in the brightness of the soldering part for each index data is the most suitable for the index data among the images of the parts generated in the inspection of a plurality of substrates. Extract the image and register at least the image of the soldered part in the extracted image as a sample image.
A method for registering a sample image in board inspection, characterized in that: In response to the distribution of the inspection result from the substrate inspection apparatus that inspects the inspected part on the substrate using the image obtained by imaging the substrate that has undergone one step related to the manufacture of the component mounting substrate by the imaging means, An apparatus for creating a sample image representing a state of a region to be inspected according to an inspection result,
A delivery acceptance means for accepting delivery of the inspection result and the image used for the inspection from the substrate inspection apparatus;
For at least one site to be inspected, index data indicating the contents to be represented by the sample image of the site to be inspected and a memory capable of storing a plurality of each of the sample images,
For the inspected part in which the index data is stored, from the image distributed from the substrate inspection apparatus, the matching image extracting means for extracting the image having the highest degree of matching with the index data for each registered index data;
A sample image creating apparatus comprising: registration means for registering the image extracted by the matching image extraction means as a sample image in the memory.

Images