JP2004279304A - Image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a data volume for correction processing to realize image processing of a relatively short processing time, in an image processing method wherein an image data obtained by image pick-up is corrected to a value favorable for image recognition by software-like processing and wherein the corrected data is used. <P>SOLUTION: A recognition base point is selected within an area of a screen, the image data is corrected only for the selected recognition base point, an image is recognized based on the corrected image data. In the correction, an outline of an object is easy to be recognized by increasing a contrast difference between the object and a background. A data about the correction is held as a look-up table. The selection of the recognition base point is effectively carried out by superposing a plate having a point, a seek line and the like correlated to a theoretical profile outline of the object with the image of the object. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method for recognizing the position of an object from image data obtained by imaging the object, and a computer program for implementing the method.
[0002]
[Prior art]
2. Description of the Related Art Image processing for recognizing an existing position of an object from image data of the object is employed in many fields. For example, the field of circuit component mounting in which electronic components and the like are mounted on a circuit board is one of them. In the component mounting work, the circuit component held by the component holding device that holds the circuit component is mounted on the surface of the circuit board whose position is fixed. In this case, the shift amount of the holding position of the circuit component by the component holding device, the shift amount of the fixed position of the circuit board, and the like are acquired. The acquisition of these shift amounts is generally performed by imaging a held reference mark attached to a circuit component or a circuit board, and performing image processing on the imaged data. As described above, even in the field of component mounting, image processing for recognizing the location of an object from the imaging data of the object is used.
[0003]
When an object is recognized by image processing, image data obtained by imaging is a set of optical characteristic values such as luminance, and those optical characteristic values are temporarily stored in a memory as values for each pixel. The recognition process is performed using the data stored in the memory. When performing image recognition, not only when the obtained optical characteristic value is used as it is, but also when the optical characteristic value is corrected to a value convenient for image recognition, and the image recognition is performed based on the corrected characteristic value. Sometimes it is convenient to do. In that case, for example, before storing in the memory, the acquired optical characteristics are converted into values convenient for image recognition using a hardware device such as a converter, and the converted values are stored in the memory. That is being done. However, this method has a problem that, in addition to requiring a separate hardware device, only uniform correction can be performed. In consideration of this, the data once stored in the memory is converted into a convenient value by processing with software using a conversion table or the like, and the contents of the previous memory are rewritten with the converted data or another memory is used. Is also stored. However, this method has a problem that a long processing time is required for converting data of all pixels.
[0004]
On the other hand, the present applicant defines a point of image recognition, such as a point pair, a seek line, and a negative seek line, in JP-A-8-180191, JP-A-11-352727, JP-A-2000-205826, and the like. An image processing method is proposed in which a template having reference points and the like is superimposed on a screen including an object, and an existing position of the object is recognized based on data of a recognition base defined thereby. These template-based image processing methods only have to perform processing based on image data corresponding to the recognition base, and thus have an advantage that the target can be easily recognized.
[0005]
[Patent Document 1]
JP-A-8-180191
[Patent Document 2]
JP-A-11-351827
[Patent Document 3]
JP 2000-205826 A
[0006]
Problems to be Solved by the Invention, Means for Solving Problems, and Effects
In view of the above-described problem, the present invention provides an image processing method that corrects image data obtained by imaging to a value convenient for image recognition by software processing and uses the corrected data to perform correction processing. One object of the present invention is to realize image processing with a relatively short processing time by reducing the amount of data to be performed. Further, the present invention provides an image data corrected to a value suitable for image processing. Another object is to improve the above-described template-based image processing from the viewpoint of image processing based on the template.
[0007]
According to the present invention, an image processing method, an image processing program, and the like in the following aspects can be obtained. As in the case of the claims, each aspect is divided into sections, each section is numbered, and the number of another section is cited as necessary. This is for the purpose of facilitating the understanding of the present invention and should not be construed as limiting the technical features and combinations thereof described in the present specification to those described in the following sections. . In addition, when a plurality of items are described in one section, the plurality of items need not always be adopted together. It is also possible to select and adopt only some items.
[0008]
In the following items, (1) corresponds to claim 1, (2) to claim 2, (3) to claim 3, (4) to claim 4, (4) to (4), The combination of items 5) and (6) corresponds to claim 5, the combination of items (8) and (9) corresponds to item 6 and the combination of items (11) and (12) The combination corresponds to claim 7. In addition, claim (21) corresponds to claim 8.
[0009]
(1) An image processing method for recognizing a target from image data of a screen including the target obtained by imaging,
An image data storing step of storing the image data as a set of optical characteristic values for each pixel,
A base selection step of selecting a plurality of recognition bases in the screen;
A base characteristic value, which is an optical characteristic value at each of the plurality of recognition bases, is calculated by using an optical characteristic value stored in one or more pixels corresponding to each of the plurality of recognition bases and the optical characteristic value. A site characteristic value acquisition step of acquiring based on characteristic value correction data that is corrected to a value suitable for recognition of the target object,
An object recognition step of recognizing the object based on the acquired base characteristic values of the respective recognition bases;
An image processing method including:
[0010]
In the mode described in this section, of the image data obtained by imaging, only the image data corresponding to the recognition root base is used for object recognition. Since only the data to be used is corrected and used, image processing can be performed in a shorter time than in the case where all the image data is corrected as described above.
[0011]
The “screen including an object” means, for example, a screen in which an object and an object other than the object such as a background are imaged. The object is not limited to a single independent object. For example, a part of the one object may be set as an object, and the other part may be set as something other than the object. For example, when imaging an electronic component and recognizing the image, parts such as leads, bumps, and electrodes of the electronic part may be set as objects, and parts other than those parts may be set as outside the object.
[0012]
In the present invention, the imaging device is not particularly limited, and may be any device that can acquire two-dimensional image data, such as a CCD camera and a line scan camera. The “optical characteristic value” as the image data is, for example, a characteristic value such as luminance, lightness, hue, and saturation. In the present invention, one or more of them may be adopted as the characteristic value. When these characteristic values are stored or used in image recognition processing, they may be analog values representing the magnitude of the electric signal for each image sensor, or digital values represented as discrete values (for example, gray scales). Data, etc.).
[0013]
The stored optical characteristic values are data for each pixel, but the optical characteristic values used in image recognition may be data for each pixel, and data of another element than the pixel, for example, Data for each unit different from the pixel may be used. For example, an image based on image data obtained by imaging and an image on which image recognition processing is performed may be considered as different images. In other words, when an image based on data for each pixel is a physical image and a screen on which the physical image exists is a physical screen, the image recognition processing is performed on a virtual image that is an image different from the physical image. Alternatively, the processing may be performed on a virtual screen which is a screen on which the virtual image exists. The physical screen is a matrix in which pixels are arranged. The virtual screen may be a matrix in which elements finer than pixels or elements wider than pixels are arranged, and are represented by analog coordinates such as XY coordinates. May be a screen defined as a virtual plane. To give an example, the virtual image is regarded as data in which the data of each pixel of the physical screen represents the optical characteristic value of the center point of the pixel, and the surface (the virtual screen of the virtual screen) that satisfies the optical characteristic values of these many points. When a surface existing in a three-dimensional space defined by two-dimensional coordinates and one-dimensional coordinates of optical characteristic values is assumed, it is considered as a set of continuous optical characteristic values that define the surface. It can be an image.
[0014]
When the target object recognition processing is performed in the virtual screen, the recognition base, which is the point to which the image data used for the recognition processing belongs, is a point in the virtual image and is a point in the virtual screen. If the virtual screen is in a different unit from the pixels on the physical screen, the recognition base and the pixel may not correspond one-to-one, in which case, for example, the optical characteristic value at the recognition base is A value complemented based on each of the optical characteristic values stored in the plurality of pixels may be used.
[0015]
In the mode described in this section, "recognition of the object" refers to the shape, size, surface condition, defect, position of the object in the screen, etc. It means recognizing one or more things. In the present specification, the “position of the object” and the “position of the object” on the screen are changed, for example, by the position where the center of gravity of the object exists, that is, the object moves along a plane. However, the concept does not mean only the position of the object in the plane, but also includes a position at which the object changes by rotating around an arbitrary point (for example, the center of gravity), that is, a rotation angle position. . Put simply, the concept includes the azimuth position and the posture position of the target object.
[0016]
The “characteristic value correction data” in the mode described in this section is not particularly limited, and may be a “value suitable for target object recognition”, for example, a target object recognition accuracy that can quickly perform image processing. Any data may be used to convert the stored optical characteristic value into a value that provides at least one of various advantages such as being able to be improved. As described above, the “characteristic value correction data” refers to an optical characteristic value stored as physical screen data when the target object recognition process is performed on a virtual screen different from the physical screen as described above. Before converting the optical characteristic value as virtual screen data, the stored optical characteristic value may be corrected, or the optical characteristic value converted as data on the virtual screen may be corrected. May be used.
[0017]
(2) The characteristic value correction data corrects the optical characteristic value in a direction in which a contrast difference between an image of the object and an image other than the object adjacent to the image of the object is enlarged. The image processing method according to item (1).
(3) The characteristic value correction data corrects the optical characteristic value to a larger value when the value is larger than a set threshold, and to a smaller value when the value is smaller than the set threshold (1). Item) or the image processing method according to item (2).
[0018]
The modes described in the above two sections are modes in which a limitation regarding the characteristic value correction data is added. For example, when recognizing the shape and size of an object, the position of the object in a screen, and the like, an image that makes the object stand out can be easily recognized. In the case where the contour (edge) of the object is specified and recognition is performed based on the result of the specification, the position of the contour can be specified more accurately. The mode described in the item (2) is a mode suitable for such a case, for example, in which the optical characteristic value of the portion where the target object is present on the screen and the optical characteristics of the portion adjacent to the target object such as the background. This is an embodiment in which correction data for correcting so as to increase the contrast with the target characteristic value is used. The mode of “enlarging the contrast difference” includes, for example, a mode in which, if the target object is an image brighter than the background, the target object is brighter and the background is darker. Examples include a mode in which a luminance difference, a brightness difference, a saturation difference, and the like are increased, and a difference in hue is made clearer. By some means, the optical characteristic value of the target object and the optical characteristic value of the non-target object are obtained in advance, and based on the obtained characteristic values, appropriate correction data may be appropriately set according to the purpose of recognition. Can also be determined. The mode described in the item (3) is a mode in which, when the optical characteristic value changes at some boundary in the image, the correction is performed in a direction to make the change more abrupt. Similar to the mode described in the section (2), this is a mode suitable for specifying the contour of the object. For example, if the optical characteristic values at each of the positions on both sides close to the contour of the object are known, setting the intermediate value between them as the threshold makes it easy to specify the outline of the object. It can be carried out. In the embodiments described in the above two items, the correction data exists in a format such as a conversion table, and the conversion table or the like may be referred to at the time of correction. And the correction may be performed by an operation according to the equation or the like. Further, the number of correction data is not limited to one, and a plurality of correction data can be selectively used according to the purpose of recognition, the location of recognition, the type of image adjacent to the target object, and the like.
[0019]
(4) The site selection step includes at least one of a plurality of reference points and one or more reference lines associated with a theoretical contour line that is a contour line when the object has a theoretical shape and size. A template use selection that uses a template, superimposes the template on the screen at a set position, and selects a point in the screen defined by at least one of the reference point and the reference line as the plurality of recognition sites. The image processing method according to any one of the above modes (1) to (3), including a step.
[0020]
In short, the mode described in this section is a mode in which a template is used in selecting a recognition base. When recognizing the shape and size of an object, its position in the screen, and the like, an object having a theoretical shape and size (hereinafter, sometimes referred to as a “theoretical object”) is superimposed on an image, and the theoretical object and the actual In many cases, it is convenient to perform recognition processing of the actual object from the state of difference from the object. The embodiment described in this section is a preferred embodiment in such a case. The mode described in this section includes a mode in which the “template-based image processing” according to the invention of the present applicant described above is improved so as to be performed based on the corrected optical characteristic values. In addition, when performing the target object recognition processing on the physical screen, the template corresponding to the pixel may be superimposed on the physical screen, and when performing the recognition processing on the virtual screen, the template may be superimposed on the virtual screen. The theoretical contour does not necessarily have to be actually set on the template, but may be a virtual one for setting a reference line and a reference point. Further, the template is not limited to one corresponding to all the contours of the object, and may be one corresponding to a part of the contour of the object.
[0021]
(5) In the template use selecting step, a template having a plurality of reference point pairs positioned on both sides of the theoretical contour line as the plurality of reference points is used, and a plurality of templates defined by each of the plurality of reference point pairs is used. The image processing method according to (4), wherein a point is selected as the plurality of recognition sites.
(6) In the object recognition step, when the difference state between the base characteristic values of the two recognition bases defined by one reference point pair exceeds a set threshold state, one of the two recognition bases is determined. Suppose that the object belongs to the contour of the object and the other belongs to the outside of the contour of the object. The image processing method according to the mode (5), further comprising a conformity determining step of determining that the target conforms to the template in the case of.
(7) When the target object recognition step determines that the target object matches the template in the target object determination step, the existence position of the target object in the screen corresponds to the theoretical contour line. The image processing method according to item (6), further including an existence position estimation step of estimating that the position is substantially the same as the position.
[0022]
The modes described in the above three sections are, for example, modes suitable for searching for an object on a screen. The reference point pair in the term (5) can be called a point pair. One of the reference points constituting the point pair is expected to be located inside the object, and the other reference point is outside the object. It is intended to be located in. Therefore, when the relationship between the optical characteristic value in the image of the target object and the image outside the target object and the optical characteristic value is known, the optical characteristic value of the recognition base defined by each reference point is When there is such a relationship, it can be determined that each reference point is located with the contour of the object interposed therebetween. The template described in (5) can be used to make such a determination. Various determinations according to the purpose of recognition can be made by changing the number of point pairs, changing the distance of each reference point from the theoretical contour line, or the like.
[0023]
The mode described in the above item (6) relates to one mode of recognition processing using the template described in the item (5). If some of the plurality of point pairs are in the matching state, it can be considered that the theoretical contour and the contour of the actual object match to some extent. For example, a recognition error is taken into consideration, and various other situations are assumed, and a matching state is determined based on a matching state that is equal to or greater than a set number.In this mode, a matching state is determined when all the point pairs are in the matching state. included. The concept of “conformity with the template” is a concept including that the position of the theoretical object defined by the theoretical contour at the position where the template is superimposed and the position of the actual object somewhat match. In addition, when the theoretical object overlaps the position where the actual object exists, the concept includes that the outline of the object substantially coincides with the outline of the theoretical object. The mode described in the paragraph (7) relates to the former of them, and includes, for example, a mode of specifying the approximate position of the target object on the screen by matching with the template. As for the latter, it can be used to confirm whether or not the target object is a recognition target, that is, to confirm that the target object itself is not different from the intended one. When the position of the object is unknown and the object is searched for, the position where the template is superimposed is changed according to the setting conditions (including translation and rotation), and the judgment is made until the matching judgment is obtained. You just have to repeat it.
[0024]
(8) In the template use selecting step, a template having a plurality of reference intersection lines intersecting with the theoretical contour line is used as the one or more reference lines, and is arranged on a line defined by each of the plurality of reference intersection lines. The image processing method according to (4), wherein a plurality of points are selected as the plurality of recognition sites.
(9) The object recognizing step specifies a position of an outline of the object on a line defined by each of the plurality of reference intersection lines, based on the characteristic values of each of the plurality of recognition sites. The image processing method according to the above mode (8), comprising a position specifying step.
(10) The object recognition step recognizes at least one of an existing position of the object in the screen and an error with respect to a theoretical shape size based on the position of the contour specified in the contour position specifying step. The image processing method according to the mode (9), which includes a step of recognizing an error in an existing position and a shape dimension.
[0025]
The modes described in the above three sections are suitable modes for specifying, for example, the contour of the object, that is, the edge of the object. The reference intersection line in the item (8) is a line that is supposed to intersect with the outline of the object, and is a tool for searching for the outline on the line. In many cases, an optical characteristic value forming an image continuously changes in the vicinity of an edge of an object. Therefore, searching for an edge based on a change in the optical characteristic value of the recognition base along the seek line, and specifying the position, the shape characteristic of the edge (for example, a characteristic such as a sharp edge or a rounded edge) and the like. Is possible.
[0026]
The mode described in the above item (9) is a mode for specifying the position of the edge on the seek line. For example, on the seek line, the position where the optical characteristic value changes most rapidly can be specified as the position where the edge of the object exists. The mode described in the above item (10) is one mode of the recognition processing using the position of the specified edge. Based on the relationship between the theoretical contour and the position of the edge of the specified object, it is possible to recognize geometric errors such as distortion of the outline of the object and dimensional errors such as a difference in the size of the object. it can. In addition, when the position where the template is superimposed is known, the existence position of the theoretical object in the screen is known, and the displacement of the actual object with respect to the theoretical object can be recognized. It is possible to recognize the position of the object in the screen.
[0027]
(11) In the template use selecting step, a template having a reference line extending along the theoretical contour line outside or inside the theoretical contour line as the one or more reference lines is used, and is defined by the reference line. The image processing method according to (4), wherein a plurality of points arranged on a line are selected as the plurality of recognition bases.
(12) The image processing method according to (11), wherein the object recognition step includes a contour shape recognition step of recognizing a contour shape of the object based on each of the characteristic values of the plurality of recognition points. .
[0028]
The modes described in the above two sections are suitable modes for recognizing, for example, a shape defect of the contour of the object, that is, a defect existing at the edge. The reference railroad in (11) is a line that is not expected to intersect with the outline of the object, and when the outline of the object intersects on that line, it can be determined that the object has something unusual. This reference line can also be called a seek line because it performs the function of searching for the incident. In order to distinguish from the seek line in the item (8), the preceding one is called a positive seek line, and the one in this item is called a negative seek line. For example, when the negative seek line is set outside the theoretical target, and when a part of the edge protrudes due to bending or the like, the contour line of the target intersects the negative seek line. Conversely, when a negative seek line is set in a theoretical object, when a part of the edge is lost, the contour line of the object intersects the negative seek line.
[0029]
In the mode described in the above item (12), the contour shape of the target object, particularly the contour shape, is utilized by utilizing the characteristics of the negative seek line and detecting the change in the optical characteristic value of the recognition base defined by the negative seek line. In this case, for example, a defect existing at an edge can be found. Recognition processing using a negative seek line is not limited to edge defects, etc., and if the line intersects the contour of the object at any position on the seek line, the It is also possible to carry out the embodiment in such a manner that it is determined that the condition has occurred. In other words, if turned over, it is possible to search for an object using a negative seek line.
[0030]
(13) A component mounting operation for mounting a circuit component held by the component holding device on a circuit board having a fixed position, which is performed, and a shift in the holding position of the circuit component held by the component holding device or fixing of the circuit board. Any of the above items (1) to (12) is applied to image processing for recognizing a reference mark provided on a circuit board or a held circuit component as the object in order to detect a positional shift. The image processing method described in the above.
[0031]
The component mounting work is required to be performed at a high speed, and it is desired to quickly detect a circuit component and a positional shift amount of the circuit component by image recognition processing. Since the image processing method of the present invention can perform image processing by using only image data of a limited point in a screen called a recognition base, it is possible to detect the above-described displacement amount in a relatively short time. It is suitable for component mounting work. In addition, while high mounting accuracy is required due to miniaturization of electronic circuits and the like, the image processing method of the present invention, which performs image processing using data corrected to a state suitable for recognition, requires high recognition accuracy. Therefore, it is suitable for component mounting work from the viewpoint of mounting accuracy.
[0032]
(21) A program executed by a computer to perform image processing for recognizing a target from image data of a screen including the target obtained by imaging.
Image data storage step of storing the image data as a set of optical characteristic values for each pixel,
A base selection step of selecting a plurality of recognition bases in the screen;
A base characteristic value, which is an optical characteristic value at each of the plurality of recognition bases, is calculated by using an optical characteristic value stored in one or more pixels corresponding to each of the plurality of recognition bases and the optical characteristic value. A site characteristic value acquisition step of acquiring based on characteristic value correction data for correcting to a value suitable for recognition of the target,
An object recognition step of recognizing the object based on the acquired base characteristic values of the respective recognition bases;
An image processing program including:
(31) A recording medium on which the image processing program according to (21) is recorded so as to be readable by a computer.
[0033]
The aspects described in the above two sections respectively relate to a computer program for realizing the above-described image processing method and a recording medium on which the program is recorded. The description of these two items is the same as the previous description, and thus the description is omitted here. In addition, it is also possible to implement in a mode in which the technical features described in the above items (2) to (13) are added to each of the above two items.
[0034]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described by taking a case where it is applied to a component mounting operation as an example. Further, the following embodiment relates to the above-mentioned “template use type” image processing, the details of which are described in detail in the earlier application by the present applicant described in the section of [Prior Art]. Since it has been described, some description will be omitted. Note that the present invention is not limited to the following embodiments. In addition to the following embodiments, the present invention is not limited to the embodiments described in the above section [Problems to be Solved by the Invention, Problem Solving Means and Effects]. Various modifications and improvements can be made based on the knowledge of those skilled in the art.
[0035]
<Configuration of circuit component mounter and component mounting work>
FIG. 1 is an overall perspective view of a circuit component mounter 10 for performing a component mounting operation to which the present invention is applied. The circuit component mounter 10 (hereinafter, may be simply referred to as “mounter 10”) includes a base module 12, a plurality of mounting modules 14 arranged adjacent to each other and aligned on the base module 12, The control module 16 controls the entire mounting machine 10. Each of the mounting modules 14 has the same configuration in terms of hardware. The direction in which the mounting modules 14 are arranged is the direction in which the circuit board is transported. The circuit board is transported from the mounting module 14 located on the left side to the work module 14 located on the right side in the drawing, and sequentially. In each mounting module 14, the component mounting operation is performed.
[0036]
FIG. 2 shows a portion where two mounting modules 14 are arranged, and the mounting module 14 on the right side is shown with an outer plate or the like removed. As shown in this figure, each mounting module 14 is configured to include a frame 20 functioning as a module frame, and various devices arranged on the frame 20. For example, a plurality of tape feeders (hereinafter, may be abbreviated as “feeders”) 22 that are arranged side by side and supply circuit components one by one at a predetermined component supply position, and a function of transporting the circuit board 18 are provided. It has a conveyor unit 24 for holding the circuit board 18 in a predetermined working position and a mounting head 26, and moves the mounting head 26 in the work area to cause the mounting head 26 to perform component mounting work. The mounting device 28 and the like. In the present embodiment, the mounting head 26 holds and takes out the circuit components supplied from the feeder 22 and mounts the circuit components on the circuit board 18 fixed to the conveyor unit 24.
[0037]
The mounting module 14 includes circuit components mainly held by a mounting head 26 between a group 30 of a plurality of feeders 22 (hereinafter, may be referred to as “feeder group”) 30 and a conveyor unit 24. A component camera (which is a CCD camera) 32 for taking an image is provided. Furthermore, each mounting module 14 has a module control device 38 (omitted in FIG. 2, see FIG. 4) for controlling itself, and the above-described devices and the like operate while being controlled by the module control device 38. Each of the mounting modules 14 has an operation / display panel 40 as an input / output device on the upper part, and this operation / display panel 40 is connected to a module control device 38, and an operator input for various commands, information, and the like. And displays status information of the mounting module 14, information related to image processing described later, and the like.
[0038]
Each of the plurality of feeders 22 has a generally well-known structure, and will be briefly described here. However, a circuit component taping which is a taped circuit component (called an electronic component taping in the case of an electronic component). ) Is held, and the taping extending from the reel 48 is sent by the component holding pitch in accordance with the operation of the mounting device 28, and the cover tape is peeled off, so that the circuit components are removed. It is configured to supply one by one at the supply position.
[0039]
The conveyor unit 24 mainly includes a conveyor device around which a conveyor belt circulates, and includes two conveyor devices, a front conveyor 60 and a rear conveyor 62. Each of the front conveyor 60 and the rear conveyor 62 has a structure in which a conveyor rail 64 moves and the width thereof can be changed in accordance with the width of the circuit board 18. Then, the circuit board 18 is transferred while being supported by the conveyor belt. The conveyor unit 24 is provided in each of the plurality of mounting modules 14, and each of the conveyor units 24 is arranged on the mounting machine 10 in a straight line. The conveyor units 24 of each mounting module 14 can transport the circuit board 18 in cooperation with each other, and each conveyor unit 24 stops the circuit board 18 transferred to its own work area by a set stop. Stop at the working position. The conveyor unit 24 has a circuit board support plate (hereinafter, may be abbreviated as “support plate”) 76 that can be moved up and down by a lifting device (not shown) at a lower portion. A support pin to be omitted is provided so as to be changeable to an arbitrary position. When the support plate 76 is raised, the circuit board 18 is supported by the support pins and lifts to disengage with the conveyor belt, and is held between a part of the conveyor rail 64 and the support pins. 18 is fixed in the working position.
[0040]
As shown in FIG. 3, the mounting device 28 includes a mounting head 26 and a head moving device 80 that moves the mounting head 26 along a substantially flat surface in a work area. The head moving device 80 is an XY robot-type moving device, and includes a Y slide device 82 that moves the mounting head 26 in the front-rear direction and an X slide device 84 that moves the mounting head 26 in the left-right direction. The mounting head 26 is moved by the head moving device 80 over the feeder group 30 and the circuit board 18 fixed in the conveyor unit 24. Note that a mark camera (which is a CCD camera) 100 is provided below the X slide 96. The mark camera 100 mainly captures images of reference marks and the like provided on the surface of the circuit board 18. The mark camera 100 is moved by the head moving device 80 together with the mounting head 26.
[0041]
The mounting head 26 includes a plurality of mounting units 110 each having a generally axial shape, and is of a type that rotates in an index. Each mounting unit 110 has a suction nozzle 112 at the lower end thereof as a component holding device for sucking and holding a circuit component. Each mounting unit 110 is rotatable and can be moved up and down at an elevating station at one index position. To briefly explain the operation of the mounting head 26, one of the mounting units 110 is lowered in a state where the mounting head 26 is located above the feeder group 30, and the suction nozzle 112 of the mounting unit 110 The mounting unit 110 takes out the circuit component by sucking and holding the circuit component supplied at the component supply position. The mounting units 110 are intermittently circulated around the intermittent rotation axis of the mounting head 26, and each mounting unit 110 sequentially takes out circuit components. With each mounting unit 110 holding the circuit components, the mounting head 26 is moved above the circuit board 18 fixedly held on the conveyor unit 24, and one of the mounting heads 26 located at the elevating station above the circuit board 18 is moved. The mounting unit 110 is lowered after being rotated to adjust the rotational position of the circuit component, and the circuit component held by the mounting unit 110 is placed on the surface of the circuit board 18. The mounting units 110 are rotated, and the circuit components held by each mounting unit 110 are sequentially mounted on the circuit board 18.
[0042]
The mounting machine 10 is controlled by a module control device 38 arranged in each mounting module 14 for controlling each mounting module 14 and a control module 16 as a working machine control device that controls each of the working modules 14. However, the central part of the control of the work exists in each module control device 38. FIG. 4 is a block diagram related to the control of the module control device 38.
[0043]
The module control device 38 is a control device mainly composed of a computer 120. The computer 120 has a CPU 122, a ROM 124, a RAM 126, an input / output interface 128, and a bus 130 connecting these components to each other. The input / output interface 128 is connected to each of the feeders 22 arranged as the feeder group 30, the conveyor unit 24, the mounting head 26, and the head moving device 80 via a drive circuit 132, respectively. The input / output interface 128 is connected to an image processing unit 140 that performs image processing mainly using a computer, and the component camera 32 and the mark camera 100 are connected to the image processing unit 140. The image processing unit 140 will be described later in detail. The operation / display panel 40 and an external storage device 142 mainly composed of a hard disk are also connected to the input / output interface 128. Since the mounting machine 10 operates by exchanging various signals with each of the plurality of mounting modules 14, the other mounting modules 14 are connected to the input / output interface 128 via the communication circuit 144. Have been. In addition, the control module 16 which functions as a host is also connected via the communication circuit 200, and communication of signals, information, and the like is enabled with the control module. The external storage device 142 stores a basic operation program of the mounting module 14, an application program such as a mounting program corresponding to the circuit board 18, and various data related to the circuit board 18, circuit components, and the like. At the time of work or the like, necessary items are transferred to the RAM 126 and stored therein, and mounting work or the like is performed based on the stored contents of the RAM 126. The control module 16 is also mainly composed of a computer, but is not closely related to the present invention, so that the description is omitted.
[0044]
The component mounting operation by one mounting module 14 will be described below. First, the circuit board 18 transferred from the upstream side is stopped by the conveyor unit 24 at a set work position in the work area. The stopped circuit board 18 is fixed and held by the conveyor unit 24 at that position by raising the circuit board support plate 76 by the elevating device.
[0045]
Next, the mark camera 100 is moved above the reference mark provided on the circuit board 18 by the head moving device 80, and the reference mark is imaged. More specifically, the circuit board 18 is provided with two fiducial marks at positions set diagonally, and the mark camera 100 is provided with a fiducial mark when the circuit board 18 is fixed at a regular position. It is moved to a position directly above each, and each of the reference marks is imaged at that position. When the circuit board 18 is at the proper position, each reference mark is located at the center of the imaging field of view, that is, at the center of the imaging screen. However, in general, the circuit board 18 is displaced in a parallel movement and also displaced in the rotational angle position. Therefore, each of the reference marks is imaged off the center of the shooting screen. In the image processing to be described later, the above-mentioned shift amount from the center of the screen of each reference mark is obtained. Then, a geometric calculation process is performed based on the shift amount, and the shift amount of the fixed position of the circuit board 18 is detected.
[0046]
Next, the mounting head 26 is moved above the feeder group 30, and the circuit components are suction-held by the suction nozzles 112 in accordance with the set removal order. Specifically, the mounting unit 110 located at the elevating station is positioned above the component removal position of the feeder 22 that supplies the circuit component to be held, and the mounting unit 110 is lowered at that position, and A negative pressure is supplied to the suction nozzle 112 held at the tip, and the circuit component is sucked and held. Then, the mounting unit 110 is rotated, and the same component removal operation for the next mounting unit 110 is performed. In this manner, the component removal operation is sequentially performed on the mounting units 110 included in the mounting head 26.
[0047]
Next, the mounting head 26 holding the circuit components is moved above the component camera 32. At that position, the component camera 32 captures an image of all the circuit components held in each mounting unit 110 within one field of view. The image is taken so that the intermittent rotation center of the mounting head 26 is located at the center of the imaging field of view of the component camera, that is, at the center of the shooting screen. Since the positional relationship between the rotation center (axial center) of each mounting unit 110 with respect to the intermittent rotation center is determined, the position of the rotation center of each mounting unit 110 in the shooting screen is also known. The rotation angle of each mounting unit 110 at the time of imaging is also recognized. In image processing to be described later, how much positional deviation (deviation between the XY direction position and the rotational position) of each of the circuit components held in each mounting unit 110 from the rotation center of each mounting unit 110 in the photographing screen is described. It is required that it has occurred. Thereby, the shift amount of the holding position of each circuit component is detected.
[0048]
Subsequently, the mounting head 26 is moved to a position above the circuit board 18, and the held circuit components are placed on the surface of the circuit board 18 according to the set mounting order. More specifically, first, the mounting unit 110 located at the elevating station is rotated to adjust the rotation angle position of the circuit component, and the mounting unit 110 is set at a position where the circuit component can be placed at the predetermined mounting position. The mounting head 26 is moved. At this time, the rotation angle of each mounting unit 110 is optimized based on the detected shift amount of the fixed position of the circuit board 18 and the shift amount of the holding position of the circuit component, and the mounting head is adjusted. The movement position of 26 is optimized. Then, at that position, the mounting unit 110 is lowered by a predetermined distance, a positive pressure is supplied to the suction nozzle 112, and the held circuit component is placed on the surface of the circuit board 18. Subsequently, the mounting unit 110 is rotated, and the same mounting operation is performed for the next mounting unit 110. That is, in the component mounting operation, the circuit components are placed while the correction based on the detected shift amount of the fixed position of the circuit board 18 and the detected shift amount of the circuit component holding position are performed. Is accurately mounted at the set mounting position on the circuit board 18.
[0049]
The mounting head 26 is reciprocated between the feeder group 30 and the circuit board 18 until the mounting work for all the planned circuit components is completed, and the above-described component removal operation and component placement operation are repeatedly performed. After the mounting of all the circuit components is completed, the fixed holding of the circuit board 18 is released, and the circuit board 18 is transferred to the downstream side by the conveyor unit 24. Thus, the component mounting operation by the mounting module 14 scheduled on the circuit board 18 is completed.
[0050]
<Image processing unit>
Image processing for controlling the imaging of the reference mark and the circuit board by the mark camera 100 and the component camera 32 and detecting the shift amount of the fixed position of the circuit board 18 and the shift amount of the holding position of the circuit component as described above are as follows. This is performed by the image processing unit 140. FIG. 5 shows a block diagram of the image processing unit 140.
[0051]
The image processing unit 140 is mainly composed of a computer. It has an overlay display memory 166, which is interconnected by an internal bus (not shown) on the substrate 167.
[0052]
A two-channel serial interface 170 is connected to the internal bus, and an input device 172 is connectable. Normally, manual input by the operator is performed from the operation / display panel 40 of the mounting module 14. However, when adjusting the image processing unit 140 or the like, an input device 172 is separately connected and necessary input from the input device 172 is performed. Information can be entered.
[0053]
An Ethernet interface 174 (Ethernet is a registered trademark) and a memory card interface 176 are also connected to the bus. The Ethernet interface 174 exchanges data with the module control device 38 (or with an external control device optionally connected) via the P1 connector 168. Specifically, data relating to circuit components and the circuit board 18, template setting data described later, and the like are exchanged via the Ethernet interface 174. The memory card stores a program created in advance for performing image processing. When the memory card is set in the image processing unit 140, the CPU 154 uses the PROM 160 to execute programs and programs in the memory card. The necessary data is read out through the memory card interface 176 and stored in the DRAM 156.
[0054]
Further, a CCD camera interface 180 is connected to the bus, and the component camera 32 and the mark camera 100 are connected to the bus. The image data of the circuit components and the reference marks obtained by the imaging by the cameras 32 and 100 are stored in the frame grabber memory 164 via the CCD camera interface 180, respectively. Although omitted in the figure, a plurality of frame grabber memories 164 are provided, and a plurality of image data are sequentially stored in each frame grabber memory 164. An LSO bus 182 and a line sensor board interface 184 are also connected to the frame grabber memory 164 so that image data obtained by imaging by the line sensor is stored in the frame grabber memory 164. Image processing based on image data captured by a line sensor such as a line scan camera instead of a CCD camera is also possible.
[0055]
Further, a CRT interface 186 is connected to the bus, and a monitor CRT device 188 is connectable. Normally, information from the image processing unit 140 is displayed on the operation / display panel 40 described above. However, at the time of adjustment or the like, a monitor CRT device 188 is connected, and display on the monitor CRT device 188 is enabled. It is. The output image from the image processing unit 140 can be displayed in both color and monochrome. As described above, image data of a plurality of monochrome images obtained by imaging with the cameras 32 and 100 are stored in parallel in the frame grabber memory 164, while the overlay display memory 166 stores the image data. Are provided for a plurality of planes capable of storing color image data for displaying the image data in 16 colors. On the monitor CRT device 188, the one corresponding to the monochrome image among the four color images is displayed superimposed on the monochrome image, and the progress and the result of the image processing are displayed on the operation / display panel 40 or the monitor CRT device. 188 can be displayed.
[0056]
<Image processing>
Hereinafter, as an embodiment of the present invention, an image processing using a circuit component held in one mounting unit 110 as an object will be described as an example. FIG. 6 shows an image of the circuit component 200 held by one mounting unit 110 captured by the component camera 32. In the present embodiment, for simplicity of description, the circuit component 200 has a rectangular shape. Further, it is assumed that the image of the circuit component 200 is captured with higher luminance than the background portion 202 adjacent to the periphery. In the recognition process, the dimensional error 1 with respect to the theoretical circuit component 204, which is a circuit component having a theoretical shape and size, is obtained. _x −l _x0 , L _y −l _y0 , The shift amount of the holding position, that is, the normal position (the center O of the circuit component 200 is ₀ (A position located on the upper side) in the X direction, a deviation .DELTA.Y in the Y direction, and a deviation .theta. In the rotational angle position, and whether the shape of the circuit component 200 is good or bad. It is assumed that the shape of the edge 206 which is the outline of the shape is inspected.
[0057]
i) Physical screen, virtual screen and image processing program
In the present embodiment, image processing is performed using two conceptual screens, a physical screen and a virtual screen. FIG. 7 conceptually shows the relationship between the physical screen, the virtual screen, and the image processing program. The physical screen 210 is a screen on which a physical image based on image data obtained by imaging and stored in the frame grabber memory 64 exists, and is configured as a matrix corresponding to the pixels of the component camera 32. The virtual screen 212 is a screen in which a virtual image that is not constrained by pixels, in other words, does not have an image unit.
[0058]
Many parts of the image processing are performed by the image processing unit 150 executing a predetermined image processing program. The above-described recognition processing such as detection of a shift amount of the holding position of the circuit component 200 is performed on a virtual image on a virtual screen, and a recognition processing program for performing the recognition processing is a kind of image processing application software 214. Yes, as described above, it is stored in the memory card and read into the DRAM 156 prior to the mounting operation. In performing the recognition process, it is necessary to convert data on the physical screen into data on the virtual screen. Each time, a predetermined image data conversion program (image data conversion subroutine) is called and executed. . The conversion program is a kind of the physical screen / virtual screen conversion driver 216, and is stored in a memory card like the recognition processing program, and is read into the DRAM 156 prior to the mounting operation.
[0059]
In the present embodiment, luminance, which is a kind of optical characteristic value, is used as data of a physical image and data of a virtual image. That is, the frame grabber memory 64 stores luminance data corresponding to pixels, and the conversion program converts the luminance data into luminance data of a virtual image. In this embodiment, the luminance data is stored as gradation values of 0 to 255, and the converted luminance data is also gradation values.
[0060]
Further, as will be described later in detail, in the present embodiment, the luminance data is corrected to a value suitable for the recognition processing at the time of the conversion. This correction is performed according to the determined conditions. The memory card stores luminance correction data as optical characteristic value correction data in a table format. The correction data is read into the DRAM 156 prior to the mounting operation, so that the lookup table 218 is stored in the DRAM 156. Is set. The conversion of the luminance data is performed with reference to the look-up table 218.
[0061]
Hereinafter, the image processing according to the present embodiment will be described separately for the image data conversion processing and the recognition processing.
[0062]
ii) Conversion of image data
As described above, the conversion of the image data is performed by executing the image data conversion program (image data conversion subroutine). Specifically, the brightness data at the recognition site, which is the point selected on the virtual screen by the recognition processing program, is calculated based on the brightness data at the recognition site on the physical screen and the correction data. . FIG. 8 shows a flowchart of the image data conversion program. Hereinafter, description will be made according to this flowchart. FIG. 9 conceptually shows a method of a complementary operation in the conversion of image data. Hereinafter, description will be made with reference to this figure as appropriate.
[0063]
First, in step 1 (hereinafter abbreviated as "S1"; the same applies to other steps), a pixel corresponding to the recognition base selected in the recognition processing program is selected. The physical screen is a matrix defined by rectangular coordinates (hereinafter, referred to as “UV coordinates” to distinguish it from the XY coordinates on the virtual screen later), and the image data on the physical screen is defined by the center point of the pixel. The luminance value is associated with the coordinate position (u, v) (u and v both take integer values). The recognition base is selected as a point (x, y) (x, y is not limited to an integer value) on the virtual screen defined by the XY coordinates associated with the UV coordinates. In S1, as shown in FIG. 8, a point (u) corresponding to the selected recognition base (x, y) is set. ₀ , V ₀ ) (U ₀ , V ₀ Are not always integers). The four pixel center points (u ′, v ′), (u ′ + 1, v ′), (u ′, v ′ + 1), (u ′ + 1, v ′ + 1) surrounding the pixel Selected (these are all integer values). The four pixel center points are points (completion base points) that will be the basis of linear interpolation described later.
[0064]
Next, in S2, the luminance data f at each of the four complementary base points is calculated. ₀ Is read, and the luminance data is corrected with reference to a luminance correction table which is a lookup table. FIG. 10 shows the contents of the correction data of the luminance correction table. This correction data is for a certain circuit component 200 and is an example. More specifically, the correction data shown in the figure is obtained by capturing an image on the physical screen of the target circuit component 200 with a brightness value of about 170 under a predetermined imaging condition, and a background portion 202 adjacent thereto. It is known that the image is photographed with a luminance value of about 60 (the higher the luminance value, the higher the luminance). The correction data is data that increases the contrast difference between the circuit component 200 and the background portion 202 in that case. It is also known that the luminance of the edge of the circuit component 200 is approximately 120. When the luminance is equal to or higher than the threshold value, the luminance is further increased, and the luminance of the luminance lower than the threshold value is increased. In such a case, the luminance is corrected so as to be further reduced. The brightness correction table is set in several patterns, and is selected according to the circuit component 200 and the purpose and method of the recognition process. Which pattern is to be adopted is determined in the above-described mounting program, and the image processing unit 140 acquires the selection information from the module control device 38, and determines the luminance correction table to be adopted based on the selection information. Is done.
[0065]
The correction value δf is the luminance value f before correction. ₀ And the corrected luminance value f is
f = f ₀ + Δf
It is obtained by the formula. Brightness value f before correction of the selected four complementary base points ₀ (U ', v'), f ₀ (U '+ 1, v'), f ₀ (U ', v' + 1), f ₀ (U ′ + 1, v ′ + 1) is acquired, and based on the above equation, the corrected luminance values f (u ′, v ′), f (u ′ + 1, v ′), f ( u ′, v ′ + 1) and f (u ′ + 1, v ′ + 1) are calculated.
[0066]
In the next S3, the point (u) is calculated based on the corrected luminance value f of the four complementary base points. ₀ , V ₀ The brightness value in () is obtained by linear interpolation. First, the point (u ₀ , V ₀ ) Is located in the four pixel center points, specifically, the values of α and β shown in FIG. 9 are obtained. Then, using the values of α and β,
f (u ₀ , V ₀ ) = F (u ′, v ′) (1-α) (1-β) + f (u ′ + 1, v ′) α (1-β) + f (u ′, v ′ + 1) (1-α) β + f (U ′ + 1, v ′ + 1) αβ
According to the equation, linear interpolation is performed, and the point (u ₀ , V ₀ ) At the luminance value f (u ₀ , V ₀ ) Is calculated. This f (u ₀ , V ₀ ) Is a luminance value f (x, y) as a base characteristic value at the recognition base (x, y).
[0067]
As described above, according to the image data conversion program, the luminance data as the base characteristic value at the recognition base on the virtual screen is obtained according to the above steps. Note that by performing linear interpolation as in the present embodiment, image data finer than the pixel pitch can be obtained, and an object with high accuracy can be recognized.
[0068]
iii) Circuit component recognition processing
The circuit component recognition processing is performed by executing the above-described recognition processing program. FIG. 11 shows a flowchart of the recognition processing program. The recognition process can be divided into five processes of a search process, a re-search process, a measurement process, a re-measurement process, and an inspection process, and each process is a search routine and a re-search, which are five routines of S10 to S50. A routine, a measurement routine, a re-measurement routine, and an inspection routine are sequentially executed and performed. Hereinafter, each step will be described in order.
[0069]
a) Search process
The search step is a step for recognizing approximately where the circuit component 200 is located on the screen. FIG. 12 shows a flowchart of a search routine executed to perform the search step.
[0070]
In the search step, first, in S11, a search template is set. FIG. 13 shows a conceptual diagram of the search template. The search template 220 includes a plurality of reference points SP associated with a theoretical contour 222 which is a line indicating the edge of the theoretical circuit component 204 having the theoretical shape and dimensions. ₁ ~ SP ₈ , SP ₁ '~ SP ₈ 'have. Specifically, a point pair SP, which is eight pairs of reference points, is provided near each corner of the theoretical circuit component 204. ₁ −SP ₁ '~ SP ₈ −SP ₈ '(Hereinafter collectively referred to as "SP _n −SP _n '". )have. One reference point SP constituting each point pair ₁ ~ SP ₈ Is a point scheduled to be located in the image of the circuit component 200, and the other reference point SP ₁ '~ SP ₈ 'Is a point which is scheduled to be located outside the image of the circuit component 200, that is, in the background portion 202. In other words, each point pair SP _n −SP _n 'Each reference point SP _n , SP _n 'Is positioned so as to sandwich the theoretical contour line 222. In this search template 220, the reference point SP ₁ And SP ₈ , Some points are located at the same position. In the present embodiment, each point pair SP _n −SP _n 'Two reference points SP _n , SP _n 'Indicates that the straight line connecting them is orthogonal to the theoretical contour line 222 and that each reference point SP _n , SP _n The distance of the 'from the theoretical contour line 222 is made equal.
[0071]
The setting data of the search template 220 differs for each circuit component, and is stored in the external storage device 142 of the module control device 38. The setting data for the circuit components to be mounted is selected, sent to the image processing unit 150 and stored in the DRAM 156 prior to the mounting operation. In S11, from among the data stored in the DRAM 156, the data of the circuit component 200 for performing the image processing is selected, and the data is set based on the selected data.
[0072]
Next, in S12, the search template 220 is superimposed on the initial position on the virtual screen (see FIG. 6). The initial position is a position where the theoretical circuit component 204 is located at the normal position, that is, a position where there is no shift in the holding position. When the search template 220 is superimposed, the point pair SP _n −SP _n 'Each reference point SP _n , SP _n 'Is a recognition base on the virtual screen 212. That is, the recognition base is selected by overlapping the templates.
[0073]
Subsequently, in S13, the suitability of the search template 220 and the circuit component 200 (strictly, a virtual image of the circuit component 200) is determined. FIG. 14 illustrates the conforming state and the non-conforming state. FIG. 14A shows a conforming state, in which each point pair SP _n −SP _n ', One of the points SP _n Is located in the circuit component 200 (belongs to the outline of the circuit component 200) and the other point SP _n 'Is located outside the circuit component 200 (belongs to the outside of the contour of the circuit component 200). On the other hand, in FIG. 14B showing the non-conforming state, one of the point pairs SP _n −SP _n ', The two reference points SP constituting it _n , SP _n 'Are both located inside the circuit component 220 or outside the circuit component 200.
[0074]
The determination of suitability is specifically made as follows. First, each point pair SP that was set as a recognition base _n −SP _n 'Each reference point SP _n , SP _n Is obtained by executing the above-described image data conversion subroutine. Note that, as described above, this brightness value is corrected to enlarge the contrast difference. Next, each point pair SP _n −SP _n 'About each recognition base SP _n , SP _n ', The difference f (SP _n ') -F (SP _n ) Is determined, and the difference is determined. The outline 206 of the circuit component 200 is a point pair SP _n −SP _n , There should be a considerable difference in luminance between them, and in this embodiment, 50 is adopted as the set threshold (a type of threshold state), and all the point pairs SP _n −SP _n '
| F (SP _n ') -F (SP _n ) |> 50
Is satisfied, the circuit component 200 is determined to be compatible with the search template 220 superimposed. In the present embodiment, the above condition must be satisfied for all the point pairs.However, if the predetermined condition is satisfied for a predetermined number of not all the point pairs, the conforming state is determined. It is also possible to perform a process for determining.
[0075]
If the above condition is not satisfied in any of the point pairs, the search template 220 is moved in S14, and the determination in S13 is performed again at the moved position, and the processing is performed until the matching state is reached. Repeated. The movement of the search template 220 in S14 is performed according to the set conditions. FIG. 15 schematically shows how the search template 220 moves. First, the search template 220 keeps the center position fixed, and within a set rotation angle position range (for example, -45 ° to 45 °), the set angle (for example, 1 °, 5 °, etc.) around the center position. Can be rotated at a time. At each rotation angle position, the above-described matching determination is performed. Next, it is moved by a predetermined distance in the positive direction in the X direction, and at that position, the suitability determination at each rotation angle position within the set rotation angle position range is performed. Subsequently, it is moved in a rectangular spiral from the initial position, for example, by the predetermined distance in the Y direction, and subsequently by the predetermined distance in the reverse direction in the X direction. Is done.
[0076]
When it is determined in the matching determination in S13 that the vehicle is in the matching state, a search end process is performed in S15. In the search ending process, the moving position and the rotation angle position of the search template 220 in the matching state are stored. A search end process is performed, and the search process ends. Although omitted in the flowchart, when the search template 220 is moved to the set movement range, if the matching state is not obtained at any of the movement positions, the circuit component is determined to be inappropriate, The mounting operation for the circuit component is not performed. For example, when a circuit component is sucked on a surface different from the normal surface (so-called “chip standing”), the circuit component is not determined to be in an appropriate state, and a mounting failure due to such an event is prevented beforehand. It is possible.
[0077]
b) Re-search process
In the search step, the approximate location of the circuit component 200 was recognized. In this re-searching step, the existence position of the circuit component 200 is searched to recognize the more accurate existence position. FIG. 16 shows a flowchart of a re-search routine executed to perform the re-search step.
[0078]
In the re-search process, first, in S21, a re-search template is set. FIG. 17 conceptually shows a state where the re-search template is superimposed on circuit component 200. The re-search template 230 includes eight reference intersection lines SL associated with the theoretical contour 222 and intersecting the theoretical contour 222 at right angles. ₁ ~ SL ₈ (Collectively, "SL _n "). In the re-searching process, the reference intersection line SL of the edge of the circuit component 200 is used. _n Identify the position on the top. In this step, in other words, the reference intersection line SL _n Since it includes a step of searching for the position of the edge 206 above, the reference intersection line SL _n The seek line SL _n I will call it. In addition, each seek line SL ₁ ~ SL ₈ Is a point indicating the position of the edge 206 at the edge point E ₁ ~ E ₈ (Collectively, "E _n "). Seekline SL _n Is a line segment directed from inside the circuit component 200 to the outside thereof, and its start point and end point are each a point pair SP in the search template 220. _n −SP _n 'Each reference point SP _n , SP _n ' Note that the setting data of the re-search template 230 is sent from the external storage device 142 of the module control device 38 and stored in the DRAM 156 of the image processing unit 140 prior to the mounting operation, similarly to the search template 220. The circuit components to be performed are also selected.
[0079]
In the next S22, the re-search template 230 is superimposed on the virtual screen as shown in FIG. The overlapping position is the position where the search template was present at the end of the search process. FIG. 18 shows one seek line SL. _n And the relationship between the edge of the circuit component 200 and the theoretical contour line 222 are schematically shown. As shown, seek line SL _n Above, 15 reference points P that divide it into equal pitches ₁ ~ P _Fifteen (Collectively, "P _n "). The reference contour 222 is a seek line SL _n And the reference point P ₈ In, seek line SL _n Cross at right angles. In S22, these seek lines SL _n Reference point P ₈ Are selected as recognition sites.
[0080]
In the following S23, a certain seek line SL _n The edge point E at _n Is specified. First, the reference point P, which is the recognition base, _n (In this process, "Recognition base P _n ) May be obtained. Specifically, as described above, the image data conversion subroutine is used to convert each of the recognition points P on the virtual screen from the luminance value on the physical screen. _n Is calculated. FIG. 19 shows each recognition point P obtained by the calculation. _n The luminance values in the table are shown in a table. Note that, in this table, in addition to the luminance value subjected to the above-described correction for increasing the contrast difference, a luminance value obtained when the correction is not performed is listed for reference. Next, the adjacent recognition base P _n The difference between the luminances is calculated. Specifically, the brightness difference Δf (P _{n + 1} −P _n ) Is
Δf (P _{n + 1} −P _n ) = F (P _{n + 1} ) -F (P _n ・ ・ ・ ・ ・ (3)
Required by FIG. 20 shows the calculation result including the case where the above correction is not performed. FIG. 21 is a graph showing the calculation result.
[0081]
As can be seen from FIG. 19, one seek line SL _n In each recognition base P _n The luminance value becomes smaller from the circuit component 200 side toward the background portion 202 side. This seek line SL _n Edge point E on _n Is the recognition base P _n , The section where the change in the luminance value is the largest, that is, the adjacent base luminance difference Δf (P _{n + 1} −P _n ) Is present in the section where the absolute value is largest. As can be seen from FIGS. 19 and 20, when no correction is performed, the interval P ₆ −P ₇ , Section P ₇ −P ₈ , Section P ₈ −P ₉ Since both have the same difference value, it cannot be specified which section. However, when the correction for enlarging the contrast difference is performed as in the present embodiment, the section P ₇ −P ₈ Is large and the edge point E _n Can be easily specified.
[0082]
One seek line SL in S23 _n The edge point E at _n Of the seek line SL is determined by the determination of S24. _n For each edge point E in them _n Is specified. In S23, the edge point E _n Is the above recognition base P _n It is estimated that it exists at the position of the midpoint of the section delimited by.
[0083]
Next, in S25, the position where the circuit component 200 exists on the virtual screen is calculated. The superimposed position of the search template 230 is known, and each seek line SL _n , Each recognition base P _n Is also known, the edge point E on the virtual screen is _n Is found by a geometric operation. Specifically, each side of the rectangle that forms the outline of the circuit component 200 has two edge points E corresponding to the side. _n From the result, the position of the center O of the circuit component 200 in the X and Y directions and the rotation angle position at the coordinates constituting the virtual screen are specified. In the subsequent re-search termination processing in S26, the specified location is stored, and the re-search step is terminated.
[0084]
c) Measurement process
In the measurement process, the dimensional error of the circuit component 200 with respect to the theoretical circuit component 204 is measured, and the more accurate position of the circuit component 200 on the virtual screen is recognized. FIG. 22 shows a flowchart of a measurement routine executed to perform the measurement process.
[0085]
In the measurement process, first, in S31, a measurement template is set. FIG. 23 conceptually shows a measurement template. The measurement template 240 also has a seek line SL, which is a plurality of reference intersection lines that are related to the theoretical contour line 222 and intersect the theoretical contour line 222 at right angles. Compared to the re-search template, the number of seek lines SL is larger, and the length of each seek line SL is shorter. The position of the seek line SL is different from that of the re-search template 230 in order to recognize the accurate position to some extent. Like the search template 220, the setting data of the measurement template 240 is sent from the external storage device 142 of the module control device 38 and stored in the DRAM 156 of the image processing unit 140 prior to the mounting operation, and the image processing circuit performs image processing. Parts are also selected.
[0086]
In the next S32, the measurement template 240 is superimposed on the virtual screen. The overlapping position is determined based on the existing position of the circuit component 200 obtained in the re-search process. In the case of the measurement step, a plurality of reference points for dividing each of the seek lines SL are set at a smaller equal pitch than in the case of the re-search step. In S32, a point defined by each of the reference points of the seek line SL is selected as a recognition base.
[0087]
In the following S33, the position of the edge point E in one seek line SL is specified. First, as in the case of the re-searching step, first, the image data conversion subroutine is executed, and the luminance value at the recognition point P is obtained. In the measurement step, the edge point E is specified based on the degree of change in the luminance value at each recognition point P. However, unlike the section specified in the re-search step, the edge point E is complemented based on the luminance value of each recognition point P. By performing a mathematical operation, the coordinate position of the edge point E is specified. Although a specific calculation method is omitted, the position of the edge point E is recognized at a position finer than the pitch of the recognition base P. As in the case of the re-search step, the existence position of each edge point E is specified for all the seek lines SL by S34.
[0088]
In the next step S35, a dimensional error with respect to the theoretical size of the circuit component 200 is calculated. As will be briefly described, the circuit component 200 has a rectangular shape, and a plurality of pairs of seek lines SL set at positions facing each other exist on each side facing each other. The distance between the edge points E on the seek line SL forming the pair is determined, and the average of the distance between the edge points E for the pair of the seek lines SL on the long side and the short side is determined. By comparison, the aforementioned dimensional error l _x −l _x0 , L _y −l _y0 (See FIG. 6).
[0089]
In S36, the position of the circuit component 200 on the virtual screen is calculated. In brief, first, the dimensional error l determined earlier _x −l _x0 , L _y −l _y0 , The theoretical contour 222 is changed. Next, a displacement of the edge point E from the changed contour line is obtained, and based on this displacement, the position of the circuit component 200 is calculated according to a geometrical method. A specific calculation method may follow, for example, the method described in the above-mentioned publication of the present applicant's application. In this specification, a specific description of a calculation method is omitted to avoid redundant description. By the processing of S36, the above-described shift amounts ΔX, ΔY, θ of the holding positions (see FIG. 6) are obtained.
[0090]
In the measurement termination process of S37, the obtained shift amounts ΔX, ΔY, θ of the holding positions are stored, and the measurement process ends. Although omitted in the flowchart, when the dimensional error exceeds a set threshold error, the mounting operation of the circuit component 200 is stopped.
[0091]
d) Re-measurement process
The re-measurement step is a step of recognizing a more accurate existing position of the circuit component 200. Although the flowchart of the re-measurement routine is omitted, it may be generally considered that S35 in the measurement routine is omitted. In the re-measurement template used in the re-measurement step, the seek line SL and the reference point P on the seek line SL are set more finely, and the edge point E is specified more accurately. The re-measurement template is superimposed based on the position of the circuit component obtained in the measurement process. Since the processing in each step is the same as the measurement step, the description is omitted here. Note that the program may be a program that skips the re-measurement step and proceeds to the next step, or may be a program that repeats the re-measurement step a plurality of times to further improve recognition accuracy.
[0092]
e) Inspection process
The inspection step is a step of inspecting the circuit component 200 whose existence position has been specified for the contour shape of the circuit component 200. For example, when there is chipping of an edge, bending of an edge, or attachment of a foreign substance to an edge, it can be detected. FIG. 24 shows a flowchart of an inspection routine executed for performing the inspection process.
[0093]
In the inspection process, first, in S51, an inspection template is set. FIG. 25 conceptually shows an inspection template. The inspection template 250 has two seek lines NSL, which are reference lines associated with the theoretical contour line 222. These two seek lines NSL are reference lines extending in the vicinity of the theoretical contour line 222 and parallel to the theoretical contour line 222, and are not expected to intersect with the edge 206 of the circuit component 200. Therefore, it is called a negative seek line NSL. In the inspection template 250, the negative negative seek line located inside the theoretical contour line 222 and the NSL _I And the outer negative seek line NSL located outside _O And two lines. Note that, in a strict sense, the negative seek line NSL is not the theoretical contour 222 but the dimensional error l at which the theoretical contour 222 is recognized in the measurement process. _x −l _x0 , L _y −l _y0 Is a line extending along the modified theoretical contour line modified based on Like the search template 220, the setting data of the inspection template 250 is sent from the external storage device 142 of the module control device 38 and stored in the DRAM 156 of the image processing unit 140 prior to the mounting operation. A component is selected, and an inspection template 250 is set based on the selected data and the previously recognized dimensional error of the circuit component 200. In the present embodiment, the negative seek line NSL extends over the entire circumference of the circuit component 200. However, it is also possible to set a negative seek line only on a part of the contour of the object.
[0094]
Next, in S52, the inspection template 250 is superimposed on the circuit component 200 on the virtual screen. The superimposed position is the existing position of the circuit component obtained in the re-measurement step, and is the position where the corrected theoretical contour 222 matches the contour of the circuit component 200. A plurality of reference points arranged at a fine pitch are set on the negative seek line NSL, and each of the plurality of reference points is selected as a recognition base.
[0095]
In S53, the luminance value of each recognition point is obtained, and the deviation of the luminance value for each negative seek line NSL is calculated. The luminance value of each recognition point is determined by executing the image data conversion routine described above. FIG. 26 schematically shows a change in the luminance value of each recognition site along each negative seek line NSL as a line graph. The change in the luminance value shown in FIG. 26 is at the edge 206 shown schematically in FIG.
[0096]
FIG. 26 (a) shows the negative seek line NSL. _I , And there is a portion where the brightness value is low. Negative seek line NSL _I Is expected to exist inside the edge 206 of the circuit component 200, and normally, a high luminance value should be continuous at each recognition base. However, as shown in FIG. 27, in the case where a defect 254 is generated at a part of the edge and the background portion 202 is imaged at the defect 254, the negative negative seek line NSL is used. _I The luminance value is reduced in a portion where the image overlaps the background portion 202. FIG. 26B shows an outer negative seek line NSL. _O , And conversely, there is a part where the luminance value is high. Outer negative seek line NSL _O Is expected to exist outside the edge 206 of the circuit component 200, and normally, a low luminance value should be continuous at each recognition base. However, as shown in FIG. 27, foreign matter 256 adheres to a part of the edge 206, and the outer negative seek line NSL _O However, the luminance value becomes higher in the portion passing through the foreign matter 256.
[0097]
In S53, a difference between the maximum value and the minimum value of the luminance in one seek line is determined as the deviation of the luminance value, and the difference is estimated as the luminance deviation σf. In subsequent S54, when the estimated luminance deviation σf exceeds a predetermined threshold value (for example, 50), it is determined that the edge shape of the circuit component 200 is defective, and when not, it is determined that the edge shape is good. . In this way, the inspection routine ends, but if it is determined that the edge shape is defective, the mounting operation of the circuit component 200 is stopped.
[0098]
iv) Correspondence between the present invention and the above embodiment
If the above embodiment is associated with the image processing of the present invention, the following is achieved. The step of storing the image data captured by the component camera 32 in the frame grabber memory 164 corresponds to an image data storing step of storing the image data as a set of optical characteristic values for each pixel. In each of the search process to the inspection process, each template is set, and the process of superimposing the template on the image of the circuit component 200 is performed by a site selection process of selecting a plurality of recognition sites on the screen. , And each corresponds to a template use selecting step. In each of the search process to the inspection process, the acquisition of the luminance value at the recognition site performed by executing the image data conversion subroutine corresponds to a site characteristic value acquisition process. Further, in each of the search process to the inspection process, each of the processes of performing the recognition process on the specifications of the circuit component based on the luminance value of the recognition base corresponds to an object recognition process. Specifically, the process of the process performed by executing S13 and the like of the search routine corresponds to the presence position estimation process, and the process of S23 and S33 of the re-search routine and the measurement routine is performed and executed. The step of performing the steps S25 and the like, S35, S36 and the like in the contour position specifying step corresponds to the existence position / dimension shape recognition error. Further, the steps of the processing performed by executing S53, S54 and the like of the inspection routine correspond to the contour shape recognition step.
[0099]
v) Modification of image processing
In the above embodiment, for simplification of the description, the image processing for a circuit component having an extremely simple shape has been described. For example, as shown in FIG. 28A, in a chip component 262 having electrodes 260 at both ends, many of the electrodes 260 are made of metal and imaged with higher brightness than the main body 264. May be performed as image processing. In this case, for example, a search template 266 as shown in FIG. 28B and a measurement template 268 as shown in FIG. 28C can be used.
[0100]
Further, in an electronic component (for example, a small outline package (SOP), a quad flat package (QFP), etc.) in which a plurality of leads extend outside the main body, image processing is performed on the lead portion as an object. Thus, image processing for recognizing the position of the electronic component can be performed. Similarly, in an electronic component such as a ball grid array (BGA) or a chip scale package (CSP), image processing can be performed on a portion such as a solder ball bump or a land.
[0101]
In the case where the plurality of leads 270 extend outside the package 272 and are arranged side by side, as shown in FIG. 29, a negative seek line NSL parallel to the lead 270 of the circuit component having a theoretical shape is provided between the leads ( 29 (a)) or inside the outline of each lead (FIG. 29 (b)), and performing a recognition process along the negative seek line NSL to detect the bending of the lead 270. Can be.
[0102]
In the above embodiment, the image processing of the circuit component is performed. However, the image processing can be performed using the reference mark provided on the circuit board as an object. For example, as shown in FIG. 30A, a reference mark 282 is provided on each of two diagonally opposite corners on the circuit board 280. Each of the reference marks 282 is imaged by the mark camera 100, a positional deviation of each of the reference marks 282 from the theoretical position is detected based on the imaged data, and a deviation of the fixed position of the circuit board is determined based on the detection result. It is calculated. Most of the reference marks 282 attached to the circuit board 280 are circular as shown in FIG. 30B, in which case, a search template 284 as shown in FIG. 30C and shown in FIG. 30D. The recognition process may be performed using such a measurement template 286. In the case of a circular reference mark 282, in the recognition processing of the one reference mark 282, a shift in the rotational angle position cannot be detected, and only a position shift in the X and Y directions is detected.
[0103]
In the above embodiment, various templates are used, and point pairs, seek lines or negative seek lines are set in the various templates. The number of these point pairs and the like, the positional relationship with the theoretical contour, and the like can be various depending on the purpose of the recognition processing and the like.
[Brief description of the drawings]
FIG. 1 is an overall perspective view showing a circuit component mounter that performs a component mounting operation to which image processing of the present invention is applied.
FIG. 2 is a perspective view showing a configuration of a mounting module of the circuit component mounter.
FIG. 3 is a perspective view illustrating a mounting device included in each mounting module.
FIG. 4 is a block diagram related to control of a module control device that controls each mounting module.
FIG. 5 is a block diagram of an image processing unit arranged in each mounting module and performing image processing.
FIG. 6 is a conceptual diagram illustrating an image obtained by capturing a circuit component held by one mounting unit with a component camera.
FIG. 7 is a conceptual diagram showing a relationship between a physical screen, a virtual screen, and an image processing program in image processing according to an embodiment of the present invention.
FIG. 8 is a flowchart of an image data conversion subroutine that forms part of the image processing program according to the present embodiment.
FIG. 9 is a conceptual diagram showing a method of performing a complementary operation in image data conversion.
FIG. 10 is a table showing data contents of a correction table for correcting image data.
FIG. 11 is a flowchart of an image processing program according to the present embodiment, which is a recognition processing program for recognizing circuit components.
FIG. 12 is a flowchart of a search routine executed to perform a search step in a recognition processing program.
FIG. 13 is a conceptual diagram showing a search template used in a search step.
FIG. 14 is a diagram exemplifying a matching state and a mismatching state between the search template and the circuit component.
FIG. 15 is a diagram schematically showing how a search template moves.
FIG. 16 is a flowchart of a re-search routine executed to perform a re-search step in the recognition processing program.
FIG. 17 is a conceptual diagram showing a state in which a re-search template used in the re-search step is overlaid on a circuit component.
FIG. 18 is a schematic diagram showing a relationship between one seek line of a re-search template, an edge of a circuit component, and a theoretical contour line.
FIG. 19 is a table showing luminance values of respective recognition points on a seek line obtained by calculation.
FIG. 20 is a table showing a luminance difference between respective recognition points on a seek line.
FIG. 21 is a graph of a luminance difference shown in FIG. 20;
FIG. 22 is a flowchart of a measurement routine executed to perform a measurement step in the recognition processing program.
FIG. 23 is a conceptual diagram showing a measurement template used in a measurement process.
FIG. 24 is a flowchart of an inspection routine executed for performing an inspection step in the recognition processing program.
FIG. 25 is a conceptual diagram of an inspection template used in an inspection process.
FIG. 26 is a line graph schematically showing a change in a luminance value at a recognition site along a negative seek line.
FIG. 27 is a schematic diagram showing a defect existing at an edge and a negative seek line crossing the defect.
FIG. 28 is a conceptual diagram showing an electronic component having electrodes at both ends and a template used for image processing of the electronic component.
FIG. 29 is a conceptual diagram showing a negative seek line used in image processing for detecting lead bending of an electronic component with leads.
FIG. 30 is a conceptual diagram showing a reference mark attached to a circuit board and a template used in image processing using the reference mark as an object.
[Explanation of symbols]
10: Circuit component mounting machine 14: Mounting module 18: Circuit board 26: Mounting head 28: Mounting device 32: Component camera 38: Module control device 100: Mark camera 110: Mounting unit 140: Image processing unit 156: DRAM 164: Frame Grabber memory 200: Circuit component 202: Background part 204: Circuit component of theoretical shape / dimension 206: Edge 210: Physical screen 212: Virtual screen 214: Image processing application program 216: Physical screen / virtual screen conversion driver 218: Look-up table 220 : Search template 222: theoretical contour 230: re-search template 240: measurement template 250: inspection template 260: electrode 262: chip component 270: lead 274: circuit component 280: circuit board 282 : Reference mark 284: Search template 286: Search template 288: Measurement template SP: Reference point forming a point pair SL: Seek line P: Reference point on seek line E: Edge point NSL: Negative sea line

Claims (8)

An image processing method for recognizing a target from image data of a screen including the target obtained by imaging,
An image data storing step of storing the image data as a set of optical characteristic values for each pixel,
A base selection step of selecting a plurality of recognition bases in the screen;
A base characteristic value, which is an optical characteristic value at each of the plurality of recognition bases, is calculated by using an optical characteristic value stored in one or more pixels corresponding to each of the plurality of recognition bases and the optical characteristic value. A site characteristic value acquisition step of acquiring based on characteristic value correction data that is corrected to a value suitable for recognition of the target object,
An object recognition step of recognizing the object based on the acquired base characteristic value of each of the recognition bases. The characteristic value correction data is for correcting the optical characteristic value in a direction in which a contrast difference between an image of the object and an image other than the object adjacent to the image of the object is enlarged. 2. The image processing method according to 1., The said characteristic value correction data correct | amends the said optical characteristic value to a larger value when the value is larger than a set threshold value, and when the value is smaller than the set threshold value, it corrects to the smaller value. Item 3. The image processing method according to Item 2. The location selecting step uses a template having at least one of a plurality of reference points and one or more reference lines associated with a theoretical contour, which is a contour when the object has a theoretical shape and size. And a template use selecting step of superimposing the template on the screen at a set position and selecting a point in the screen defined by at least one of the reference point and the reference line as the plurality of recognition sites. The image processing method according to claim 1. In the template use selecting step, using a template having a plurality of reference point pairs positioned across the theoretical contour line as the plurality of reference points, the plurality of points defined by each of the plurality of reference point pairs is When a plurality of recognition sites are selected, and the object recognition step is performed when the difference state between the base characteristic values of the two recognition sites defined by one reference point pair exceeds a set threshold state, the two recognition sites are recognized. Assuming that one of the bases is in a suitable state belonging to the outline of the object and the other is outside the outline of the object, a set number of a plurality of recognition base pairs defined by a plurality of reference point pairs is set. The image processing method according to claim 4, further comprising a conformity determining step of determining that the object conforms to the template when the above objects are in a conformable state. In the template use selecting step, a template having a plurality of reference intersection lines intersecting with the theoretical contour line is used as the one or more reference lines, and a plurality of points arranged on a line defined by each of the plurality of reference intersection lines. Are selected as the plurality of recognition points, and the object recognition step is based on the characteristic values of each of the plurality of recognition points, and the object on a line defined by each of the plurality of reference intersection lines. 5. The image processing method according to claim 4, further comprising a contour position specifying step of specifying a position of the contour. In the template use selecting step, a template having a reference line extending along the theoretical contour line outside or inside the theoretical contour line as the one or more reference lines is used, and is arranged on a line defined by the reference line. A plurality of points are selected as the plurality of recognition points, and the object recognition step includes a contour shape recognition step of recognizing a contour shape of the object based on each base characteristic value of the plurality of recognition points. The image processing method according to claim 4, comprising: A program executed by a computer to perform image processing for recognizing the target from image data of a screen including the target obtained by imaging,
Image data storage step of storing the image data as a set of optical characteristic values for each pixel,
A base selection step of selecting a plurality of recognition bases in the screen;
A base characteristic value, which is an optical characteristic value at each of the plurality of recognition bases, is calculated by using an optical characteristic value stored in one or more pixels corresponding to each of the plurality of recognition bases and the optical characteristic value. A site characteristic value acquisition step of acquiring based on characteristic value correction data for correcting to a value suitable for recognition of the target,
An object recognition step of recognizing the object based on the acquired base characteristic values of the respective recognition bases.

Images