JP7258259B1 - Alignment system, alignment method and program - Google Patents

Abstract

The alignment system (1) includes an image processing unit (400) for detecting an identification target from a photographed image of a workpiece (10) having the identification target by coarse search and detailed search using different search methods, and image processing. a motion control unit (200) for controlling drive devices (111, 112, 113) for moving the workpiece (10) based on the real coordinates of the object to be identified detected by the unit (400). The detection of identification targets by the image processing unit (400) and the control of the drive devices (111, 112, 113) by the motion control unit (400) are repeatedly executed. The fine search is executed in a narrower search range than the rough search range, and the next search when the identification target is detected by the fine search is executed without the rough search.

Description

The present disclosure relates to alignment systems, alignment methods, and programs.

2. Description of the Related Art In the field of FA (Factory Automation), an alignment technique is used to align the position of a workpiece, which is an object to be controlled, with a reference position. For example, there is a control system that detects a work from an image captured by an imaging device and aligns the position of the work with a reference position (for example, Patent Document 1).

In the actuator control system described in Patent Document 1, a first position detection process for detecting the position of the target object with a first precision and a position of the target object with a higher precision than the first precision are performed with respect to the captured image. A command for driving the actuator is issued based on the data detected by executing both the second position detection process and the second position detection process.

In this actuator control system, the actuator is preliminarily driven based on the intermediate data obtained by the first position detection process, and the actuator is further driven according to the difference between the final data obtained by the second position detection process and the intermediate data. It is explained that this makes it possible to increase the speed of the control because the drive control is preliminarily started before the completion of the computation of the final data.

Japanese Patent Application Laid-Open No. 2020-110916

In the actuator control system disclosed in Patent Document 1, a rough search is used as the first position detection process, and a fine search is used as the second position detection process. A rough search is intended to roughly detect the position of a target object from a wide range of images, and pattern matching is used, for example. Fine search, on the other hand, aims to accurately detect the position of a target object from a narrow-range image, and uses, for example, edge detection.

Since the fine search is based on searching in a narrow range, it is necessary to limit the search range. , the processing time may be long. For this reason, alignment control, in which control instructions are issued based on data detected by executing both rough search and fine search, has a problem that the processing time for the entire control becomes long.

The present disclosure has been made in view of the circumstances described above, and aims to provide an alignment system, an alignment method, and a program that enable high-speed and high-precision positioning.

In order to achieve the above object, the alignment system of the present disclosure includes an image processing unit that detects an identification target from a captured image of a workpiece having the identification target by coarse search and fine search using different search methods; An operation control unit for controlling a driving device for moving a workpiece based on the real coordinates of an identification target detected by the processing unit, and a setting terminal for setting the image processing unit . The detection of the identification target by the image processing unit and the control of the drive device by the motion control unit are repeatedly executed. The fine search is performed in a narrower search range than the coarse search range, and the next fine search is performed without the coarse search when the identification target is detected in the fine search. Coarse search includes pattern matching processing for matching patterns possessed by the workpiece with pattern models registered in advance by the setting terminal, and fine search involves registering pattern models used by the setting terminal in pattern matching processing for coarse search. It is characterized in that it is executed using search logic for detailed search generated based on the shape of the pattern model .

According to the present disclosure, when the identification target is detected by the detailed search, the next search executes the detailed search without performing the rough search, so that the overall processing time can be reduced, and the positioning can be performed at high speed and with high accuracy. becomes possible.

1 is a block diagram showing an example of the overall configuration of an alignment system according to Embodiment 1; FIG. Block diagram showing a configuration example of the motion control unit Block diagram showing a configuration example of an image processing unit Diagram for explaining the alignment control method Diagram for explaining the alignment control method FIG. 2 is a block diagram showing a functional configuration example of the alignment system according to the first embodiment; 4 is a flowchart showing alignment control processing according to the first embodiment; Flowchart showing search processing according to the first embodiment A diagram showing an example of a search range A diagram showing an example of the search angle range Diagram showing synchronization method by transmission path delay measurement FIG. 4 is a diagram for explaining shutter timing control of an imaging device; FIG. 5 is a block diagram showing an example of the functional configuration of an alignment system according to Embodiment 2; 5 is a flowchart showing alignment control processing according to the second embodiment; Flowchart showing search processing according to the second embodiment FIG. 4 is a block diagram showing an example of the functional configuration of an alignment system according to Modification 1; Flowchart showing alignment control processing according to modification 1 Flowchart showing search processing according to modification 1 FIG. 11 is a diagram for explaining prediction of an identification target according to Modification 3; FIG. 11 is a diagram for explaining prediction of an identification target according to Modification 3; Diagram for explaining rough search according to modification 4 FIG. 11 is a diagram for explaining detailed search according to Modification 4; Diagram for explaining rough search according to modification 5 FIG. 11 is a diagram for explaining detailed search according to Modification 5; Flowchart showing search processing according to modification 6

(Embodiment 1)
Below, Embodiment 1 for carrying out the present disclosure will be described in detail with reference to the drawings. The same reference numerals are given to the same or corresponding parts in the drawings.

FIG. 1 is a block diagram showing an overall configuration example of an alignment system 1 according to Embodiment 1. As shown in FIG. The alignment system 1 includes an alignment mechanism 100 that moves a workpiece 10 to be controlled, an operation control unit 200 that controls the alignment mechanism 100, an imaging device 300 that captures an image of the workpiece 10, and processes images captured by the imaging device 300. and a setting terminal 500 for performing various settings used for processing in the image processing unit 400 .

Driving devices 111 , 112 , and 113 are connected to the alignment mechanism 100 for applying driving force in each direction to move the workpiece 10 . Further, drive control devices 121, 122 and 123 are connected to the drive devices 111, 112 and 113, respectively. Devices 111, 112, and 113 are driven.

The drive control devices 121, 122, 123, the motion control unit 200, the image processing unit 400, and the imaging device 300 are connected for communication with each other. The communication means may be any conventional communication means such as Ethernet (registered trademark), CameraLink, CoaxPress (registered trademark), USB (Universal Serial Bus, registered trademark). Ethernet-based industrial networks such as /field™, CC-Link IE/TSN™, etc. are suitable.

The alignment mechanism 100 has a mounting table 101 on which the workpiece 10 is mounted, and a mechanism for moving the mounting table 101 . For example, the mounting table can be translated in X and Y directions that are orthogonal to each other in the horizontal direction, and can be rotated in the θ direction, which is the rotational direction on the horizontal plane. In this embodiment, the driving device 111 coupled to the alignment mechanism 100 translates in the X direction, the driving device 112 translates in the Y direction, and the driving device 113 rotates in the θ direction.

The drive devices 111, 112, and 113 are arbitrary drive devices (actuators) capable of precisely driving the alignment mechanism 100, such as servo motors. The drive control devices 121, 122, 123 are control devices, such as servo amplifiers, that control the driving of the drive devices 111, 112, 113, respectively, based on control signals from the motion control unit 200. FIG.

The drive devices 111, 112, and 113 have arbitrary position sensors inside or outside, and detect and output positions actually changed by driving the drive devices 111, 112, and 113. FIG. A position sensor is, for example, an encoder attached to the actuator. The encoder output signals are input to the motion control unit 200 via the drive control devices 121 , 122 and 123 . In this embodiment, a case will be described in which the drive devices 111, 112, 113 are servo motors, the drive control devices 121, 122, 123 are servo amplifiers, and the position sensors are encoders.

The motion control unit 200 is a motion controller that issues commands relating to the motion of the drive devices 111, 112, and 113 to the drive control devices 121, 122, and 123, and includes, for example, a PLC (Programmable Logic Controller). The motion control unit 200 generates control signals indicating commands based on the information acquired from the drive control devices 121 , 122 , 123 and the image processing unit 400 and outputs them to the drive control devices 121 , 122 , 123 .

The operation control unit 200 comprises a processor 210, a volatile memory 220, a non-volatile memory 230, a clock 240, and a communication interface 250, as shown in FIG. Processor 210, volatile memory 220, nonvolatile memory 230, clock 240, and communication interface 250 are communicatively connected to each other via bus B1.

The processor 210 is, for example, a CPU (Central Processing Unit), and reads and executes a control program 231 stored in a nonvolatile memory 230 to generate a position information generator 211, a movement amount calculator 212, a command It functions as part 213 .

A position information generating unit 211 of the processor 210 generates position information corresponding to the position of the mounting table 101 on which the workpiece 10 is mounted based on the output signals of the position sensors of the driving devices 111 , 112 and 113 . The movement amount calculator 212 calculates the movement amount of the work 10 in the X direction, Y direction and θ direction based on the position of the identification target detected from the captured image by the image processing unit 400 . The command unit 213 outputs control signals based on the movement amount calculated by the movement amount calculation unit 212 to the drive control devices 121 , 122 and 123 .

The volatile memory 220 is a work memory from which data can be read and written at high speed during arithmetic processing executed by the processor 210, and is, for example, a RAM (Random Access Memory). The nonvolatile memory 230 stores a control program 231 for realizing various functions of the operation control unit 200, and control data 232 including parameters used when executing the control program 231, past detection data, and command data. do. The nonvolatile memory 230 is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a nonvolatile semiconductor memory such as a flash memory, a magnetic disk, or an optical disk.

The clock 240 measures the local time by counting the clock signals of the clock elements of the operation control unit 200, and synchronizes with the drive control devices 121, 122, 123, the imaging device 300, and the image processing unit 400. . Accordingly, the motion control unit 200 has time information synchronized with the drive control devices 121 , 122 , 123 , the imaging device 300 and the image processing unit 400 .

The communication interface 250 is an interface for the operation control unit 200 to communicate with the drive control devices 121, 122, 123, the imaging device 300, and the image processing unit 400. For example, CC-Link IE/field, CC-Link IE/ It is an interface conforming to communication standards such as TSN.

The imaging device 300 is an imaging device that captures images of the workpiece 10 from above the alignment mechanism 100 at regular time intervals, and is, for example, a camera that has sufficient resolution to realize the alignment accuracy of the alignment system 1 . Although the number of imaging devices 300 is arbitrary, the number of imaging devices 300 is determined according to the number and positions of identification targets used for alignment. FIG. 1 shows a case in which two imaging devices 300 are provided. The object to be identified may be any object that indicates the position of the work 10 , such as an alignment mark provided on the work 10 , a corner end of the work 10 , or a corner end of the mounting table 101 .

The imaging device 300 also has time information synchronized with the drive control devices 121, 122, and 123, the motion control unit 200, and the image processing unit 400, and the drive control devices 121, 122, and 123, the motion control unit 200, and the image processing unit 400. and a communication interface for communicating.

The image processing unit 400 performs a process of detecting an identification target on the captured image acquired from the imaging device 300 by coarse search and fine search using different search methods. A coarse search is performed in the entire range of the captured image acquired from the imaging device 300, and a fine search is performed in a narrower search range than the range of the coarse search. The image processing unit 400 outputs the real coordinates of the identification target to the operation control unit 200 when the identification target is detected by the detailed search. Here, the actual coordinates are coordinates in reference coordinates that are consistent with the movement amount calculation section 212 of the motion control unit 200 .

The image processing unit 400 includes a processor 410, a volatile memory 420, a nonvolatile memory 430, a clock 440, and a communication interface 450, as shown in FIG. Processor 410, volatile memory 420, nonvolatile memory 430, clock 440, and communication interface 450 are communicatively connected to each other via bus B2.

The processor 410 is, for example, a CPU (Central Processing Unit), and by reading and executing a control program 431 stored in a nonvolatile memory 430, an image acquisition unit 411, a range determination unit 412, and a search unit 413. function as

The image acquisition unit 411 of the processor 410 acquires the image captured by the imaging device 300 . The range determination unit 412 determines a range for searching for an identification target in the captured image acquired by the image acquisition unit 411 based on information including position information generated by the position information generation unit 211 of the operation control unit 200 . The search unit 413 performs a detailed search for the identification target in the image within the search range determined by the range determination unit 412, and outputs the real coordinates of the identification target to the operation control unit 200 when the identification target is detected. do.

The volatile memory 420 is a work memory from which data can be read and written at high speed during arithmetic processing executed by the processor 410, and is, for example, a RAM (Random Access Memory). The nonvolatile memory 430 stores a control program 431 for realizing various functions of the image processing unit 400, and control data 432 including parameters used when executing the control program 431 and past detection data. The nonvolatile memory 430 is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a nonvolatile semiconductor memory such as a flash memory, a magnetic disk, or an optical disk.

The clock 440 measures the local time by counting the clock signal of the clock element of the image processing unit 400, and synchronizes with the drive control devices 121, 122, 123, the operation control unit 200, and the imaging device 300. . Accordingly, the image processing unit 400 has time information synchronized with the drive control devices 121 , 122 , 123 , the motion control unit 200 and the imaging device 300 .

The communication interface 450 is an interface for the image processing unit 400 to communicate with the drive control devices 121, 122, 123, the motion control unit 200, and the imaging device 300. For example, CC-Link IE/field, CC-Link IE/ It is an interface conforming to communication standards such as TSN.

The setting terminal 500 is a terminal installed with an application program corresponding to the control program 431 of the image processing unit 400, and is, for example, a personal computer. The setting terminal 500 has management functions for the image processing unit 400 including inputting or changing parameters stored in the non-volatile memory 430 of the image processing unit 400 . A communication interface for communication connection between the setting terminal 500 and the image processing unit 400 is an arbitrary interface corresponding to the interface of the image processing unit 400, such as a USB interface or an RS232C interface.

The motion control unit 200 calculates the amount of movement based on the difference between the actual coordinates of the object to be identified detected by the image processing unit 400 and the target coordinates of the alignment destination, and outputs control signals based on the amount of movement to the drive control devices 121, 122, . 123.

The operation of the alignment system 1 having the configuration described above will be described. First, an example of an alignment method based on the position of an identification target, which is a premise, will be described with reference to FIGS. 4 and 5 are diagrams for explaining an alignment method when the objects to be identified are the alignment marks 1001 and 1002 attached to the end of the workpiece 10. FIG.

Although the shape of the alignment marks 1001 and 1002 is arbitrary, it is preferable to use a shape that allows the positions and rotation angles of the reference points to be clearly determined. For example, the cross marks shown in FIGS. Moreover, the number of alignment marks is arbitrary, and may be one as shown in FIG. 4. As shown in FIG. good too.

When one alignment mark 1001 is used, one imaging device 300 acquires an image of the imaging range 1300 . When two alignment marks 1001 and 1002 are used, and when the two alignment marks 1001 and 1002 cannot be imaged in one imaging range 1300, images of the imaging ranges 1300 shifted from each other are acquired by the two imaging devices 300. do.

When one alignment mark 1001 is used as shown in FIG. uses three pieces of information of the angle difference between .

On the other hand, when two alignment marks 1001 and 1002 are used as shown in FIG. Three pieces of information are used: the coordinates and the angle difference between the straight line connecting the alignment marks 1101 and 1102 and the straight line connecting the alignment marks 1001 and 1002 .

In both cases of FIGS. 4 and 5, the angular difference Δθ is the amount of movement of the θ-axis. Therefore, as shown in FIG. 4, the motion control unit 200 obtains the virtual coordinates of the reference point of the virtual mark 1201 after the work 10 is rotated by the angle .DELTA..theta. After that, the amount of movement ΔX in the X-axis direction and the amount of movement ΔY in the Y-axis direction are calculated from the difference between the virtual coordinates and the target coordinates of the alignment target mark 1101 . The command section 213 of the motion control unit 200 transmits control signals for realizing the movements of Δθ, ΔX, and ΔY thus obtained to the drive control devices 121, 122, and 123, thereby issuing commands.

As shown in FIG. 4, the method of using one alignment mark 1001 can be used when the rotation of the alignment mark 1001 can be clearly determined like a cross mark. On the other hand, as shown in FIG. 5, when two alignment marks 1001 and 1002 are used, it is not necessary to detect the rotation of the alignment marks themselves. can. Further, since the angle difference Δθ in the θ direction is calculated by detecting two points sufficiently separated from each other, highly accurate control can be performed.

In this manner, the motion control unit 200 calculates Δθ, ΔX, and ΔY based on the coordinates of the reference points of the alignment marks 1001 and 1002 to be identified and the angular difference, and the drive control unit 121 uses these values to , 122 and 123 control the drive devices 111 , 112 and 113 to move the workpiece 10 . Normally, however, the difference from the target coordinates often does not fall within the allowable range at one time, so the detection of the identification target and the control of the drive devices 111, 112, 113 are repeatedly executed.

In such repetitive processing, the burden of processing to search for an identification target from an image captured by the imaging device 300 is very heavy. Therefore, the alignment system 1 according to this embodiment reduces the processing load by limiting the range in which the search is executed.

Detailed processing will be described below with reference to FIGS. FIG. 6 is a block diagram showing a functional configuration example of the alignment system 1 according to this embodiment, FIG. 7 is a flowchart of alignment control processing executed by the motion control unit 200, and FIG. 8 is an image processing unit. 4 is a flowchart of a search process performed by 400;

First, the motion control unit 200 issues a command to move the workpiece 10 to a preset approximate target position (FIG. 7, step S101). Specifically, the motion control unit 200 outputs control signals to the drive control devices 121, 122, and 123 to realize movement in each direction. Then, the drive devices 111, 112, and 113 are driven by the control of the drive control devices 121, 122, and 123 based on the control signals, and the mounting table 101 on which the workpiece 10 is mounted is moved.

After the movement is completed, the motion control unit 200 instructs the image processing unit 400 to acquire and search for captured images (step S102), and acquires position information based on the outputs of the position sensors of the drive devices 111, 112, and 113. is generated and output (step S103). After that, the operation control unit 200 waits until the image processing unit 400 finishes searching (step S104).

The image processing unit 400 instructed by the operation control unit 200 to search for an identification target in step S102 executes the processing shown in FIG. Since the imaging is the first time here (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 acquires the captured image captured by the imaging device 300 (step S202). After that, the search unit 413 performs a rough search of the identification target (step S203).

The rough search is performed, for example, by pattern matching processing using a pre-registered pattern model of the alignment mark 1001 . The search unit 413 detects the alignment mark 1001 when the match rate of pattern matching is equal to or higher than a predetermined threshold value, and determines the search range of the detailed search based on the position or rotation angle of the reference point of the alignment mark 1001. (step S204). The shape or size of the search range at this time is set in advance according to the shape or size of the alignment mark 1001 , and may be set by the user's input to the setting terminal 500 .

For example, in the case of the cross-shaped alignment mark 1001 shown in FIG. 9A, the position of the center, which is the reference point, and the rotation angle of the cross are specified by pattern matching, and the size of the alignment mark 1001 is increased by a predetermined margin. is added to determine a search range 1301 . Also, the search range (search angle range) related to the angles shown in FIG. 9B is determined.

Next, the search unit 413 executes a detailed search (fine search) on the images within the search range determined in step S204 (step S205). As the detailed search, for example, edge detection is performed to detect an accurate straight line or curve, thereby obtaining more accurate real coordinates of the identification target. Here, the real coordinates of the identification target include the position (XY coordinates) and the rotation angle (θ coordinates) of the reference point of the identification target.

If the search unit 413 cannot detect the identification target in the detailed search of step S205 (step S212: No), the process returns to step S201 and repeats the processes of steps S202 to S205. On the other hand, if the identification target is successfully detected in the detailed search of step S205 (step S212: Yes), the real coordinates of the identification target acquired in step S205 are output to the operation control unit 200 (step S213).

Returning to the flowchart of FIG. 7, after the search processing by the image processing unit 400 is completed (step S104: Yes), the motion control unit 200 acquires the actual coordinates from the image processing unit 400 (step S105), Calculate the difference between If the difference between the actual coordinates and the target coordinates is equal to or less than the threshold (step S106: Yes), the alignment control process is terminated. Here, the target coordinates are the coordinates of the alignment destination, and include the center position (XY coordinates) and the rotation angle (θ coordinates).

When the difference between the actual coordinates and the target coordinates exceeds the threshold (step S106: No), the command unit 213 executes a command to correct the position of the alignment mechanism 100 (step S107; command step). Specifically, it outputs a control signal to drive control devices 121 , 122 , and 123 to match the actual coordinates with the target coordinates.

After the drive devices 111, 112, and 113 have been driven, the process returns to step S102. The motion control unit 200 again instructs imaging and searching (step S102), and outputs position information (step S103).

Returning to the flowchart of FIG. 8 again, since the imaging is the second time or later and detection was successful last time (step S201: No), the image processing unit 400 acquires position information (step S206). The range determination unit 412 of the image processing unit 400 calculates predicted coordinates of the identification target at that time (step S207).

Specifically, the range determination unit 412 identifies the correspondence relationship between the position information acquired from the motion control unit 200 during the initial alignment control and the actual coordinates detected in the initial detailed search. For example, when the XY coordinates of the initial position information are represented by (x ₁ , y ₁ ) and the XY coordinates of the actual coordinates are represented by (X ₁ , Y ₁ ), the relationship between them is given by the following equation (1 ) is represented by a constant matrix A.

Let (x ₂ , y ₂ ) be the XY coordinates of the positional _information acquired from the motion control unit ₂₀₀ during the second alignment control. Using the constant matrix A represented by (1), it can be represented by the following equation (2). Also, the n-th time including the third and subsequent times can be similarly represented by Equation (3).

Note that the constant matrix A representing the correspondence relationship between the position information and the real coordinates may be updated each time based on the position information and the real coordinates acquired in repeated alignment control, or may be averaged multiple times. good. Alternatively, a correspondence relationship between position information and actual coordinates may be constructed in advance by prior calibration. For example, in the case of alignment control using a corner end portion of the mounting table 101 as an identification target, since the error for each execution is small, such pre-calibration is effective.

In this way, the correspondence relationship between the XY coordinates of the reference point of the identification target detected from the captured image and the position information is specified in advance, and the XY coordinates are predicted from the newly acquired position information using this correspondence relationship (step S207). After that, a search range having a predetermined size is determined centering on the predicted XY coordinates (step S208). Similarly, with respect to the θ coordinate, the correspondence relationship between the actual coordinates detected in the detailed search and the position information is specified in advance, and then the θ coordinate indicating the rotation angle of the identification target is determined based on the newly acquired position information. Prediction is performed (step S207), and a search angle range having a predetermined angle width centered on the predicted .theta. coordinate is determined (step S208). Note that the search range may not be a range centered on the predicted coordinates, and the range including the predicted coordinates may be set as the search range depending on the alignment conditions or the form of the identification target.

Here, the size of the search range may be set by the user's input to the setting terminal 500, or may be set automatically. For example, the size of the search range may be a size obtained by adding a margin set automatically or manually based on the moving speed of the drive devices 111, 112, and 113 to the shape or size of the object to be identified. good. Alternatively, the size of the search range may be determined statistically based on past alignment control results. The size may be obtained by adding, as a margin, the average value of the difference between the predicted coordinates obtained from the calculation and the actual coordinates detected from the captured image.

In other words, if the second or later detection has not failed the previous time, the range determination unit 412 does not execute the rough search in step S203, and calculates the predicted coordinates calculated from the position information based on the position sensor output in step S207. Determines the search range centered at . The processing for determining the search range based on this positional information has a much smaller amount of processing than the processing for determining the search range by the rough search executed in step S203. Therefore, high-speed control is possible compared to conventional alignment control that starts from rough search each time.

Next, the image acquisition unit 411 acquires an image (step S209; image acquisition step), and performs simple processing on the image in the search range determined in step S208 among the acquired images (step S210). Simplified processing is arbitrary processing before performing a detailed search. For example, the search unit 413 performs a rough search in a range wider than the search range determined in step S208, but narrower than the entire range, and repeats the search range. may decide. Alternatively, if the predicted coordinates of the XY coordinates have been calculated in step S207 and the search range in the XY coordinates has been determined in step S208, rough search of the θ coordinates is performed as simple processing to determine the search angle range. good too. Note that the simplified processing may be omitted.

After that, the search unit 413 executes a detailed search for the search range determined in step S208 or the range determined in step S210 (step S211: search step). The detailed search method is the same as in step S205. As a result of executing the detailed search in step S211, if the identification target could not be detected (step S212: No), the process returns to step S201. Execute. Here, the reason why the coarse search of the entire range is restarted is to avoid loss of control due to repeated detection failures in the detailed search.

If the identification target is successfully detected in the detailed search in step S211 (step S212: Yes), the real coordinates of the identification target acquired in step S211 are output to the motion control unit 200. FIG.

Returning to the flowchart of FIG. 7 again, the motion control unit 200 acquires the actual coordinates from the image processing unit 400 (step S105), and calculates the difference from the target coordinates. If the difference between the actual coordinates and the target coordinates is equal to or less than the threshold (step S106: Yes), the alignment process is terminated.

If the difference between the actual coordinates and the target coordinates exceeds the threshold (step S106: No), the command unit 213 executes a command to correct the position of the alignment mechanism 100 (step S107). Specifically, it outputs a control signal to the drive control devices 121 , 122 , 123 to match the actual coordinates with the target coordinates. After that, the process returns to step S102 to continue the process.

As described above, the alignment system 1 according to the present embodiment includes the image processing unit 400 for detecting an identification target from an image of the workpiece 10 having the identification target, and the real coordinates of the identification target detected by the image processing unit 400. and an operation control unit for controlling the drive devices 111, 112, 113 for moving the workpiece 10 based on the following. The image processing unit 400 identifies in advance the correspondence relationship between the position information based on the position sensor outputs of the driving devices 111, 112, and 113 and the real coordinates of the identification target detected from the captured image, and uses this correspondence relationship to perform the following: The predicted coordinates of the identification target are calculated from the acquired position information, the search range including the predicted coordinates is determined, and the identification target is detected from the image of the search range. As a result, it is possible to omit rough search processing for the entire range and perform positioning with high speed and high accuracy.

(Embodiment 2)
Below, Embodiment 2 for carrying out the present disclosure will be described in detail with reference to the drawings. The same reference numerals are given to the same or corresponding parts in the drawings.

The overall configuration of the alignment system 2 according to the second embodiment and the hardware configuration of each component are the same as those of the first embodiment. In the alignment system 2 according to the second embodiment, the image acquisition unit 411 of the image processing unit 400 acquires a captured image from the imaging device 300 by issuing a transfer instruction specifying a transfer range, and obtains a captured image within the transfer range. differs from the first embodiment in that the search unit 413 performs search processing for .

Also in this embodiment, as in the first embodiment, the drive control devices 121, 122, 123, the motion control unit 200, the image processing unit 400, and the imaging device 300 are connected for communication with each other. However, since this embodiment requires higher responsiveness than the first embodiment, it is desirable to establish a communication connection through an industrial network such as CC-Link IE/field, CC-Link IE/TSN.

In industrial networks such as CC-Link IE/field and CC-Link IE/TSN, in order to ensure synchronism, while achieving a certain or more punctuality in H / W equipment, the transmission path delay is statistically It has a mechanism that performs measurement and synchronizes the timing between devices with microsecond accuracy.

A synchronization method based on transmission path delay measurement is a method that uses transmission path delay measurement values from a master station to a device station to achieve higher-precision synchronization. FIG. 10 shows a synchronization method based on transmission line delay measurement. In the transmission path delay measurement method, synchronization is performed at synchronization points. A transmission control (MyStatus) frame transmitted from the master station 40 is delayed as the distance increases.

The master station 40 calculates the transmission path delay time in each device station 50, 60, 70 from the master station time when the response signal was received from each device station 50, 60, 70, and transmits the transmission path delay time to each device station 50, 60, 70. do. The synchronization point is the time after a certain time (Tsync) has passed since the master station 40 transmitted the transmission control (MyStatus) frame. Each device station 50, 60, 70 synchronizes after Tps time (Tsync-delay time), which is the time obtained by subtracting the transmission path delay time, after receiving the transmission control (MyStatus) frame.

In this embodiment, using this transmission path delay measurement method, for example, with the operation control unit 200 as the master station 40, the timings of the drive control devices 121, 122, 123, the operation control unit 200, the imaging device 300, and the image processing unit 400 are calculated. control is performed. That is, the drive control devices 121, 122, 123, the motion control unit 200, the imaging device 300, and the image processing unit 400 are connected to communicate with each other via an industrial network (communication line) in which the transmission line delay time is measured, and are synchronized. time information.

The operation control unit 200, which is the master station 40, sets a specific time after the longest transmission path delay time has passed since the transmission time of the instruction as a synchronization point. Then, the image processing unit 400 determines a range including predicted coordinates at a specific time in the future as a transfer range. and instruct.

Furthermore, it is necessary to perform timing control in consideration of the shutter speed for the imaging device 300 as well. FIG. 11 is a diagram for explaining timing control of the shutter of the imaging device 300. As shown in FIG. The imaging device 300 automatically determines the imaging timing according to the setting information obtained from the image processing unit 400 or in advance using the synchronized time information that is the source of calculation of the synchronization point. At this time, the image processing unit 400 can issue an earlier image capturing instruction within the grace period as necessary in order to perform image capturing at a specific time in the future. It is also useful to align the timing of reserved imaging with the center of the exposure time range.

FIG. 12 is a block diagram showing a functional configuration example of the alignment system 2 according to this embodiment, FIG. 13 is a flowchart of alignment control processing executed by the motion control unit 200, and FIG. 14 is an image processing unit. 4 is a flowchart of a search process performed by 400; The operation of the alignment system 2 will be described along the flow charts of FIGS.

First, the motion control unit 200 issues a command to move the workpiece 10 to a preset approximate target position (FIG. 13, step S101). Specifically, the motion control unit 200 outputs control signals to the drive control devices 121, 122, and 123 to realize movement in each direction. Then, the drive devices 111, 112, and 113 are driven by the control of the drive control devices 121, 122, and 123 based on the control signals, and the mounting table 101 on which the workpiece 10 is mounted is moved.

After issuing the movement command, the operation control unit 200 determines the imaging time of the imaging device 300 to be time T _n (n=1) (step S 122 ), and designates it to the image processing unit 400 . Further, the position information generator 211 of the motion control unit 200 generates position information based on the outputs of the position sensors included in the driving devices 111, 112, and 113, and outputs the position information (step S103). After that, the operation control unit 200 waits until the image processing unit 400 finishes searching (step S104).

The image processing unit 400 that has acquired the position information from the motion control unit 200 executes the processing shown in FIG. Since the imaging is the first time here (step S201: Yes), the image acquisition unit 411 of the image processing unit 400 instructs the imaging device 300 to perform the imaging at time T _n (n=1) and the captured image of the entire range. is instructed to transfer (step S221). After that, the image acquisition unit 411 acquires the captured image of the entire range (step S222).

Next, the search section 413 of the image processing unit 400 performs a rough search of an identification target on the captured image of the entire range (step S203). The rough search is performed, for example, by pattern matching processing using a pre-registered pattern model of the alignment mark 1001 . The search unit 413 detects the alignment mark 1001 as an identification target when the matching rate of pattern matching is equal to or higher than a predetermined threshold, and performs a detailed search based on the position or rotation angle of the reference point of the alignment mark 1001. A range is determined (step S204). The shape or size of the search range at this time is determined in advance based on the shape or size of the alignment mark 1001 , and may be set by the user's input to the setting terminal 500 .

Next, a detailed search (fine search) is executed for the image within the search range determined in step S204 (step S205). In the detailed search, for example, edge detection is performed to detect an accurate straight line or curve, thereby obtaining more accurate real coordinates of the object to be identified. Here, the real coordinates of the identification target include the position (XY coordinates) and the rotation angle (θ coordinates) of the reference point of the identification target.

If the detailed search in step S205 fails to detect an identification target (step S212: No), the process returns to step S201 and repeats steps S221, S222, and steps S203 to 205. On the other hand, if the identification target is successfully detected in the detailed search of step S205 (step S212: Yes), the real coordinates of the identification target acquired in step S205 are output to the operation control unit 200 (step S213).

Returning to the flowchart of FIG. 13, after the search processing of the image processing unit 400 is completed (step S104: Yes), the motion control unit 200 acquires the actual coordinates from the image processing unit 400 (step S105), and calculates the difference from the target coordinates. Calculate If the difference between the actual coordinates and the target coordinates is equal to or less than the threshold (step S106: Yes), the alignment control process is terminated. Here, the target coordinates are the coordinates of the alignment destination, and include the center position (XY coordinates) and the rotation angle (θ coordinates).

When the difference between the actual coordinates and the target coordinates exceeds the threshold (step S106: No), the command unit 213 executes a command to correct the position of the alignment mechanism 100 (step S107; command step). Specifically, it outputs a control signal to the drive control devices 121 , 122 , 123 to match the actual coordinates with the target coordinates.

After issuing the correction command to the alignment mechanism 100, the motion control unit 200 returns to step S122, determines the imaging time to be time T _n (n=2) (step S122), outputs position information (step S103), and outputs the image. It waits until the search process of the processing unit 400 ends (step S104).

Returning to the flowchart of FIG. 14 again, the next imaging is the second or subsequent imaging, and the previous detection was successful (step S201: _No ). n=2) is instructed (step S223). Also, the image processing unit 400 acquires position information from the operation control unit 200 (step S206). The range determination unit 412 of the image processing unit 400 predicts the coordinates of the identification target at that time (step S207).

The coordinate prediction method is the same as in the first embodiment, and specifies the correspondence relationship between the position information acquired from the motion control unit 200 during the initial alignment control and the actual coordinates detected in the initial detailed search. The range determination unit 412 uses the specified correspondence relationship to predict the XY coordinates of the reference point to be identified from the position information acquired in step S206 (step S207). After that, the range determination unit 412 determines a transfer range having a predetermined size centered on the XY coordinates predicted in step S207, and instructs the imaging device 300 to transfer the captured image with the transfer range specified. . (Step S226). Note that the transfer range may not be a range centered on the predicted XY coordinates, and the transfer range may be a range including the predicted XY coordinates depending on the alignment conditions or the form of the identification object.

Here, the size of the transfer range may be set by the user's input to the setting terminal 500 . For example, since there is a trade-off between robustness and processing speed, the transfer range may be set small when priority is given to throughput by maximizing the average processing speed while allowing fluctuations in processing. On the other hand, if it is desired to make the processing time constant by giving priority to the jitter characteristics over the processing speed, the transfer range may be set large.

Alternatively, the size of the transfer range may be a size obtained by adding a margin set automatically or manually based on the moving speed of the drive devices 111, 112, and 113 to the shape or size of the object to be identified. good. Alternatively, the range determining unit 412 may statistically determine the size of the transfer range based on past alignment control results. and the average value of the difference from the real coordinates of the identification object detected from the image as a margin may be added as the size of the transfer range.

In this way, if the imaging is the second or subsequent time and the previous detection has not failed, the imaging device 300 does not transfer the image of the entire range, and the prediction based on the position information of the position sensor obtained in step S226. Transfer only the image within the transfer range centered on the coordinates. The data volume of the image transferred here is significantly smaller than the data volume of the image of the entire range transferred in step S221. Therefore, compared to the case where the image of the entire range is transferred every time, the data transfer capacity can be reduced, the transfer time can be shortened, and high-speed control is possible because the coarse search of the entire range is not executed.

The image acquisition unit 411 acquires the image in the transfer range that has been transferred by the transfer instruction in step S226 (step S209; image acquisition step), and the search unit 413 performs simple processing on the image within the transfer range ( step S210). Simplified processing is arbitrary processing before detailed search is performed. For example, the search range may be determined by performing coarse search on the image within the transfer range acquired in step S209. In particular, a rough search of the θ coordinate may be performed as a simple process to determine the search angle range. Note that the simplified processing may be omitted.

After that, a detailed search is executed for the image within the transfer range acquired in step S209 or for the search range determined in step S210 (step S211; search step). The detailed search method is the same as in step S205. As a result of executing the detailed search in step S211, if the identification target could not be detected (step S212: No), the process returns to step S201. 205 is executed. Here, the reason why the coarse search of the entire range of the image is restarted is to avoid loss of control due to repeated detection failures in the detailed search.

If the identification target is successfully detected in the detailed search in step S211 (step S212: Yes), the real coordinates of the identification target acquired in step S211 are output to the operation control unit 200 (step S213).

Returning to the flowchart of FIG. 13 again, the motion control unit 200 acquires the actual coordinates from the image processing unit 400 (step S105) and calculates the difference from the target coordinates. If the difference between the actual coordinates and the target coordinates is equal to or less than the threshold (step S106: Yes), the alignment process is terminated.

If the difference between the actual coordinates and the target coordinates exceeds the threshold (step S106: No), the command unit 213 executes a command to correct the position of the alignment mechanism 100 (step S107). Specifically, it outputs a control signal to the drive control devices 121 , 122 , 123 to match the actual coordinates with the target coordinates. After that, the process returns to step S122 to continue the process.

In this manner, the range determination unit 412 of the image processing unit 400 determines the transfer range of the image based on the position information, and the search unit 413 performs detailed search for the image transferred from the imaging device 300 . This can significantly reduce transfer time and search time.

The transfer range from the image processing unit 400 to the imaging device 300 can be specified by specifying the imaging parameters including the transfer range together with the imaging instruction, or by specifying the transfer range in advance for the imaging device 300 and then performing imaging. A trigger may be output to the imaging device 300 .

When specifying the transfer range together with the former imaging instruction, the above-described transmission path delay measurement synchronization method is used to match the timing of imaging by the imaging device 300 and the acquisition of position information, thereby reducing the waiting time. It is possible to search within an appropriate transfer range. On the other hand, in the latter case of specifying the transfer range in advance, it is possible to adjust the imaging timing with high accuracy by outputting only the imaging trigger through the dedicated line.

Further, in the image transfer of the entire range designated in step S221 of FIG. 14 and the image transfer of the transfer range designated in step S226, the image may be reduced by the imaging device 300 and transferred. The reduction method can be any conventional method, for example subsampling or binning is used. Binning, in particular, is relatively light in processing and has the effect of improving the signal-to-noise ratio (S/N ratio) of pixel signals. preferred.

When the transfer range is limited (trimmed) by the transfer instruction in step S226, if the actual coordinates and the target coordinates are separated at an early stage of alignment control, the transfer range is limited and reduced, and the actual coordinates and the target coordinates are reduced. After the distance from the . Also, when reducing the image within the entire range or the transfer range, a model suitable for the reduced data is prepared in advance as a pattern model for search, and the search unit 413 performs a rough search or a detailed search using this model. may be performed.

Further, in parallel with the detailed search (step S211) after acquiring the captured image of the transfer range according to the transfer instruction in step S226, the captured image of the entire range may be acquired. It is possible to reduce the time taken to acquire the captured image of the entire range when the identification target cannot be detected in the detailed search in step S211 (step S212: No). In this case, when the identification target is detected by the detailed search, the transfer of the captured image of the entire range may be stopped or the transferred image may be deleted.

As described above, in the alignment system 2 according to the present embodiment, the image processing unit 400 predetermines the correspondence relationship between the position information of the driving devices 111, 112, and 113 and the real coordinates of the identification target detected from the captured image. Using this correspondence relationship, the predicted coordinates of the identification target are calculated from the position information to be acquired next, the transfer range of the image including the predicted coordinates is determined, and the transfer range transferred from the imaging device 300 is determined. We decided to search the image in detail and detect the object to be identified. As a result, it is possible to reduce the transfer time and the capacity of the image to be transferred, and to omit the rough search of the entire range of the image, so that the alignment control can be performed at high speed and with high accuracy.

(Modification)
Various modifications of the above embodiment are possible. Modifications are described below.

[Modification 1]
In Embodiments 1 and 2, the range determination section 412 of the image processing unit 400 determines the range including the predicted coordinates calculated from the position information based on the position sensor outputs of the drive devices 111, 112, and 113 as the search range or transfer range. I thought I would. However, the search range or transfer range may be determined in other ways. Modification 1 describes another method for determining the search range or the transfer range.

In this modification, command information relating to driving of the driving devices 111, 112, and 113 is used to calculate predicted coordinates for determining the search range or transfer range. Specifically, the range determination unit 412 of the image processing unit calculates the predicted coordinates based on the command information, and determines the search range or transfer range including the predicted coordinates.

In this modified example, timing control of imaging by the imaging device 300, driving of the drive devices 111, 112, and 113 based on the command information, and acquisition of the position sensor output is performed while ensuring synchronism. High-speed control becomes possible. FIG. 15 is a block diagram showing a functional configuration example of the alignment system 3 according to this modification, FIG. 16 is a flowchart of alignment control processing executed by the motion control unit 200 according to this modification, and FIG. 4 is a flowchart of search processing executed by the processing unit 400. FIG.

Here, FIGS. 15 to 17 are modifications of the second embodiment, showing forms in which the transfer range of an image is determined based on command information. A form in which the search range of the image is determined by using the same method can also be realized. The operation of the alignment system 3 according to this modified example will be described below with reference to the flowcharts of FIGS.

First, the motion control unit 200 issues a command to move the workpiece 10 to a preset approximate target position (FIG. 16, step S101). The command information at this time is passed to the image processing unit 400 for use in determining the transfer range. After that, the driving devices 111, 112, and 113 start driving based on the control signal related to the command information, and the mounting table 101 on which the work 10 is mounted moves.

After issuing the movement command, the operation control unit 200 determines the future specific time T _n (n=1) as the imaging time of the imaging device 300 (step S 122 ), and designates it to the image processing unit 400 . Further, the position information generator 211 of the motion control unit 200 designates recording of position information at time _Tn based on the outputs of the position sensors of the drive devices 111, 112, and 113 (step S123). After that, the operation control unit 200 waits until the image processing unit 400 finishes searching (step S104). The subsequent operation of the image processing unit 400 is the same as in the second embodiment, and the processing of steps S221, 222, 203-205, 212, and 213 in FIG. 17 is executed.

Returning to the flowchart of FIG. 16, when the search processing of the image processing unit 400 is completed (step S104: Yes), the movement amount calculator 212 of the motion control unit 200 acquires the actual coordinates from the image processing unit 400 (step S105). ), the position information generator 211 generates and records position information based on the position sensor output at time _Tn (step S124). The movement amount calculation unit 212 calculates the difference between the actual coordinates and the target coordinates, and if the difference between the actual coordinates and the target coordinates is equal to or less than the threshold (step S106: Yes), the alignment control process ends.

When the difference between the actual coordinates and the target coordinates exceeds the threshold (step S106: No), the command unit 213 executes a command to correct the position of the alignment mechanism 100. At this time, the specific time recorded in step S124 The position information in _Tn is referenced, and a command is executed in which the deviation between the position indicated by the command and the actual position indicated by the position information is also compensated (step S125).

After executing the command to the alignment mechanism 100 by the command unit 213, the process returns to step S122, the imaging time is determined to be the time T _n (n=2) (step S122), and the position at the specific time T _n (n=2) is determined (step S122). The recording of information is specified (step S123), and the process waits until the search processing of the image processing unit 400 is completed (step S104).

Returning to the flowchart of FIG. 17 again, the next imaging is the second or subsequent time, and the previous detection was successful (step S201: _No ). n=2) is instructed (step S223). Also, the image processing unit 400 acquires command information at the time _Tn from the operation control unit 200 (step S224). The range determination unit 412 of the image processing unit 400 calculates predicted coordinates of the identification target at that time (step S225).

In the method of calculating the predicted coordinates by the range determination unit 412, first, instead of the position information of the second embodiment, the correspondence relationship between the command information during the initial alignment control and the actual coordinates detected in the initial detailed search is specified. . The range determination unit 412 uses the specified correspondence relationship to calculate the predicted coordinates (XY coordinates) of the reference point to be identified from the command information related to the scheduled position at the specific time _Tn acquired in step S224 (step S225). ). The prediction method using correspondence is the same as in the second embodiment.

After that, the range determination unit 412 determines a transfer range having a predetermined size centered on the XY coordinates predicted in step S225, and instructs the imaging device 300 to transfer an image within the determined transfer range. . (Step S226). Note that the transfer range may not be a range centered on the predicted XY coordinates, and the transfer range may be a range including the predicted XY coordinates depending on the alignment conditions or the form of the identification target.

The image acquisition unit 411 acquires the image of the transfer range transferred by the transfer instruction in step S226 (step S209), and performs simple processing (step S210). After that, the search unit 413 executes a detailed search for the image within the transfer range acquired in step S209 or the image subjected to the simple processing in step S210 (step S211). As a result of executing the detailed search in step S211, if the identification target could not be detected (step S212: No), the process returns to step S201. 205 is executed.

If the identification target is successfully detected in the detailed search in step S211 (step S212: Yes), the real coordinates of the identification target acquired in step S211 are output to the operation control unit 200 (step S213). Subsequent processing by the operation control unit 200 is the same as in the second embodiment.

In this manner, the range determination section 412 of the image processing unit 400 determines the transfer range of the image based on the command information of the operation control unit 200, and the search section 413 performs a detailed search for the image transferred from the imaging device 300. and Accordingly, since the search range or transfer range is determined based on the command information regarding the future planned position, more efficient timing control can be performed, and faster alignment control can be performed.

[Modification 2]
Modification 2 describes still another method for determining the search range or transfer range. In the first and second embodiments, the range determination unit 412 determines the search range or transfer range centered on the predicted coordinates calculated based on the position information. Determine the search range or transfer range to be used.

Specifically, the range determination unit 412 detects the identification target previously detected in the detailed search by the search unit 413 of the image processing unit 400 and stored in the nonvolatile memory 230 without acquiring position information or command information. The range centered on the reference point of is determined as the transfer range. Alternatively, the range determination unit 412 determines a range centered on the stored reference point of the identification target or an angle range centered on the rotation angle of the identification target as the search range. Note that the transfer range or search range may not be a range centered on the previously detected actual coordinates, and depending on the alignment conditions or the form of the identification target, a range including the previously detected actual coordinates may be used as the transfer range or search range. good.

The size of the search range or transfer range may be determined based on the moving speed of the real coordinates of the identification target detected in the past. Alternatively, it may be determined based on the moving speed of each driving device based on the setting parameters of the alignment control. For example, the size may be determined by multiplying the maximum moving speed by the imaging interval of the imaging device 300 . When the moving distance between imaging intervals is short because the imaging interval is set short, the search range or transfer range can be sufficiently limited by using the previous detection result. And compared with Modification 1, the processing can be simplified.

[Modification 3]
Modification 3 describes yet another method for determining the search range or transfer range. In this modification, the range determination unit 412 determines the search range or the transfer range based on the trajectory of the actual coordinates detected in the past by the detailed search by the search unit 413 . In other words, without acquiring the position information or command information as in the first and second embodiments, the identification object detected in the past by the detailed search by the search unit 413 of the image processing unit 400 and stored in the nonvolatile memory 230 Calculate the predicted coordinates based on the locus of the actual coordinates, and determine the range centered on the predicted coordinates as the search range or the transfer range.

Specifically, the range determination unit 412 determines a range having a predetermined size centered on the XY coordinates predicted based on the locus of the XY coordinates of the reference point to be identified as the transfer range. Alternatively, the range determining unit 412 may be a range having a predetermined size centered on the XY coordinates predicted based on the locus of the XY coordinates of the reference point to be identified, or a θ coordinate predicted based on the locus of the θ coordinates. is determined as the search range. 18A and 18B are diagrams for explaining prediction of an identification target. Note that the transfer range or search range may not be a range centered on the predicted coordinates, and depending on the alignment conditions or the form of the identification target, a range including the predicted coordinates may be used as the transfer range or search range.

Calculation of the predicted coordinates (XYθ coordinates) is performed by prediction at the time of imaging by the imaging device 300 using two or more times before the previous time and the actual coordinates at each time. For example, as shown in FIG. 18A, the XY coordinates at the time of imaging are predicted by performing linear prediction based on the XY coordinate data for two times, the previous time and the time before last time. Alternatively, as shown in FIG. 18B, the XY coordinates at the time of imaging are predicted by performing secondary prediction based on the XY coordinate data for three times, ie, the previous time, the time before the time before the time, and the time before the time before the time before.

The size of the search range or transfer range may be determined based on the moving speed of the real coordinates of the identification target detected in the past. Alternatively, it may be determined based on the moving speed of the driving devices 111, 112, 113. For example, the size may be determined by multiplying the maximum moving speed by the imaging interval of the imaging device 300 . Linear prediction is less accurate than quadratic prediction or even higher-order prediction, so the size of the search range or transfer range must be increased.

Also, the size of the search range or transfer range may be determined based on statistical information of errors with respect to past detection results. For example, for the shape and size of the identification target, the average value of the difference between the predicted coordinates based on the trajectory of the real coordinates and the real coordinates detected from the captured image at corresponding points in the past is added as a margin. It can be any size.

According to this modification, when the imaging interval is long and the movement distance between the imaging intervals is long, the prediction accuracy is improved, and the search range or transfer range can be narrowed.

[Modification 4]
In the first and second embodiments, the alignment marks 1001 and 1002 provided on the workpiece 10 have been described as objects to be identified, but the processing may be partially changed according to the form of the object to be identified. FIG. 19A is a diagram for explaining coarse search according to this modification, and FIG. 19B is a diagram for explaining fine search according to this modification.

For example, when the work 12 is an IC (Integrated Circuit) having many terminals as shown in FIG. A rough search is performed by performing processing ( FIG. 8 , step S203).

At this time, the search unit 413 determines the search range 1314 for the detailed search based on the position or rotation angle of the pattern 1313 when the match rate of pattern matching is equal to or greater than a predetermined threshold (FIG. 8, step S204). ). In this pattern matching, as shown in FIG. 19A, there are cases where a plurality of patterns with matching rates equal to or higher than a predetermined threshold are detected. , may determine the search range 1314 .

In the example shown in FIG. 19B, the search unit 413 determines a search range 1314 having a predetermined positional relationship from the pattern 1313 specified by the rough search as the search range for the fine search (FIG. 8, step S204). The search unit 413 executes a detailed search for the search range determined in step S204 (FIG. 8, step S205).

Here, in Embodiments 1 and 2, in the second and subsequent imaging, if the previous detection was successful (FIG. 8, step S201: No), the search including the predicted coordinates calculated from the position information A range is determined (FIG. 8, step S208), and a detailed search is executed (FIG. 8, step S211). However, if it is possible to calculate the predicted coordinates with a certain degree of accuracy like the IC shown in FIG. The finer search may be repeated. Then, only when the detection by the fine search fails, the process of determining the search range by the rough search may be performed.

According to this modified example, it is possible to simplify the processing according to the form of the object to be identified. Note that the application of the setting terminal 500 may classify the form to be identified and automatically select a search processing method suitable for the classification.

[Modification 5]
Modification 5 describes a case where the identification target has a further different form. FIG. 20A is a diagram for explaining coarse search according to this modification, and FIG. 20B is a diagram for explaining fine search according to this modification. In this modified example, the processor of the setting terminal 500 executes a dedicated application program to automatically perform coarse search model registration and detailed search logic generation.

For example, when the workpiece 13 is a lens as shown in FIG. 20A, the processor of the setting terminal 500 extracts the outer shape of the lens from the captured image and registers the pattern model 1322 . Further, the processor automatically generates search logic for the fine search based on the shape of the pattern model 1322 . For example, when the processor of the setting terminal 500 detects that the shape of the pattern model 1322 includes an arc, as shown in FIG. and automatically generate logic for detecting the center of the circle as the reference point 1323 .

The image processing unit 400 executes the search processing described in the first and second embodiments using the coarse search pattern model and the fine search logic generated by the application of the setting terminal 500 . According to this modified example, it is possible to perform an optimum coarse search and fine search according to the form of the object to be identified. In addition, it is possible to automate the preliminary setting for executing the search processing of the image processing unit 400, and it is possible to reduce the user's burden of setting input to the setting terminal 500. FIG.

[Modification 6]
In Embodiments 1 and 2, a coarse search is performed on the first captured image, a fine search is performed in a search range determined based on the result of the coarse search, and the next fine search is performed based on the result of the fine search. Compute predicted coordinates of the object to be identified for determining the range or transfer range. However, if the prediction accuracy of the coordinates to be identified is sufficiently high, preliminary calibration of the alignment control may omit the rough search for the first captured image. In particular, in the case of alignment control using the corner end of the mounting table 101 on which the work 10 is mounted, since the error for each execution is small, such pre-calibration is effective.

FIG. 21 is a flowchart of search processing when rough search for the first captured image is omitted. After the image acquisition unit 411 acquires the first captured image (step S301), the range determination unit 412 calculates the predicted coordinates of the identification target based on the position information using the calibration information in advance (step S302). A search range including the predicted coordinates calculated by the range determination unit 412 is determined (step S303), and a detailed search is performed for the determined search range (step S304).

If the detailed search in step S304 succeeds in detecting the real coordinates of the object to be identified (step S305: Yes), the real coordinates are output to the motion control unit 200 (step S309), and the motion control unit 200 are used to control the drive devices 111 , 112 , 113 . After the work 10 is moved, a detailed search is executed for the same search range as the previous one for the next picked-up image. In this way, when the fine search succeeds in detection, the fine search is repeatedly executed without executing the coarse search.

While repeating the detailed search, only when the detection of the identification target fails (step S305: No), the coarse search is executed (step S306) to determine the search range (step S307), and the detailed search is performed in the determined search range. A search is executed (step S308), and the actual coordinates are output (step S309). As described above, according to this modified example, the detailed search is repeated based on the calibration information in advance, and the rough search is executed only when the identification target cannot be detected. This simplifies the process and enables high-speed alignment control.

It should be noted that the hardware configuration and flowcharts shown in the above embodiments and modifications are examples, and can be arbitrarily changed and modified. For example, in the above embodiments and modifications, the operation control unit 200, the imaging device 300, the image processing unit 400, and the setting terminal 500 are configured independently, but at least two of them may be integrated.

Further, the search processes shown in the above embodiments and modifications may be executed in arbitrary combinations. Alternatively, the setting terminal 500 may be configured to be able to select one of the search processes according to the above-described embodiment and modifications. Alternatively, the image processing unit 400 may automatically select one of the search processes according to the above-described embodiments and modifications according to conditions such as the type of work and the form of an identification target.

Also, the division of each function realized by the processors in the operation control unit 200 and the image processing unit 400 in the above-described embodiment and modification is an example, and the division may be arbitrarily changed. Moreover, each function realized by the processor of the operation control unit 200 or the image processing unit 400 can be realized using a normal computer system without using a dedicated system.

Further, the program for executing the operation of the above embodiment can be stored in a computer-readable CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), MO (Magneto Optical Disc), memory card, or the like. A computer that can implement each function may be configured by storing the program in a recording medium, distributing it, and installing the program in the computer. When each function is shared between an OS (Operating System) and an application, or by cooperation between the OS and an application, only portions other than the OS may be stored in the recording medium.

This disclosure is capable of various embodiments and modifications without departing from the broader spirit and scope of this disclosure. Moreover, the embodiments described above are for explaining the present disclosure, and do not limit the scope of the present disclosure. That is, the scope of the present disclosure is indicated by the claims rather than the embodiments. Various modifications made within the scope of the claims and within the scope of equivalent disclosure are considered to be within the scope of the present disclosure.

Reference Signs List 1,2,3 alignment system, 10,12,13 workpiece, 11 alignment target, 40 master station, 50,60,70 device station, 100 alignment mechanism, 101 mounting table, 111,112,113 drive device, 121,122 , 123 drive control device, 200 operation control unit, 210 processor, 211 position information generation unit, 212 movement amount calculation unit, 213 command unit, 220 volatile memory, 230 nonvolatile memory, 231 control program, 232 control data, 240 clock , 250 communication interface, 300 imaging device, 400 image processing unit, 410 processor, 411 image acquisition unit, 412 range determination unit, 413 search unit, 420 volatile memory, 430 nonvolatile memory, 431 control program, 432 control data, 440 Clock 450 Communication interface 500 Setting terminal 1001, 1002 Alignment mark 1101, 1102 Alignment destination mark 1200 θ-axis rotation center 1201 Virtual mark 1300 Imaging range 1301, 1314 Search range 1312, 1322 Pattern model 1313 Pattern, 1323 reference point.

Claims (10)

an image processing unit that detects an identification target from a captured image of a workpiece having the identification target by coarse search and fine search using different search methods;
a motion control unit that controls a driving device that moves the workpiece based on the real coordinates of the identification target detected by the image processing unit;
a setting terminal for setting the image processing unit,
the detection of the identification target by the image processing unit and the control of the drive device by the operation control unit are repeatedly executed,
the fine search is performed in a search range narrower than the range of the coarse search, and the next search when the identification target is detected in the fine search is performed without the coarse search , and
The rough search includes pattern matching processing for matching patterns of the work with pattern models registered in advance by the setting terminal,
The fine search is executed using search logic for the fine search generated based on the shape of the pattern model when the setting terminal registers the pattern model to be used in the coarse search pattern matching process.
alignment system. When the pattern model used in the coarse search pattern matching process is registered, the setting terminal searches for an edge as logic for the fine search when detecting that the shape of the pattern model includes an arc. generating said search logic for performing circle approximation on the detected edge and detecting the center of the circle as a reference point for said identification target;
The image processing unit performs the fine search using the search logic generated by the setting terminal.
The alignment system of Claim 1. When the identification target is detected by the detailed search, the image processing unit executes the next detailed search for a search range determined based on the real coordinates of the identification target obtained by the detailed search. ,
The alignment system of Claim 1. When the detailed search within the search range determined by the rough search detects the identification target, the image processing unit executes the next detailed search for the search range searched by the fine search.
The alignment system of Claim 1. The image processing unit executes the detailed search within the search range determined by newly executing the rough search in the next search when the real coordinates of the identification target cannot be detected in the detailed search.
The alignment system of Claim 1 . The image processing unit executes the fine search in a search range determined by pre-calibration, and executes the rough search to determine the search range when the identification target cannot be detected in the fine search. performing said deep exploration in
The alignment system of Claim 1 . The rough search determines the range of the fine search based on the position or rotation angle of the pattern for which the matching rate of the pattern matching process is equal to or greater than a predetermined threshold.
The alignment system of Claim 1 . When there are a plurality of patterns whose matching rate in the pattern matching process is equal to or higher than a predetermined threshold, the coarse search is performed based on the position or rotation angle of the pattern with the highest matching rate. determine the range of
The alignment system of Claim 1 . a search step of detecting the identification target by a rough search or a detailed search from a captured image of a workpiece having the identification target;
a command step of commanding a driving device for moving the workpiece based on the real coordinates of the identification target detected in the search step;
The searching step and the commanding step are repeatedly performed,
In the search step, when the identification target can be detected by the fine search, the next search is performed by the fine search without performing the rough search,
The rough search includes pattern matching processing for matching patterns of the workpiece with pre-registered pattern models,
The fine search is performed using search logic generated based on the shape of the pattern model used in the coarse search pattern matching process.
alignment method. In an alignment system that detects an identification target from a captured image of a workpiece having the identification target and performs alignment control of the workpiece based on the detected real coordinates of the identification target, a computer that performs image processing of the captured image is provided. ,
The identification target is detected from the captured image by coarse search or fine search, and when the fine search successfully detects the discrimination target, the next search is performed as a search unit that executes the fine search without performing the coarse search. A program for functioning ,
The rough search includes pattern matching processing for matching patterns of the workpiece with pre-registered pattern models,
The fine search is performed using search logic generated based on the shape of the pattern model used in the coarse search pattern matching process.
program.

Images