JP3322278B2 - 3D object position and orientation detection method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention obtains three-dimensional shape data of a target object by distance measurement using a range sensor or the like in advance.
A 3D object position and orientation detection method that detects the currently unknown position and orientation of the target object using the distance data measured on the assumption that the target object is placed in an unknown posture. It is about.

[0002]

2. Description of the Related Art Generally, three-dimensional data of an object is obtained by a slit light projection method (Nagashima, Sakauchi, Nakajima, "Acquisition, accumulation, and display of three-dimensional image data", Journal of the Institute of Image Electronics Engineers, Vol. 20, No. 4, 1991).
Year pp. 508-522) and so on. As shown in FIG. 14A, the slit light 2 projected from the projection unit 1 is irradiated on the object α, the irradiated light is photographed by the light receiving unit 3, and the portion irradiated with the slit light 2 by the principle of triangulation. The distance of α1 is calculated. Here, by obtaining the distance data while rotating the rotary table 4, all the three-dimensional shape data of the object model α 'is created.

In the case of an object α composed only of a plane as shown in FIG. 14A, a plane is allocated to the obtained distance data, and as shown in FIG. express α '. FIG. 14D shows the relationship between adjacent planes.

Next, when the object α is placed in an unknown posture, the position and posture of the object model α ″ can be detected by the same method as that used to create the data of the object model α ′. Expressing α ″, the relationship between adjacent planes is as shown in FIG.

When the planes are allocated, the correspondence between the plane on the object model α 'and the plane on the object model α ″ can be obtained by paying attention to the shape and area. For example, FIG.
If it is found that the plane S1 and the plane P1 correspond to each other in FIG. 14B and FIG. 14C, and that the plane S2 and the plane P5 correspond to each other, it can be seen from FIG. 14D and FIG. Then, the position and orientation in the plane three-dimensional space allocated on the distance data of the object model α ″ having the unknown posture become clear, and the position and orientation of the object α are calculated.

[0006]

However, the conventional detection method can only be applied to an object α by combining planes. Further, prior to the detection of the position and orientation of the object α, the obtained distance data is Must be reconstructed and the entire shape of the object model α ″ in an unknown posture state must be reconstructed, and the surface of the object model α ′ and the surface of the reconstructed object model α ″ must be correlated. There are problems such as a large processing time required.

Further, as a limitation of the method, a detection method that can be used only for a specific type such as a range sensor that can be used is limited to a combination of only a point sensor or only a line sensor. Cannot be detected. Here, in view of the conventional problem, the present invention measures the three-dimensional shape data of the target object in advance by using a range sensor, and assumes that the target object is placed in an unknown posture state. Using three-dimensional shape data obtained by a three-dimensional distance measuring device by an arbitrary combination such as an area sensor,
It is an object of the present invention to provide a three-dimensional object position / posture detection method capable of detecting a position and a posture of an object having an arbitrary shape as well as an object whose surface is not limited to a plane.

[0008]

The object of the present invention can be attained by adopting the following novel characteristic configuration of the present invention. That is, a first feature of the present invention is a method of detecting the position and orientation of an object based on an object model that is the shape of the object. First, the shape of the target object is measured in advance to create the object model. In this case, an inverted model obtained by inverting the sign of each coordinate axis of the object model is generated, and then, for the target object placed at an unknown position and orientation, the position and the measurement direction are determined in advance. The distance of the target object is observed by an arbitrary number of two or more and an arbitrary type of distance sensor, and subsequently, a point on the inverted model with respect to a point of interest on the object or the object model is matched with the distance sensor observation point. The surface of the inverted model at that time is obtained for each of the distance sensor measurement points, and all common areas are obtained, so that the surface as an arbitrary point on the object model is obtained. Define the position and direction of the point, so to confirm the position and orientation of the object 3
This is a two-dimensional object position and orientation detection method.

A second feature of the present invention is a method for detecting the position and orientation of an object based on an object model that is the shape of the object. First, the shape of the target object is measured in advance to create the object model. In addition, an inverted model obtained by inverting the sign of each coordinate axis of the object model is generated in advance, and then, for the target object placed at an unknown position and orientation, the position and the measurement direction are determined in advance. The distance of the target object is observed by two or more arbitrary number and arbitrary types of distance sensors, and subsequently, the point on the inverted model with respect to the point of interest on the object or the object model matches the distance sensor observation point. The surface of the inverted model at this time is obtained for each of the distance sensor measurement points, and all common areas are obtained to obtain any one point on the object model. Define the position and orientation of the interest point,
Furthermore, by restricting the existence candidate area of the point of interest as an arbitrary point on the object model by the existence position of the work table on which the object is placed or the jig member for fixing the object, the position of the object And a three-dimensional object position / posture detection method in which the position and posture are determined.

A third feature of the present invention is a method for detecting the position and orientation of an object based on an object model that is the shape of the object. First, the shape of the target object is measured in advance to create the object model. In addition, an inverted model obtained by inverting the sign of each coordinate axis of the object model is generated in advance, and then a cross-sectional contour pattern obtained by cutting the object in advance on a plane horizontal to the worktable is set at a distance from the worktable. In accordance with the above, a plurality of arbitrary numbers and arbitrary types of distances whose positions and measurement directions are determined in advance with respect to the target object placed at an unknown position and orientation at the time of posture detection are continuously determined. After observing the distance of the target object by the sensor, and further performing coordinate transformation on the first measurement coordinate for each point on the cross-sectional contour pattern with respect to the first measurement point, When the cross-sectional contour pattern for the measurement point after the point is referred to, and the value of the referenced pattern indicates the cross-sectional contour, it is regarded as one of position and orientation candidates, and the value of the referred pattern does not indicate the cross-sectional contour. In
The position of the point of interest as an arbitrary point on the object model is obtained by obtaining the cross-sectional contour pattern common area superimposed according to the coordinates of the distance sensor measurement points by repeating the operation of excluding the point from the candidates. The three-dimensional object position and orientation detection method is to determine the position and orientation of the object by determining the position and orientation of the object.

A fourth feature of the present invention is a method for detecting the position and orientation of an object based on an object model that is the shape of the object. First, the shape of the target object is measured in advance to create the object model. In addition, an inverted model obtained by inverting the sign of each coordinate axis of the object model is generated in advance, and then a cross-sectional contour pattern obtained by cutting the object in advance on a plane horizontal to the worktable is set at a distance from the worktable. In accordance with the above, a plurality of arbitrary numbers and arbitrary types of distances whose positions and measurement directions are determined in advance with respect to the target object placed at an unknown position and orientation at the time of posture detection are continuously determined. After observing the distance of the target object by the sensor, and further performing coordinate transformation on the first measurement coordinate for each point on the cross-sectional contour pattern with respect to the first measurement point, When the cross-sectional contour pattern for the measurement point after the point is referred to, and the value of the referenced pattern indicates the cross-sectional contour, it is regarded as one of position and orientation candidates, and the value of the referred pattern does not indicate the cross-sectional contour. In
The point is determined as an arbitrary point on the object model by obtaining a common area of the cross-sectional contour pattern superimposed in accordance with the coordinates of each of the distance sensor measurement points by repeating the operation of excluding from the candidates. Determine the position and direction, and further limit the existence candidate area of the point of interest as an arbitrary point on the object model by the existence position of the work table on which the object is placed or the jig member for fixing the object. This is a three-dimensional object position and orientation detection method in which the position and orientation of the object are determined.

[0012]

According to the present invention, in order to detect the position and orientation of an object based on a model, the object is first placed on a turntable in order to create a target model. An object model is created by performing distance measurement while placing and rotating the table, and then using the obtained object model, an inverted model and a cross-sectional shape pattern of the object are created.

After performing the above processing as preprocessing, a posture detection process is performed when an object having the same shape as the model is placed in an unknown posture. The number and type of sensors used for posture detection can be arbitrarily combined. A plurality of various sensors such as a point sensor and a line sensor are arranged according to the shape of the object and the work target location.

Measurement is performed by each sensor for an object whose posture detection position and posture are unknown. Since the position and orientation of the object itself can be determined by determining the position and direction of an arbitrary point of interest on the object model, the position of the point of interest may be obtained. Posture detection processing is performed by detecting the position data observed by an arbitrary number and type of sensors, and the range in which the object can be present when the work table or the like on which the object is placed contacts the object at any part of the object surface. Is restricted within a limited range.

A common part of the candidate area of the inversion model for each sensor measurement point or a common area of the sectional shape pattern is obtained. By this processing, the position / posture of the object can be obtained by integrating the posture candidates by the sensors. The detection accuracy can be improved by sequentially narrowing the region of the existence candidate or gradually changing the step size of rotation. This process may be used as a rough positioning process of the object, and may be provided with a post-process as a fine mode posture detection method for minimizing a square error of a distance between the model and the observation data.

In the above process flow, in the object position / posture detection method of the present invention, the operation for obtaining the position / posture of the object is to obtain the position / posture of the point of interest on the object.
This operation is an efficient operation for obtaining a common area of the inverted model for each sensor measurement point when the point of interest on the inverted model of the three-dimensional object is positioned at the sensor measurement point.
In addition, it can be performed flexibly without depending on the shape of the object.

More specifically, by determining the position and direction of a point of interest as an arbitrary point on the object model,
That the position and orientation of the object itself can be determined,
Position data observed by an arbitrary number and type of sensors, and the work table on which the object is placed, etc., are areas where the area where the object can exist by contact with the object at any part of the object surface is limited This is a method for detecting the position and orientation of an object based on a model based on the principle of being confined within.

That is, an inverted model obtained by inverting the sign of each coordinate axis of the object model is generated, and the surface of the inverted model when a point on the inverted model with respect to a point of interest on the object coincides with the sensor observation point. Is obtained for each sensor measurement point, and the position and orientation of the object are determined by obtaining all common regions.

Further, in detecting the posture of the three-dimensional object,
When the point of interest on the inverted model is matched with the sensor measurement point, in order to find the common area of the inverted model for each sensor measurement point, a cross-sectional contour pattern obtained by cutting the object in advance on the worktable on a horizontal plane Are prepared according to the distance of the sensor, and at the time of posture detection, in order to obtain a common area of the cross-sectional contour pattern superimposed according to the coordinates of each sensor measurement point, the cross-sectional contour pattern for the first measurement point is determined. After performing coordinate transformation on the measurement coordinates of the first point for each point of, the cross-sectional contour pattern of the second point transition is referred to, and when the value of the referred pattern indicates the cross-sectional contour, it is determined as a position and orientation candidate point. If the value of the referenced pattern does not indicate a cross-sectional contour, the operation of excluding the point from the candidate points is repeated to limit the candidate area, thereby determining the position and orientation of the object. That, due to extremely simpler configuration.

Therefore, by integrating the measurement results of a plurality of sensors without depending on the difference in the shape of the object, the type and the number of the sensors, the position and orientation detection processing of the object can be performed at high speed, and the number of sensors and the observation conditions can be controlled. Even when the position and orientation cannot be limited due to lack, candidates for the position and orientation of the object can be shown as an area.

[0021]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. Here, prior to the description of the present embodiment, the basic principle will be described with reference to the drawings. FIG. 1 (a) (b) (c)
Are the object model diagram, the distance measurement diagram of the object by a plurality of sensors, and the trajectory diagram of the point of interest that may exist with respect to the measuring point of one point sensor. FIGS. 2A and 2B are the inverted model diagram, respectively. FIG. 9 is a diagram illustrating a procedure for detecting a region of interest of interest using an inversion model. In the figure, β is a plane object, β '
Is a plane object model, β ″ is an inverted plane model, 5a to 5c
Is a point sensor.

As shown in FIG. 1 (a), an object plane object β constituted by an arc β1 and three straight lines β2 to β4 is measured by point sensors 5a to 5c, and a plane object model β 'is created based on the measured points. create. The position and orientation of the plane object model β ′ can be represented by three values of coordinates and rotation on a two-dimensional plane. These are three values of x, y coordinates and rotation of an arbitrary point on the planar object model β ′, and for example, a point A shown in FIG. 1A may be selected.

As shown in FIG. 1A, it is assumed that distance data is obtained by three point sensors 5a to 5c (the point sensor measures the coordinate value of only one point corresponding to the plane object β). The plane object model β 'always contacts measurement points K1 to K3 measured by the point sensors 5a to 5c. Assuming that the rotation of the plane object model β ′ is fixed in the direction shown in FIG. 1A, the plane object model β ′ that can be measured by the point sensor 5a has various positions as shown in FIG. I can take it.

At this time, an area L in FIG. 1C is created as a trajectory of the point A on the fixedly rotated plane object model β '. Similarly, the point sensors 5b, 5c
Also, a locus of point A (not shown) is created. These trajectories are the candidate areas L of the existence range of the point A, and the actual position of the point A can be determined by obtaining the intersection of the trajectories of the point A with respect to the point sensors 5a to 5c.

In order to obtain the candidate area L of the point A on the plane object model β 'for the point sensor 5a, the plane object model β' is brought into contact with the measurement point K1 of the point sensor 5a. An arbitrary point on the model β ′ is -in the x direction and -x and y direction in the x and y planes with respect to the surface shape of the planar object model β '.
Draw a shape with y inverted.

For example, the shape of the surface of the plane object model β 'of the two-dimensional object plane object β is represented by y = f (x) (however, it passes through the origin and f (0) = 0), and each of the point sensors 5a to 5c
Let P (xp, yp) be the point measured by, and the point of origin (0, 0) on plane object model β ′ is (xp−x) when the surface of plane object model β ′ is in contact with point P. , Yp-
y), which is a shape obtained by inverting the plane object model β 'point-symmetrically around the origin and further translating it by (xp, yp).

More generally, assuming that a plane object model β ′ is expressed by f (x, y) = 0 on the x, y plane, a point A of interest for posture detection is represented by ( x _A ,
y _A ) . The point A is a point on the plane object model β ′. Here, the measurement points of the respective point sensors 5a to 5c when observing the target plane object β placed at an unknown position are
Let P (x _p , y _p ) . Locus of points A when in contact with all the outer shape of the planar object model beta '(any point of the X0 on contour) to the measurement point P is as follows, indicated as (x _T, y _T).

[0028]

(Equation 1) Since the point X0 satisfies f (x0, y0), the following equation is obtained.

(Equation 2) The above equation shows the outer shape of the inverted plane model β ″ when the point A ′ on the inverted plane model β ″ is translated to the measurement point P.

However, due to the restriction on the sensing direction, the surface range of the target plane object β that can be measured by one point sensor 5a is determined by the normal vector of the target plane object β surface and the direction vector of the point sensors 5a to 5c. The inner product is in a negative range (the direction vector of the point sensor 5a points in the measurement direction with the sensor body as a starting point).

Here, the plane object model β 'in FIG. 1A is inverted in the x and y directions, and the obtained model is shown in FIG.
As shown in FIG. 1A, assuming an inverted plane model β ″, FIG.
The point on the inverted plane model β ″ in FIG. 2A with respect to the point A on the plane object model β ′ in (a) is the point A ′. Thus, on the plane object model β ′ for one point sensor 5a. In order to obtain the trajectory of point A, the point A 'of the inverted plane model β "is made coincident with the measurement point K1 of the point sensor 5a, and the normal direction vector of the surface of the plane object model β' and the direction of the point sensor 5a By drawing a portion of the surface that can be measured according to the relationship with the vector, an area L is obtained as a candidate area for the existence range of the point A shown in FIG.

By performing this for all the point sensors 5a to 5c as shown in FIG. 2B, areas L1 to L3, which are the trajectories of the points A 'corresponding to the point sensors 5a to 5c, are obtained. It is a candidate for the existence range of the point A on the model β ′. The actual position of the point A is a point where these candidates for the existence range match, that is, the regions L1, L2, L3.
And the position of the point A of the planar object model β 'is obtained as the point C1 shown in FIG.

Next, in order to remove the constraint in the rotation direction of the plane target object β, every time the inverted plane model β ″ is rotated by a small amount, the above-described processing for obtaining the existence range of the point A is performed. The orientation of the object model β ′ can be determined, and the following processing is performed in accordance with the rotation angle increment θ when rotating the inverted plane model β ″ in order to improve the processing speed. That is, in FIG. 2B, a hatched portion obtained by rotating the area L1 which is the locus of the point A obtained by one point sensor 5a around the measurement point K1 of the point sensor 5a by θ as shown in FIG. of
The area is a candidate area for the point sensor 5a.
The same applies to the other point sensors 5b and 5c.
In addition, the areas of L2 and L3 are respectively rotated around K2 and K3 by θ.
Areas obtained by rotation are point sensors 5b, 5
c is a candidate area. Thus, three sensors 5
If there is a common range in the candidate area for a, 5b, 5c
For example, the accuracy of detecting the position and orientation of the planar object model β ′ can be improved by reducing the rotation step width θ within that range and obtaining the common existence range again.

In order to determine the unknown position and orientation of a plane object using different types of sensors as shown in FIG. 4A, one point sensor 5a and one line sensor 6a (the plane of the projection light) are used. Measure the coordinate values of a series of points on the line of intersection between the object and the planar object surface)
When F is a plane target object γ and its position and orientation are detected, first, as shown in FIG. 4A, distance measurement of the target plane object γ is performed by the point sensor 5d and the line sensor 6a, and the measurement point K4 is determined. The plane object model γ ′ is created based on the result of the point sequence t1 as an end point and the contact with the target plane object γ at the measurement point K5. Here, it is assumed that the target plane object γ does not rotate.

As shown in FIG. 4B, the point F 'on the inverted plane model γ "with respect to the point F of the plane object model γ' is adjusted to the measurement points K4 and K5, and the point F When the candidate regions L4 and L5 in the existence range are obtained, a trajectory as the candidate region L4 by the line sensor 6a and a trajectory as the candidate region L5 by the point sensor 5d are obtained, and at the intersection of the trajectories as the common regions L4 and L5. A certain point C2 is the position of the point F of the target plane object γ. However, since the locus of the point F with respect to the line sensor 6a as the candidate area L4 has the width indicated by the point sequence t1 in the actual measurement area. E 'from the point D' on the inverted plane model γ "by the length of the point sequence t1.
The range is from the point displaced in the direction to the point E '.

Next, it is assumed that the target plane object γ can rotate.
And, when, for example, the sides DE of a right triangle DEF becomes horizontal, 'as to determine the position of the point D, point D of the inversion plane model gamma "as shown in FIG. 4 (c)' planar object model gamma to The trajectories corresponding to the measurement points K6 and K7 are obtained as the candidate areas L6 and L7 of the point D in the same manner as described above, and the point C3 which is the intersection of the trajectories is obtained as the position of the point D of the target plane object γ. The state of the planar object model γ ′ obtained by is shown by the broken line in FIG.

As a result of measurement by the two types of sensors 5d and 6a shown in FIGS. 4A and 4B, a plurality of posture candidates were obtained by changing the angle condition. It can be said that it is possible to obtain a posture candidate of the object model γ ′.

Further, the plane object γ is moved to the constraint surface 7 such as a worktable.
The case placed above will be described with reference to the drawings.
The target plane object γ is placed on the constraint surface 7 as shown in FIG. However, the side EF of the target plane object γ is the constraint surface 7
Shall be placed in contact with It is assumed that as a result of the measurement by the point sensors 5e and 5f, it is found that the point sensors 5e and 5f come into contact with the target plane object γ at the measurement points K8 and K9. Thus, a plane object model γ ′ is created, and as described above, a candidate area of the existing range of the point F is obtained by the inverted plane model γ ″.

In this case, since the target plane object γ cannot cross the constrained surface 7, an area L8 is obtained as a candidate area of the point F with respect to the measurement point K8 as shown in FIG. An area L9 is obtained as an F candidate area.
The position of the point F on the planar object model γ ′ is determined by the intersection C4, which is a common area between the two candidate areas L8 and L9.
Is required.

If it is known that the side EF of the target plane object γ is in contact with the constraint surface 7, the point F is
Therefore, the intersection of the region L8 or the region L9 with the measurement point K8 or the measurement point 9 in only one direction and the constraint surface 7 can be set as the position of the point C4. Next, FIG.
As shown in FIG. 5C, the target object γ in FIG.
It is assumed that it is in a state of being rotated by 0 degrees. However, the measurement points by the point sensors 5e and 5f are the same points K8 and K9.

As shown in FIG. 5D, the areas L10 and L11 are obtained as candidate areas of the point F for the measurement point K8 from the inverted plane model γ ″, and the area L12 is obtained as a candidate area of the point F for the measurement point K9. Region L11 and region L1
2 intersect at a point C5, but the plane object model γ ′ is at a distance d1 or more from the constraint surface 7 (d1 is the side D of the target plane object γ).
(Length of F), the point C5 does not become the position of the point F of the plane object model γ ′.

FIG. 6 (a) shows the plane object model γ 'when the side DE of the target plane object γ in FIG. 5 (a) is in a state of being parallel to the constraint surface 7 at a certain interval. The candidate areas of the point D are obtained, and the areas L13 and L1 which are the candidate areas of the point D of the plane object model γ 'with respect to the measurement points K8 and K9 in the same manner as described above, as shown in FIG.
The point C6 is obtained as a common area of the four. The dashed line shown in FIG. 6B indicates the obtained attitude of the planar object model γ ′.

[0042]

[Outside 1]

An embodiment applied to the following three-dimensional object position / posture detection method of the present invention based on the procedure for determining the unknown position and posture of the two-dimensional object plane object γ will be described.

Embodiment 1 A first embodiment of the present invention will be described with reference to the drawings. FIG. 7 (a)
(B) is an explanatory view of a measurement perspective view of a three-dimensional object with a line sensor and a point sensor and a procedure of obtaining the position of a point of interest using an inverted model in the present embodiment.

In the figure, δ is an object, δ 'is an object model, 5g
Is a point sensor, 6b is a line sensor, and 8 is a workbench. In this embodiment, a pentahedron D as shown in FIG.
A case will be described in which the position and orientation of the three-dimensional target object δ that is EFGHI is detected using the point sensor 5g and the line sensor 6b.

When the target object δ is three-dimensional, the same operation can be performed as in the case of the two-dimensional target plane object γ. FIG.
Assume that the target object δ is placed on the worktable 8 as shown in FIG. On the other hand, it is assumed that a point sequence t2 having the measurement point K17 by the point sensor 5g and the measurement point K18 by the line sensor 6b as end points is obtained. Based on this, an object model δ ′ is created.

As shown in FIG. 7B, the point D 'on the inverted model δ "with respect to the point D on the object model δ' is
17 and the measurement point K18, the candidate area of the existence range of the point D on the object model δ 'with respect to the measurement point K17 is the plane D'H'G' on the inversion model δ "with respect to the plane DHG. The candidate area of the existence range of the point D on the object model δ ′ with respect to K18 becomes the plane F′G′H′I ′ on the inverted model δ ″ with respect to the plane FGHI, and the intersection of the two planes is on the object model δ ′. Is a candidate area for the point D.

However, since the target object δ is placed on the worktable 8, the point D exists on the worktable 8, and as shown in FIG. 7B, the candidate area for the measurement point K17 is The candidate area for the sensor area t2 having the area L19 and the measurement point K18 as end points is the area L18, and the intersection C8 of the two areas L18 and L19 is the position where the upper point D of the object model δ 'is obtained .

(Embodiment 2) A second embodiment of the present invention will be described with reference to the drawings. FIGS. 8A, 8B, and 8C show two-dimensional objects when the three-dimensional object is placed on the work table in the present embodiment.
3A and 3B are a perspective view of measurement by two line sensors, a view of obtaining a position of a point of interest using an inverted model, and a plan view of FIG. In the figure, δ is an object, δ 'is an object model,
6c and 6d are line sensors, and 8 is a work table.

In this embodiment, two line sensors 6
The position and orientation of the three-dimensional target object δ are detected using c and 6d. Similarly to the above, the target object δ is assumed to be placed on the worktable 8 as shown in FIG. 8 (a), and the point arrays t3 and t4 by the two line sensors 6c and 6d.
Is obtained. Based on this, an object model δ ′ is created.

When the region where the point of interest on the object model is present is obtained from the measurement result of the line sensor, the following procedure may be used. As shown in FIG. 8A, a pentahedron is placed in an unknown posture, and two line sensors 6c are used.
It is assumed that the point series t3 and the point series t4 are obtained as a result of the measurement by the steps 6 and 6d. The two end points of the point sequence t3 are K19 and K20, and the height from the worktable 8 is h1 and h2, respectively.
And Also, one end point of the point sequence t4 is K21, one intermediate point is K22, and the end point is K23, and the height from the worktable 8 is h3, h4, and h5, respectively. Here, it is assumed that the position of the point E on the object δ is obtained using the inverted model δ ″.

Since the point sequences t3 and t4 are literally a set of points, each point on the point sequence is processed in the same manner as the above-described process of acquiring the existence candidate area for the measurement point of the point sensor.
By applying the process, it is possible to obtain a candidate region where a point of interest exists for the point sequence. For example, the measurement point K on the point sequence t4
A point E ′ on the inverted model δ ″ is set along the point sequence t4, such as δ2 ″ for the inverted model δ ″ obtained by combining the point E ′ of the inverted model δ ″ with 21 and δ3 ″ for the measurement point K22. The surface on the inverted model δ ″ corresponding to the surface DEIH on the object model δ ′ also slides along with this, but the common range of the surfaces on all the slid inverted models δ ″ is the point E to be obtained. Is an existence range candidate area.

The same processing is performed for the point sequence t3, and the common part of the candidate range of the point of interest on the inverted model δ ″ obtained by all the sensors 6c and 6d is obtained, thereby obtaining the point of the object model δ ′. The position of E, ie
The posture of the object δ in the unknown posture can be determined. Even if the posture of the object δ cannot be determined to one state due to the limitation of the number of sensors, the possible object
Can be obtained.

As shown in FIG. 8B, the point sequences t3 and t4
The candidate area can be obtained by combining the inversion model δ ″ for each of the above points and finding the common range of the corresponding candidate area, but the point on the object δ of interest is
If it is known that it exists above, the processing can be simplified and shown as follows.

[Outside 2] Model at the height of the observation point (height from the workbench) by the sensors 6c and 6d

[Outside 3]

The measurement point K by the sensors 6c and 6d
19, K20, K21, K22, K

[Outside 4]

[Outside 5] Is

(Equation 3) Is obtained.

The principle of equation (3) will be described with reference to FIG. FIG. 8C is a view of FIG. 8A viewed from directly above. As an example, the sensor measurement point K19 in FIG.
When the point E ′ on the inverted model δ ″ corresponding to the point E is aligned, the intersection line between the work table 8 and the inverted model δ ″ becomes the horizontal plane of the height of K19, ie, the height of the object model δ ′. The cross section obtained by cutting at the plane of h1 is inverted. As a result, as shown in FIG. 8C, the upper side of the rectangle J1 is obtained as a candidate area corresponding to K19.
The upper side of the rectangle J2 is obtained as a candidate area corresponding to 0.

As a candidate common area when the point E 'of the inverted model δ "is slid along the sensor measurement point sequence t3, L20 in FIG. 8C is obtained. By performing the same process,
For example, the left sides of rectangles U1, U2, and U3 in FIG. 8C are obtained as candidate areas for measurement points K21, K22, and K23, and L21 is obtained as a common area. The common point C9 between L20 and L21 is the position of point E to be determined.

The width of the region where the point of interest corresponding to each measurement point exists may be variable according to the difference between the measurement points. For example,
Depending on the relationship between the sensor observation direction and the normal direction of the object plane, if the reliability of the distance data to be observed is different, the candidate area should be narrowed in high reliability parts and low reliability parts Then, the width of the candidate area may be further increased.

(Embodiment 3) A third embodiment of the present invention will be described with reference to the drawings. FIG. 9A is a representation diagram of an object model using a sequence of points on a plane parallel to the work plane in the present embodiment, and FIGS. 9B, 9C, and 9D are explanatory diagrams of cross-sectional contour patterns and intersection coordinate values, respectively. FIGS. 10 (a) and 10 (b) are an explanatory view and a table of intersection coordinate values, respectively, for a procedure for obtaining a common area of a cross-sectional contour pattern on a work table in this embodiment.

In the execution procedure of this embodiment, as shown in FIG. 14A, the data of the object model α ′ obtained by acquiring the distance data while rotating the table 4 is used, as shown in FIG. As shown in a), the model can be easily re-created by the cross-sectional shape parallel to the horizontal plane. That is, it is possible to obtain a contour constituted by a sequence of points such as H1, H2,... Hi,.
If only the data obtained by the operation shown in FIG. 14A cannot represent the outline of one round of each horizontal plane due to the roughness of the rotation, the data is interpolated to Hi.
(I = 1, 2,...) May be created.
The height of the horizontal section may be set arbitrarily.

The outline of each horizontal section can be expressed as a pattern shown in FIG. Contour value is 1 and others are 0
Indicated by. Points having a value of 1 can be tabulated as a table as shown in FIG. 9C. Further, depending on the accuracy of the external shape of the model of the object, a thickness may be provided on the outline of the horizontal cross section shown in FIG. 9B. Further, the value is not limited to two values of 0 or 1, and as shown in FIG. 9D, the value of the portion representing the contour is multi-valued. For example, a high value is set at the center of the cross section, and the peripheral portion is set. May be set to a low value.

After the above preparation, the posture can be detected according to the principle shown in FIG. The rectangles J1, J2, U3, U2, U3, etc. in FIG. 8C correspond to the contour of the cross-sectional shape in FIG. 9B. As previously mentioned,
The detection principle is to find a common part of the candidate points on the inversion model δ ″ obtained according to each observation point.
When the pattern (b) is used, as shown in FIG. 10A, regions H1, H2, and H3 are formed for each observation point as a candidate region of a point of interest on the object. These common portions S are the positions of the points of interest to be obtained. As an operation at this time, first, as an initial state, the values of the respective regions H1, H2, and H3 are cumulatively added to a work table in which the values of all elements are 0, and the cumulative value is the same as the number of candidate regions. An operation of obtaining an area (3 in this case) may be performed.

In order to speed up the processing for obtaining the common area of the cross-sectional shape, it is possible to judge the existence of the common area when the cross-sectional shapes are further overlapped only in the area in which the values are added and accumulated. For example, in FIG.
Only for the common area of H2, it may be checked whether or not the common area with H3 exists.

In the example shown in FIG. 10B, there is no region in which all the horizontal cross-sectional shapes have a common region. Assume that H1 and H2 have a common area at point 1 and point 2. FIG.
As shown in (c), an intersection table indicating the coordinates of point 1 and point 2 can be created. Subsequently, when determining the common area with H3, it is sufficient to check whether or not the common area with H3 exists only for the coordinates described in the intersection table. The check of the existence of the common area can be realized as a two-dimensional pattern reference operation as follows.

[0065]

[Outside 6] And Further, a pattern (corresponding to the pattern in FIG. 9B) indicating the horizontal cross-sectional contour with respect to the height of the observation point 1 is represented by H
1 and the point on H1 is (XH1, YH1). So

[Outside 7] yj, zj). Also, a point on the pattern Hj indicating the horizontal cross-sectional contour with respect to the height of the observation point j is (XHj, yHj)
And Here, a matrix Aθ for the rotation θ is determined as follows. That is,

(Equation 4) And

Assuming that the rotation of the object is θ, 3
Reference is made to the value of the horizontal cross-sectional contour pattern Hj at the point (XGj, YGj) obtained by performing the conversion on the dimensional coordinates by the following equation.

(Equation 5)

That is, if the value referred to is a value ("1" or "ON") indicating the existence of a contour, it can be said that a common area exists. Further, when the cross-sectional contour is represented by multi-values as shown in FIG. 9D, the extent to which the common region exists can be obtained depending on the magnitude of the value referred to.
The point where the referenced value is ON (or more than a specified value in the multi-value pattern of FIG. 9D)
This is described in the intersection table shown in FIG. At the time of intersection determination after the next time, the point (XH1, YH
As 1), determination may be made only for the points described in the intersection table, and the intersection table may be updated according to the result of the two-dimensional pattern reference operation.

When referring to the multi-value pattern shown in FIG. 9D, both the coordinate value and the reference value may be described in the intersection table. In this case, for example, even if the reference value is low in the common area check between H1 and H2, if the reference value is high in the common area check with another cross-sectional profile, processing can be performed so as not to be deleted from the candidates.

By performing the above operation while changing the rotation θ in small increments, the position and orientation of the object can be obtained. Depending on the observation direction of the sensor,
As shown in (b) and the like, the region of the existence candidate of the point of interest is
Since the cross-sectional contour of the object is not the whole but only a part, the method of FIGS. 9 and 10 may be used to check for the existence of the common area in consideration of the observation direction of the sensor.

In the first to third embodiments described above,
If the orientation of the object is limited to a certain range and the range on the object model to which the measurement points correspond is limited, the orientation of the object can be determined by the least square method or the like. That is, by the position and orientation detection method of the object according to the first to third embodiments,
As shown in FIG. 11, the correspondence between the observation points of the point sensor and the line sensor and the points on the object model is limited to a certain local area as shown by oblique lines. In this case, in order to calculate a position and orientation with higher accuracy, a point at a corresponding position on the object model is translated and rotated in the x and y directions by using all measurement point data from a plurality of sensors. The translation amount and the rotation angle may be determined so that the sum of squares of the difference between the position and the position measured by the sensor is minimized.

[0071]

[Outside 8] When

(Equation 6)

[Outside 9] In the above description, a model whose entire shape is known is used as the object.However, if it is sufficient that the position / posture of the local part of the object is known by the work, it is not necessary to take the entire shape of the object as a model. The data of only the part may be stored as a model.

The sensor used does not depend on the measurement method, such as a non-contact sensor using a laser or an ultrasonic wave or a contact sensor using a tactile sensor. In the case of three dimensions,
Area sensors can be used as sensors. Further, in the above description, at the observation point by the line sensor, 3 is indicated by the line of intersection between one plane and the object.
Although the dimensional point sequence is assumed to be observed, the present invention is not limited to this, and an arbitrary set of point sequences may be observed. For example, a point sequence measured while the direction of the point sensor is swung in an arbitrary direction can be used.

The flow of the method will be described with reference to FIG. First, the shape of the target is input by measuring the distance by placing the target object on the turntable. Subsequently, an inverted model and a sectional shape pattern of the object are created using the obtained distance data. Depending on the shape of the object and the location of the work,
Arrange multiple sensors. Up to this point,
Performed as preprocessing. That is, for an object having the same shape, the above-described processing is performed once, and then the following processing is performed for an object having the same value shape placed in an unknown posture.

The posture detection position and posture of an object whose unknown position is measured by each sensor. Since the position and orientation of the object can be indicated by the position and orientation of a point on the object, in order to find the position of the point of interest, the common part of the inverted model candidate area for each sensor measurement point or the cross-sectional shape Find the common area of the pattern. By this processing, the position / posture of the object can be obtained by integrating the posture candidates by the sensors. The detection accuracy can be improved by sequentially narrowing the region of the existence candidate or gradually changing the step size of rotation. Accuracy can be improved by this process alone.However, using the above results as a rough positioning process, the posture detection in the fine mode can be performed by a method such as minimizing the square error of the distance between the model and the observation data. Can do it.

In the method of checking whether or not the cross-sectional patterns have a common area, the method of performing the reference processing of the binary pattern has been described. However, the present invention is not limited to this. For example, as shown in FIG. A circumscribed rectangular frame is created hierarchically in advance, the intersection determination process between the rectangular frames is performed, and if they intersect, a similar intersection determination is performed for the lower-level rectangular frame. (Reference: Ballard Brown: “Computer Vision”, p.
p. 307-312, Japan Computer Association).

[0076]

Thus, according to the present invention, the following excellent effects can be obtained with a very simple configuration. In other words, the position and orientation of the object can be detected by a unified method without depending on the difference in the shape of the object or the type of sensor, and the orientation can be determined by integrating the measurement results of multiple sensors. In addition, even if the position and orientation cannot be limited due to the shortage of the number of sensors and the observation conditions, candidates for the position and orientation of the object can be indicated as regions. In addition, since it is possible to obtain candidates for the position and orientation of the object by checking whether or not there is a common area of the inverted model or the cross-sectional shape pattern, high-speed processing is possible.

[Brief description of the drawings]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory view of the basic principle of the present invention, wherein (a) is a plan view of an object, (b) is an observation view of an object by a plurality of point sensors, and (c) is an image of one point sensor. FIG. 4 is a trajectory diagram of a point of interest that can exist with respect to an observation point.

FIG. 2A is a plan view of an inverted model, and FIG. 2B is a diagram illustrating a procedure of detecting a region where a point of interest exists using the inverted model;

FIG. 3 is a diagram showing a spread of a region of a candidate for existence of a point of interest according to a rotation step width;

4A and 4B are explanatory diagrams of the present invention using a two-dimensional planar object, wherein FIG. 4A is a measurement diagram of a right-angled triangular planar object by two types of sensors using a point sensor and a line sensor;
(B), (c) is an explanatory view of a posture determination procedure using the same and inverted plane models.

5A and 5B are measurement diagrams in which a plane that restricts the position of a plane object is present when a plane object is placed on the restriction plane, and FIGS. 5B, 5C, and 5D are the same and inverted. It is explanatory drawing of the attitude | position determination procedure which used each plane model.

FIG. 6A is a measurement diagram in which the DE base of a right-angled triangular planar object is placed parallel to a constraint surface, FIG. 6B is an explanatory diagram of a posture determination procedure using the same inverted model, and FIG. Is the same
FIG. 9D is a measurement diagram in which the bottom of F is rotated by 30 ° with respect to the constraint surface, and FIG.

FIGS. 7A and 7B show a first embodiment of the present invention, in which FIG. 7A is a perspective view of a three-dimensional object measured by a line sensor and a point sensor, and FIG. FIG. 4 is an explanatory diagram of a procedure for obtaining the.

8A and 8B show a second embodiment of the present invention, in which FIG. 8A is a perspective view of measurement using two line sensors when a three-dimensional object is placed on a workbench, and FIG. FIG. 7C is an explanatory diagram of a procedure for determining a point of interest position by obtaining a common area of an object cross-sectional shape according to the height of a sensor observation point, and FIG. 9C is a diagram viewed from directly above FIG.

FIG. 9 shows a third embodiment of the present invention, in which (a) is a representation diagram of an object model by a sequence of points on a plane parallel to a work plane;
(B), (c), and (d) are explanatory diagrams of cross-sectional contour patterns and table of intersection coordinate values, respectively.

FIGS. 10A and 10B are explanatory diagrams illustrating a procedure for obtaining a common area of a cross-sectional contour pattern on a work table, and FIG. 10C is a diagram of intersection coordinate values;

FIG. 11 is a diagram showing a local region on the object model corresponding to each of the sensor observation points for calculating the posture in the fine mode.

FIG. 12 is a flowchart from the acquisition of an object model to the detection of a posture in the embodiment of the present invention.

FIG. 13 is a diagram illustrating a case where a strip tree is applied to a cross-sectional shape contour pattern.

14A is a perspective view illustrating a distance measurement slope of a three-dimensional object by slit light projection, and FIGS. 14B, 14C, 14D, and 14E are explanatory diagrams of a conventional polyhedral posture detection procedure.

[Explanation of symbols]

 α, δ, δ2, δ3 ... object α1 ... part α ', α ", δ' ... object model α", δ "... inverted model β, γ ... plane object β ', γ' ... plane object model β", γ Inverted plane model β1 Arc β2 to β4 Straight line θ Width 1 3D object 2 Projection unit 3 Light receiving unit 4 Table 5a to 5g Point sensors 6a to 6d Line sensor 7 Constraining surface 8 Work table A, A ', D, D', E, E ', F, F', Q ... Points C, C1 to C9 ... Intersections K1 to K23, R, j ... Measurement points H1 to H3, L, L1 L21 area d1, d2 distance h1 to hi height t1 to t4 point sequence

Claims (4)

(57) [Claims] In a method for detecting the position and orientation of an object based on an object model that is the shape of the object, first, the shape of the target object is measured in advance to create the object model, Generate an inversion model obtained by inverting the sign of the coordinate axis, and then, for the target object placed at an unknown position and orientation,
Observe the distance of the target object by a distance sensor of two or more arbitrary types and arbitrary types whose positions and measurement directions are determined in advance, and subsequently, a point on the inverted model with respect to a point of interest on the object or the object model The position of the point of interest as an arbitrary point on the object model by obtaining the surface of the inverted model when matching the distance sensor observation point for each of the distance sensor measurement points, and obtaining all common regions, A three-dimensional object position / posture detection method, wherein a direction is determined and the position and orientation of the object are determined. 2. A method for detecting the position and orientation of an object based on an object model, which is the shape of the object, comprises: measuring the shape of a target object in advance to create the object model; Generate an inversion model obtained by inverting the sign of the coordinate axis, and then, for the target object placed at an unknown position and orientation,
Observe the distance of the target object by a distance sensor of two or more arbitrary types and arbitrary types whose positions and measurement directions are determined in advance, and subsequently, a point on the inverted model with respect to a point of interest on the object or the object model The position of the point of interest as an arbitrary point on the object model by obtaining the surface of the inverted model when matching the distance sensor observation point for each of the distance sensor measurement points, and obtaining all common regions. Determine the direction, further, by limiting the existence candidate area of the point of interest as an arbitrary point on the object model by the existence position of the work table on which the object is placed or a jig member for fixing the object, A three-dimensional object position / posture detection method, characterized in that the position and orientation of the object are determined. 3. A method for detecting the position and orientation of an object based on an object model that is the shape of the object, wherein the object model is created by measuring the shape of a target object in advance, and A reversal model obtained by reversing the sign of the coordinate axis is generated in advance, and then a plurality of cross-sectional contour patterns obtained by cutting the object on a plane horizontal to the work table are prepared in advance according to the distance from the work table. When the posture is detected, the distance between the target object and the target object placed at an unknown position / posture is determined by two or more arbitrary numbers and arbitrary types of distance sensors whose positions and measurement directions are determined in advance. And further, for each point on the cross-sectional contour pattern with respect to the first measurement point,
After performing the coordinate transformation on the first measurement coordinate, the section contour pattern for the second and subsequent measurement points is referred to, and when the value of the referred pattern indicates the section contour, one of the position and orientation candidates is obtained. If the value of the referenced pattern does not indicate the cross-sectional contour, the operation of excluding the point from the candidates is repeated, so that the cross-sectional contour pattern common according to the coordinates of the distance sensor measurement points is common. A three-dimensional object position / posture detection method characterized by determining the position and orientation of a point of interest as an arbitrary point on the object model by obtaining an area, thereby determining the position and orientation of the object. 4. A method for detecting a position and a posture of an object based on an object model that is a shape of the object, wherein the shape of the target object is measured in advance and the object model is created. A reversal model obtained by reversing the sign of the coordinate axis is generated in advance, and then a plurality of cross-sectional contour patterns obtained by cutting the object on a plane horizontal to the work table are prepared in advance according to the distance from the work table. When the posture is detected, the distance between the target object and the target object placed at an unknown position / posture is determined by two or more arbitrary numbers and arbitrary types of distance sensors whose positions and measurement directions are determined in advance. And further, for each point on the cross-sectional contour pattern with respect to the first measurement point,
After performing the coordinate transformation on the first measurement coordinate, the section contour pattern for the second and subsequent measurement points is referred to, and when the value of the referred pattern indicates the section contour, one of the position and orientation candidates is obtained. In the case where the value of the referenced pattern does not indicate the cross-sectional contour, the operation of excluding the point from the candidates is repeated to repeat the operation of removing the cross-sectional contour pattern according to the coordinates of the distance sensor measurement points. Determine the position and direction of the point of interest as an arbitrary point on the object model by obtaining a common area, and further, depending on the work table on which the object is placed or the position of the jig member for fixing the object A position and orientation of the object is determined by limiting a candidate area of a point of interest as an arbitrary point on the object model; Detection methods.