JP2022042866A - System and method of obtaining positional information for picking - Google Patents

Abstract

To efficiently obtain positional information on an object in picking, and to efficiently enable the automatic picking of transparent containers including PET bottles.SOLUTION: A system for obtaining three-dimensional position posture information for picking each bottle from an aggregation of a plurality of bottles, comprises a sensor that obtains an image in the aggregation and controlling means. The controlling means comprises means for detecting a mouth position and posture of each bottle in the obtained image from the mouth shape of the bottle that is recorded in advance, and obtaining the three-dimensional position posture information. The controlling means further comprises: an ultraviolet light irradiation device to the aggregation; and means for detecting the mouth position and posture of each bottle in the obtained image while being irradiated with ultraviolet light to obtain the three-dimensional position posture information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a system and a method for acquiring position information for picking from a predetermined aggregate. In particular, the present invention relates to a system and a method that enable automatic picking of transparent PET bottles having different shapes and types.

Conventionally, as for beverage bottles such as PET bottles, bottles that do not meet the conditions in the inspection at the time of manufacture are once collected in a box and re-inspected. Further, in recycling or the like, beverage bottles such as PET bottles are once collected in a box and then sent to a recycling device. In order to be placed in these processes such as re-inspection and recycling, it is necessary to pick and palletize each bottle from the accumulated aggregate or place it on the line.

On the other hand, as a picking technique, there is a picking system that searches for position information from a pre-recorded shape by using a 3D sensor or the like. In the picking system using the 3D sensor, objects are photographed by two cameras whose positional relationship with each other is fixed, the objects in the obtained image are associated with each other, and triangulation is performed using parallax. Judging the position to do. That is, a technique for searching for a specific object from the acquired image and recognizing the three-dimensional position of the found object is disclosed (see, for example, Patent Document 1).

Beverage bottles such as PET bottles have different shapes depending on the manufacturer and product, and in order to perform picking, it is necessary to recognize in advance the CAD data and the like of each bottle shape existing in the aggregate. However, the shape of PET bottles is different for each product, and many bottles of different types and shapes may overlap each other in the aggregate, so it is not efficient to record for each shape.

Furthermore, where many of the beverage bottles in recent years are PET bottles, these PET bottles are often transparent. Since such a transparent PET bottle is transparent, it is not possible to extract the target edge in the sensor image. Even if the pattern projection (point cloud projection) method or the light cutting (edge extraction by strong striped pattern projection) method is used, the edge cannot be extracted because the object is transmitted. Therefore, there is a problem that shape recognition and position recognition cannot be performed in the first place.

For this reason, the current situation is that picking from an aggregate in which transparent PET bottles are accumulated is finally performed by a person. On the other hand, from the viewpoint of business feasibility and utilization of resources, human resources, time and cost cannot be wasted in the picking process before re-inspection or recycling.

Japanese Unexamined Patent Publication No. 2019-181573

Therefore, an object of the present invention is to efficiently acquire the position information of the target in picking. Furthermore, it is an object of the present invention to efficiently enable automatic picking of a transparent container including a PET bottle.

In order to solve the above problems, the system of the present invention is a system that acquires three-dimensional position / orientation information for picking each bottle from an aggregate of a plurality of bottles, and includes a sensor that acquires an image in the aggregate. A control means is provided, and the control means is provided with a means for detecting the position and posture of the mouth of each bottle in the acquired image from the shape of the mouth of the bottle recorded in advance and acquiring the three-dimensional position / posture information. It consists of things.

Further, the aggregate is further provided with an ultraviolet irradiation device, and a means for detecting the position and orientation of the drinking spout of each bottle in the acquired image and acquiring three-dimensional position / orientation information in the state of being irradiated with ultraviolet rays. Is preferable.
Further, the sensor is composed of two cameras, and it is preferable that the three-dimensional position / orientation information is acquired by using the parallax of the image information acquired from the two cameras.
Further, it is preferable that the mouthpiece shape includes a support ring.

Further, a picking device and its control means are further provided, and the control means of the picking device can acquire the three-dimensional position / posture information of the detected drinking spout position and posture of the bottle, and the picking device can pick the bottle. Suitable.
Further, when the control means of the picking device detects that a bottle other than the assumed bottle is held, the holding of the assumed bottle is stopped and the picking of the held bottle is instructed to be continued. Is preferable.
Further, the bottle holding means is an adsorption means, and it is preferable to provide a means for detecting whether or not a bottle other than the assumed bottle is held by detecting the fluctuation of the air flow rate.

Further, the present invention is a method of providing position information for picking each bottle from an aggregate of a plurality of bottles, using a system provided with a sensor for acquiring an image in an aggregated state and a control means thereof in advance. It includes a process of detecting the position and posture of each bottle in the acquired image from the recorded shape of the mouth of the bottle and grasping the three-dimensional position / posture information.

Further, the system further includes an ultraviolet irradiation device for the aggregate, irradiates the aggregate with ultraviolet rays, and detects the position and posture of the drinking spout of each bottle in the acquired image in the state of being irradiated with ultraviolet rays. However, it is preferable to include a process of grasping three-dimensional position / orientation information.
Further, it is preferable that the sensor is composed of two cameras and the three-dimensional position / orientation information is acquired by using the parallax of the image information acquired from the cameras.
Further, it is preferable that the mouthpiece shape includes a support ring.

Further, the present invention is a method for acquiring three-dimensional position / orientation information for picking each container from an aggregate including a transparent container made of a material that fluoresces by ultraviolet irradiation, and is a sensor for acquiring an image in the aggregate. And its control means, and a system equipped with ultraviolet irradiation means, the position and orientation of the common part recorded in advance are detected from the image acquired in the state of being irradiated with ultraviolet rays, and the three-dimensional position and orientation information is acquired. It consists of including the process of doing.

Further, it is preferable that the container is a beverage bottle.
Further, it is preferable that the pre-recorded intersection has a spout shape.
Further, it is preferable that the material is PET, polycarbonate, or porestyrene.

The present invention focuses on the fact that the shape of the main body of a beverage bottle differs from product to product, while the shape of the mouthpiece of the product is almost the same and the direction thereof can be clearly specified. More specifically, PET bottles have various capacities and shapes, but the shape and structure of the mouthpiece of PET bottles are almost the same regardless of their capacities and types, and are supported from the bottom. The ring, lead ring, screw, and opening are configured in this order toward the top. Therefore, when the shape of the spout was recorded by the control means and only the spout was detected from the acquired image, the screw structure of the spout was found regardless of the shape and size of the entire PET bottle and the presence or absence of the vent slit. Regardless of the difference, it is possible to properly detect the direction including the direction and transmit the three-dimensional position / attitude information to a picking device such as a robot. As the mouthpiece shape information to be recorded, it is desirable that the shape can be utilized as three-dimensional information, and CAD data is typically used.

In the present application, as a sensor for acquiring an image in an aggregated state, a sensor capable of specifying a three-dimensional position in an image together with a control means is used. Typically, two cameras are used to shoot an object with two cameras whose positional relationship with each other is fixed. The captured image data is transmitted to a control device including a known configuration such as a memory, a CPU, and a communication interface. The control device searches for the mouth shape recorded in advance from the transmitted image data, and transmits the detected three-dimensional position / orientation information to the picking device. Therefore, in the present invention, even if the aggregate is randomly filled with various bottles, the position information can be accurately recognized without the need to recognize the shape of each bottle.

In particular, the present invention includes means and steps for providing an ultraviolet irradiation device and projecting onto an aggregate in order to enable picking of a transparent PET bottle. Some transparent PET bottles have a high visible light transmittance of 80% or more, but they turn blue by irradiating with ultraviolet rays (10 to 400 nm, preferably near ultraviolet rays (200 to 400 nm in consideration of fluorescence)). It fluoresces. Then, we focused on the fact that the contrast of the bottle shape acquired by the camera increases in the fluorescent state, and edge extraction is possible with visible light. The strength of the contrast at the edge is strongly expressed in the spout, especially in the support ring and the threaded part, and even if the aggregate is randomly filled with various bottles, it is not necessary to recognize each shape accurately. I found that I can recognize the location information. It is preferable that the box body of the aggregate is a non-glossy black color under visible light in order to sufficiently exhibit the contrast.

The picking device uses a device that can hold and pick up each bottle based on the acquired position / orientation information. Typically, it is composed of a robot having a multi-axis and a three-dimensional movable arm. A robot equipped with a vacuum gripper at the tip so as not to damage or change the shape of the object during picking, picking by adsorption by air pressure, and a means capable of detecting fluctuations in air flow rate or air pressure. Is preferable. If necessary due to restrictions on the work space and the equipment used, it is also possible to use other gripping means such as a chuck for re-attitude control and palletizing.

The picking order is not particularly limited as long as the objects in the aggregate can be picked, but preferably, the picking is performed in order from the one having the highest height coordinate (Z coordinate) of the mouthpiece. In this case, the robot recognizes the bottle position based on the three-dimensional position / orientation information of the drinking spout, but in a randomly arranged state, it moves to acquire the bottle having the drinking spout that is supposed to be acquired. During, it may come in contact with or adsorb to another bottle. In such a case, if the purpose is to perform all picking regardless of the order of acquisition, stop the approach to the bottle of the drinking spout that is supposed to be acquired, and pick the contacted or adsorbed bottle as it is. Efficient picking can be performed with. Specifically, if adsorption or contact is detected before reaching the target after the approach to the target is started, the approach to the target is stopped and the adsorbed target is transported to a predetermined destination.

Further, in the present invention, when the mouthpiece portion cannot be recognized or picked in a posture in the aggregate, a process in which the robot shakes the box or stirs the aggregate may be appropriately inserted. In addition, depending on the space and the restrictions of the equipment used, in picking, once placed in the temporary placement area from the aggregate, picking is performed again by changing the gripping means, and the posture is adjusted, and the posture is placed at the final destination in the desired posture. , The mode can be selected according to the situation.

Furthermore, the present invention is also applicable to non-transparent bottle aggregates, mixtures with transparent PET bottles, and aggregates of containers having a partially common shape. Further, it can be applied not only to PET bottles but also to polycarbonate and polystyrene bottles and containers that can be fluorescent with ultraviolet rays.

According to the present invention, it is possible to efficiently acquire the position information of the target in picking for an aggregate in which a plurality of objects are randomly input. In particular, automatic picking of a transparent container including a PET bottle can be efficiently performed.

It is a conceptual diagram which shows the structure of this invention. It is a figure which shows the arrangement structure at the time of carrying out of this invention. It is a figure which shows the fluorescent PET bottle assembly and the box. It is a figure which shows the Example which includes the temporary storage area. It is a figure which shows the work flow including a temporary storage area. It is a figure which shows the work flow when the acquisition of an unexpected PET bottle is detected. It is a figure which shows the shape of the drinking mouth of a PET bottle.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a conceptual diagram showing the configuration of the present invention. FIG. 2 is a diagram showing an arrangement configuration at the time of carrying out the present invention. FIG. 3 is a diagram showing a fluorescent PET bottle assembly and a box.

As shown in FIGS. 1 to 3, in the picking system 1 of the present invention, a large number of transparent PET bottles 3 are piled up in bulk in the box 2 to form an aggregate 4. Two cameras 5, 5'are provided for the box 2 including the aggregate 4, and an image is acquired. The acquired image is sent to the control device 6. The control device 6 can detect an object by performing triangulation using two parallax units and grasp it as three-dimensional information. The control device 6 stores the mouth shape 19 (CAD information) of the PET bottle 3, and detects the position and posture of the mouth of the PET bottle 3 from the acquired image regardless of the shape and type.

An ultraviolet irradiation device 7 is provided for the box 2 including the aggregate 4. When irradiated with ultraviolet rays, the transparent PET bottle 3 fluoresces to the extent that it can be seen with the naked eye, as shown in FIG. The photo is black and white because it is grayscale, but it actually emits blue light. In particular, the mouthpiece with many irregularities and curved surfaces fluoresces clearly. The switch for ultraviolet irradiation and the adjustment of the intensity are performed by the lighting controller 8. The fluorescent state enables the control device 6 to detect the three-dimensional position / orientation information of the drinking spout of the transparent PET bottle 3 with visible light.

The control device 6 sends the acquired three-dimensional position / orientation information of the drinking spout of the PET bottle 3 to the robot controller 9. The robot controller 9 orders the acquisition of the PET bottle to be acquired and the transportation to a predetermined position from the acquired three-dimensional position / attitude information. The robot 10 acquires the designated PET bottle and transports it to a predetermined position.

The robot 10 uses a vacuum gripper 11 having a rubber member at the end as a means for acquiring the PET bottle 3. This configuration is useful for avoiding damage or deformation of the PET bottle 3 during acquisition. In addition, since the inside of the gripper cannot be completely evacuated due to the presence of irregularities in the spout such as the screw portion, suction is performed while continuously sucking air. Since the flow rate of air clearly changes when the suction state is reached, the robot is provided with a flow switch 12 capable of detecting the flow rate, and when a large decrease in the flow rate is observed, it is transmitted to the robot controller 9 as being adsorbed. Further, an air chuck 13 is provided to facilitate the re-position adjustment.

A specific layout configuration in which wiring and the like are omitted is shown in detail in FIG. The box 2 and the robot 10 are arranged on the gantry 14. Further, a frame 15 is provided so as to surround the gantry 14. In the frame 15, a pair of inner frames 152 extend from the outer frame 151 to the vicinity of the box 2. An ultraviolet irradiation device 7 is provided at the end of the pair of inner frames 152. The entire box 2 can be irradiated with ultraviolet rays by the two ultraviolet irradiation devices 7. A camera holding frame 153 is provided on the upper portion of the outer frame frame 151, and two cameras 5, 5'are provided on the camera holding frame 153, and each of them acquires an image in the box 2. Further, in the box 2, a temporary storage area 16 (see FIG. 4) for temporarily placing the PET bottle 3 once acquired is provided in order to improve the accuracy in picking and valetizing.

In this embodiment, the priority of the PET bottle 3 to be acquired is selected from the one having the highest height coordinate (Z coordinate). However, in the present invention, it is assumed that the transparent PET bottles 3 are picked in a state of being randomly stacked in bulk. Therefore, there is a possibility of adsorbing another PET bottle 3 before reaching the highest drinking spout. This detection can be performed by a change in flow rate or pressure as long as it is by adsorption as in this embodiment, but other means such as contact detection may be adopted.

An embodiment including the temporary storage area 16 and a work flow thereof are shown in FIGS. 4 and 5. The two cameras 5, 5'recognize the inside of the box 2, and the robot 10 operates. The box 2 is provided with an area 17 that is piled up in bulk and a temporary storage area 16 for correcting the posture and regaining its posture. The robot 10 places the PET bottles 3 picked from the bulk-loaded area 17 in the temporary storage area 16, picks them again, and puts them in the final destination 18 while adjusting them in a desired direction in the posture adjustment area 20. Since only one PET bottle 3 exists in the posture adjusting area 20 in a predetermined direction, by gripping the drinking spout with the air chuck 13, accurate palletizing can be performed while avoiding damage. The temporary storage area 16 and the posture adjustment area 20 are not essential elements because they are assumed to be necessary due to restrictions on space and equipment in actual application.

As the work flow of the transparent PET bottle, after starting the work, the inside of the box 2 is recognized (S1), and it is determined whether or not the drinking spout is present (S2). If there is no spout, the work is completed (S3). If there is a spout, it is determined whether the spout exists in the temporary storage area 16 (S4), and if it does not exist, the bottle of the spout at the highest height position (Z axis) from the bulk picking area 17. Is selected, picked (S5), carried to the temporary storage area 16 (S6), and returned to S1. If there is a bottle in the temporary storage area 16, pick the bottle from the temporary storage area 16 (S7), switch to the air chuck 13 in the posture adjustment area 20 to adjust the posture (S8), and place it at the final destination 18. (S9), return to S1. In the above process, the robot 10 shakes the box 10 and / / Alternatively, the process of stirring the aggregate can be appropriately inserted as a pre-step of, for example, S3, S5, S7.

Further, FIG. 6 shows a work flow in which another PET bottle is adsorbed before reaching the highest drinking spout. When picking is started and the approach to the target PET bottle 3 is started (S10), if adsorption is detected before reaching the target PET bottle (S11), the approach to the target PET bottle is stopped and the adsorption PET bottle is left as it is. Adsorb and transport to the destination (S12) to finish. On the other hand, if this detection is not performed, the target PET bottle is adsorbed and transported to the destination (S13), and the process ends.

An example of the mouth shape 19 of the PET bottle 3, which is CAD data to be recorded in the control device 6, is shown in FIG. In the PET bottle 3, a support ring 192 for the lid is provided on the base 191 of the neck. A lead ring 193 and a threaded portion 194 are provided on the lead ring 193, and an opening 195 at the upper end portion is provided. In the present invention, the shape above the support ring 192 is stored and detected. By including the support ring 192 in the shape information of the spout, it is possible to detect the direction at the same time as facilitating the detection. It is desirable to remove the shape information below the base of the neck 191 because the shape differs for each PET bottle.

A picking experiment was performed using the above system in which only the spout shape 19 above the support ring 192 was recorded as a search target. Twenty 500 ml transparent PET bottles 3 were randomly stacked in the bulk loading area 17 in the box 2. The box 2 is provided with a temporary storage area 16, and the picked PET bottles are once transported to the temporary storage area 16, picked again, and the posture is adjusted by the box having an inclination toward the bottom and being adjustable in direction. The purpose was to change the posture to the air chuck 13 in the area 20 and arrange the posture at the final destination 18 in a desired posture.

The box 2 was irradiated with near-ultraviolet rays by the ultraviolet irradiation device 7, and picking was performed in a state where the transparent PET bottle 3 was fluorescent. As a result, the robot 10 was able to arrange all the transparent PET bottles 3 at the final destination 18 in a desired posture.

From the above results, it was found that even with a transparent PET bottle, it is possible to efficiently acquire the position / orientation information of the target in picking and efficiently perform automatic picking. Further, since it was possible to recognize the position and orientation of other shapes and sizes with the same mouthpiece shape information, the present invention can be applied even if non-transparent bottles or containers are mixed. The present invention can be applied to picking using a common portion of containers having different shapes (particularly transparent containers). Further, since fluorescence by ultraviolet irradiation can be performed not only with PET but also with polycarbonate and polystyrene, the present invention can be applied to these position recognition and picking.

1 Picking system 2 Box 3 PET bottle 4 Assembly 5, 5'Camera 6 Control device 7 Ultraviolet irradiation device 8 Lighting controller 9 Robot controller 10 Robot 11 Vacuum gripper 12 Flow switch 13 Air chuck 14 Frame 15 Frame 151 Outer frame frame 152 flame
153 Camera holding frame 16 Temporary placement area 17 Bulk area 18 Final destination 19 Drinking mouth shape 191 Base of neck 192 Support ring 193 Lead ring 194 Threaded part 195 Opening 20 Posture adjustment area

Claims (15)

It is a system that acquires 3D position / orientation information for picking each bottle from an aggregate of multiple bottles.
It is equipped with a sensor for acquiring images in an aggregate and its control means.
The control means
From the pre-recorded bottle mouth shape,
A system comprising a means for detecting the drinking mouth position and posture of each bottle in the acquired image and acquiring three-dimensional position / posture information. The system according to claim 1.
Further equipped with an ultraviolet irradiation device for the aggregate,
A system characterized by comprising means for detecting the drinking mouth position and posture of each bottle in the acquired image in a state of being irradiated with ultraviolet rays and acquiring three-dimensional position / posture information. The sensor is composed of two cameras.
The system according to claim 1 or 2, wherein the three-dimensional position / orientation information is acquired by using the parallax of the image information acquired from the two cameras. The system according to any one of claims 1 to 3, wherein the mouthpiece shape includes a support ring. The system according to claim 1 further comprises a picking device and a control means thereof.
A system characterized in that a control means of the picking device acquires three-dimensional position / posture information of a detected drinking spout position and posture of the bottle, and the picking device picks the bottle. The control means of the picking device is
When it is detected that a bottle other than the expected bottle is held,
The system according to claim 5, further comprising means for instructing the holding of the assumed bottle to be stopped and the picking of the held bottle to be continued. The system according to claim 6, wherein the bottle holding means is an adsorption means, and the means is provided for detecting whether or not a bottle other than the bottle assumed by detecting the fluctuation of the air flow rate is held. A method of providing location information for picking each bottle from a collection of multiple bottles.
Using a system equipped with a sensor for acquiring images in a gathered state and its control means,
From the pre-recorded bottle mouth shape,
A method comprising a process of detecting the drinking mouth position and posture of each bottle in the acquired image and grasping the three-dimensional position / posture information. The system further comprises an ultraviolet irradiation device for the aggregate.
Irradiate the aggregate with ultraviolet rays and
The method according to claim 8, further comprising a process of detecting the drinking mouth position and posture of each bottle in the acquired image in a state of being irradiated with ultraviolet rays and grasping the three-dimensional position / posture information. The sensor is composed of two cameras.
The method according to claim 8 or 9, wherein the three-dimensional position / orientation information is acquired by utilizing the parallax of the image information acquired from the camera. The method according to any one of claims 8 to 10, wherein the mouthpiece shape includes a support ring. It is a method of acquiring three-dimensional position / orientation information for picking each container from an aggregate including a transparent container made of a material that fluoresces when irradiated with ultraviolet rays.
Sensors that acquire images in the aggregate, control means, and
Using a system equipped with UV irradiation means,
A method comprising a process of detecting a position and a posture of a pre-recorded common portion from an image acquired in a state of being irradiated with ultraviolet rays and acquiring three-dimensional position / posture information. 12. The method of claim 12, wherein the container is a beverage bottle. 13. The method of claim 13, wherein the pre-recorded intersection is in the shape of a mouthpiece. The method according to any one of claims 12 to 14, wherein the material is PET, polycarbonate, or porestyrene.

Images