JP7364285B1 - How the robot handling system works - Google Patents

Abstract

An object of the present invention is to provide an operating method for a robot operation system that can reduce the effort and man-hours required for robot operation. [Solution] When a three-dimensional model (9) displayed on a display section (8) is moved to a predetermined position using an operating means (61), the three-dimensional model (9) is moved to a predetermined position accordingly. Even if the moving step and the three-dimensional model (9) move to a predetermined position, the control means (60) will keep the robot (2) at the predetermined position unless a permission operation is performed using the operating means (61). The robot (2) is moved to a predetermined position by the control means (60) when a permission operation is performed using the operation means (61) without moving the robot (2) to the robot (2). . [Selection diagram] Figure 2

Description

The present invention relates to a method of operating a robot operating system.

BACKGROUND OF THE INVENTION Conventionally, before using an industrial robot that performs work such as welding on a workpiece, teaching is performed to make the robot memorize the motions for moving to a destination in advance as a preparation.

Generally, two teaching methods are known: an online teaching method using a teach pendant (see FIG. 1 of Patent Document 1) and an offline teaching method (Patent Document 2).

The online teaching method is a method in which a robot is taught operations using a teach pendant while directly viewing the workpiece at the work site.

On the other hand, the offline teaching method uses offline teaching software installed on a computer to create CAD (computer-aided design) data of the workpiece, robot, and surrounding equipment, and displays a pseudo-workpiece on the screen. This is a method of teaching the robot how to operate by drawing an image of the robot and surrounding equipment and having the worker input data such as coordinate positions using a keyboard, mouse, etc.

Japanese Patent Application Publication No. 2013-132728 Japanese Patent Application Publication No. 9-179624

By the way, there is a demand for directly operating a robot without performing preparatory teaching (for example, when manufacturing a small number of products with different shapes). However, when operating the robot directly, there are two methods: the online teaching method, in which the robot is directly viewed at the work site and a teach pendant, and the offline teaching method, in which the robot is operated using a teach pendant. There are also drawbacks to each.

First, the online teaching method has the disadvantage that it requires consideration of safety while performing highly difficult tasks.

First of all, the teach pendant is equipped with a large number of buttons that let you know which direction you want the robot to move in on the predetermined three-dimensional coordinates, and how fast you want the robot to move. You need to use these multiple buttons properly, paying attention to whether or not to Therefore, operating the teach pendant is a highly difficult task that requires skill. Furthermore, while operating the teach pendant, care must also be taken to ensure that the robot does not interfere with surrounding equipment. In this way, online teaching using a teach pendant has the disadvantage that it is necessary to simultaneously operate the teach pendant, which is highly difficult, and check for interference in order to move the robot safely. .

On the other hand, the offline teaching method does not use a teach pendant, but only moves the robot's CAD data on the computer, so it is possible to generate a program to operate the robot, but it is not possible to directly operate the robot. I can't. Furthermore, this method has a drawback in that if there is a change in the layout of the workpiece or equipment, the change needs to be reflected in the CAD data of the offline teaching software.

SUMMARY OF THE INVENTION In view of the above problems, it is an object of the present invention to provide a method for operating a robot operating system that allows a robot to operate with easy operation.

The above object of the present invention is achieved by the following means. Note that reference numerals of embodiments to be described later are given in parentheses, but the present invention is not limited thereto.

The method for operating the robot operating system according to claim 1 includes:
A robot (for example, robot 2 shown in FIG. 2) that performs a predetermined processing operation on a workpiece (for example, workpiece W shown in FIG. 2),
a first camera (for example, first cameras 31 to 39 shown in FIG. 2) that can image the robot (for example, the robot 2 shown in FIG. 2) and the workpiece (for example, the workpiece W shown in FIG. 2);
A three-dimensional model storage unit (for example, the three-dimensional model 9 shown in FIG. 4, the three-dimensional model 9A after movement shown in FIGS. 5 and 6) of the robot (for example, the robot 2 shown in FIG. 2). ROM62) shown in FIG.
Images of the robot (for example, robot 2 shown in FIG. 2) captured by the first camera (for example, first cameras 31 to 39 shown in FIG. 2) (for example, images P1 to P3 of the work site shown in FIGS. 4 to 6); and A display section (for example, the display section 8 shown in FIG. 2) that displays a three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4, the moved three-dimensional model 9A shown in FIGS. 5 and 6) in an overlapping manner;
an operating means (for example, the input unit 61 of the operating terminal 6 shown in FIG. 2) for operating the three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4, the moved three-dimensional model 9A shown in FIGS. 5 and 6);
A control means (for example, the CPU 60 shown in FIG. 2) that controls the robot (for example, the robot 2 shown in FIG. 2);
A method of operating a robot operation system comprising:
The robot (for example, the robot 2 shown in FIG. 4) and the three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4) are displayed on the display section (for example, the display section 8 shown in FIG. 2) in a superimposed state. When the three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4) is moved to a predetermined position using the operating means (for example, the input section 61 of the operating terminal 6 shown in FIG. 2), the corresponding three-dimensional model is moved to a predetermined position. Although a three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4) is moved to a predetermined position, the robot does not move (for example, the robot 2 shown in FIG. 5), and the robot that is not moving (for example, the robot 2 shown in FIG. A step of superimposing and displaying the robot 2 shown in FIG. For example, step S4) shown in FIG.
having a worker check the display on the display section (for example, the display section 8 shown in FIG. 2) in order to determine whether the robot (for example, the robot 2 shown in FIG. 5) may be moved to the predetermined position; ,
Even if the three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4, the three-dimensional model 9A after movement shown in FIGS. 5 and 6) moves to a predetermined position, the operating means (for example, the operating terminal 6 shown in FIG. 2) moves to a predetermined position. The control means (for example, the CPU 60 shown in FIG. 2) controls the robot (for example, the robot 2 shown in FIG. ) is not moved to the predetermined position, and the permission operation (for example, step S6: Y shown in FIG. 3) is performed using the operation means (for example, the input unit 61 of the operation terminal 6 shown in FIG. 2), A step of moving the robot (for example, the robot 2 shown in FIG. 2) to the predetermined position by the control means (for example, the CPU 60 shown in FIG. 2) (for example, step S7 shown in FIG. 3). shall be.

The method for operating the robot operating system according to claim 2 includes:
a second camera (for example, A second camera 28) shown in FIG.
When the robot (for example, the robot 2 shown in FIG. 2) is performing the predetermined processing operation on the workpiece (for example, the workpiece W shown in FIG. 2), the second camera (for example, the second camera shown in FIG. 8) 28) (for example, a camera housing 28a and a shielding plate 28d shown in FIG. 8).

The method for operating the robot operation system according to claim 3 includes:
Display control means (for example, of the imaging processing terminal 5 shown in FIG. 2) that controls images (for example, images P1 to P3 of the work site shown in FIGS. 4 to 6) to be displayed on the display section (for example, the display section 8 shown in FIG. 2) further equipped with CPU50),
The first camera (for example, the first cameras 31 to 39 shown in FIG. 2) is a plurality of first cameras (for example, the first cameras 31 to 39 shown in FIG. 2),
Based on the position information of the three-dimensional model (for example, the moved three-dimensional model 9A shown in FIG. 5) displayed on the display section (for example, the display section 8 shown in FIG. 2), the display control means (for example, the display section 8 shown in FIG. 2) a step (for example, step S103 shown in FIG. 3) of selecting one of the plurality of first cameras (for example, first cameras 31 to 39 shown in FIG. 2) by the CPU 50 of the imaging processing terminal 5 shown in FIG.
The image (for example, the image P2 of the work site shown in FIG. 5) captured by the selected first camera (for example, the first camera 31 shown in FIG. 2) is displayed on the display control means (for example, the image capture processing terminal shown in FIG. 2). The present invention is characterized by further comprising a step (for example, step S104 shown in FIG. 3) of displaying the information on the display section (for example, the display section 8 shown in FIG. 2) by the CPU 50 of No. 5 (for example, the display section 8 shown in FIG. 2).

The method for operating the robot operating system according to claim 4 includes:
The three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4, the three-dimensional model 9A after movement shown in FIGS. 5 and 6) has a portion corresponding to the arm of the robot (for example, the arm 22 of the robot 2 shown in FIG. 6). It is characterized in that the three-dimensional model 9b) of the portion corresponding to the image is transmitted through and displayed on the display section (for example, the display section 8 shown in FIG. 2).

Next, the effects of the present invention will be explained with reference numerals in the drawings. Note that reference numerals of embodiments to be described later are given in parentheses, but the present invention is not limited thereto.

According to the invention according to claim 1, before moving the robot (for example, the robot 2 shown in FIG. 2) to a predetermined position, the worker displays the robot (for example, the The three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4) that is displayed superimposed on the robot 2) shown in FIG. 2 is moved to a predetermined position. As a result, in order to move a robot (for example, robot 2 shown in FIG. 2) to a predetermined position, the operator only has to move the 3D model (for example, 3D model 9 shown in FIG. 4) to a predetermined position, which is difficult. The robot can be operated without using a highly sophisticated teach pendant.

Further, according to the invention according to claim 1, the 3D model after movement (for example, the 3D model after movement 9A shown in FIG. 5) that has been moved as described above is Images of the robot (for example, robot 2 shown in FIG. 5 ) and workpieces (for example, workpiece W shown in FIG. 2) and surrounding equipment (for example, the work shown in FIG. 5) before movement are captured by the first cameras 31 to 39 shown in FIG. It is displayed superimposed on the image P2) of the scene. Therefore, as described above, the operator completes the operation of moving the three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4) to a predetermined position, and then moves the three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 5) after the movement. While visually checking the actual workpiece (for example, the workpiece W shown in FIG. 2) and surrounding equipment that are superimposed with the 3D model 9A after the movement on the display unit 8 , the robot before the movement (for example, the workpiece shown in FIG. 5) It is possible to check whether or not interference will occur with surrounding equipment when the robot 2) shown in FIG. In other words, it is not necessary to perform the movement operation and the confirmation work at the same time, and these tasks can be performed separately, so that safety can be enhanced while reducing the difficulty of the operation.

Moreover, what is displayed in superimposition is the actual workpiece (for example, the workpiece W shown in FIG. 2) imaged by the first camera (for example, the first cameras 31 to 39 shown in FIG. 2) and surrounding equipment, so the workpiece Even if there is a change in the layout of the computer or surrounding equipment, there is no need to take special measures such as modifying CAD data, which was required with conventional offline teaching software.

Furthermore, according to the invention according to claim 1, unless the operator determines that there is no problem in moving to the predetermined position and performs the permission operation (for example, step S6: Y shown in FIG. 3), the control means (for example, the The CPU 60 shown in FIG. 2 does not move the robot (for example, the robot 2 shown in FIG. 5). Therefore, the operator must redo the operation of moving only the 3D model (for example, the 3D model 9A after movement shown in FIG. 5) without moving the robot (for example, the robot 2 shown in FIG. 5) to find the optimal You can find a way to move. At this time, since the robot (for example, robot 2 shown in FIG. 5) is not being moved, the time and effort of returning the robot (for example, robot 2 shown in FIG. 5) to its original position and redoing the movement is not required.

Therefore, according to the invention according to claim 1, it is possible to provide a method for operating a robot operation system that allows a robot to operate with easy operation.

According to the invention as claimed in claim 2, the worker moves the tip of the work tool (for example, the tip 24a shown in FIG. 7) while checking the image of the tip of the work tool (for example, the tip 24a shown in FIG. 7) and the vicinity of the tip. The arm (for example, the arm 22 shown in FIG. 2) can be moved so that the tip end 24a) shown in FIG. Furthermore, protection members (for example, the camera housing 28a and shielding plate 28d shown in FIG. 8) protect the second camera (for example, the camera housing 28a and shielding plate 28d shown in FIG. 8) from welding spatter, arc light, drill chips, and other flying debris from processing operations. The second camera 28) shown in FIG.

According to the invention according to claim 3, the first camera (for example, the first camera 31 shown in FIG. 2) is selected based on the position information of the three-dimensional model (for example, the moved three-dimensional model 9A shown in FIG. 5). , since the image (for example, the image P2 of the work site shown in FIG. 5) is displayed on the display section (for example, the display section 8 shown in FIG. 2), the 3D model (for example, the 3D model 9A after movement shown in FIG. 5) is displayed. An image of the work site corresponding to the position information (for example, image P2 of the work site shown in FIG. 5) is displayed.

According to the invention according to claim 4, the three-dimensional model (for example, the three-dimensional model 9 shown in FIG. 4, the three-dimensional model after movement shown in FIGS. 5 and 6) displayed on the display section (for example, the display section 8 shown in FIG. 2), In order to move the dimensional model 9A) to a predetermined position, the 3D model of the part corresponding to the tip of the work tool (for example, the 3D model 9a of the part corresponding to the tip 24a shown in FIG. 6) is moved to the arm of the robot. It may happen that it is hidden by the 3D model of the corresponding part (for example, the 3D model 9b of the part corresponding to the arm 22 shown in FIG. 6) and cannot be confirmed on the display section (for example, the display section 8 shown in FIG. 2). do not have.

1 is an overall diagram of a robot operation system according to an embodiment of the present invention. FIG. 1 is an overall schematic diagram of a robot operation system according to the same embodiment. It is a flowchart figure showing the operation contents of the robot operation system concerning the same embodiment. FIG. 3 is a diagram showing a display section that displays a state in which a three-dimensional model and a robot are superimposed. FIG. 3 is a diagram showing a display section that displays a state in which a three-dimensional model has been moved to a predetermined position. FIG. 6 is a diagram showing a display section that displays a state in which the robot has moved to a predetermined position in response to the operator performing a permission operation after moving the three-dimensional model to a predetermined position. FIG. 2 is a perspective view showing a state in which a camera housing is attached to the tip of a robot arm. (a) is a vertical cross-sectional view of the camera housing, and (b) is a vertical cross-sectional view showing a state in which a shielding plate of the camera housing is rotated upward.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, one embodiment of a robot operation system using a method of operating a robot operation system according to the present invention will be specifically described with reference to the drawings. In the following description, when referring to directions such as up, down, left, and right, the directions are meant to be up, down, left, and right when viewed from the front in the drawing.

<Overview of robot operation system>
First, an overview of the robot operation system 1 in this embodiment will be explained. As shown in FIG. 1, the robot operation system 1 in this embodiment includes a turntable T on which a workpiece W (see FIG. 2) is placed, and a robot 2 that performs predetermined processing operations such as welding and polishing on the workpiece W. is imaged by a plurality of first cameras 31-39. Then, the worker operates the robot 2 using the operating terminal 6 (see FIG. 2) while viewing the robot 2, the workpiece W, and the surrounding situation captured by the first cameras 31 to 39.

The configuration of the robot operation system 1 in this embodiment will be described in more detail using FIG. 2. First, as shown in FIG. 2, the robot 2 is connected to the operation terminal 6 via a programmable logic controller (hereinafter referred to as PLC 29) via a network N. On the other hand, the first cameras 31 to 39 are connected to the imaging processing terminal 5 via a network N. Furthermore, the imaging processing terminal 5 and the operation terminal 6 are also connected via a network N.

Further, the imaging processing terminal 5 is connected to a projection device 70, and the operation terminal 6 is connected to a projection device 71 through a network N. Also, a display section 8 on which an image from the projection device 70 (images captured by the first cameras 31 to 39) and an image from the projection device 71 (displayed on the display section 64 of the operating terminal 6) are displayed in a superimposed manner. Equipped with

Here, a method for moving the robot 2 using the robot operating system 1 configured as described above will be briefly described. The operator first uses the operating terminal 6 to move the three-dimensional model 9 (see FIG. 4) of the robot 2 displayed on the display unit 64 of the operating terminal 6 by dragging or the like. As a result, the robot 2, the workpiece W, the surrounding situation, and the moved three-dimensional model 9A are displayed on the display unit 8 in a superimposed manner (see FIG. 5). The operator checks the display on the display unit 8, and if he or she determines that a problem such as interference will occur if the robot 2 is moved to the position and orientation of the moved three-dimensional model 9A, the operator presses a cancel button (not shown). Then, the operator can move the three-dimensional model 9 again without moving the robot 2. On the other hand, if the operator determines that there is no problem in moving the robot 2 to the position and orientation of the moved three-dimensional model 9A, the operator presses an OK button (not shown). As a result, the robot 2 moves to the position of the moved three-dimensional model 9A (see FIG. 6).

The outline of the robot operation system in this embodiment is as described above. Next, each part of FIG. 2 will be explained in detail.

<Robot description>
The robot 2 is an articulated general-purpose robot with six degrees of freedom. To explain in more detail, this robot 2 includes a base 20 fixed to the floor of a workplace, an arm 22 whose base end is rotatably connected to the base 20, and a distal end 22a of the arm 22. A work tool 24 is provided for performing work such as welding, sealing, painting, cutting, polishing, handling, etc. on the work W mounted on the turntable T. In this embodiment, the robot 1 is a six-axis multi-joint general-purpose robot, but the robot 1 is not limited to this, and any robot may be used.

<Description of the first camera>
The first cameras 31 to 39 are digital cameras that can take parallax images as well as color images. To install the first cameras 31 to 39 at the work site, in this embodiment, as shown in FIG. 1, the camera installation frame 30 is installed inside the safety fence D where the robot 2 and turntable T are installed. There is. The camera installation frame 30 consists of a first frame 30b, a second frame 30c, a third frame 30d, and a common horizontal member 30a that connects the first frame 30b, second frame 30c, and third frame 30d from the right side in the figure. The first frame 30b, the second frame 30c, and the third frame 30d each include a pair of left and right vertical members and a horizontal member that connects the upper ends of both vertical members. The common horizontal member 30a is provided so as to connect the centers of the respective horizontal members of the first frame 30b, the second frame 30c, and the third frame 30d.

First cameras 31 and 32 are installed vertically at intervals on the vertical member on the left side of the first frame 30b in the figure, and first cameras 33 and 34 are installed at vertical intervals on the vertical member on the right side in the figure. is set up. Further, a first camera 36 is installed on the vertical member on the left side of the second frame 30c in the drawing, and a first camera 37 is installed on the vertical member on the right side in the drawing. Further, a pair of left and right vertical members of the second frame 30c are provided with members 30c1 and 30c2 extending toward the left side in the drawing, and first cameras 38 and 39 are installed at the tip portions of the members 30c1 and 30c2.

<Description of the second camera>
Although not shown in FIG. 2, a second camera 28 shown in FIG. 8 is provided at the tip 22a of the arm 22. More specifically, as shown in FIG. 7, a mounting bracket 26 for mounting the working tool 24 is provided at the tip 22a of the arm 22. A camera housing 28a is attached and fixed to the horizontally extended portion of the mounting bracket 26 using screws or the like. As shown in FIG. 8(a), the camera housing 28a is formed in a hollow shape and shaped like a warhead in cross section. As shown in FIG. 8(b), an opening 28b is formed in the front part (left part in the figure) of the camera housing 28a. As shown in FIG. 8, the opening 28b is opened and closed by a shielding plate 28d rotatably attached to a rotation axis 28c provided at the front part (left part in the figure) of the camera housing 28a. It is now possible to do so. Although not shown, the rotation shaft 28c can be driven by a solenoid or the like.

As shown in FIG. 8, the second camera 28 is installed and fixed in the camera housing 28a, and is capable of capturing an image of the tip 24a of the working tool 24.

Although not shown, the second camera 28 is communicably connected to the operation terminal 6 (see FIG. 2).

<Description of the imaging processing terminal>
The imaging processing terminal 5 is an information processing device configured with, for example, a PC (Personal Computer). Specifically, as shown in FIG. 2, a CPU 50 and an input unit 51 that can input predetermined data into the imaging processing terminal 5 from the outside using a mouse, a keyboard, a touch panel, a USB connector, a Lightning connector, etc. , a ROM 52 consisting of a writable flash ROM or the like that stores a predetermined application program, etc., a RAM 53 that functions as a work area, a buffer memory, etc., a display section 54 that can display various images, and a wireless LAN and wired LAN. , and a communication unit 55 that can be connected to the network N by communication means such as dial-up.

<Description of the operating terminal>
The operation terminal 6 is an information processing device configured with, for example, a PC (Personal Computer). Specifically, it has the same configuration as the imaging processing terminal 5, and is composed of a CPU 60, an input section 61, a ROM 62, a RAM 63, a display section 64, and a communication section 65, as shown in FIG. Note that the ROM 62 stores a robot operation program used by the operator to move the robot 2 according to this embodiment, and a three-dimensional model 9 (see FIG. 4) used in the program.

<Display section description>
The display section 8 consists of one screen. The display unit 8 displays the image received by the imaging processing terminal 5 from one of the first cameras 31 to 39 selected from among the first cameras 31 to 39, and the robot operation program started by the operator on the operation terminal 6. The screen displayed on the display section 64 is projected onto the display section 8 by the projection device 70 and the projection device 71, respectively, and displayed in a superimposed manner.

<Explanation of usage examples of robot operation system>
A usage example of the operating method of the robot operation system 1 configured as described above will be specifically described with reference to FIG. 3. Note that the following content is just an example and is not limited thereto.

The CPU 50 (hereinafter sometimes abbreviated as the imaging processing terminal 5) of the imaging processing terminal 5 shown in FIG. Image reception is started (step S101).

Next, the imaging processing terminal 5 transmits an image of a predetermined one first camera among the images received from the first cameras 31 to 39 shown in FIG. (Step S102). Note that which first camera image is to be displayed on the display unit 8 first may be stored in advance in the ROM 52 of the imaging processing terminal 5. Here, it is assumed that the image P1 (see FIG. 4) captured by the first camera 31 is displayed on the display unit 8.

On the other hand, the CPU 60 of the operation terminal 6 shown in FIG. 2 (hereinafter sometimes abbreviated as the operation terminal 6) starts a robot operation program stored in the ROM 62 (step S1).

Next, the operating terminal 6 reads out the three-dimensional model 9 (see FIG. 4) stored in the ROM 62 of the operating terminal 6 (step S2).

Next, as shown in FIG. (step S3). Specifically, the operating terminal 6 generates a virtual space corresponding to the work site where the robot 2 and the workpiece W are placed as a three-dimensional coordinate space, and converts the three-dimensional model 9 read out in step S2 into a virtual space. Place in space. Then, the operating terminal 6 depicts the three-dimensional model 9 seen from the position corresponding to the first camera 31 in the virtual space, and displays it on the display section 8 (or the display section 64 of the operating terminal 6).

Next, the operation terminal 6 moves the three-dimensional model 9 (see FIG. 4) according to the operator's operation (step S4). Specifically, while looking at the display unit 8, the worker performs a predetermined operation using the input unit 61 (for example, dragging or (and drop) to move the three-dimensional model 9 to a desired position and orientation. The operation terminal 6 moves the three-dimensional model 9 according to the operator's operation. As a result, the three-dimensional model 9 is displayed on the display unit 8 in a state in which it assumes a position and orientation different from that of the robot 2 in the work site image P2, like the three-dimensional model 9A after movement shown in FIG. .

Next, the operation terminal 6 displays the position information of the three-dimensional model 9 displayed on the display unit 8 (depicts the three-dimensional model 9 from which position in the virtual space) and displays the position information on the display unit 8 (or 64) is transmitted to the imaging processing terminal 5 (step S5).

To explain this point in detail, the operating terminal 6 can depict the three-dimensional model 9 seen from any position in the virtual space and display it on the display unit 8 (or the display unit 64 of the operating terminal 6). . Therefore, when moving the three-dimensional model 9 in step S4, the operator must move the three-dimensional model 9 not only as seen from the side (see FIG. 4) but also as seen from above or diagonally below (see FIG. (not shown) can be displayed on the display section 8 (or the display section 64 of the operating terminal 6) for confirmation.

In step S5, positional information (for example, three-dimensional coordinates) from which position in the virtual space the three-dimensional model 9 is depicted and displayed on the display unit 8 (or the display unit 64 of the operating terminal 6) is transmitted by the operating terminal 6 to the imaging processing terminal 5.

Next, based on the received position information, the imaging processing terminal 5 (see FIG. 2) selects the first camera 31 to 39 ( (see FIG. 2) (step S103). Specifically, the operation terminal 6 uses the first camera 31 when the position information is greater than or equal to the threshold value A and less than B, and the first camera 32 when it is greater than or equal to the threshold value B and less than C, for example. The position information received from the operation terminal 6 and the first cameras 31 to 39 are stored in advance in association with each other. Then, the first camera corresponding to the received position information is selected from the first cameras 31 to 39. Here, it is assumed that the first camera 31 is selected.

The imaging processing terminal 5 receives the image of the first camera 31 selected as described above, and displays it on the display section 8 via the projection device 70.

Next, the operating terminal 6 detects whether an OK button (not shown) is pressed (step S6). Specifically, as shown in FIG. 5, the worker views the moved three-dimensional model 9A displayed on the display unit 8, and the robot 2, workpiece W, and surrounding situation in the image P2 of the work site. confirm. Then, the worker checks whether or not interference will occur with surrounding equipment at the work site when the robot 2 is moved according to the moved three-dimensional model 9A.

Furthermore, the worker also confirms whether the robot 2 is in a position and posture that it can take. Note that the operating terminal 6 may store in the ROM 62 in advance the range of positions and postures that the robot 2 can take. If the worker moves the 3D model 9 to take a position/orientation that exceeds this range, the robot 2 determines that the position/orientation is not possible, and the operating terminal 6 moves the 3D model 9 The color may be changed to a predetermined color (for example, red) to alert the operator or cancel the operation.

Next, when the operator presses a cancel button (not shown) (step S6: N), the operation terminal 6 returns to step S4. Specifically, since the operator has determined that the robot 2 should not be moved to the position and orientation of the three-dimensional model 9A shown in FIG. 5 after movement, the operator presses a cancel button (not shown). In response to this, the operating terminal 6 returns to step S4 without moving the robot 2 to the position of the moved three-dimensional model 9A, and moves the three-dimensional model 9 again according to the operator's operation.

On the other hand, when the operating terminal 6 detects that the OK button (not shown) has been pressed by the worker (step S6: Y), the operating terminal 6 moves the robot 2 (step S7). Specifically, the operating terminal 6 transmits parameters indicating the position and orientation of the three-dimensional model 9A after movement shown in FIG. (while the robot 2 is moving), the robot 2 is controlled to move to the position and orientation of the moved three-dimensional model 9A. As a result, as shown in FIG. 6, the robot 2 assumes a position and orientation that overlaps the moved three-dimensional model 9A.

Note that if the robot 2 is likely to interfere with surrounding devices while moving, the operator can avoid the interference by releasing the long-pressed OK button (not shown).

Next, the operating terminal 6 receives and displays the image of the second camera 28 shown in FIG. 7 (step S8). That is, the operator may not be able to confirm the detailed positional relationship between the tip 24a of the work tool 24 and the workpiece W using the image taken by the first camera 31 (see FIG. 6). Therefore, the operation terminal 6 causes the second camera 28 to take an image of the tip 24a of the work tool 24 and the workpiece W, receives the taken image from the second camera 28, and displays the image on the display section 8 (or the display section 6 of the operation terminal 6). ) (not shown). Thereby, the operator can confirm the position of the tip 24a of the work tool 24 and the workpiece W in more detail, and move the tip 24a to a predetermined position on the workpiece W more accurately. At this time, the PLC 29 rotates the rotation shaft 28c shown in FIG. 8, rotates the shielding plate 28d of the camera housing 28a, and opens the opening 28b as shown in FIG. 8(b). I have to.

Next, the operating terminal 6 instructs the robot 2 to perform processing operations such as welding (step S9). Specifically, the operating terminal 6 instructs the robot 2 to perform processing operations such as welding using a robot operating program or other predetermined program. At this time, the PLC 29 rotates the rotation shaft 28c shown in FIG. 8, rotates the shielding plate 28d of the camera housing 28a, and closes the opening 28b as shown in FIG. 8(a). There is. This makes it possible to protect the second camera 28 from welding spatter, arc light, drill chips, and other flying objects during the processing operation.

When the predetermined processing work is completed, the imaging processing terminal 5 and the operation terminal 6 end the processing.

Note that, after this, when processing a workpiece W of a different shape, the operator only has to place the workpiece W of a different shape on the turntable T. Since the first cameras 31 to 39 shown in FIG. 2 take images of the workpiece W, the operator can operate the robot 2 in the above-described operating method while checking the workpiece W on the display unit 8.

Therefore, according to the present embodiment described above, it is possible to provide a method for operating a robot operation system that allows a robot to operate with easy operation.

In other words, it has traditionally been possible to operate the robot using a teach pendant while directly viewing the workpiece at the work site, but operating the teach pendant is highly difficult and requires interference to move the robot safely. It is necessary to simultaneously check the presence or absence of On the other hand, offline teaching software does not use a teach pendant, but only moves the robot's CAD data on the computer, so it cannot actually move the robot. Furthermore, if there is a change in the layout of the work or equipment, it is necessary to reflect the change in the CAD data of the offline teaching software.

However, according to the present embodiment, the robot can be moved to a predetermined position by operating the three-dimensional model 9 displayed on the display unit 8 using the input unit 61 such as a mouse, so the teach pendant cannot be operated. The robot can be operated using simple operations like offline teaching software without having to worry about it.

Furthermore, according to the present embodiment, the three-dimensional model 9 is displayed superimposed on the image P2 of the work site captured by the first cameras 31 to 39. Therefore, after moving the 3D model 9, it is possible to visually check the 3D model 9, the actual workpiece W, and surrounding equipment to check whether there will be any interference with the surrounding equipment. . That is, it is not necessary to perform the movement operation and the confirmation work at the same time, and these tasks can be performed separately. Note that if there is a change in the layout of the workpiece W or surrounding equipment, the change can be immediately confirmed on the work site image P2, and there is no need to modify the CAD data using the application.

Furthermore, according to the present embodiment, when it is found as a result of visually checking the moved three-dimensional model 9, the actual workpiece W, and surrounding equipment that it is likely to be in an unfavorable state such as likely to interfere with surrounding equipment, By pressing the cancel button, the three-dimensional model 9 can be moved again. In this way, the optimal movement method can be found while redoing the operation of moving only the three-dimensional model 9A without moving the robot 2. At this time, since the robot 2 is not being moved, the time and effort of returning the robot 2 to its original position and redoing the movement is not required.

From the above, according to the present embodiment, there is provided an operating method for a robot operating system that overcomes the drawbacks of the online teaching method and the offline teaching method, takes advantage of their advantages, and allows the robot to operate with easy operation. can do.

<Explanation of modification example>
Note that the operating method of the robot operating system 1 shown in this embodiment is merely an example, and various modifications and changes are possible within the scope of the gist of the present invention as set forth in the claims. For example, in this embodiment, although the shielding plate 28d is illustrated, the present invention is not limited thereto, and a light shielding plate may be used. In this way, it is possible to protect the camera 28 from welding spatter, arc light, drill chips, and other flying debris from the machining work, while still allowing the camera 28 to take an image of the machining work.

Moreover, the camera housing 28a illustrated in this embodiment is just an example, and may have any shape. In short, it is sufficient to protect the camera 28 from welding spatter, arc light, drilling chips, and other flying objects generated during processing operations.

Further, in this embodiment, the entire three-dimensional model 9 is represented in a single color, but the operating terminal 6 allows the three-dimensional model 9b of the portion corresponding to the arm 22 of the robot 2 shown in FIG. 6 to be transparent. , may be displayed on the display section 8 (or the display section 64 of the operating terminal 6). In this way, the 3D model 9a of the portion corresponding to the tip 24a of the work tool 24 shown in FIG. 6 is hidden by the 3D model 9b of the portion corresponding to the arm 22 of the robot 2 and cannot be seen. There is no.

Further, in this embodiment, the imaging processing terminal 5 and the operation terminal 6 are two information processing devices. Dividing into two devices in this way has the advantage of faster processing speed. However, the processing of both the imaging processing terminal 5 and the operation terminal 6 may be performed by one information processing device.

When processing both the imaging processing terminal 5 and the operation terminal 6 as described above with one information processing device, the information processing device receives the images captured by the first cameras 31 to 39, and the received images The three-dimensional model 9 may be superimposed within the information processing device and displayed on the display section 64 or the display section 8 of the operation terminal 6.

Further, in this embodiment, the display unit 8 is one screen, but it may be one monitor. In that case, the imaging processing terminal 5 and the monitor may be connected with a monitor cable (not shown), and the image of the imaging processing terminal 5 may be displayed on the monitor. Then, the screen on which the robot operation program started by the operator on the operation terminal 6 is displayed on the display section 64 is projected onto the display section 8 by the projection device 71 and superimposed on the image displayed on the monitor. good.

In addition, in the present embodiment, in step S103, the imaging processing terminal 5 selects one of the first cameras 31 to 39 as the receiving source of the image displayed on the display unit 8, but the free viewpoint image may be generated and displayed on the display unit 8.

For example, the imaging processing terminal 5 generates a free viewpoint image as follows.

First, the imaging processing terminal 5 depicts the three-dimensional model 9 from which position in the virtual space it is viewed and displays the positional information (for example, the three-dimensional coordinates) from the operating terminal 6 (see step S5).

Next, the imaging processing terminal 5 calculates which position on the work site the received positional information corresponds to, and displays the images received from the first cameras 31 to 39 so that the images are viewed from the calculated position. calibrate. Note that in converting the position information indicating the position in the virtual space to the position of the work site, calculation may be performed using a known conversion method such as a method using a perspective projection matrix.

Finally, the imaging processing terminal 5 causes the display unit 8 to display an image (free viewpoint image) obtained by combining the images obtained as described above.

A free viewpoint image may be generated and displayed on the display unit 8 as described above.

Furthermore, instead of generating free viewpoint images, the first cameras 31 to 39 may be moved.

Specifically, in step S5, the imaging processing terminal 5 calculates which position in the work site the position information received from the operation terminal 6 corresponds to, and selects the first camera that can move to the calculated position as the first camera. 31 to 39, the selected first camera 31 to 39 is moved to the calculated position, and the image is displayed on the display unit 8.

As described above, any one of the first cameras 31 to 39 may be moved and displayed on the display unit 8. In addition, in this embodiment, although the turntable T was illustrated, it is applicable not only to this but to any thing, such as a two-axis positioner.

2 Robot 22 Arm 24 Work tool 28 Second camera 28a Camera housing 28 (protective member)
28d Shielding plate (protective member)
31 to 39 First camera 50 CPU (display control means) of the imaging processing terminal 5
60 CPU (control means) of operation terminal 6
61 Input section of operation terminal 6 (operation means)
62 ROM of operation terminal 6 (3D model storage unit)
8 Display parts 9, 9A 3D model 9b 3D model W of the part corresponding to the arm 22 Works P1 to P3 Images of the work site

Claims (4)

A robot that performs predetermined machining work on a workpiece,
a first camera capable of capturing images of the robot and the work;
a three-dimensional model storage unit that stores a three-dimensional model of the robot;
a display unit that superimposes and displays an image of the robot captured by the first camera and a three-dimensional model;
operating means for operating the three-dimensional model;
control means for controlling the robot;
A method of operating a robot operation system comprising:
When the three-dimensional model, which is displayed on the display section with the robot and the three-dimensional model superimposed , is moved to a predetermined position using the operating means, the three-dimensional model is moved to a predetermined position accordingly. a step of moving to a position;
When the three-dimensional model displayed on the display section is moved to a predetermined position using the operating means, the three-dimensional model is moved to the predetermined position accordingly, but the robot does not move. displaying the robot that has not been moved and the three-dimensional model that has been moved to the predetermined position in a superimposed manner on the display unit ;
having a worker check the display on the display unit in order to determine whether or not to move the robot to the predetermined position;
Even if the three-dimensional model moves to a predetermined position, the control means does not move the robot to the predetermined position unless a permission operation is performed using the operating means; An operating method for a robot operating system, comprising the step of causing the control means to move the robot to the predetermined position when the permission operation is performed. a second camera capable of capturing an image of a tip of a work tool provided on the arm of the robot;
The method of operating a robot operating system according to claim 1, further comprising: a protection member that protects the second camera when the robot performs the predetermined processing operation on the workpiece. further comprising display control means for controlling an image to be displayed on the display section,
The first camera is a plurality of first cameras,
selecting one of the plurality of first cameras by the display control means based on position information of the three-dimensional model displayed on the display unit;
2. The method of operating a robot operating system according to claim 1, further comprising the step of causing the display control unit to display an image captured by the selected first camera on the display unit. 2. The method of operating a robot operating system according to claim 1, wherein the three-dimensional model is displayed on the display section with a portion corresponding to the arm of the robot being transparent.

Images