JP2014188600A - Remote visual recognition device and remote visual recognition operation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to operate a master slave manipulator in an ordinary attitude.SOLUTION: According to an embodiment, a remote visual recognition device 100 for operating a manipulator device 3 for allowing an operator 40 to remotely handle a target 20 comprises: a view point measurement mechanism part 6 measuring a three-dimensional position and a visual line direction of the eyes of the operator 40 in real time; a monitor 5 attached to cover a shielding window 1a for observing a target 20 side from an operator 40 side of a shielding wall 1; a plurality of camera mechanism parts 51a and 51b including cameras 8a and 8b photographing the target 20 and camera drive parts 52a and 52b for the cameras 8a and 8b, respectively, and three-dimensionally grasping a shape of the target 20; and a control processing part 50 outputting operation instruction signals to the view point measurement mechanism part 6 and the camera mechanism parts 51a and 51b on the basis of an instruction from the operator 40, processing images picked up by the camera mechanism parts 51a and 51b, and outputting a display signal to the monitor 5.

Description

  Embodiments described herein relate generally to a remote viewing device and a remote viewing operation system using the remote viewing device.

  In facilities that handle radioactive materials, for example, reprocessing facilities for spent fuel assemblies, there are devices and rooms that have high radiation levels and cannot be directly accessed by people. In such facilities, mechanical and electric manipulators are used, and maintenance and repair work such as disassembly of various devices, replacement of parts, and assembly are performed by remote control.

  In order to prevent radiation exposure, a room for remote maintenance and repair is surrounded by a shielding wall such as lead or concrete. For example, a master-slave manipulator (hereinafter abbreviated as “MSM”) has a shielding wall. It is fixed to the shielding wall.

  In MSM, when an operator outside the shielding wall operates the master arm, the slave arm inside the shielding wall moves according to the movement of the master arm, thereby enabling remote operation. In normal MSM operation, there is a shielding window in front of the shielding wall to which the MSM is attached, and the operator performs remote work while directly confirming the working status of the slave side through this shielding window. Therefore, the remote visual recognition device for the remote maintenance work of the MSM operation is mainly a shielding window.

Japanese Patent Laid-Open No. 9-234685

  In the above-described remote visual recognition system, it is common to place an object to be remotely operated mainly in front of a shielding window and handle these work objects with the MSM. The MSM penetrates the shielding wall at a height of, for example, 2 to 3 m above the floor, and the shielding window is usually located directly below the shielding wall to which the MSM is fixed and the eye level at the operator's standing position. Is provided.

  In such a layout, the slave arm falls from top to bottom in front of the shielding window, and during work, the work object is often seen from the side of the slave arm as seen by the operator and is not visible through the shielding window. Will occur.

  In this conventional configuration, when looking at the tip of the grip attached to the tip of the slave arm, it is necessary to take a posture in which the wrist is twisted to either the left or right side. I had to plan my work. Originally, if the working state can be seen from the shielding window, remote maintenance work should be possible with the grip extended straight to the front. For this reason, there are problems such as poor workability and wide space occupied by the grip.

  Therefore, an object of the embodiment of the present invention is to enable operation of a master-slave manipulator in a normal posture.

  In order to achieve the above-described object, an embodiment of the present invention is fixed to a wall that separates an area where an object is placed and an area where an operator is located, and handles the object by being remotely operated by the operator. A remote visual recognition device for operating a manipulator device having a slave arm, a visual point measurement mechanism unit for measuring the visual point information of the operator in real time, a monitor attached to the operator side of the wall, and the target A plurality of camera mechanism units each having a camera for photographing an object and a camera drive unit for driving the camera, and calculating a three-dimensional shape of the object based on video data from the camera mechanism unit, and measuring the viewpoint A portion of the object seen by the operator based on viewpoint information data from a mechanism unit that is obstructed by the slave arm and cannot be seen is complemented with an image. A control processing unit that generates a virtual two-dimensional image that an operator should be able to see if not obstructed and outputs a display signal to the monitor so as to display it in a range corresponding to the invisible portion; It is characterized by providing.

  In addition, an embodiment of the present invention includes a manipulator device that penetrates a shielding wall against radiation and is fixed to the shielding wall, and an operator remotely handles an object, and a remote visual recognition device for operating the manipulator device. A remote visual recognition operation system comprising: a visual point measurement mechanism unit that measures real-time visual point information related to a three-dimensional position of the operator's eyes and a visual line direction of the operator; and the shielding wall. A monitor attached so as to cover a shielding window for observing the object side of the shielding wall from the operator side, a camera for photographing the object, and a camera driving unit for driving the camera. A viewpoint from the viewpoint measurement mechanism unit that calculates a three-dimensional shape of the object based on a plurality of camera mechanism units and video data from the camera mechanism unit Based on the report data, a portion of the object viewed by the operator that is obstructed by the slave arm is supplemented with an image, and the virtual object that the operator should be able to see if the slave arm is not obstructed. And a control processing unit that outputs a display signal to the monitor so as to generate a two-dimensional image and display it in a range corresponding to the invisible portion.

  According to the embodiment of the present invention, the master-slave manipulator can be operated in a normal posture.

It is a top view which shows the structure of the remote visual recognition apparatus which concerns on the 1st Embodiment of this invention. It is a side view which shows the structure of the remote visual recognition apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the remote visual recognition apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the flow of a process in a video processing calculating part etc. It is a top view which shows the structure of the remote visual recognition apparatus which concerns on the 2nd Embodiment of this invention. It is a side view which shows the structure of the remote visual recognition apparatus which concerns on the 2nd Embodiment of this invention. It is a side view which shows the structure of the remote visual recognition apparatus which concerns on the 3rd Embodiment of this invention. It is a side view which shows the structure of the laser beam projection mechanism of the remote visual recognition apparatus which concerns on the 4th Embodiment of this invention. It is a side view which shows the structure of the camera of the remote visual recognition apparatus which concerns on the 5th Embodiment of this invention. It is a front view which shows the filter in the camera of the remote visual recognition apparatus which concerns on the 5th Embodiment of this invention.

  Hereinafter, a remote visual recognition device according to an embodiment of the present invention will be described with reference to the drawings. Here, the same or similar parts are denoted by common reference numerals, and redundant description is omitted.

[First Embodiment]
FIG. 1 is a plan view showing the configuration of the remote visual recognition apparatus according to the first embodiment of the present invention. FIG. 2 is a side view showing the configuration of the remote visual recognition apparatus according to the first embodiment of the present invention.

  The remote visual recognition operation system includes a master slave manipulator (hereinafter abbreviated as “MSM”) 3 and a remote visual recognition device 100 for operating the MSM 3. The remote viewing device 100 includes a see-through monitor 5, camera mechanism units 51a and 51b, a viewpoint measurement mechanism unit 6, a light projecting mechanism unit 7, and a control processing unit 50.

  A shielding wall 1 is provided between the object 20 and the operator 40. The MSM 3 is provided in a state of being fixed to the shielding wall 1 through the shielding wall 1. The MSM 3 has a master arm 3a operated by the operator 40 on the outside of the shielding wall 1, that is, on the operator 40 side.

  Further, the MSM 3 has a slave arm 3b that directly handles the object 20 inside the shielding wall 1, that is, on the object 20 side. The slave arm 3b follows the movement of the master arm 3a and performs the same movement as the operation of the operator 40. A position detection sensor such as a position sensor or a resolver is attached to an operation axis such as a rotation axis or a telescopic axis of the slave arm 3b of the MSM 3, and position information of each part of the MSM 3 is output.

  A shielding window 1 a is fitted and fixed to the shielding wall 1 so as to penetrate the shielding wall 1. The shielding window 1a is a window for viewing the object 20 side from the operator 40 side, and is located at a position where the operator 40 can see the object 20 side on the opposite side of the shielding wall 1 from which the necessary range can be seen. It is a size. Further, the shielding window 1a has a function of shielding radiation from the object 20 side to the operator 40 side.

  A see-through monitor 5 is attached to the side of the operator 40 of the shielding window 1a so as to cover the shielding window 1a. When the monitor display is not performed on the see-through monitor 5, the operator 40 can see the object 20 side of the shielding wall 1 through the see-through monitor 5 and the shielding window 1a.

  When a monitor display signal in a predetermined range of the see-through type monitor 5 is input to the see-through type monitor 5, the monitor video by the see-through type monitor 5 is displayed for this predetermined display range, and other ranges are displayed. The operator 40 can see the object 20 side of the shielding wall 1 through the see-through monitor 5 and the shielding window 1a.

  The see-through monitor 5 may be a liquid crystal monitor, for example. For example, it is known that by using an organic EL element and a transparent electrode, it is possible to manufacture a transparent monitor having a high transparency like transparent glass when not displayed.

  Camera mechanism portions 51 a and 51 b are attached to the oblique upper portions of the shielding window 1 a on the inner wall surface of the shielding wall 1. The camera mechanism units 51a and 51b have camera drive units 52a and 52b that adjust the orientation of the camera 8a and the camera 8b, respectively. Both the cameras 8a and 8b are oriented in such a direction that the object 20 and the slave arm 3b are within the imaging range.

  As shown in this embodiment, a plurality of cameras are necessary. That is, in order to grasp the object 20 three-dimensionally, images from at least two different places are necessary. In the present embodiment, the case where two cameras are provided has been described, but three or more cameras may be used as necessary.

  Further, when the size of the object 20 and the range of movement of the slave arm 3b in order to access related tools for work are taken into consideration, the cameras 8a, 8b Is preferably a wide-angle camera.

  A viewpoint measuring mechanism 6 is provided outside the shielding wall 1. The viewpoint measurement mechanism unit 6 measures the eye position of the operator 40 and the direction of the line of sight in real time as information on the direction of the eye of the operator 40. The viewpoint measurement mechanism unit 6 has an eye tracking function. This can be realized by a combination of a laser range finder, a camera, an image processing apparatus, and the like. For example, a highly accurate motion detection device is commercially available as a hobby consumer product, and such a device may be used.

  Above the shielding window 1a on the inner wall surface of the shielding wall 1, a light projecting mechanism portion 7 is attached. The light projecting mechanism unit 7 projects light into a range including the object 20. Here, when it is necessary to accurately grasp the object 20, the light is desirably a laser beam. In this case, the light projecting mechanism unit 7 includes a laser transmitter (not shown).

  The light may include invisible light (for example, infrared light). In this case, the cameras 8a and 8b are also sensitive to invisible light. About invisible light, you may project the grid light on a grid line, for example by providing the grid light projection mechanism part 7c as shown in FIG. In this case, the cameras 8a and 8b can capture the grid lines irradiated on the object 20 by the grid light, which is effective in improving the accuracy of grasping the shape of the object 20. In addition, since the grid light is projected with invisible light, the grid lines on the object 20 are not visible to the eyes, so that it is possible to avoid a situation where the grid lines are added and difficult to see in operation. Alternatively, instead of projecting grid light, unidirectional light projection may be scanned.

  A control processing unit 50 that constitutes a part of the remote visual recognition device 100 is provided outside the shielding wall 1. The position information signal of each part of the MSM 3 is sent to the control processing unit 50 in real time. Further, the camera image signals from the camera mechanism units 51a and 51b, the eye position of the operator 40 and the line-of-sight information from the viewpoint measurement mechanism unit 6 are sent to the control processing unit 50 in real time. The line-of-sight information is sent from the control processing unit 50 to the viewpoint measurement mechanism unit 6 in real time. Further, a signal related to monitor display is sent from the control processing unit 50 to the see-through monitor 5 in real time.

  Transmission / reception of these signals is performed via the cable 12. In the present embodiment, the case where all of the input / output of the above information is via the cable 12 is shown, but the present invention is not limited to this. For example, part or all of input / output may be performed wirelessly.

  FIG. 3 is a block diagram showing the configuration of the remote visual recognition apparatus according to the first embodiment of the present invention. As described above, the remote visual recognition device 100 includes the see-through monitor 5, the camera mechanism units 51a and 51b, the viewpoint measurement mechanism unit 6, the light projecting mechanism unit 7, and the control processing unit 50. In addition, the control processing unit 50 includes an input unit 13, a video capturing unit 15, a video processing calculation unit 16, and a 2D video output unit 18.

  The input unit 13 receives a signal relating to the eye position and line-of-sight information of the operator 40 output from the viewpoint measurement mechanism unit 6. Further, the position information of each part of the MSM 3 output from the MSM 3 is input to the input unit 13.

  The video capturing unit 15 receives captured video signals of the cameras 8a and 8b output from the camera mechanism units 51a and 51b, respectively.

  The video processing calculation unit 16 receives the eye position and line-of-sight information of the operator 40 input to the input unit 13, position information of each part of the MSM 3, and the captured video signals of the cameras 8 a and 8 b input to the video capturing unit 15. Is done.

  The video processing calculation unit 16 includes a mapping calculation unit 61 and a two-dimensional video calculation unit 62. The mapping calculation unit 61 of the video processing calculation unit 16 calculates three-dimensional data of the object 20 and the slave arm 3b based on the captured video signals of the cameras 8a and 8b. That is, shape data such as the shape of the object 20 in the three-dimensional space and the outline line of the slave arm 3b are calculated.

  Specifically, a plurality of representative points of the object 20 are identified as selection points, and for each selection point, a three-dimensional position in real space is calculated based on the principle of triangulation, and created by the calculated point group. A 3D composite image is mapped onto a curved surface in a 3D space and a 3D shape is output.

  In addition, the 2D video computation unit 62 of the video processing computation unit 16 includes the 3D composite video synthesized by the mapping computation unit 61, the information about the eye position and line-of-sight direction of the operator 40 received from the viewpoint measurement mechanism unit 6. Based on the position information signal of each part of the MSM 3, when the object 20 is viewed from the operator 40, a range in which the object 20 cannot be seen by the slave arm 3b is calculated. Further, for this range, when the object 20 is viewed from the operator 40 based on the result of the above calculation, a two-dimensional image of the range in which the object 20 cannot be seen is synthesized by the slave arm 3b.

  The two-dimensional video output unit 18 displays the image synthesized by the video processing arithmetic unit 16 at a place where the target 20 cannot be seen by the slave arm 3b when the operator 40 views the target. Then, the image signal in this range is output to the see-through monitor 5.

  As a result, on the see-through monitor 5, an image in a range where the object 20 cannot be seen by the slave arm 3b is displayed in real time. It should be noted that the display of the see-through monitor 5 is shown when the operator 40 has no image, that is, when the slave arm 3b blocking the object is seen and when the image of that part of the object 20 is being projected. For example, the switching operation may be performed.

  Here, if the operator 40 can visually perform the work while grasping the object 20 three-dimensionally, the operation becomes easy. For this purpose, the viewpoint measurement mechanism unit 6 measures the positions and line-of-sight directions of the left and right eyes of the operator 40 separately.

  In addition, the video processing calculation unit 16 displays two-dimensional images for the two types when the object 40 is viewed from the left and right eyes of the operator 40 in the range where the object 20 cannot be seen by the slave arm 3b. Synthesize. The two-dimensional two types of images are alternately output and a synchronization signal is output.

  The operator 40 receives the synchronization signal and wears the synchronization glasses 21 that can synchronize with this video. The synchronization glasses 21 are opened and closed separately for the left and right sides that receive the synchronization signals for the left and right sides. The operator 40 can grasp visually and three-dimensionally by wearing the synchronous glasses 21 and viewing this video. As the synchronization glasses 21, for example, glasses with a liquid crystal shutter may be used.

  Since the operation of the MSM 3 is mainly performed in front of the shielding window 1a, part of the light from the light projecting mechanism unit 7 is blocked by the slave arm 3b during the work in the MSM 3, and the surface of the object 20 The shape of the non-projecting part of the slave arm 3b is projected as a shadow.

  FIG. 4 is a flowchart showing the flow of processing in the video processing calculation unit and the like.

  By using this non-projected portion, that is, the boundary outside the shadow picture, the image processing calculation unit 16 can grasp the target object with higher accuracy. Specifically, for example, there are the following methods.

  The video processing calculation unit 16 extracts a shadow edge image (step S1).

  After step S1, the representative point of the outer boundary of the shadow image, which is a non-projected part of the captured images (hereinafter referred to as the left and right images) of the cameras 8a and 8b, is identified as a selected point. The identification is derived from the correlation between the left and right images including singular points and the like. Note that, as the selection point, for example, there are a number of points divided on an edge portion or a side (step S2).

  After step S2, for each identified selected point, the position (X, Y, Z) of each point in the actual space according to the principle of triangulation from the two-dimensional position of the left and right images, that is, the (x, y) coordinate position. ) Is calculated (step S3).

  The video obtained in step S3 is mapped to a virtual three-dimensional space, and if there is a synthesized three-dimensional image, it is added to the three-dimensional image to create a three-dimensional digital video (step S4).

  Based on the 3D digital image mapped in the virtual 3D space obtained in step S4, the 2D digital image when viewing the 3D digital image with the eye position and line-of-sight direction of the operator 40 as the virtual camera. Is created (step S5). Note that the synthesis of the two-dimensional digital image is an image from the dominant position of the operator 40. Which one is dominant may be input, or may be determined at the time of the first calibration.

  The two-dimensional video output unit 18 extracts only the range blocked by the slave arm 3b when viewed from the virtual camera in the virtual space from the two-dimensional digital video obtained in step S5, and sees the video of the extracted range in a see-through monitor. 5 (step S6).

  Next, the operation of the present embodiment configured as described above will be described. When the operator 40 handles the object 20 using the MSM 3, when the operator 40 views the object 20 or the like, a portion that is not visible due to being blocked by the slave arm 3b occurs.

  On the other hand, the viewpoint measurement mechanism unit 6 always tracks the positions of the left and right eyes of the operator 40 and the respective gaze directions by the eye tracking function, and outputs the information signal. In addition, position information of each part of the slave arm 3b itself is also output. These output signals are taken into the input unit 13 of the control processing unit 50.

  Each of the cameras 8 a and 8 b outputs an image of the object 20 and the slave arm 3 b that handles the object 20, and the image information signal is captured by the image capturing unit 15 of the control processing unit 50.

  In the control processing unit 50, information signals regarding the eyes and lines of sight of the operator 40 captured by the input unit 13, position information signals of each part of the slave arm 3 b, and video information of the cameras 8 a and 8 b captured by the video capturing unit 15. The signal is input to the video processing calculation unit 16.

  The video processing calculation unit 16 calculates three-dimensional information of the object 20 based on the video information signals of the cameras 8a and 8b. Further, based on the position and line-of-sight information of the operator 40 and the position information of each part of the MSM 3, when the object is viewed from the operator 40, a range in which the object 20 cannot be seen by the slave arm 3b is calculated.

  The video processing calculation unit 16 further synthesizes a two-dimensional image in a range where the target 20 cannot be seen by the slave arm 3b when the target 20 is viewed from the operator 40 based on the result of the above calculation. This result is output from the two-dimensional video output unit 18 to the see-through monitor 5.

  The operator 40 who sees the see-through monitor 5 can perform an operation with a sense that the entire object can be actually seen by the object 20 that is actually visible and the image on the see-through monitor 5 that covers the invisible part. .

  In addition, when the image on the see-through monitor 5 that covers the invisible portion is configured to be alternately displayed as an image corresponding to the left and right eyes, by wearing synchronous glasses synchronized with this image, this The image on the see-through type monitor 5 that covers the invisible part can be seen more stereoscopically.

  As described above, according to the present embodiment, it is not necessary to operate the MSM 3 in an unnatural posture in order to avoid that a part of the object 20 is blocked by the slave arm 3b, and the MSM 3 in a normal posture. Operation becomes possible.

[Second Embodiment]
FIG. 5 is a plan view showing the configuration of the remote visual recognition apparatus according to the second embodiment of the present invention. FIG. 6 is a side view showing the configuration of the remote visual recognition apparatus according to the second embodiment of the present invention.

  This embodiment is a modification of the first embodiment. Out of the object 20, the outer portion of the object that cannot be seen by the slave arm 3 b during the operation of the MSM 3 when viewed from the operator 40, the irradiation range and the light that receive light from the light emitting unit 7 a of the light projecting mechanism unit 7 The light projecting mechanism 7 is caused to follow so as to minimize the difference in the boundary of the non-irradiation range that does not receive the light.

  In the first embodiment, the light projecting mechanism unit 7 is provided on the inner side of the shielding wall 1, that is, on the wall surface of the shielding wall 1 on the object 20 side. However, in this embodiment, the outer side of the shielding wall 1, that is, the operator. 40 side is provided. Further, the light emitting unit 7 a of the light projecting mechanism unit 7 is provided immediately above the head of the operator 40.

  The reason why it is provided directly above is to prevent the movement of the operator 40 from being obstructed. If the movement of the operator 40 is not obstructed, the vicinity of the eyes of the operator 40 is The lower side of the front of the face of the operator 40 may be used.

  The light projecting mechanism unit 7 emits light at a position near the operator 40 following the eyes of the operator 40 that moves during the operation based on the position information of the eyes of the operator 40 obtained in real time by the viewpoint measurement mechanism unit 6. The light emitting unit position is moved in real time so that the position of the unit 7a is maintained. This function may be performed by causing the operator 40 to attach the light emitting unit 7a.

  In addition, in the imaging | photography with the cameras 8a and 8b, it is necessary to image | photograph a non-irradiation part and to image | video. When illuminance for identifying the non-irradiated part is necessary, such as when photographing of the non-irradiated part is difficult because the target is dark, the entire object 20 is separated from the light projecting mechanism unit 7 as necessary. What is necessary is just to provide the light projection apparatus which illuminates.

  In the present embodiment configured as described above, when the video processing calculation unit 16 calculates the shape information of the target 20 in the three-dimensional space based on the captured video signals of the cameras 8a and 8b, the non-target of the target 20 is determined. The accuracy when using the selection point of the boundary point of the light projecting portion is improved.

  The purpose of the synthesis of the three-dimensional shape is to synthesize a two-dimensional image in a range where the object 20 cannot be seen by the slave arm 3b when the operator 20 views the object 20. Of the two-dimensional images, a two-dimensional image can be generated efficiently and accurately in that the image of the irradiation range of the object 20 can be used as it is.

[Third Embodiment]
FIG. 7 is a side view showing the configuration of the remote visual recognition apparatus according to the third embodiment of the present invention. This embodiment is a modification of the first embodiment or the second embodiment. In the present embodiment, the outer portion of the object 20 that is not seen by the slave arm 3b during the operation of the MSM 3 when viewed from the operator 40, and the light from the light emitting unit 7a of the light projecting mechanism unit 7 are also displayed. In addition to bringing the boundary between the irradiation range received and the non-irradiation range not receiving light closer, the light projecting mechanism unit 7 is made to follow so as to match.

  In this embodiment, the half mirror 25 is provided between the see-through monitor 5 and the shielding window 1a. The light projecting mechanism unit 7 is attached above the half mirror 25 in a direction of emitting light downward.

  Here, the half mirror 25 is not limited to a case where the ratio of transmission and reflection is 50% each, and refers to a half mirror that has a function of transmitting part of light and reflecting part of light. It is out.

  Moreover, the light projection part drive mechanism 23 which supports the light projection mechanism part 7 from upper direction is provided. The light projecting unit driving mechanism 23 is fixedly supported by the shielding wall 1. The light projecting unit driving mechanism 23 adjusts the light projecting direction of the light projecting mechanism unit 7 and also adjusts the horizontal position of the light projecting mechanism unit 7.

  A signal from the viewpoint measuring mechanism unit 6 is input to the light projecting unit driving mechanism 23, and the horizontal position and the light projecting direction of the light projecting mechanism unit 7 are controlled in real time based on the position information of the eyes of the operator 40.

  In addition, in the imaging | photography with the cameras 8a and 8b, it is necessary to image | photograph a non-irradiation part and to image. When illuminance for identifying the non-irradiated part is necessary, such as when photographing of the non-irradiated part is difficult because the target is dark, the entire object 20 is separated from the light projecting mechanism unit 7 as necessary. What is necessary is just to provide the light projection apparatus which illuminates.

  In the present embodiment having such a configuration, the light projecting unit driving mechanism 23 adjusts the horizontal position and the light projecting direction of the light projecting mechanism unit 7 so that the center of the light emitted from the light projecting mechanism unit 7 is It can be equivalent to that emitted from the eye position of the operator 40.

  As a result, in the target object 20 viewed from the operator 40, a range where the light is blocked by the slave arm 3b and cannot be seen, and a non-light-projected part range where the light is blocked by the slave arm 3b and cannot be applied to the target object 20. Match.

  As a result, the operator 40 is easy to see because there is no non-projection range in the range of the object 20 to be actually viewed other than the two-dimensional image on the see-through monitor 5, and the light is applied to all.

  Further, when the video processing calculation unit 16 calculates the shape information of the object 20 in the three-dimensional space based on the captured video signals of the cameras 8a and 8b, the selection point of the boundary point of the non-projection part of the object 20 is selected. Accuracy when used is improved.

  The purpose of the synthesis of the three-dimensional shape is to synthesize a two-dimensional image in a range where the object 20 cannot be seen by the slave arm 3b when the operator 40 views the object. Of the two-dimensional images, the two-dimensional image generation can be performed efficiently and accurately in that the image of the irradiation range of the object 20 can be used as it is.

[Fourth Embodiment]
FIG. 8 is a side view showing the configuration of the laser light projecting mechanism of the remote visual recognition apparatus according to the fourth embodiment of the present invention. This embodiment is a modification of the third embodiment, and a laser light projecting mechanism 7b is supported by a light projecting unit driving mechanism 23 (see FIG. 7) instead of the light projecting mechanism unit 7 in the third embodiment. Is provided.

  The laser light projecting mechanism 7b includes a housing 24, a half mirror 27a attached to the housing 24, a half mirror 25 housed in the housing 24, lenses 26a and 26b, a half mirror 27b, a laser oscillator 28, and a camera mechanism. 29.

  As in the case of the half mirror 25 in the third embodiment, the half mirrors 27a and 27b have a function of transmission and reflection in this embodiment, including the case where the ratio of transmission and reflection is not 50% each. It is called half mirror.

  Laser light emitted from the laser oscillator 28 passes through the lens 26b, passes through the half mirror 27b and the half mirror 27a, and is projected.

  On the other hand, the image viewed from the camera mechanism 29 is transmitted through the half mirror 27a, reflected by the half mirror 27b, reflected by the half mirror 25, and passes through the lens 26a.

  In the present embodiment configured as described above, the same image as the image seen from the operator 40 is displayed in addition to the fact that the laser light appears to be emitted from the eye position of the operator 40 equivalently. Photographed by mechanism 29.

  As a result, the video processing calculation unit 16 calculates the shape information in the three-dimensional space of the object 20 based on the captured video signals of the cameras 8a and 8b, and when the operator 40 views the object 20, the slave arm 3b By directly using the video from the camera mechanism 29 when setting the range that cannot be seen, the calculation process can be greatly reduced, and more accurate two-dimensional image composition can be performed. .

[Fifth Embodiment]
FIG. 9 is a side view showing the configuration of the camera of the remote viewing device according to the fifth embodiment of the present invention. This embodiment is a modification of the first to third embodiments. In the first to third embodiments, a laser beam projecting mechanism 7b is provided instead of the projecting mechanism unit 7, or a laser beam is used. In this embodiment, laser light is transmitted depending on whether or not the light projecting mechanism 7b is further provided.

  The camera mechanism 29 in this embodiment includes a camera 29a and a filter mechanism 31. The filter mechanism 31 includes a motor 32, a control circuit 33, a rotation mechanism 34, and a disk 36.

  FIG. 10 is a front view showing a filter in the camera of the remote visual recognition apparatus according to the fifth embodiment of the present invention. The disk 36 has an annular shape, and two types of filters, a filter 35a and a filter 35b, are provided in a semicircular shape inside thereof.

  Of the two types of filters, one filter 35a cuts off infrared rays that are invisible light and passes only visible light. The other filter 35b is a filter for passing only the infrared wavelength from the laser oscillator. In addition, in this embodiment, although the case of the infrared of a long wavelength side is shown about invisible light, it is not limited to this. For example, ultraviolet rays on the short wavelength side may be used.

  The control circuit 33 controls the motor 32 to rotate exactly once every video frame, that is, every 1/30 seconds, for example, so that the filters 35a and 35b alternately turn to the front of the lens 30 every half frame. To do.

  In addition, the control circuit 33 drives the left and right cameras 8a and 8b in synchronization with the synchronization signal from the control processing unit 50, so that a visible light image and an infrared image can be alternately captured on the left and right simultaneously.

  The use of invisible light and visible light may be different from each other, for example, invisible light may be emitted by projecting grid light on a grid line, and visible light may be normal illumination. In this case, the camera 29a can capture the grid lines irradiated on the object 20 by the grid light, which is effective in improving the accuracy of grasping the shape of the object 20. In addition, since the grid light is projected with invisible light, the grid lines on the object 20 are not visible to the eyes, so that it is possible to avoid a situation where the grid lines are added and difficult to see in operation. Alternatively, instead of projecting grid light, unidirectional light projection may be scanned.

  By providing such a camera mechanism 29 at a location where the plurality of cameras 8a and 8b according to the first to third embodiments are provided, it is possible to more easily and easily detect the shadow portion of the laser light from the camera image. Can be extracted.

[Other Embodiments]
As mentioned above, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. For example, in the embodiment, the range that is obstructed by the slave arm 3b in the object 20 is displayed as an image, but is not limited to the object 20. For example, the object to be visualized may include mechanical instruments for handling the object. Moreover, you may combine the characteristic of each embodiment.

  Furthermore, these embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations of the embodiments can be made without departing from the spirit of the invention. . These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Shielding wall, 1a ... Shielding window, 3 ... Master slave manipulator (MSM, manipulator apparatus), 3a ... Master arm, 3b ... Slave arm, 5 ... See-through type monitor (monitor), 6 ... Viewpoint measurement mechanism part, 7 ... Light projecting mechanism, 7a ... Light emitting unit, 7b ... Laser light projecting mechanism, 7c ... Grid light projecting mechanism, 8a, 8b ... Camera, 12 ... Cable, 13 ... Digital input interface board (input unit), 15 ... Image capturing unit, 16 ... Video processing operation unit, 18 ... 2D image output unit, 20 ... Target device (object), 21 ... Synchronous glasses, 23 ... Projection unit drive mechanism, 24 ... Housing, 25 ... Half mirror 26a, 26b ... lens, 27a, 27b ... half mirror, 28 ... laser oscillator, 29 ... camera mechanism, 29a ... camera, 30 ... lens, 31 ... fill -Mechanism, 32 ... motor, 33 ... control circuit, 34 ... rotating mechanism, 35a, 35b ... filter, 36 ... disk, 40 ... operator, 50 ... control processing unit, 51a, 51b ... camera mechanism unit, 52a, 52b ... Camera drive unit 61 ... Mapping calculation unit 62 ... 2D image calculation unit 100 ... Remote visual recognition device

Claims (12)

A remote visual recognition device for operating a manipulator device having a slave arm that is fixed to a wall separating an area where an object is disposed and an area where an operator is located and is remotely operated by the operator and handles the object There,
A viewpoint measurement mechanism unit that measures the viewpoint information of the operator in real time;
A monitor attached to the operator side of the wall;
A plurality of camera mechanisms each having a camera for photographing the object and a camera driving unit for driving the camera;
A three-dimensional shape of the object is calculated based on video data from the camera mechanism, and is obstructed by the slave arm among the objects viewed by the operator based on viewpoint information data from the viewpoint measurement mechanism. The invisible portion is complemented with an image, and a virtual two-dimensional image that the operator should be able to see if it is not obstructed by the slave arm is generated and displayed in a range corresponding to the invisible portion. A control processing unit for outputting a display signal to the monitor;
A remote visual recognition device comprising: The control processing unit
An input unit for inputting the viewpoint information signal output from the viewpoint measurement mechanism unit and the position information signal of each part of the manipulator device output from the manipulator device;
A video capturing unit for capturing a digital video signal from the camera;
A video processing calculation unit that receives the output from the input unit and the video capturing unit, calculates the three-dimensional shape synthesized by video processing calculation, and generates and outputs the two-dimensional video;
A 2D video output unit for outputting the 2D video from the video processing operation unit to the monitor;
The remote visual recognition device according to claim 1, comprising: The cameras are provided at predetermined intervals in the space on the object side of the wall,
The video processing arithmetic unit is
A plurality of points of the object are identified as selection points, a three-dimensional position in real space is calculated for each of the selection points based on the principle of triangulation, and a curved surface in a three-dimensional space created by the calculated point group A mapping operation unit for mapping the 3D composite video to output the 3D shape;
Based on the operator's eye position and line-of-sight direction based on the 3D composite video, the operator's viewpoint information received from the viewpoint measurement mechanism, and the position information signal of each part of the manipulator device A 2D video calculation unit for creating a 2D video;
The remote visual recognition device according to claim 2, comprising: The left and right shutters are further equipped with synchronized glasses that can be opened and closed separately.
The viewpoint measurement mechanism unit measures the position and line-of-sight direction of the left and right eyes of the operator, and outputs left-eye information including information on the left-eye position and line-of-sight, and right-eye information including information on the right-eye position and line-of-sight direction. Output to the control processing unit,
The two-dimensional image calculation unit creates a two-dimensional image viewed from the left eye of the operator based on the left eye information and a two-dimensional image viewed from the right eye of the operator based on the right eye information, and displays the two-dimensional image on the monitor. Output alternately, and output synchronization signals for opening and closing each of the left and right shutters to the synchronization glasses,
The remote viewing device according to claim 3, wherein the operator can obtain stereoscopic vision by wearing the synchronous glasses. A grid light projecting mechanism for projecting grid light onto the object to generate grid lines that shine on the object;
The video processing calculation unit performs a three-dimensional calculation of the object based on information from the grid lines.
The remote visual recognition device according to any one of claims 2 to 4, wherein the device is a remote visual recognition device. The grid light is invisible;
The remote viewing device according to claim 5, wherein the camera includes a color filter and a spectroscopic device, and is capable of separately capturing two images of a visible light image and a non-visible light image. A light projecting mechanism that has a light emitting unit that projects illumination light on the object and is capable of adjusting the position of the light emitting unit;
The control processing unit minimizes the difference between the irradiation range irradiated with light from the light emitting unit and the non-irradiation range not irradiated in the object, and the outline of the invisible part, A command signal is issued to the light projecting mechanism unit so that the position of the light emitting unit follows the position near the eyes of the operator.
The remote visual recognition device according to claim 1, wherein the remote visual recognition device is provided. A light projecting mechanism that has a light emitting unit that projects illumination light onto the object and can adjust the position of the light emitting unit;
A half mirror that reflects the illumination light and enables the operator to visually recognize the object side;
Further comprising
The light emitting unit is arranged toward the half mirror, and the illumination light is reflected by the half mirror to illuminate a target device on the target side of the wall,
The control processing unit is configured to minimize a difference between a boundary between an irradiation range irradiated with light from the light emitting unit and a non-irradiation range that is not irradiated in the object, and an outline of the invisible portion. A command signal is issued so that the light projecting mechanism section follows so that the origin of light from the light emitting section is the position of the eyes of the operator.
The remote visual recognition device according to claim 1, wherein the remote visual recognition device is provided. A light projecting mechanism that has a light emitting unit that projects illumination light on the object and is capable of adjusting the position of the light emitting unit;
The control processing unit minimizes the difference between the irradiation range irradiated with light from the light emitting unit and the non-irradiation range not irradiated in the object, and the outline of the invisible part, A command signal is issued to the light projecting mechanism unit so that the position of the light emitting unit follows the position near the eyes of the operator.
The remote visual recognition device according to any one of claims 3 to 6, wherein the remote visual recognition device is provided. A light projecting mechanism that has a light emitting unit that projects illumination light onto the object and can adjust the position of the light emitting unit;
A half mirror that reflects the illumination light and enables the operator to visually recognize the object side;
Further comprising
The light emitting unit is arranged toward the half mirror, and the illumination light is reflected by the half mirror to illuminate a target device on the target side of the wall,
The control processing unit is configured to minimize a difference between a boundary between an irradiation range irradiated with light from the light emitting unit and a non-irradiation range that is not irradiated in the object, and an outline of the invisible portion. A command signal is issued so that the light projecting mechanism section follows so that the origin of light from the light emitting section is the position of the eyes of the operator.
The remote visual recognition device according to any one of claims 3 to 6, wherein the remote visual recognition device is provided.   The said selection point is a point of a boundary with the irradiation range of the said target object, and the non-irradiation range which the said slave arm does not receive by the said slave arm, The Claim 9 or Claim 10 characterized by the above-mentioned. Remote viewing device. A remote vision operation system comprising a manipulator device that penetrates a shielding wall against radiation and is fixed to the shielding wall so that an operator remotely handles an object, and a remote vision device for operating the manipulator device,
The remote viewing device includes:
A viewpoint measurement mechanism unit that measures in real time viewpoint information regarding the three-dimensional position of the operator's eyes and the line-of-sight direction of the operator;
A monitor attached to cover a shielding window for observing the object side of the shielding wall from the operator side of the shielding wall;
A plurality of camera mechanisms each having a camera for photographing the object and a camera driving unit for driving the camera;
A three-dimensional shape of the object is calculated based on video data from the camera mechanism, and is obstructed by the slave arm among the objects viewed by the operator based on viewpoint information data from the viewpoint measurement mechanism. An invisible portion is complemented with an image, and a virtual two-dimensional image that the operator should be able to see if it is not obstructed by the slave arm is generated and displayed in a range corresponding to the invisible portion. A control processing unit for outputting a display signal to the monitor;
A remote visual recognition operation system characterized by comprising:

Images