JP5968178B2 - Image processing apparatus for remote control, remote control system, and image processing method for remote control - Google Patents

Description

  The present invention relates to an image processing device for remote control, a remote control system, and an image processing method for remote control.

For example, when performing remote control of an object to be controlled such as a vehicle, a laser radar or the like that can detect an object in three dimensions is installed in the vehicle as a detection unit that detects terrain or obstacles on which the vehicle travels. Then, a driver who remotely controls the vehicle remotely controls the vehicle while confirming on the monitor a three-dimensional map composed of images based on the detection result by the detection means. Specifically, a three-dimensional map corresponding to the three-dimensional coordinates of the object detected by the detecting means is displayed on the monitor.
As an example of such a remote monitoring device, Patent Document 1 discloses that an infrared image of a specific range ahead of a vehicle is taken by an infrared camera installed in the vehicle, and a specific range of 3 is measured by a three-dimensional shape measuring device installed in the vehicle. Detecting the three-dimensional shape, the road surface area where the vehicle is easy to pass from the three-dimensional shape is detected by the arithmetic processing device, and a corrected infrared image is formed on the infrared image so that the road surface region looks smooth and corrected by the display device An apparatus for displaying an infrared image is disclosed.

JP 2011-39727 A

FIG. 11 shows an example of a screen that is detected by the laser radar 102 installed in the vehicle 100 and is displayed on the monitor 106 of the remote control device 104. A screen A shown in FIG. 11 shows an accurate screen to be displayed to a pilot who performs remote control, and one voxel 108 indicating a detection target is displayed. However, when the detection target is a moving object such as a person or a vehicle, the laser radar 102 accumulates the position information of the detection target at that time each time the three-dimensional coordinates of the detection target are acquired, as shown in FIG. As shown in the screen B, the movement history of the detection target is displayed as it is on the monitor 106 as a plurality of voxels 108.
When the display as shown in the screen B is displayed on the monitor 106, there is a possibility that the operator may hinder the operation of the vehicle 100.

  The present invention has been made in view of such circumstances, and is capable of accurately displaying the periphery of an operation target on display means disposed at a remote place, an image processing apparatus for remote control, a remote control system, and a remote control An object is to provide an image processing method for steering.

  In order to solve the above problems, the image processing apparatus for remote control, the remote control system, and the image processing method for remote control of the present invention employ the following means.

  An image processing device for remote control according to the first aspect of the present invention is a remote control image processing device for displaying an image for remotely controlling a control object on a display means, and is installed on the control object and is a detection target. The three-dimensional detection means for detecting the position of the three-dimensionally in the vertical, horizontal, and depth, and the coordinates detected by the three-dimensional detection means as coordinates to be displayed on the screen of the display means by applying an image of a predetermined shape Out of the three-dimensional coordinates for each detection target detected by the three-dimensional detection means and the storage means for storing the three-dimensional coordinates for each detection target, the three-dimensional coordinates not stored in the storage means are extracted. Additional coordinate extraction means for adding and storing in the storage means, and determining whether or not the detection target has moved based on a time change of the three-dimensional coordinates, and corresponding to the detection target determined to have moved That and a deletion coordinate extracting means for extracting for deletion from the 3-dimensional coordinates said storing means.

  The image processing apparatus for remote control according to this configuration is an apparatus that displays an image for remotely maneuvering the operation target on the display unit, and the position of the detection target is vertically moved by the three-dimensional detection unit installed on the operation target. , Lateral, and depth in three dimensions. The three-dimensional detection means is, for example, a laser radar. The operation target is, for example, a vehicle. The detection target is terrain or an obstacle that the control target moves.

  The remote control image processing apparatus stores the three-dimensional coordinates for each detection target detected by the three-dimensional detection means in the storage means as coordinates to be displayed on the screen of the display means by applying an image of a predetermined shape. The image having the predetermined shape is, for example, a cube (voxel). That is, by displaying the cube at a position on the screen of the display means corresponding to the three-dimensional coordinates stored in the storage means, the pilot can grasp the periphery of the operation target at a remote place.

  Of the three-dimensional coordinates for each detection target detected by the three-dimensional detection means, the three-dimensional coordinates that are not stored in the storage means are extracted by the additional coordinate extraction means, and are added and stored in the storage means.

  However, simply adding and storing the three-dimensional coordinates in the storage means sequentially causes a cube indicating the position of the detection target to be added and displayed on the display means along with the movement of the detection target. That is, since the detection target that does not exist is displayed on the display unit, the periphery of the maneuvering target is not accurately displayed on the display unit.

  Therefore, it is determined by the deletion coordinate extraction means whether or not the detection target has moved based on the time change of the three-dimensional coordinates, and the three-dimensional coordinates corresponding to the detection target determined to have moved are deleted from the storage means. Extracted as

  As a result, the three-dimensional coordinates extracted as the deletion target are deleted from the storage unit, and the detection target that does not exist is prevented from being displayed on the display unit. Displayed on the display means.

  In the first aspect, the deletion coordinate extraction unit obtains a depth indicated by the three-dimensional coordinates for each region obtained by dividing the range detected by the three-dimensional detection unit, and the depth is It is preferable to extract the three-dimensional coordinates corresponding to the region deeper than the depth obtained in the past as a deletion target from the storage unit.

  This structure calculates | requires the depth shown by a three-dimensional coordinate for every area | region obtained by dividing | segmenting the range detected by a three-dimensional detection means. This depth is the distance from the operation target to the detection target. When the depth is deeper than the depth obtained in the past, this indicates that the detection target has moved. This is because the detection target detected in the past by the three-dimensional detection means moves, and then the three-dimensional detection means detects what is behind the detection target. That is, when the depth is deeper than the depth obtained in the past, in other words, when the position where the detection target is detected moves further back, it is determined that the detection target of the corresponding region has moved.

Then, three-dimensional coordinates corresponding to a region whose depth is deeper than the depth obtained in the past are extracted as a deletion target from the storage unit.
Therefore, this configuration can easily extract the deletion target.

  In the first aspect, the deletion coordinate extracting means stores the three-dimensional coordinates detected in the past located on a straight line connecting the three-dimensional detection means and the three-dimensional coordinates indicating the detection target with the storage means. It is preferable to extract as an object to be deleted.

This configuration has already moved a previously detected three-dimensional coordinate on a straight line connecting the three-dimensional detection means and the three-dimensional coordinate indicating the detection target, that is, on a vector that faces the newly detected three-dimensional coordinate. It is determined as a detection target and extracted as a deletion target from the storage means.
Therefore, this configuration can easily extract the deletion target.

  In the first aspect, the deleted coordinate extraction unit is the three-dimensional coordinate detected in the past located on a straight line connecting the three-dimensional detection unit and the three-dimensional coordinate indicating the detection target, It is preferable to extract the three-dimensional coordinates detected in the past located in the deletion range based on the detection error by the three-dimensional detection means as the deletion target.

  In this configuration, since the error is included in the detection by the three-dimensional detection means, the three-dimensional coordinates located in the range affected by the detection error of the three-dimensional detection means are not targeted for deletion, so there is no need to delete them. It is possible to prevent deleting the three-dimensional coordinates.

  In the first aspect, the image information acquired by the imaging unit is provided on the steering target, the imaging unit including an imaging unit that captures the detection target and acquires two-dimensional image information. It is preferable that the three-dimensional coordinates corresponding to the detection target determined to have moved are also extracted as the deletion target from the storage unit on the basis of the color change indicated by.

This configuration is a detection target that is difficult to detect by the three-dimensional detection means, for example, a detection target that is substantially the same distance from the control target and that one does not move and the other moves (moves along the wall) Color change can be detected by the imaging means.
As described above, the present configuration extracts and deletes the detection target that has moved from the color change indicated by the image information acquired by the imaging unit, and thus can more reliably extract the three-dimensional coordinates to be deleted. it can.

  An image processing device for remote control according to the second aspect of the present invention is a remote control image processing device for displaying an image for remotely controlling a control object on a display means, which is installed on the control object and is a detection target. A three-dimensional detection means for detecting the position of the object in three dimensions of length, width, and depth; an imaging means that is installed on the steering object and images the detection object to acquire two-dimensional image information; and a predetermined shape As the coordinates to be displayed on the screen of the display means by applying the image of the above, the storage means for storing the three-dimensional coordinates for each detection target detected by the three-dimensional detection means, and the detection detected by the three-dimensional detection means Of the three-dimensional coordinates for each detection target, the three-dimensional coordinates that are not stored in the storage unit are extracted and added to the storage unit to be stored, and the additional coordinate extraction unit that is acquired by the imaging unit Extraction means for extracting the moved detection object based on the image information, and deletion coordinate extraction for extracting the three-dimensional coordinates corresponding to the detection object extracted by the extraction means as deletion objects from the storage means Means.

  The image processing apparatus for remote control according to this configuration is an apparatus that displays an image for remotely maneuvering the operation target on the display unit, and the position of the detection target is vertically moved by the three-dimensional detection unit installed on the operation target. , Lateral and depth are detected in three dimensions, and the imaging target is imaged by the imaging means to acquire two-dimensional image information. The three-dimensional detection means is, for example, a laser radar. The operation target is, for example, a vehicle. The detection target is terrain or an obstacle that the control target moves.

  The remote control image processing apparatus stores the three-dimensional coordinates for each detection target detected by the three-dimensional detection means in the storage means as coordinates to be displayed on the screen of the display means by applying an image of a predetermined shape. The image having the predetermined shape is, for example, a cube (voxel). That is, by displaying the cube at a position on the screen of the display means corresponding to the three-dimensional coordinates stored in the storage means, the pilot can grasp the periphery of the operation target at a remote place.

  Of the three-dimensional coordinates for each detection target detected by the three-dimensional detection means, the three-dimensional coordinates that are not stored in the storage means are extracted by the additional coordinate extraction means, and are added and stored in the storage means.

  However, only by sequentially adding and storing the three-dimensional coordinates in the storage means, a cube indicating the detection target is added with the movement of the detection target every moment and displayed on the display means. That is, since the detection target that does not exist is displayed on the display unit, the periphery of the maneuvering target is not accurately displayed on the display unit.

  Therefore, the moving detection target is extracted based on the image information acquired by the imaging unit by the extraction unit, and the three-dimensional coordinates corresponding to the extracted detection target are deleted from the storage unit by the deletion coordinate extraction unit. Extracted as

  As a result, the three-dimensional coordinates extracted as the deletion target from the storage unit are deleted, and the detection target that does not exist is prevented from being displayed on the display unit. Displayed on the display means.

  A remote control system according to a third aspect of the present invention is a remote control system including a control target and a remote control device, wherein the control target is the remote control according to any one of claims 1 to 6. The steering image processing apparatus, the additional coordinates that are the three-dimensional coordinates additionally stored in the storage means by the additional coordinate extracting means, and the deleted coordinates that are the three-dimensional coordinates extracted by the deleted coordinate extracting means Transmitting means for transmitting to the remote control device, wherein the remote control device displays an image for remotely operating the operation target, and the additional coordinates and the deletion coordinates from the transmission means. And a storage means for display in which the additional coordinates received by the receiving means are stored and the deleted coordinates received by the receiving means are deleted. , And a display screen generation means for displaying on said display means by applying an image of the predetermined shape in the three-dimensional coordinates stored in the display storage section.

  The remote control system according to this configuration includes a control target and a remote control device. The control target and the remote control device transmit and receive various information necessary for remote control, for example, wirelessly.

  The control target including the remote control image processing device described above is an additional coordinate that is a three-dimensional coordinate that is additionally stored in the storage unit by the additional coordinate extraction unit, and a three-dimensional coordinate that is extracted by the deletion coordinate extraction unit. A certain deletion coordinate is transmitted from the transmission means to the remote control device. As described above, as the information for updating and displaying the image of the predetermined shape on the screen of the display means, only the added coordinates and the deleted coordinates are transmitted as update information from the operation target. Thereby, the amount of information transmitted from the control target to the remote control device can be reduced.

  On the other hand, the remote control device receives the addition coordinates and the deletion coordinates from the transmission means included in the control target by the reception means. Then, the display storage means stores the additional coordinates received by the receiving means, and the deleted coordinates received by the receiving means are deleted from the display storage means. Thereby, since the three-dimensional coordinates stored in the storage means included in the operation target and the display storage means included in the remote control device are the same, the control of the control target by the remote control device can be performed more accurately.

  Then, the display screen creation means applies an image of a predetermined shape to the three-dimensional coordinates stored in the display storage means and displays the image on the display means.

  From the above, since the three-dimensional coordinates extracted as the deletion target are deleted from the storage unit and the non-existing detection target is prevented from being displayed on the display unit, this configuration can Can be accurately displayed on the display means.

  An image processing method for remote control according to a fourth aspect of the present invention includes a three-dimensional detection unit that is installed on the control target and detects the position of the detection target in three dimensions of length, width, and depth, and a predetermined shape. Storage means for storing three-dimensional coordinates for each of the detection targets detected by the three-dimensional detection means as coordinates to be displayed on the screen of the display means by applying an image, and for remotely maneuvering the control target An image processing method for remote control by a remote control image processing apparatus for displaying an image on a display means, wherein among the three-dimensional coordinates for each detection target detected by the three-dimensional detection means, stored in the storage means The first step of extracting the three-dimensional coordinates that have not been extracted and adding and storing them in the storage means, and determining whether or not the detection object has moved based on the time change of the three-dimensional coordinates, moved The determined the three-dimensional coordinates corresponding to the detection target and a second step of extracting for deletion from the storage means.

  The image processing method for remote control according to the fifth aspect of the present invention includes a three-dimensional detection unit that is installed on the control target and detects the position of the detection target in three dimensions of length, width, and depth, and the control target. An imaging unit that captures a detection target and acquires two-dimensional image information; and coordinates that are displayed on the screen of the display unit by applying an image of a predetermined shape, and detected by the three-dimensional detection unit An image processing method for remote control by a remote control image processing apparatus, comprising: storage means for storing three-dimensional coordinates for each detection target; and displaying images for remote control of the control target on a display means, A first step of extracting the three-dimensional coordinates that are not stored in the storage means out of the three-dimensional coordinates for each of the detection targets detected by the three-dimensional detection means, and adding and storing them in the storage means; Based on the image information acquired by the imaging means, a second step of extracting the moved detection object, and the three-dimensional coordinates corresponding to the detection object extracted by the second step from the storage means And a third step of extracting as a deletion target.

  According to the present invention, there is an excellent effect that the periphery of the maneuvering target can be accurately displayed on the display unit disposed in a remote place.

1 is a diagram showing a remotely operated vehicle according to a first embodiment of the present invention. It is a block diagram which shows the structure of the remote control system which concerns on 1st Embodiment of this invention. It is a schematic diagram which shows an example of the screen displayed on the monitor which concerns on 1st Embodiment of this invention. It is a schematic diagram which shows the division area of the range detected with the laser radar which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structure of the remote control system which concerns on 2nd Embodiment of this invention. It is a schematic diagram which shows the voxel used as the deletion object which concerns on 2nd Embodiment of this invention. It is a schematic diagram which shows the example of the voxel deleted accidentally. It is a schematic diagram which shows the deletion range which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structure of the remote control system which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the structure of the remote control system which concerns on 4th Embodiment of this invention. It is a schematic diagram which shows the image displayed on the monitor of the conventional remote monitoring apparatus.

  Hereinafter, an embodiment of an image processing device for remote control, a remote control system, and an image processing method for remote control according to the present invention will be described with reference to the drawings.

[First Embodiment]
The first embodiment of the present invention will be described below.

  FIG. 1 is a diagram illustrating a vehicle 12 that is a target of a remote control system 10 according to the first embodiment.

  The vehicle 12 can run unattended, and is remotely controlled by a remote control device 20 (see also FIG. 2) of the remote control system 10.

  The vehicle 12 is provided with a laser radar 14 as a three-dimensional detection means for detecting the position of the detection target in three dimensions of length, width, and depth. The detection target is a terrain on which the vehicle 12 moves or an obstacle including a person. In addition, although the installation position of the laser radar 14 is the roof of the vehicle 12 as an example, it is not limited to this and may be another installation position.

  The laser radar 14 emits a predetermined number of lasers, and the position (x, y, z) of the reflection point of each laser is set as a detection target position. As shown in FIG. 1, the detection range of the laser radar 14 is the circumferential direction (yaw direction) of the vehicle 12 and the height direction (pitch direction) of the vehicle 12, and the laser radar 14 itself is angled with respect to the circumferential direction of the vehicle 12. The periphery of the vehicle 12 is detected by rotating at θn (for example, 360 degrees) and rotating at an angle φn in the height direction. Thereby, the detection target around the vehicle 12 is detected.

  FIG. 2 is a block diagram showing the configuration of the remote control system 10 according to the first embodiment. The block diagram shown in FIG. 2 mainly shows the configuration used for displaying the surroundings of the vehicle 12 on the monitor 22 provided in the remote control device 20. The monitor 22 is for the operator to grasp the surroundings of the vehicle 12 by the remote control device 20.

  In addition to the laser radar 14, the vehicle 12 includes a vehicle position information acquisition unit 24, a remote control image processing device 26, and a transmission unit 28.

  The vehicle position information acquisition unit 24 is a GPS (Global Positioning System), and specifies the position of the vehicle 12 by latitude and longitude.

  The remote control image processing device 26 includes a voxel creation unit 30, an additional voxel extraction unit 32, a depth map creation unit 34, a deletion voxel extraction unit 36, and a vehicle storage unit 38.

  The voxel creation unit 30 indicates the position of the detection target detected by the laser radar 14 based on the detection result (x, y, z) by the laser radar 14 and the position information of the vehicle 12 by the vehicle position information acquisition unit 24. The dimensional coordinate is converted into the world coordinate system, and the converted three-dimensional coordinate is applied to an image of a predetermined shape as a coordinate to be displayed on the screen of the monitor 22 included in the remote control device 20. An image having a predetermined shape is, for example, a cube, which is hereinafter referred to as a voxel.

  That is, the voxel is displayed at a position on the screen of the monitor 22 corresponding to the three-dimensional coordinates, so that the operator can grasp the surroundings of the vehicle 12 at a remote place.

  More specifically, when a plurality of laser beams emitted from the laser radar 14 are reflected, for example, on a building, three-dimensional coordinates are obtained for each position of the reflection point. And the building is represented as FIG. 3 shows by arrange | positioning the obtained voxel 40 centering on the some three-dimensional coordinate on the screen of the monitor 22. FIG. In the example of FIG. 3, only the building is displayed by the voxel 40, but all detection targets of the laser radar 14 including the ground and the like are represented by the voxel 40. In this way, the periphery of the vehicle 12 is displayed on the monitor 22. For ease of explanation, in FIG. 3, the center (black dot) of the voxel 40 is displayed on the screen of the monitor 22, but the center of the voxel 40 may not actually be displayed.

  The additional voxel extraction unit 32 extracts three-dimensional coordinates that are not stored in the vehicle storage unit 38 from the three-dimensional coordinates for each detection target detected by the laser radar 14 and adds them to the vehicle storage unit 38. Remember. The extracted three-dimensional coordinates are output to the transmission unit 28 together with the vehicle storage unit 38.

  By the extraction of the three-dimensional coordinates by the additional voxel extraction unit 32 and the storage in the vehicle storage unit 38, a voxel map indicating the position where the voxel 40 is arranged in the world coordinate system is constructed in the vehicle storage unit 38; Become.

  The depth map creation unit 34 determines the position of the detection target detected by the laser radar 14 based on the detection result (x, y, z) by the laser radar 14 and the position information of the vehicle 12 by the vehicle position information acquisition unit 24. Convert the three-dimensional coordinates shown to the world coordinate system. The depth map creation unit 34 obtains the depth indicated by the three-dimensional coordinates for each region obtained by dividing the range detected by the laser radar 14 (hereinafter referred to as “divided region”). This depth is the distance from the vehicle 12 to the detection target.

For example, as shown in FIG. 4, the detection range of the laser radar 14 is, for example, 360 degrees in the circumferential direction of the vehicle 12 and from N degrees to −N degrees in the height direction, and is divided into eight in the circumferential direction as divided areas. Then, by dividing into three in the height direction, 24 divided regions are obtained.
A map in which the depth D _t (θn, φn) is obtained for each divided region is a depth map.

The deletion voxel extraction unit 36 obtains a divided region in which the depth D ₁ (θn, φn) indicated by the depth map is deeper than the depth D ₀ (θn, φn) obtained in the past, and corresponds to the divided region. The three-dimensional coordinates are extracted as a deletion target from the vehicle storage unit 38.

  The transmission unit 28 remotely transmits additional coordinates, which are three-dimensional coordinates added and stored in the vehicle storage unit 38 by the additional voxel extraction unit 32, and deletion coordinates, which are three-dimensional coordinates extracted by the deletion voxel extraction unit 36. Transmit to the control device 20.

  In addition to the monitor 22, the remote control device 20 includes a receiving unit 42, a display storage unit 44, and a display screen creation unit 46.

  The receiving unit 42 receives the added coordinates and the deleted coordinates from the transmitting unit 28.

  The display storage unit 44 stores the additional coordinates received by the receiving unit 42 and deletes the deleted coordinates received by the receiving unit 42.

  The display screen creation unit 46 applies the voxels 40 to the three-dimensional coordinates stored in the display storage unit 44 and causes the monitor 22 to display them.

  Note that the remote control image processing device 26 and the remote control device 20 include, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), a computer-readable recording medium, and the like. A series of processes for realizing various functions is recorded on a recording medium or the like in the form of a program as an example, and the CPU reads the program into a RAM or the like to execute information processing / arithmetic processing. As a result, various functions are realized.

  Next, the operation of the remote control system 10 according to the first embodiment will be described.

  By simply adding and storing three-dimensional coordinates in the vehicle storage unit 38 by the remote control image processing device 26, a voxel 40 indicating the position of the detection target is added along with the movement of the detection target every moment and is added to the monitor 22. Will be displayed. That is, since the detection target that does not exist is displayed on the monitor 22, the periphery of the vehicle 12 is not accurately displayed on the monitor 22.

  Therefore, the deletion voxel extraction unit 36 determines whether or not the detection target has moved based on the time change of the three-dimensional coordinates, and the three-dimensional coordinates corresponding to the detection target determined to have moved are stored in the vehicle storage unit 38. It is extracted as a deletion target. In the following description, the deletion of the three-dimensional coordinate and the deletion of the voxel 40 are synonymous.

  As described above, the deleted voxel extraction unit 36 according to the first embodiment obtains, from the vehicle storage unit 38, the three-dimensional coordinates corresponding to the divided region where the depth indicated by the depth map is deeper than the depth obtained in the past. To be deleted. When the depth is deeper than the depth obtained in the past, it indicates that the detection target has moved. This is because the detection target detected in the past has moved, and the laser radar 14 has subsequently detected an object located behind the detection target. That is, when the depth is deeper than the depth obtained in the past, in other words, when the position where the detection target is detected (the reflection point of the laser) moves deeper, it is determined that the detection target of the corresponding divided region has moved. Is done.

  Thereby, the deletion voxel extraction unit 36 according to the second embodiment can easily extract the deletion target.

  The vehicle 12 then adds the additional coordinates, which are the three-dimensional coordinates added and stored in the vehicle storage unit 38 by the additional voxel extraction unit 32, and the deletion coordinates, which are the three-dimensional coordinates extracted by the deletion voxel extraction unit 36. The data is transmitted from the transmission unit 28 to the remote control device 20. Thus, as information for updating and displaying the voxel 40 on the screen of the monitor 22, only the additional coordinates and the deleted coordinates are transmitted as update information from the vehicle 12. Thereby, the amount of information transmitted from the vehicle 12 to the remote control device 20 can be reduced.

  On the other hand, the remote control device 20 receives the additional coordinates and the deleted coordinates from the transmission unit 28 included in the vehicle 12 by the reception unit 42. Then, the display storage unit 44 stores the additional coordinates received by the reception unit 42, and the deletion coordinates received by the reception unit 42 are deleted from the display storage unit 44. As a result, the three-dimensional coordinates stored in the vehicle storage unit 38 included in the vehicle 12 and the display storage unit 44 included in the remote control device 20 are the same, so that the operation of the operation target by the remote control device 20 is more accurate. Can be done.

  The display screen creation unit 46 applies the voxel 40 to the three-dimensional coordinates stored in the display storage unit 44 and displays the voxel 40 on the monitor 22.

  As described above, the remote control system 10 according to the first embodiment deletes the three-dimensional coordinates extracted as the deletion target from the vehicle storage unit 38 and the display storage unit 44, so that the non-existing detection target is detected. The display on the monitor 22 is prevented, and the periphery of the vehicle 12 can be accurately displayed on the monitor 22 disposed in a remote place.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described.

  FIG. 5 shows a configuration of the remote control system 10 according to the second embodiment. In FIG. 5, the same components as those in FIG. 2 are denoted by the same reference numerals as those in FIG.

  Although the remote control system 10 according to the first embodiment described above includes the depth map creation unit 34, the remote control system 10 according to the second embodiment does not include the depth map creation unit 34, and The function of the deletion voxel extraction unit 36 is different from that of the deletion voxel extraction unit 36 according to the first embodiment.

  The deletion voxel extraction unit 36 according to the second embodiment includes a vehicle storage unit 38 and a display of previously detected three-dimensional coordinates located on a straight line connecting the laser radar 14 and a three-dimensional coordinate indicating a detection target. Extracted as a deletion target from the storage unit 44.

  As a result, the previously detected three-dimensional coordinates located on the straight line connecting the laser radar 14 and the three-dimensional coordinates indicating the detection target, that is, on the vector that faces the newly detected three-dimensional coordinates, have already moved. The deletion voxel extraction unit 36 determines that the object is an object, and the object is extracted as an object to be deleted. Therefore, the deletion voxel extraction unit 36 according to the second embodiment can easily extract the deletion target.

  FIG. 6 is a schematic diagram showing voxels 40 (three-dimensional coordinates) extracted as deletion targets by the deletion voxel extraction unit 36 according to the second embodiment. As shown in FIG. 6, a voxel 40 </ b> B that is located between the newly created voxel 40 </ b> A and the vehicle 12 and has been created in the past is a deletion target.

  Further, when the laser radar 14 irradiates, for example, 64 lasers and rotates at 360 degrees in the circumferential direction at intervals of x degrees, the above determination is made at (360 / x) × 64 locations. .

  Here, the detection by the laser radar 14 includes an error. Therefore, as shown in the example of FIG. 7, if an error is included in the voxel 40C (three-dimensional coordinates) obtained by the new detection, it is located between the voxel 40C and the laser radar 14 (vehicle 12). In some cases, voxels 40D and 40E that are voxels 40 and are not originally targeted for deletion may be targeted for deletion.

  Therefore, the deleted voxel extraction unit 36 according to the second embodiment is a three-dimensional coordinate detected in the past located on a straight line connecting the laser radar 14 and the three-dimensional coordinate indicating the detection target. As shown, three-dimensional coordinates detected in the past located in the deletion range L ′ based on the detection error of the laser radar 14 are extracted as deletion targets. As a result, the deleted voxel extraction unit 36 according to the second embodiment does not need to delete the three-dimensional coordinates (voxel 40) located in the range affected by the detection error of the laser radar 14, and thus does not need to be deleted. It is possible to prevent deleting the three-dimensional coordinates.

The deletion range L ′ is calculated from the distance L between the laser radar 14 and the newly detected three-dimensional coordinate as shown in the following equation (1).

V is the size of one side of the voxel 40 (hereinafter referred to as “voxel size”), and σ _L is the error standard deviation at the distance L.

  Here, the reason why the voxel size V is included in the deletion range L ′ is as follows.

'Error standard deviation sigma _{L in'} deletion range L is shown in the following equation (2).

If the error standard deviation σ _L ′ is larger than the voxel size V, it may be deleted erroneously. Therefore, σ _L ′ ≦ V is set as a deletion condition, and as an example, σ _L ′ = V in equation (2), thereby obtaining equation (1).

  As described above, the remote control system 10 according to the second embodiment uses the three-dimensional coordinates detected in the past located on the straight line connecting the laser radar 14 and the three-dimensional coordinates indicating the detection target for the vehicle. Since it is extracted as a deletion target from the storage unit 38, the deletion target can be easily extracted.

[Third Embodiment]
Hereinafter, a third embodiment of the present invention will be described.

  FIG. 9 shows the configuration of the remote control system 10 according to the third embodiment. In FIG. 9, the same components as those in FIG. 2 are denoted by the same reference numerals as those in FIG.

  The vehicle 12 according to the third embodiment is provided with a camera 50 that captures a detection target and acquires two-dimensional image information. The coordinates of the imaging range by the camera 50 and the coordinates of the detection range by the laser radar 14 are calibrated (calibrated) so as to match.

The camera 50 is a detection object that is difficult to be detected by the laser radar 14, for example, a detection object that is substantially the same distance from the vehicle 12 and one of which does not move and the other moves (for example, moves along the wall). Even if there are people), these can be detected by color change.
Therefore, the remote control image processing device 26 includes a color change determination unit 52.

  The color change determination unit 52 determines the area where the color has changed from the time change of the image information acquired by the camera 50. For example, if the color change is equal to or greater than the threshold value K, the color change determination unit 52 determines that the color has changed.

  The deleted voxel extraction unit 36 according to the third embodiment stores the three-dimensional coordinates corresponding to the divided area where the depth indicated by the depth map is deeper than the depth obtained in the past, as in the first embodiment. This is extracted as a deletion target from the unit 38. In addition to this, the deletion voxel extraction unit 36 according to the third embodiment also extracts the three-dimensional coordinates corresponding to the region determined by the color change determination unit 52 as a deletion target from the vehicle storage unit 38.

  That is, the deletion voxel extraction unit 36 according to the third embodiment also uses the three-dimensional coordinates corresponding to the detection target determined to have moved based on the change in color indicated by the image information acquired by the camera 50. It is extracted as a deletion target from the storage unit 38.

  Therefore, even if it is a detection object that has moved, even if it is a three-dimensional coordinate that has not been deleted from the detection result of the laser radar 14, it is determined as a deletion object based on the image information acquired by the camera 50. Thereby, the remote control system 10 according to the second embodiment can more reliably extract the three-dimensional coordinates to be deleted.

[Fourth Embodiment]
The fourth embodiment of the present invention will be described below.

  FIG. 10 shows the configuration of the remote control system 10 according to the fourth embodiment. Note that the same components in FIG. 10 as those in FIG. 2 are denoted by the same reference numerals as those in FIG.

  The remote control image processing device 26 according to the fourth embodiment does not include the depth map creation unit 34, while the vehicle 12 is provided with a camera 50, and the remote control image processing device 26 has a moving object extraction. A part 60 is provided.

  The camera 50 captures a detection target and acquires two-dimensional image information. The coordinates of the imaging range by the camera 50 and the coordinates of the detection range by the laser radar 14 are calibrated (calibrated) so as to match.

  The moving body extracting unit 60 extracts the moved detection target based on the image information acquired by the camera 50. In addition, the method of extracting the detection target which moved based on image information may be a conventionally known method.

  Then, the deletion voxel extraction unit 36 extracts the three-dimensional coordinates corresponding to the detection target extracted by the moving body extraction unit 60 as the deletion target from the vehicle storage unit 38.

  Thereby, in the remote control system 10 according to the fourth embodiment, the three-dimensional coordinates extracted as the deletion target are deleted from the vehicle storage unit 38 and the display storage unit 44, and the non-existing detection target is displayed on the monitor 22. Is prevented. Therefore, the remote control system 10 according to the first embodiment can accurately display the periphery of the vehicle 12 on the monitor 22 disposed in the remote place.

  As mentioned above, although this invention was demonstrated using said each embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various changes or improvements can be added to the above-described embodiments without departing from the gist of the invention, and embodiments to which the changes or improvements are added are also included in the technical scope of the present invention.

  For example, in each of the above-described embodiments, the form in which the object to be controlled by the remote control system 10 is the vehicle 12 has been described. However, the present invention is not limited to this, and the form in which the object to be controlled is, for example, an aircraft or a robot. It is good.

  In each of the above embodiments, a description has been given of a mode in which a laser radar is used as a three-dimensional detection means for detecting the position of a detection target in three dimensions of length, width, and depth, but the present invention is limited to this. Instead of this, a stereo camera or the like may be used as the three-dimensional detection means.

  Further, in each of the above embodiments, the mode in which the laser radar 14 is rotated at 360 degrees in the circumferential direction has been described. However, the present invention is not limited to this, and the laser radar 14 is within a predetermined range of less than 360 degrees. It is good also as a form to rotate.

DESCRIPTION OF SYMBOLS 10 Remote control system 12 Vehicle 14 Laser radar 20 Remote control device 26 Remote control image processing device 28 Transmitter 32 Additional voxel extraction unit 36 Deletion voxel extraction unit 38 Vehicle storage unit 42 Reception unit 44 Display storage unit 46 Display screen creation 50 Camera 60 Moving object extraction unit

Claims (9)

An image processing apparatus for remote control that displays on a display means an image for remotely controlling a control object,
A three-dimensional detection means that is installed in the steering target and detects the position of the detection target in three dimensions of length, width, and depth;
Storage means for storing three-dimensional coordinates for each detection target detected by the three-dimensional detection means as coordinates to be displayed on the screen of the display means by applying an image of a predetermined shape;
Additional coordinate extraction means for extracting the three-dimensional coordinates not stored in the storage means out of the three-dimensional coordinates for each detection target detected by the three-dimensional detection means, and adding and storing them in the storage means When,
Deletion that determines whether or not the detection target has moved based on a time change of the three-dimensional coordinates, and extracts the three-dimensional coordinates corresponding to the detection target that has been determined to have moved as a deletion target from the storage unit Coordinate extraction means;
An image processing apparatus for remote control comprising:   The deletion coordinate extraction unit obtains a depth indicated by the three-dimensional coordinate for each region obtained by dividing the range detected by the three-dimensional detection unit, and the depth is obtained from a depth obtained in the past. The remote control image processing apparatus according to claim 1, wherein the three-dimensional coordinates corresponding to the deepest region are extracted as deletion targets from the storage unit.   The deletion coordinate extracting means extracts the three-dimensional coordinates detected in the past located on a straight line connecting the three-dimensional detection means and the three-dimensional coordinates indicating the detection target as a deletion target from the storage means. The image processing apparatus for remote control according to claim 1.   The deletion coordinate extracting means is the three-dimensional coordinates detected in the past located on a straight line connecting the three-dimensional detection means and the three-dimensional coordinates indicating the detection target, and is detected by the three-dimensional detection means. The image processing apparatus for remote control according to claim 3, wherein the three-dimensional coordinates detected in the past located in a deletion range based on an error are extracted as the deletion target. An imaging unit that is installed on the steering target and that captures the detection target to acquire two-dimensional image information;
The deletion coordinate extraction means also deletes the three-dimensional coordinates corresponding to the detection target determined to have moved based on the color change indicated by the image information acquired by the imaging means from the storage means. The image processing device for remote control according to any one of claims 1 to 4, wherein the image processing device is extracted as a target. An image processing apparatus for remote control that displays on a display means an image for remotely controlling a control object,
A three-dimensional detection means that is installed in the steering target and detects the position of the detection target in three dimensions of length, width, and depth;
An imaging unit installed on the steering target and capturing the two-dimensional image information by imaging the detection target;
Storage means for storing three-dimensional coordinates for each detection target detected by the three-dimensional detection means as coordinates to be displayed on the screen of the display means by applying an image of a predetermined shape;
Additional coordinate extraction means for extracting the three-dimensional coordinates not stored in the storage means out of the three-dimensional coordinates for each detection target detected by the three-dimensional detection means, and adding and storing them in the storage means When,
Extraction means for extracting the detected object that has moved based on the image information acquired by the imaging means;
A deletion coordinate extraction unit that extracts the three-dimensional coordinates corresponding to the detection target extracted by the extraction unit as a deletion target from the storage unit;
An image processing apparatus for remote control comprising: A remote control system including a control object and a remote control device,
The pilot object is
The image processing device for remote control according to any one of claims 1 to 6,
The additional coordinates, which are the three-dimensional coordinates additionally stored in the storage means by the additional coordinate extraction means, and the deleted coordinates, which are the three-dimensional coordinates extracted by the deletion coordinate extraction means, are transmitted to the remote control device. Sending means to
With
The remote control device includes:
Display means for displaying an image for remotely operating the maneuvering object;
Receiving means for receiving the additional coordinates and the deleted coordinates from the transmitting means;
Storing the additional coordinates received by the receiving means, and display storage means for deleting the deleted coordinates received by the receiving means;
Display screen creation means for applying the image of the predetermined shape to the three-dimensional coordinates stored in the display storage means and causing the display means to display;
Remote control system with As the coordinates that are installed on the steering object and detect the position of the detection object in three dimensions of length, width, and depth, and coordinates to be displayed on the screen of the display means by applying an image of a predetermined shape, Storage means for storing three-dimensional coordinates for each detection target detected by the three-dimensional detection means, and remote control by a remote control image processing device for displaying an image for remote control of the control target on the display means Image processing method for
A first step of extracting the three-dimensional coordinates not stored in the storage means out of the three-dimensional coordinates for each of the detection targets detected by the three-dimensional detection means, and adding and storing them in the storage means; ,
A determination is made as to whether or not the detection object has moved based on a time change of the three-dimensional coordinates, and the three-dimensional coordinates corresponding to the detection object that has been determined to have moved are extracted as deletion objects from the storage means. Two steps,
An image processing method for remote control including: A three-dimensional detection unit that is installed on the control target and detects the position of the detection target in three dimensions of length, width, and depth; and a two-dimensional image information that is set on the control target and images the detection target. Imaging means for obtaining; and storage means for storing three-dimensional coordinates for each detection target detected by the three-dimensional detection means as coordinates to be displayed on the screen of the display means by applying an image of a predetermined shape. An image processing method for remote control by a remote control image processing device for displaying an image for remotely controlling a control object on a display means,
A first step of extracting the three-dimensional coordinates not stored in the storage means out of the three-dimensional coordinates for each of the detection targets detected by the three-dimensional detection means, and adding and storing them in the storage means; ,
A second step of extracting the detected detection object based on the image information acquired by the imaging means;
A third step of extracting the three-dimensional coordinates corresponding to the detection target extracted in the second step as a deletion target from the storage unit;
An image processing method for remote control including:

Images