JP2017162251A - Three-dimensional noncontact input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional noncontact input device capable of performing noncontact input, without applying a load to a computer as much as possible.SOLUTION: A three-dimensional noncontact input device 10 for transmitting the operation of an operator to a computer 1 includes a plurality of cameras 11 and 12 which are directed to an operator side from a different position and a processing part 13 which transmits the information of a specific part 102 of the operator on the basis of an acquisition image 100 obtained from the plurality of cameras 11 and 12. The processing part 13 includes a depth image forming part 21 which forms a depth image 102 from the acquisition image 100 obtained from the plurality of cameras 11 and 12 and an object recognition part 23 which performs the coordinate recognition and shape recognition of the specific part 102 at a predetermined distance to the plurality of cameras 11 and 12 on the basis of the depth image 102 and can transmit the information of the coordinate and shape of the specific part 102 obtained by the object recognition part 23 to the computer 1.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a three-dimensional non-contact input device, and more particularly to a three-dimensional non-contact input device having a plurality of cameras.

  2. Description of the Related Art Conventionally, a non-contact input system has been proposed in which an operator's movement is captured by a camera or sensor, and predetermined input is performed without touching a display device, a mouse, or a keyboard based on the information (see, for example, Patent Document 1). ).

JP 2009-104297 A

  However, in the conventional apparatus such as the above-described Patent Document 1, an image captured by a camera or the like is sent to a computer, and the computer processes the image to calculate input information to specify a position or the like. It was. Therefore, there has been a problem that the load on the computer that is also performing other processing is too large.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a three-dimensional non-contact input device that can perform non-contact input without imposing a load on the computer as much as possible.

  In order to solve this problem, the invention described in claim 1 is a three-dimensional non-contact input device that transmits an operation of an operator to a computer, and is a plurality of cameras directed to the operator from different positions. And a processing unit that transmits information on the specific part of the operator based on acquired images obtained from the plurality of cameras, and the processing unit obtains the acquisition obtained from the plurality of cameras. A depth image forming unit that forms a depth image from an image, and an object recognition unit that performs object recognition processing including coordinate recognition and shape recognition of the specific part at a predetermined distance with respect to the plurality of cameras based on the depth image And a three-dimensional non-contact input device capable of transmitting information on the coordinates and shape of the specific part obtained by the object recognition unit to the computer.

  Moreover, in addition to the structure of Claim 1, the invention of Claim 2 deletes the image of the depth which remove | deviates from the predetermined distance in the said depth image obtained by the said depth image generation part in the said process part. The present invention is characterized in that a three-dimensional non-contact input device having an image preprocessing unit that performs filter processing is provided.

  According to a third aspect of the present invention, in addition to the configuration of the first or second aspect, the object recognition unit performs the object recognition process on the entire image for the first time, and the object from the second time. A three-dimensional non-contact input device that performs the object recognition process only on a predetermined range around the specific part obtained by the previous object recognition process in the recognition unit is provided.

  According to a fourth aspect of the present invention, in addition to the configuration according to any one of the first to third aspects, the object recognition unit compares the image with a pre-stored template image to compare the object. A three-dimensional non-contact input device adapted to perform recognition processing is characterized.

  According to a fifth aspect of the present invention, in addition to the configuration according to any one of the first to fourth aspects, the processing unit includes coordinate averaging for the specific part obtained by the object recognition unit. And an image post-processing unit for adjusting a moving curve.

  According to a sixth aspect of the present invention, in addition to the configuration according to any one of the first to fifth aspects, the computer is recognized and operated as a USB standard HID class. It is characterized by being a dimension non-contact input device.

  According to a seventh aspect of the present invention, in addition to the configuration according to any one of the first to sixth aspects, the plurality of cameras and the processing unit are configured as a one-piece structure. It is a device.

  According to the first aspect of the present invention, a depth image is formed from acquired images from a plurality of cameras, and after performing coordinate recognition and shape recognition of a specific part based on the depth image, information obtained is obtained. Since the transmission is made to the computer, the load on the computer side can be reduced as much as possible, and smooth non-contact input can be performed.

  According to the second aspect of the present invention, since it is possible to erase a portion that is considered unnecessary for non-contact input from the depth image, it is possible to recognize the specific part reliably and quickly.

  According to the invention described in claim 3, since the process is performed only on the periphery of the specific part obtained by the first object recognition process from the second object recognition process, Fast processing can be performed.

  According to the fourth aspect of the present invention, since the coordinate recognition and shape recognition of the specific part are performed by comparison with the template image stored in advance, more reliable recognition power can be exhibited.

  According to the invention described in claim 5, since the coordinate averaging and the movement curve adjustment are performed for the recognized specific part, the movement of the specific part can be led to a smooth one.

  According to the invention described in claim 6, it can be used by connecting to a computer with a versatile standard that does not require a special driver or the like.

  According to the seventh aspect of the present invention, since the plurality of cameras and the processing unit are configured as a single unit, non-contact input can be performed with an apparatus having a compact configuration.

It is the schematic which shows the computer which connected this and the three-dimensional non-contact input device of embodiment of this invention. It is a block diagram of the three-dimensional non-contact input device of the embodiment. In the three-dimensional non-contact input device of the embodiment, (a) an example of an acquired image taken by a camera, (b) an example of a depth image formed from the acquired image of (a), and (c) and (b) It is explanatory drawing which shows the example of the specific image formed from the depth image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  1 to 3 show an embodiment of the present invention.

  As shown in FIG. 1, the three-dimensional non-contact input device 10 of this embodiment is connected to a computer 1, and by performing a predetermined operation in front of the three-dimensional non-contact input device 10, the computer 1 The input is performed without contact.

  First, the computer 1 will be described. In this embodiment, the computer 1 is connected to the display device 2, and a display is made on the display device 2 by an output signal from the computer 1. Further, the computer 1 is provided with a USB standard terminal 4 as an external terminal.

  Next, the three-dimensional non-contact input device 10 of this embodiment will be described. The three-dimensional non-contact input device 10 includes left and right cameras 11 and 12 and a main body 16 in which a processing unit 13 is built. A cord 14 protrudes from the main body 16 and a USB terminal 15 is provided at the tip thereof. The terminal 15 is inserted into the terminal 4 of the computer 1.

  In addition, the cameras 11 and 12 of this embodiment, which have a predetermined number of pixels, are attached in a state where they are separated from each other by a predetermined distance. Here, the two cameras 11 and 12 are fixed in a state where they face each other. Here, the acquired images 100 (see FIG. 3A) captured by the two cameras 11 and 12 are transmitted as signals to the processing unit 13 in the main body unit 16.

  The processing unit 13 of this embodiment processes the acquired image 100 (see FIG. 3A) sent from the plurality of cameras 11 and 12 and sends it to the computer 1 as coordinate recognition data and shape recognition data. It is. As shown in FIG. 2, the processing unit 13 includes a depth image generation unit 21, an image preprocessing unit 22, an object recognition unit 23, an image postprocessing unit 24, and a device adaptation unit 25. Details will be described below.

  The depth image generator 21 forms a depth image 101 as shown in FIG. 3B from the acquired images 100 of the left and right cameras 11 and 12 as shown in FIG. 3A by the stereo matching method. . Here, using the principle of triangulation, the parallax generated between the two left and right images is obtained and converted into a distance. As a result, as shown in FIG. 3B, a depth image 101 that is whiter toward the front of the image and blacker toward the back of the image is formed.

  The image pre-processing unit 22 performs filter processing using the depth image 101 described above. Here, in the depth image 101 as shown in FIG. 3B, a portion longer than the predetermined distance from the cameras 11 and 12 (here, a portion other than the specific portion 102) is painted black, and FIG. As shown in (c), a specific image 103 in which the part other than the specific part 102 to be noticed is erased is formed.

  The object recognition unit 23 performs an object recognition process. Specifically, the shape of the object is first recognized by the Haar-Like feature algorithm for the specific image 103 that has been filtered by the image preprocessing unit 22. When the shape recognition is successful, the coordinates and size (x, y, z, h) of the object are output as a result. Here, a plurality of (for example, several hundred types) template data (template images) are stored in advance, and various shapes can be reliably recognized by comparing all of the template data with the specific image 103. It has become. Here, the hand image is recognized as the specific part 102, and the shape of the hand (for example, the shape of goo, choki, par, etc.) is compared with the template data. The coordinates and the size of the representative position (for example, the center of gravity position or the center position in the template data) of the recognized specific part 102 are measured.

  Here, in the object recognition processing when the series of acquired images 100 are captured for the first time, the shape recognition and coordinate recognition of the specific part 102 are performed by looking at all the transmitted specific images 103. In addition, for the second and subsequent times when a series of acquired images 100 are captured and processed, the time is shortened by not looking outside the predetermined range around the specific part 102 that has been recognized and coordinate-recognized in the previous object recognition process. It comes to plan.

  Further, the image post-processing unit 24 performs coordinate averaging and movement curve adjustment in order to make adjustment so that the movement of the specific part 102 whose coordinates are recognized and whose shape is recognized is smooth.

  The device adaptation unit 25 outputs the information on the coordinate recognition and shape recognition of the specific part 102 obtained by the depth image forming unit 21 to the image post-processing unit 24 as HID class data of the USB standard. is there.

  The acquired images 100 sent from the two cameras 11 and 12 to the processing unit 13 are processed in the order of the depth image generation unit 21, the image preprocessing unit 22, the object recognition unit 23, the image postprocessing unit 24, and the device adaptation unit 25. The information of the USB standard HID class of the coordinate recognition and shape recognition of the specific part 102 obtained by performing the processing is transmitted to the computer 1.

  The computer 1 that has received this information performs only the same processing as information from a normal mouse or keyboard, and issues a display instruction to the display device 2 (for example, the cursor 3 is moved and the click state is displayed). And so on).

  It should be noted that from the photographing process by the two cameras 11 and 12 to the process of converting the USB standard HID class data in the device adapting unit 25 into a so-called pipeline process, the process from image acquisition to USB output is automatic. One after another. This shortens the processing time.

  As described above, according to the three-dimensional non-contact input device 10 of the above-described embodiment, the depth image 101 is formed from the acquired images 100 from the plurality of cameras 11 and 12, and the specific part is based on the depth image 101. Since the obtained information is transmitted to the computer 1 after performing the coordinate recognition and shape recognition of 102, the load on the computer 1 side can be reduced as much as possible, and smooth non-contact input is performed. be able to.

  Further, according to the three-dimensional non-contact input device 10 of the above-described embodiment, the filtering processing of the image pre-processing unit 22 can erase a portion considered unnecessary for non-contact input from the depth image 101. The specific part 102 can be recognized reliably and quickly.

  Further, according to the three-dimensional non-contact input device 10 of the above-described embodiment, the coordinate recognition and shape recognition in the object recognition process of the first object recognition unit 23 are changed from the object recognition process of the second object recognition unit 23. Since processing is performed only on the periphery of the obtained specific part 102, reliable and faster processing can be performed.

  Further, according to the three-dimensional non-contact input device 10 of the above-described embodiment, the coordinate recognition and shape recognition of the specific part 102 are performed by comparison with template data (template image) accumulated in advance. , More reliable recognition can be demonstrated.

  In addition, according to the three-dimensional non-contact input device 10 of the above-described embodiment, since the coordinate averaging and the movement curve adjustment are performed on the recognized specific part 102, the movement of the specific part 102 can be guided smoothly. it can.

  In addition, according to the three-dimensional non-contact input device 10 of the above-described embodiment, it can be used by connecting to the computer 1 with a versatile USB standard HID class that does not require a special driver or the like.

  In addition, according to the three-dimensional non-contact input device 10 of the above-described embodiment, since the plurality of cameras 11 and 12 and the processing unit 13 are configured as a single body, non-contact input can be performed with a compact configuration device. It can be carried out.

  In addition, according to the three-dimensional non-contact input device 10 of the above-described embodiment, different operations can be set according to various operations such as the shape and movement of the operator's hand, one hand or both hands, Various operation signals can be set rather than a mouse or the like.

  The embodiment described above is described in order to facilitate understanding of the present invention, and is not described in order to limit the present invention.

  For example, in the above-described embodiment, the plurality of cameras 11 and 12 and the processing unit 13 are integrated. However, the present invention is not limited to this, and the cameras 11 and 12 and the processing unit 13 are independently separated from each other. It may be arranged at a different position. However, it is easier to carry and install one-piece.

  In the above-described embodiment, the two cameras 11 and 12 are attached with facing the front. However, the present invention is not limited to this, and one or both of them may be installed at an angle. Further, the number of cameras is not limited to two, and three or more cameras may be installed. In addition, the performance and size of the camera may be set as necessary. If the number of pixels of the camera and the resolution are high, a smaller part can be recognized as a specific part. If the number of pixels of the camera and the resolution is low, the operation can be made faster.

  Further, in the above-described embodiment, the two cameras 11 and 12 are arranged so as to be separated from each other in the left and right directions. However, the present invention is not limited to this. They may be separated in an appropriate direction. However, it is not suitable that the depth image is difficult to be created if it is almost not separated.

  In the above-described embodiment, the depth image generation unit 21, the image preprocessing unit 22, the object recognition unit 23, and the image post processing are performed on the acquired image 100 sent from the two cameras 11 and 12 to the processing unit 13. The processing is performed in the order of the unit 24 and the device adaptation unit 25. However, the present invention is not limited to this, and depending on the image, processing method, finally obtained data format, the image pre-processing unit, the image post-processing unit, and the device adaptation In some cases, there is no need for the processing of the parts. However, the processing of the depth image forming unit and the object recognition unit cannot be omitted.

  In the above-described embodiment, the object recognition process has been described by taking the object shape recognition by the Haar-Like feature algorithm as an example. However, the present invention is not limited to this, and the object recognition process by another algorithm is performed. It may be like this. In addition, shape recognition may be performed by a method other than the shape recognition method of comparing a plurality of template data stored in advance with an image.

DESCRIPTION OF SYMBOLS 1 Computer 2 Display apparatus 3 Cursor 4 Terminal 10 3D non-contact input device 11 Left camera 12 Right camera 13 Processing part 14 Cable 15 Terminal 16 Body part 21 Depth image generation part 22 Image pre-processing part 23 Object recognition part 24 Image post-processing Unit 25 device matching unit 100 acquired image 101 depth image 102 specific part 103 specific image

Claims (7)

A three-dimensional non-contact input device that transmits an operator's action to a computer,
A plurality of cameras directed toward the operator from different positions;
A processing unit that transmits information on the specific part of the operator based on acquired images obtained from the plurality of cameras,
The processor is
A depth image forming unit that forms a depth image from the acquired images obtained from the plurality of cameras;
An object recognition unit that performs object recognition processing including coordinate recognition and shape recognition of the specific part at a predetermined distance with respect to the plurality of cameras based on the depth images;
Have
A three-dimensional non-contact input device characterized in that information on the coordinates and shape of the specific part obtained by the object recognition unit can be transmitted to the computer.   2. The image processing unit according to claim 1, wherein the processing unit includes an image pre-processing unit that performs a filter process for erasing an image having a depth outside a predetermined distance in the depth image obtained by the depth image generation unit. 3D non-contact input device.   The object recognition unit performs the object recognition process on the entire image for the first time, and from the second time, only a predetermined range around the specific part obtained by the previous object recognition process in the object recognition unit. The three-dimensional non-contact input device according to claim 1, wherein the object recognition process is performed on a target.   4. The object recognition unit according to claim 1, wherein the object recognition unit performs the object recognition processing by comparing the image with a template image stored in advance. 5. Dimensional non-contact input device.   5. The image processing unit according to claim 1, wherein the processing unit includes an image post-processing unit that performs coordinate averaging and movement curve adjustment for the specific part obtained by the object recognition unit. The three-dimensional non-contact input device described.   6. The three-dimensional non-contact input device according to claim 1, wherein the computer recognizes and operates as a USB standard HID class.   The three-dimensional non-contact input device according to any one of claims 1 to 6, wherein the plurality of cameras and the processing unit are formed as a single unit.