JP5812582B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to a technique for setting a region for recognizing a space gesture.

  2. Description of the Related Art Conventionally, a technique for setting a part of a moving image as a region for recognizing a gesture based on the width, vertical length, or area of a person's face extracted from the moving image is known (for example, patents). Reference 1). A touch panel for operating a device by touching a screen is also known.

JP 2003-216955 A

  However, according to the conventional technology, it is not possible to set an area for recognizing a space gesture based on a contact position according to a user's preference. Therefore, an object of the present invention is to set an area for recognizing a space gesture based on a contact position according to a user's preference.

In order to solve the above-described problem, an information processing apparatus according to the present invention includes a detection unit that detects a contact position on a display surface, and a setting that sets a region for recognizing a space gesture based on the detected contact position. A position of a region for recognizing a space gesture at a position predicted from the shape of the movement trajectory. Set. In addition, a detection unit that detects a contact position on the display surface, and a setting unit that sets a region for recognizing a space gesture based on the detected contact position, the detection unit further includes the contact The distance, the moving direction, or the moving speed of the movement trajectory of the position is detected, and the setting means sets the position of the region for recognizing the space gesture to the position predicted from the distance, the moving direction, or the moving speed of the moving trajectory. To do.

According to the present invention, it is possible to appropriately set an area for recognizing a spatial gesture input by a user following contact with a display surface .

Hardware configuration diagram and functional block diagram of the information processing apparatus 100 The figure which shows a mode that the input part 140, the distance image sensor 160, etc. were installed. The flowchart which shows the flow of the process which performs control according to space gesture The figure which shows the gesture recognition area | region set in step S308. The figure which shows the gesture recognition area | region set in step S307. The figure which shows the gesture recognition area | region set in step S306. The figure which shows the control corresponding to various space gestures The figure which shows a gesture recognition area when an object does not exist in a contact position

(First embodiment)
FIG. 1A is a diagram showing a hardware configuration of the information processing apparatus 100 according to the present invention. The information processing apparatus 100 includes a CPU 101, a ROM 102, a RAM 103, an input interface (I / F) 104, an output I / F 105, an input I / F 106, an output I / F 107, and a communication I / F 108. The above-described units are connected via a system bus 109. The information processing apparatus 100 is connected to an input unit 140, a display unit 150, a distance image sensor 160, a display unit 170, and a network 180. Hereinafter, each part of the information processing apparatus 100 and each part connected to the information processing apparatus 100 will be described.

  The CPU 101 implements various operations of the information processing apparatus 100 by expanding and reading a program or the like stored in the ROM 102 in the RAM 103. The ROM 102 stores programs for causing the information processing apparatus 100 to perform various operations, data necessary for executing the programs, and the like. The RAM 103 provides a work area for developing a program stored in the ROM 102. The input I / F 104 acquires data indicating a user's instruction input (contact position) to the input unit 140. The output I / F 105 transmits data indicating an image to be displayed on the display unit 150. The input I / F 106 acquires data indicating the distance image generated by the distance image sensor 160. The output I / F 107 transmits data indicating an image to be displayed on the display unit 170. The communication I / F 108 is connected to another computer or the like via the network 180 and transmits / receives various data.

  The input unit 140 is, for example, a touch panel, a digitizer, or the like provided so as to overlap the display unit 150, and detects the coordinates of the contact position on the display surface. The display unit 150 is, for example, a liquid crystal display (LCD), and displays and outputs images, characters, and the like. The distance image sensor 160 is formed by reflecting an object such as an infrared ray and the like, and a reference surface (for example, a surface perpendicular to the direction in which the distance image sensor measures the distance, A distance image indicating the distance to the surface on which the distance image sensor exists is generated. The distance image is an image having a pixel value as a distance between an object existing in a predetermined three-dimensional area and a reference surface. The display unit 170 is, for example, an LCD, and displays and outputs images, characters, and the like. Note that the display unit 150 and the display unit 170 may cooperate with each other and function as a multi-display.

  FIG. 1B is a diagram illustrating a functional configuration of the information processing apparatus 100. The information processing apparatus 100 includes a detection unit 111, a setting unit 112, an acquisition unit 113, a recognition unit 114, and a control unit 115. The detection unit 111 corresponds to the input I / F 104, and the acquisition unit 113 corresponds to the input I / F 106. The setting unit 112, the recognition unit 114, and the control unit 115 are realized by the CPU 101 developing and reading a predetermined program or the like stored in the ROM 102 on the RAM 103. Hereinafter, each part will be described.

  The detection unit 111 detects the coordinates of the contact position on the input unit 140. The detection unit 111 detects coordinates such as one point, a plurality of points, and a locus. The case where the coordinates of a plurality of points are detected by the detection unit 111 is, for example, the case where the coordinates of other contact positions are detected within a predetermined time after the coordinates of the first contact position are detected. It is. Also, the case where the coordinates of a plurality of points are detected by the detection unit 111 means, for example, that other contact positions are detected before the process of recognizing the space gesture is started after the coordinates of the first contact position are detected. Is detected.

  The setting unit 112 sets an area for recognizing a space gesture (gesture recognition area) based on the coordinates detected by the detection unit 111. The spatial gesture refers to an operation of moving a hand so as to cross the horizontal direction, an operation of crossing the hands, an operation of moving the hands in the vertical direction within a three-dimensional region serving as a detection region of the distance image sensor 160, An operation of moving a hand in the horizontal direction and the vertical direction. In addition, the gesture recognition area set by the setting unit 112 corresponds to a part of a three-dimensional area that is a detection area of the distance image sensor 160.

  The acquisition unit 113 acquires data indicating the distance image generated by the distance image sensor 160. The recognition unit 114 identifies a portion corresponding to the gesture recognition area from the acquired distance image, and recognizes a space gesture in the portion. Note that the spatial gesture is recognized based on, for example, changes in pixel values of a plurality of distance images obtained within a predetermined time from when the contact with the input unit 140 is detected. The control unit 115 executes a process corresponding to the space gesture recognized by the recognition unit 114, and changes, for example, images and characters displayed on the display units 150 and 170.

  FIG. 2 is a diagram illustrating a state in which the information processing apparatus 100, the input unit 140, the display unit 150, the distance image sensor 160, and the display unit 170 are installed. The information processing apparatus 100 only needs to be connected to other units, and can be installed at an arbitrary place. The input unit 140 and the display unit 150 are installed so as to overlap each other, and in this embodiment, are assumed to be installed on a desk. The distance image sensor 160 is installed on, for example, a ceiling surface (above the input unit 140 and the display unit 150), and the three-dimensional region that is the detection region of the distance image sensor 160 is indicated by a broken line, for example. A substantially frustum-shaped space (a space above the input unit 140 and the display unit 150). The display part 170 shall be installed in the side surface (wall surface).

  In this embodiment, a horizontal plane is described as an XY plane defined by an X axis and a Y axis, and a vertical axis is described as a Z axis. Further, in the present embodiment, a point in the vicinity of one vertex of the input unit 140 and the display unit 150, which is substantially the same height as the input unit 140 and the display unit 150, will be described as the origin of the XYZ space.

  FIG. 3 is a flowchart showing a flow of processing for executing control according to the recognized space gesture.

  In step S301, the detection unit 111 determines whether or not contact with the input unit 140 is detected. When contact is detected, the processing of step S302 is executed. On the other hand, if no contact is detected, the process of step S301 is repeated. In step S <b> 302, the detection unit 111 acquires information indicating the contact position, and outputs the information to the setting unit 112. The information indicating the contact position is, for example, information indicating the coordinates of the contact position, the shape of the movement locus of the contact position, the movement direction of the locus, the information indicating the movement distance of the locus, the information indicating the number of contact points, Information indicating the object displayed at the contact position, the duration of the detected contact, and the like.

  In step S303, the setting unit 112 determines whether there is an object displayed at the contact position based on the information indicating the contact position. If it is determined that the object is present, the setting unit 112 executes the process of step S304. On the other hand, if it is not determined that it exists, the series of processing ends. If it is not determined that the object displayed at the contact position exists, the entire three-dimensional area that is the detection area of the distance image sensor 160 may be set as the gesture recognition area. The gesture recognition area may be predicted based on the shape of the movement locus of the contact position.

  In step S304, the setting unit 112 determines whether or not the detection of contact continues for a predetermined time (for example, 1 second). If it is determined that the detection continues for a predetermined time, the setting unit 112 executes the process of step S306. On the other hand, if it is not determined that the predetermined time has continued, the process of step S305 is executed. In step S305, the setting unit 112 determines whether or not contact is detected again within a predetermined time after the detection of contact starting from the detection in step S301, and it is determined that contact is detected again within the predetermined time. If YES, the process of step S307 is executed. On the other hand, when contact is not detected again within the predetermined time, the process of step S308 is executed. In step S304, whether or not there is a movement of the contact position may be further determined based on whether or not the contact position has moved by a predetermined threshold or more in a state where the contact continues. That is, when the contact position (coordinate point) does not move beyond the predetermined threshold value when the contact is continued (when the movement amount of the coordinate point is not equal to or greater than the threshold value), it is determined that there is no movement of the contact position. The

  In step S306, the setting unit 112 sets a gesture recognition area based on the coordinates of one contact position, the distance of the movement locus of the contact position, and the movement direction. In step S306, a gesture recognition area may be set based on the moving speed and acceleration of the movement locus of the contact position. Note that the movement trajectory means a set of contact positions when the contact positions move in time series. In step S307, the setting unit 112 sets a gesture recognition area based on the coordinates of the plurality of contact positions. In step S308, the setting unit 112 sets a gesture recognition area based on the coordinates of one contact position. The gesture recognition area set in steps S306, S307, and S308 will be described later.

  In step S309, the recognition unit 114 recognizes a spatial gesture in a portion corresponding to the gesture recognition region set in any of steps S306, 307, and 308 in the distance image acquired by the acquisition unit 113. In step S310, the control unit 115 executes control corresponding to the recognized space gesture. Note that the types of recognized spatial gestures will be described later. The above is the flow of processing for executing control according to the recognized space gesture.

  The following modifications are also within the scope of the present invention. That is, instead of the above-described steps S304 and S305, the setting unit 112 converts the shape of the movement locus of the contact position to the shape of a line (a plurality of continuous points) or a shape of a plurality of discrete points. It is determined whether it corresponds to the shape of one point. If it is determined that the shape of the movement locus of the contact position corresponds to the shape of a line, the process of step S306 is executed. If it is determined that the shape corresponds to a plurality of discrete points, the process of step S307 is executed. If it is determined that the shape corresponds to one point, the process of step S308 is executed. The shape of the contact position is determined, for example, by comparing the feature information indicating the shape registered in advance in the ROM 102 with the movement locus (one or a plurality of coordinate points) of the detected contact position.

  Next, the gesture recognition area set in steps S308, S307, and S306 will be described with reference to FIGS. FIGS. 4, 5, and 6 (a) are views when a three-dimensional region that is a detection region of the distance image sensor 160 is viewed vertically downward. 4B, 5B, and 6B are diagrams when a three-dimensional region that is a detection region of the distance image sensor 160 is viewed in the horizontal direction.

  In FIG. 4, there is one point in contact with the input unit 140, and an object is displayed at the contact position. Therefore, a predetermined area (the area indicated by a broken line in FIG. 4) above the contact position and based on the contact position is set as a gesture recognition area. Note that the vertical, horizontal, and height of the gesture recognition area may be determined based on the length of a general human arm, or may be determined according to the size of the displayed object.

  In FIG. 5, there are three points that are in contact with the input unit 140, and objects A, B, and C are displayed at the respective contact positions. Therefore, a region above all three contact positions and including all three contact positions (a region indicated by a broken line in FIG. 5) is set as a gesture recognition region. Note that the three points in the input example of FIG. 5 are examples in which three points are input continuously in a short time. In this case, as processing, short-time contact input data is processed as one data group. That is, counting is performed from the time point when the contact detection is completed, and if there is a next input in a short time, it is processed as one data group as it is, and if a predetermined short time has passed, the contact input data is The input waiting process ends.

  In FIG. 6, there is one point in contact with the input unit 140, and the object A is displayed at the contact position. And the contact position is moving. Therefore, a region above the contact position and moved from a predetermined region (the region indicated by the broken line in FIG. 4) on the basis of the contact position on the extension of the movement locus of the contact position (the portion indicated by the broken line in FIG. 6). Region) is a gesture recognition region.

  In step S306, when the gesture recognition area is set based on the movement speed and acceleration of the movement locus of the contact position, the gesture recognition area may be set wider as the movement speed and acceleration increase. In this way, when a fine touch operation is performed, it is possible to accurately recognize a spatial gesture with a fine movement. When a dynamic touch operation is performed, a spatial gesture with a large movement is accurately detected. Can be recognized.

  FIG. 7 is an example of a table showing control corresponding to various spatial gestures recognized in the gesture recognition area. When an object is displayed at the contact position, the control to be executed differs depending on the number of touched points and whether or not the contact position has moved. Further, data corresponding to the table shown in FIG. 7 is stored in the ROM 102, for example.

  FIG. 8 is a diagram illustrating another example of the process performed when it is not determined in step S303 that the object displayed at the contact position exists. FIG. 8A is a diagram when a three-dimensional region that is a detection region of the distance image sensor 160 is viewed vertically downward. FIG. 8B is a diagram when a three-dimensional region that is a detection region of the distance image sensor 160 is viewed in the horizontal direction.

  In FIG. 8, there is one point in contact with the input unit 140, and the object A is not displayed at the contact position. And the contact position is moving. Accordingly, as a region based on the shape of the movement locus of the contact position, for example, a rectangle circumscribing the movement locus (region of a broken line portion in FIG. 8) is set as a gesture recognition region.

  In the present embodiment, the operation by one user has been described, but there may be a case where two or more users operate at the same time. For example, if the screen corresponding to the input unit 140 and the display unit 150 is a screen having a size of 50 inches or 100 inches, it is possible for two users to perform input operations at the same time. In that case, if inputs are detected simultaneously at distant points on the screen (for example, 1 m to 2 m), they may be handled as input by a plurality of users. That is, as the input by the user A and the input by the user B, the above-described series of processes may be executed for each input. In addition, when input to a plurality of distant points on the screen is detected, a distant region corresponding to each input position may be set as a gesture recognition region so that the gesture recognition regions are located exclusively. Good.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

Detecting means for detecting a contact position on the display surface;
Setting means for setting a region for recognizing a space gesture based on the detected contact position;
The detection means further detects the shape of the movement locus of the contact position,
The information processing apparatus, wherein the setting unit sets a position of an area for recognizing a space gesture at a position predicted from the shape of the movement trajectory. Detecting means for detecting a contact position on the display surface;
Setting means for setting a region for recognizing a space gesture based on the detected contact position;
The detecting means further detects a distance, a moving direction, or a moving speed of the movement locus of the contact position,
The information processing apparatus, wherein the setting unit sets a position of an area for recognizing a space gesture at a position predicted from a distance, a moving direction, or a moving speed of the movement locus. The setting means, when an object to have been the contact position detected by said detecting means at said display surface exists, claims and sets the recognized area space gesture is input to the object The information processing apparatus according to any one of 1 and 2.   4. The apparatus according to claim 1, further comprising a recognizing unit for recognizing a space gesture input by the pointing object based on a movement of the pointing object in an area set by the setting unit. The information processing apparatus according to item 1. Having an acquisition means for acquiring image information indicating a distance between a pointing object existing in a predetermined three-dimensional region and a reference surface;
The recognizing unit recognizes a spatial gesture input by the pointing object based on information of a portion corresponding to an area set by the setting unit in the image information acquired by the acquiring unit. The information processing apparatus according to claim 4.   The recognizing unit is configured to perform the instruction within a region set by the setting unit based on a plurality of pieces of image information acquired by the acquiring unit during a predetermined time from the time when the contact position is detected by the detecting unit. The information processing apparatus according to claim 5, wherein a spatial gesture input by an object is recognized. Furthermore, it has execution means for executing processing corresponding to the spatial gesture recognized by the recognition means,
When the pointing object is displayed at the contact position detected by the detection means in the predetermined three-dimensional space close to the display surface, the execution means determines whether or not the detected contact position has moved. The information processing apparatus according to any one of claims 1 to 6, wherein different processing is executed according to a space gesture recognized by the recognition unit.   The setting means sets a plurality of regions at positions based on each of the plurality of contact positions when the detection means detects a plurality of contact positions separated on the display surface. The information processing apparatus according to any one of claims 1 to 7. An information processing apparatus operating method for setting a region for recognizing a space gesture,
A detection step of detecting a contact position on the display surface, and a setting step of setting a region for recognizing a space gesture based on the detected contact position;
In the detection step, the shape of the movement locus of the contact position is further detected,
The operation method characterized in that the setting step sets a position of a region for recognizing a space gesture at a position predicted from the shape of the movement locus.   A program for causing a computer to execute the operation method according to claim 9. An information processing apparatus operating method for setting a region for recognizing a space gesture,
A detection process for detecting a contact position on the display surface;
A setting step for setting a region for recognizing a space gesture based on the detected contact position;
In the detecting step, the distance, moving direction, or moving speed of the moving locus of the contact position is further detected,
In the setting step, the position of a region for recognizing a space gesture is set at a position predicted from the distance, the moving direction, or the moving speed of the movement locus.   A program for causing a computer to execute the operation method according to claim 11.

Images