TWI469060B - Motion recognition system and motion recognition method - Google Patents

Description

Motion recognition system and motion recognition method

The present invention relates to a motion recognition system and a motion recognition method.

In recent years, due to the rapid development of the semiconductor industry, it has driven the growth and progress of the electronics industry. Especially in the development of media playback devices, there has been a great breakthrough in the performance of sound or video. However, in general, the input interface of most media playback devices is still a conventional keyboard, mouse, joystick or multi-button control handle.

Thus, in order to provide a more convenient and intuitive input interface for the user, the operator has proposed an operation mode through the image sensing device as an input interface. Please refer to FIG. 1 , which is a conventional image sensing device 1 . The image sensing device 1 has an image sensor 11, a controller 12, and a bus bar 13. The image sensor 11 is configured to capture a continuous motion image of the user, and convert the motion image into image data for transmission to the controller 12 via the bus bar 13 . Next, the controller 12 compares the received plurality of image data in sequence to determine the state of the user, and controls the media playback device to be used in accordance with the state of the user.

However, since the amount of data of the motion image sensed by the image sensor 11 is extremely large, and the controller 12 must receive all the image data and perform subsequent processing and comparison, it will cause a burden on the data operation, thereby affecting the processing. Speed and performance.

Therefore, how to provide a motion recognition system and motion recognition method, which can improve data processing speed and performance, and at the same time provide more convenient and more intuitive input operations, has become one of the important topics.

In view of the above problems, an object of the present invention is to provide a motion recognition system and a motion recognition method capable of improving data processing speed and performance while providing a more convenient and intuitive input operation.

To achieve the above object, a motion recognition system according to the present invention cooperates with an action object. The motion recognition system includes a signal generating module, a sensing module and a processing module. The signal generating module sequentially outputs a first optical signal, a second optical signal and a third optical signal. The sensing module receives the first optical signal, the second optical signal, and the third optical signal, and generates an image according to the first optical signal, the second optical signal, and the third optical signal during an operation period. The processing module is coupled to the sensing module and generates an action signal according to the image. The motion signal represents one of the motion changes of the motion object.

In an embodiment of the invention, the signal generating module includes a light emitting unit and a filter unit. The light emitting unit outputs an optical signal. The filter unit is disposed opposite to the light emitting unit, and receives the optical signal to sequentially output the first optical signal, the second optical signal, and the third optical signal.

In one embodiment of the present invention, the signal generating module has a first lighting unit, a second lighting unit, and a third lighting unit, and the first lighting unit, the second lighting unit, and the third lighting unit respectively output An optical signal, a second optical signal, and a third optical signal.

In an embodiment of the invention, the first optical signal, the second optical signal, and the third optical signal have different wavelengths or light intensities.

In an embodiment of the invention, the sensing module comprises a charge coupled device array or a complementary MOS array.

In an embodiment of the invention, the first optical signal, the second optical signal, and the third optical signal are transmitted to the sensing module via reflection of the motion object.

In an embodiment of the invention, the motion recognition system further includes a media playback device. The media playback device is coupled to the processing module and displays an object. The media playback device adjusts the moving rate or moving path of the object according to the motion signal.

In an embodiment of the invention, the motion recognition system further includes a control module. The control module is coupled to the processing module and generates a control signal according to the motion signal. The media playback device performs at least one function according to the control signal.

In an embodiment of the invention, the control module has a storage unit and stores at least one action data reference value.

In an embodiment of the invention, the filter unit is further provided with a light shielding unit, and the light shielding unit is located between the filter unit and the sensing module.

In one embodiment of the invention, the action change includes a rate of movement.

In an embodiment of the invention, the action change comprises a moving path.

To achieve the above object, a motion recognition method according to the present invention cooperates with an action object. The motion recognition method includes: sequentially outputting a first optical signal, a second optical signal, and a third optical signal by a signal generating module; and receiving, by a sensing module, the first optical signal, the second optical signal, and the first a three-light signal; the sensing module generates an image according to the first optical signal, the second optical signal, and the third optical signal during an operation period; and a processing module generates an action signal according to the image, wherein the action The signal represents an action change on the action object.

In an embodiment of the invention, the first optical signal, the second optical signal, and the third optical signal have different wavelengths or light intensities.

In an embodiment of the present invention, the motion recognition method further includes: adjusting, by a media playback device, a moving rate or a moving path of an object displayed by the media playing device according to the motion signal.

In an embodiment of the present invention, the motion recognition method further includes: generating, by the control module, a control signal according to the motion signal; and performing, by the media playback device, the at least one function according to the control signal.

In an embodiment of the invention, the motion change includes a moving rate, and the moving speed is determined according to the density of the first optical signal, the second optical signal, and the third optical signal in the image.

In an embodiment of the present invention, the action change includes a moving path, and the order of the first optical signal, the second optical signal, and the third optical signal output by the signal generating module is a first order. The order of the first optical signal, the second optical signal, and the third optical signal in the image received by the sensing module is a second order. The processing module compares the first sort and the second sort, and determines the moving path of the action change.

According to the above, a motion recognition system and a motion recognition method according to the present invention generate a first optical signal, a second optical signal, and a third optical signal output by the sensing module receiving the signal generating module. The image is further processed by the processing module to generate an action signal representing an action change generated by an action object according to the image. This enables faster data processing and performance while providing more convenient and intuitive input operations.

Hereinafter, a motion recognition system and a motion recognition method according to a preferred embodiment of the present invention will be described with reference to the related drawings, wherein like elements will be described with the same reference numerals.

First, please refer to FIG. 2A to FIG. 2C to illustrate a motion recognition system 2 in accordance with a preferred embodiment of the present invention. The motion recognition system 2 cooperates with an action object T, which is, for example, a user. The motion recognition system 2 includes a signal generating module 21, a sensing module 22 and a processing module 23.

In this embodiment, the signal generating module 21 includes a light emitting unit 211, a filter unit 212, and a light blocking unit 213. The light emitting unit 211 outputs an optical signal S ₁ . The filter unit 212 is disposed opposite to the light emitting unit 211 and receives the optical signal S ₁ . The filter unit 212 sequentially converts the optical signal S ₁ into a first optical signal S ₁₁ , a second optical signal S _{12 ,} and a third optical signal S ₁₃ , and sequentially outputs the first optical signal S ₁₁ and the second light. Signal S ₁₂ and third optical signal S ₁₃ .

In an implementation, the light-emitting unit 211 can be a light source that generates white light, and the filter unit 212 includes a color wheel. The filter unit 212 sequentially generates three different colors of red, green, and blue by continuous rotation of the color wheel. The wavelength (color) of the optical signal. The rotation speed of the color wheel is, for example, one rotation every 3 milliseconds, that is, the filter unit 212 outputs light of one color every 1 millisecond. It should be noted that the sequential output described herein may be that the first, second, and third optical signals S ₁₁ , S ₁₂ , and S ₁₃ are outputted continuously or uninterrupted, and may be first and second. The second and third optical signals S ₁₁ , S ₁₂ , and S ₁₃ are output at a fixed time interval. Shielding unit 213 disposed at one side of the filter unit 212, a first optical signal S _11, the second optical signal, S ₁₂ and S ₁₃ and the third optical signal from the signal generating module 21 is emitted by the light-blocking mode to the sensing unit 213 Group 22. In other words, the shading unit 213 is located between the filter unit 212 and the sensing group 22.

The sensing module 22 receives the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S ₁₃ output by the signal generating module 21 . In an implementation, the sensing module 22 includes a charge-coupled device (CCD) array or a complementary metal oxide semiconductor (CMOS) array, according to the first optical signal S ₁₁ , the second The optical signal S ₁₂ and the third optical signal S ₁₃ are generated during an operation period, for example, 30 milliseconds during operation, to generate an image P, which is, for example, continuous exposure during operation to be first The optical signal S ₁₁ , the second optical signal S ₁₂ and the third optical signal S _{13 are} recorded in the same image P.

In addition, it is worth mentioning that, in this embodiment, the sensing module 22 is held or mounted by the action object T to directly receive the first optical signal S ₁₁ and the second optical signal S output by the signal generating module 21 . ₁₂ and the third optical signal S _{13 are taken} as an example. However, in actual use, the signal generating module 21 can be held or mounted by the action object T, and the optical signal generated by the signal generating module 21 can be transmitted to The sensing module 22 is disposed at another location. In addition, in addition to the above-mentioned sensing module 22, the signal generating module 21 and the sensing module 22 are respectively installed in different places in a manner of directly receiving the signal output by the signal generating module 21, and the sensing module is respectively installed. The group 22 can also receive the first optical signal S ₁₁ , the second optical signal S ₁₂ and the third optical signal S ₁₃ reflected by the action object T by means of reflection, for example, the action object T (user) additionally A reflective component is worn or worn to reflect the signal output by the signal generating module 21 to the sensing module 22. In this way, the signal generating module 21 and the sensing module 22 are both disposed at the same place.

The processing module 23 is coupled to the sensing module 22 and generates an action signal S _M according to the image P output by the sensing module 22. Wherein the operation of the signal S _M T based objects that represent actions generated by changing one operation, wherein the operation of the system, for example, change or rate of movement comprising moving path. In practice, the processing module 23 can be a microprocessor or a circuit composed of multiple processors. In addition, the processing module 23 can be integrated with the sensing module 22 in the same mechanism to effectively reduce the volume of the device.

Next, please refer to FIG. 3A to FIG. 3E together with FIG. 2A to FIG. 2C to further illustrate the motion recognition system 2 of the present invention. In operation, the sensing module 22 will display different colors according to the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S ₁₃ , for example, red R, green G, and blue B. As shown in FIG. 3A, the image P is red R, green G, and blue B are sequentially arranged to the right, because the signal generating module 21 sequentially outputs the first optical signal S ₁₁ and the second light at a fixed rate. The signal S ₁₂ and the third optical signal S ₁₃ , so that the processing module 23 can generate an action signal representing the change of the action of the action object T according to the color arrangement order displayed on the image P and the range of the area included in each color. S _M . Here, the range of the area included in each of the above colors represents the length, width, or area occupied by each color on the image P. And as shown in the image P shown in FIG. 3A, the motion object T has a motion change from left to right.

As shown in FIG. 3B, the movement path of the motion object T is the same as that of FIG. 3A, and is moved to the right. The area range of red R, green G, and blue B in the image P shown in FIG. 3B is longer than the range of the red R, green G, and blue B of the image P of FIG. 3A, in other words, The movement rate of the action object T corresponding to FIG. 3B is faster than the movement rate of the action object T of FIG. 3A. In addition, FIG. 3C to FIG. 3E respectively show the motion changes generated by the motion object T, and the movement paths thereof are right to left, bottom to top, and counterclockwise. In addition, if the user does not generate an action change during the operation of generating an image P, the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S ₁₃ are projected to the sensing module 22 . The same position on the top, and the image P is rendered white.

Therefore, the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S _{13 in} the image P generated by the processing module 23 according to the sensing module 22 can be obtained by the hardware architecture. The corresponding position, through a relatively simple operation or judgment, quickly generates a change in the action representative of the action object T, such as the movement rate or the motion signal S _{M of the} movement path, thereby improving the performance of the data processing.

In addition, in actual use, the signal generation module 21 can have other different components. Referring to FIG. 4A and FIG. 4B, two other architectures of the signal generation module are illustrated. As shown in FIG. 4A, the signal generating module 31 has a first lighting unit 311, a second lighting unit 312, and a third lighting unit 313. Here, the first light emitting unit 311, the second light emitting unit 312, and the third light emitting unit 313 are respectively a red light emitting diode, a green light emitting diode, and a blue light emitting diode to generate a wavelength ( Color) different first optical signal S ₁₁ , second optical signal S ₁₂ and third optical signal S ₁₃ .

In addition, as shown in FIG. 4B, it is another signal generation module 32. The signal generating module 32 includes a light emitting unit 321 and a filter unit 322. The light emitting unit 321 outputs an optical signal S ₁ . The filter unit 322 is disposed opposite to the light emitting unit 321 and receives the optical signal S ₁ . The filter unit 322 is a rotary light intensity adjuster, so that the first optical signal S ₁₁ , the second optical signal S ₁₂ and the third optical signal S _{13 having} different light intensities are generated.

Next, please refer to FIG. 5A and FIG. 5B, which are a motion recognition system 4 according to a preferred embodiment of the present invention. The difference between the motion recognition system 4 and the motion recognition system 2 is that the motion recognition system 4 further includes a media playback device 41. The media playback device 41 is coupled to the processing module 23. In the present embodiment, the media playback device 41 is a display device and displays an object O, such as the arrow cursor shown in Figure 5B. The media playing apparatus 41 according to the operation of the processing module 23 generates the signal S _M to adjust the rate of movement of the object O or moving path, that the object O is preferably exhibit the same or very similar to the operation target T or the moving speed of the moving path. In other words, the object O will simulate the pose or motion of the action object T, and for example, can be applied to a somatosensory game. In addition, in the embodiment, the signal generating module 21 is held by the action object T, and the sensing module 22 and the processing module 23 are commonly disposed in a mechanism and mounted on the media playing device 41. One of the signal generating module 21 and the sensing module 22 is disposed on the media playing device 41.

Please refer to FIG. 6A and FIG. 6B, which are another motion recognition system 5 according to a preferred embodiment of the present invention. The difference between the motion recognition system 5 and the motion recognition system 2 is that the motion recognition system 5 further includes a media playback device 51 and a control module 52. In this embodiment, the signal output by the signal generating module 21 is transmitted to the sensing module 22 via the reflection of the action object T (user), and the signal generating module 21 and the sensing module 22 are It is disposed on the media playback device 51. The control module 52 is coupled to the processing module 23. The control module 52 has a storage unit 521, and the storage unit 521 stores at least one motion data reference value V _M . The control module 52 receives the action signal S _M generated by the processing module 23, and compares the action signal S _M with the action data reference value V _M to generate a control signal S _C . The control module 52 transmits the control signal S _C to the media playback device 51, and the media playback device 51 will perform at least one function according to the control signal S _C . In practice, the media playback device 51 can be a television or an audio player, and the media playback device 51 can perform various functions such as power on, power off, channel selection, volume adjustment, or menu switching according to the control signal S _C . In addition, the control module 52 can be integrated into the processing module 23 to form a single module or circuit, and the control module 52 can also generate the control signal S _C by performing arithmetic processing on the motion signal S _M .

Referring to the flowchart of FIG. 7 and FIG. 2A to FIG. 2C, the motion recognition method of the preferred embodiment of the present invention can be used in combination with the motion recognition system 2 and the motion object T described above. The steps of the identification method include steps S01 to S04.

In step S01, a first optical signal S ₁₁ , a second optical signal S ₁₂ and a third optical signal S _{13 are} sequentially output by a signal generating module 21 . In an implementation, the signal generating module 21 can generate a first optical signal S ₁₁ , a second optical signal S _{12 ,} and a third optical signal S _{13 of} three different wavelengths (colors) or light intensities with a white light source and a color wheel. Alternatively, the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S _{13 having} different wavelengths are generated directly by three different color light-emitting elements.

In step S02, the first optical signal S ₁₁ , the second optical signal S ₁₂ and the third optical signal S _{13 are} received by a sensing module 22 . In practice, the sensing module 22 includes a charge coupled device array or a complementary MOS array.

In step S03, the sensing module 22 generates an image P according to the first optical signal S ₁₁ , the second optical signal S ₁₂ and the third optical signal S ₁₃ during an operation period. In practice, the sensing module 22 will display corresponding colors according to the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S ₁₃ , for example, red R, green G, and blue. Color B, and sensing module 22 will, for example, produce an image P every 30 milliseconds.

In step S04, a processing module 23 generates an action signal S _M according to the image P, wherein the action signal S _M represents an action change of the action object T. In practice, since the signal generating module 21 sequentially outputs the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S ₁₃ at a fixed rate, the processing module 23 can be based on the image. The color arrangement displayed on P and the range of areas included in each color generate an action signal S _M representing the movement rate or movement path of the motion object T.

It is worth mentioning that when the action change generated by the action object T includes the moving rate, the processing module 23 is based on the density of the red R, the green G, and the blue B in the image P, that is, according to the first light. The corresponding relationship between the signal S ₁₁ , the second optical signal S ₁₂ and the third optical signal S ₁₃ in the region range presented on the image P determines the moving rate of the motion change of the motion object T. In addition, when the action change generated by the action object T includes the moving path, the order of the first optical signal S ₁₁ , the second optical signal S _{12 ,} and the third optical signal S ₁₃ output by the signal generating module 21 can be defined. For the first ordering, the order of the first optical signal S ₁₁ , the second optical signal S ₁₂ and the third optical signal S ₁₃ received by the sensing module 22 is defined as a second ordering, that is, the sensing module 22 defines the order of red R, green G, and blue B presented in image P as a second order. The processing module 23 determines the movement path of the motion change of the motion object T by comparing the first sort and the second sort.

Secondly, when used in conjunction with the above-described motion recognition system 4, the motion recognition method may further include a step of adjusting the moving rate or movement of one of the objects O displayed by the media playback device 41 according to the motion signal S _M. path. In the embodiment, the media player device 41 based on the operation of the processing module 23 generates a moving path or moving speed adjustment signal S _M of the object O, so that the object O is preferably exhibit the same or very similar operation rate of movement of the subject T Or move the path.

In addition, when used in conjunction with the motion recognition system 5 described above, the step of the motion recognition method further includes: generating, by the control module 52, a control signal S _C according to the motion signal S _M ; and controlling by a media playback device 51 The signal S _C performs at least one function. In an implementation, the control module 52 can compare the action signal S _M with the action data reference value V _M stored by the storage unit 521 to generate the control signal S _C and cause the media playback device 51 to perform the control signal S _C according to the control signal S _C . A variety of different functions such as power on, power off, channel selection, volume adjustment or menu switching. In addition, the control module 52 may also perform the arithmetic processing on the motion signal S _M by a preset function or algorithm to generate the control signal S _C .

In summary, the motion recognition system and the motion recognition method according to the present invention generate a first optical signal, a second optical signal, and a third optical signal output by the sensing module receiving the signal generating module. The image is further processed by the processing module to generate an action signal representing an action change generated by an action object according to the image. This enables faster data processing and performance while providing more convenient and intuitive input operations.

The above is intended to be illustrative only and not limiting. Any equivalent modifications or alterations to the spirit and scope of the invention are intended to be included in the scope of the appended claims.

1. . . Image sensing device

11. . . Image sensor

12. . . Controller

13. . . Busbar

2, 4, 5. . . Motion recognition system

21, 31, 32. . . Signal generation module

211, 321. . . Light unit

212, 322. . . Filter unit

213. . . Shading unit

twenty two. . . Sensing module

twenty three. . . Processing module

311. . . First lighting unit

312. . . Second lighting unit

313. . . Third lighting unit

41, 51. . . Media player

52. . . Control module

521. . . Storage unit

B. . . blue

G. . . green

O. . . object

P. . . image

R. . . red

S01～S04. . . Steps of the motion recognition method

S ₁ . . . Optical signal

S ₁₁ . . . First optical signal

S ₁₂ . . . Second optical signal

S ₁₃ . . . Third optical signal

S _M . . . Motion signal

Sc. . . Control signal

T. . . Action object

V _M . . . Motion data reference value

1 is a schematic view of a conventional image sensing device;

2A-2C are schematic diagrams of a motion recognition system in accordance with a preferred embodiment of the present invention;

3A-3E are schematic diagrams of images generated by a sensing module of a motion recognition system in accordance with a preferred embodiment of the present invention;

4A and 4B are schematic diagrams showing changes in a signal generating module according to a preferred embodiment of the present invention;

5A and 5B are schematic diagrams of a motion recognition system in accordance with a preferred embodiment of the present invention;

6A and 6B are schematic diagrams of a motion recognition system in accordance with a preferred embodiment of the present invention;

FIG. 7 is a flow chart of a motion recognition method according to a preferred embodiment of the present invention.

2. . . Motion recognition system

twenty one. . . Signal generation module

twenty two. . . Sensing module

twenty three. . . Processing module

P. . . image

S ₁₁ . . . First optical signal

S ₁₂ . . . Second optical signal

S ₁₃ . . . Third optical signal

S _M . . . Motion signal

Claims (18)

A motion recognition system, comprising: a signal generating module, sequentially outputting a first optical signal, a second optical signal and a third optical signal; and a sensing module receiving the first The optical signal, the second optical signal, and the third optical signal, and an image generated by the first optical signal, the second optical signal, and the third optical signal during an operation period; and a processing module, The action module is coupled to the sensing module and generates an action signal according to the image, wherein the action signal represents a motion change of the action object. The motion recognition system of claim 1, wherein the signal generating module comprises: at least one light emitting unit, outputting an optical signal; and a filter unit disposed opposite to the light emitting unit and receiving the optical signal And outputting the first optical signal, the second optical signal, and the third optical signal in sequence. The motion recognition system of claim 1, wherein the signal generating module has a first lighting unit, a second lighting unit, and a third lighting unit, and the first lighting unit and the second lighting unit The unit and the third lighting unit respectively output the first optical signal, the second optical signal and the third optical signal. The motion recognition system of claim 1, wherein the first optical signal, the second optical signal, and the third optical signal have different wavelengths or light intensities. The motion recognition system of claim 1, wherein the sensing module comprises an array of charge coupled devices or a complementary array of metal oxide semiconductors. The motion recognition system of claim 1, wherein the first optical signal, the second optical signal, and the third optical signal are transmitted to the sensing module via reflection of the motion object. The action recognition system of claim 1, further comprising: a media playback device coupled to the processing module and displaying an object, wherein the media playback device adjusts the moving speed of the object according to the motion signal Or move the path. The action recognition system of claim 7, further comprising: a control module coupled to the processing module, and generating a control signal according to the action signal, wherein the media playback device performs the control signal according to the control signal At least one feature. The action recognition system of claim 8, wherein the control module has a storage unit that stores at least one action data reference value. The motion recognition system of claim 2, wherein the filter unit is further provided with a light shielding unit, and the light shielding unit is located between the filter unit and the sensing module. The action recognition system of claim 1, wherein the action change comprises a movement rate. The action recognition system of claim 1, wherein the action change comprises a movement path. A motion recognition method, which cooperates with an action object, includes: a signal generating module sequentially outputting a first optical signal, a second optical signal, and a third optical signal; and receiving, by the sensing module, the first The optical signal, the second optical signal, and the third optical signal are generated by the sensing module according to the received first optical signal, the second optical signal, and the third optical signal during an operation period. An image is generated by the processing module according to the image, wherein the motion signal represents one of the motion changes of the motion object. The motion recognition method of claim 13, wherein the first optical signal, the second optical signal, and the third optical signal have different wavelengths or light intensities. The action recognition method of claim 13, further comprising: adjusting, by a media playback device, a moving rate or a moving path of an object displayed by the media playback device according to the motion signal. The action recognition method of claim 13, further comprising: generating, by the control module, a control signal according to the action signal; and performing, by the media playback device, at least one function according to the control signal. The motion recognition method of claim 13, wherein the motion change includes a moving rate, and according to the density of the first optical signal, the second optical signal, and the third optical signal in the image, Determine the rate of its movement. The action recognition method of claim 13, wherein the action change includes a movement path, and the order of the first optical signal, the second optical signal, and the third optical signal output by the signal generating module The order of the first optical signal, the second optical signal, and the third optical signal in the image received by the sensing module is a second order, and the processing module compares The first sorting and the second sorting determine a moving path of the action change.