TWI394063B - System of inputting instruction by image identification and method of the same - Google Patents

Description

Command input system and method using image recognition

The present invention relates to a data processing technique, and more particularly to a data processing technique that utilizes image recognition technology as an instruction input.

With the rapid development of data processing software and hardware technology, the use of data processing devices such as personal computers, notebook computers, and smart mobile phones as work forms for processing documents, playing briefings or multimedia messages is becoming more and more popular.

For example, a student, a professor, or a business person can use a notebook computer with an image generating device such as a projector to play a report file, a teaching content, or a briefing file for a commercial message. For example, a conventional briefing file can be, for example, a Microsoft product. Microsoft Office PowerPoint ^TM .

In the implementation process, the user must perform the input of the operation commands such as the previous page, the next page, and the file or the application by using the button of the notebook computer or the mouse connected thereto. At this time, if the user is located close to the notebook computer, it is convenient to operate. However, if there is a considerable distance between the user and the notebook, or the user must be in a moving state, it is not convenient to input an operation command through a button or a mouse.

In order to solve this shortcoming, the prior art provides a wireless remote control technology using infrared rays or blue buds, and also an input unit such as an operation button or a trackball is disposed on a device similar to a remote controller, and an operation command is issued through a wireless signal mode. The notebook receives the wireless signal to perform an operation that allows the instruction. Although this technology can solve the problem that the user can't leave the notebook too far when the briefing is made, the extra wireless remote control device is a limitation and burden for the user, and the fashion must take into consideration the notebook computer and the wireless device. The remote control matches the compatibility issue. In addition, the wireless remote control device requires a portable power source, so that it is inevitable that the power supply is exhausted and cannot be used.

In order to solve the above-mentioned lack of the prior art, the Chinese Patent Publication No. I289203 discloses a "finger pointing detection system and method", which provides a finger pointing detection system and method, which are used by several image capturing devices. The number of hand images is processed by a processor, and a plurality of hand images are captured to obtain a plurality of finger pointing lines, and a coordinate conversion program is used to convert a finger pointing line to a projection line on the pointing plane. Finally, the intersection of several projection lines is calculated, which is the target point pointed by the user's finger. This technology can detect any target point on a plane without using any positioning gloves or marking gloves. It has high resolution and high accuracy, and can be applied to the presentation system as a replacement for the laser pen or slide. Mouse human machine interface.

Although the aforementioned patents can solve the problems existing in the human-machine interface, it is necessary to use a plurality of image capturing devices and perform complex calculations such as object center of gravity confirmation and coordinate conversion, which requires relatively large use and construction costs, and the system The installation procedure is quite cumbersome. In addition, complex computational processes are also an unfavorable factor for the resources of the data processing device, which may result in a decrease in the recognition rate in a limited hardware environment.

In summary, how to provide an instruction input system with convenient installation, low cost and high recognition rate to replace the conventional command input technology is an urgent problem to be solved.

In order to solve the problems of the prior art, the present invention provides an instruction input system for applying image recognition, which is applied to a data processing device. The data processing device is overlapped with an image capturing device, and the image input command input system is applied. The method includes: a setting module for defining at least one command input area in the image data captured by the image capturing device; and a detecting module for determining a foreground image from the image data to detect the command input The status information of the foreground image appearing in the area; the storage module is for storing a control command corresponding to the status information; and the control module is configured to use the status information detected by the detection module, The storage module extracts a corresponding control command to cause the data processing device to perform a functional action through the control command.

In a preferred embodiment, the data processing device is multiplexed with an image presenting device for presenting a specific area in the image data, and the setting module defines the specific area as a display area, wherein the setting The module defines at least one command input area within or outside the range of the display area, and the command input area has a function correspondence relationship with the display area, so that the detection module detects status information in the command input area. . Preferably, the image presenting device sequentially displays the first area and the second area of different sizes, so that the area enclosed by the boundary of the first area and the boundary of the second area is defined as a border by the setting module. , and then define the area within the border as the display area.

Furthermore, the present invention provides an instruction input method for applying image recognition, which is applied to a data processing device. The data processing device is overlapped with an image capturing device. The method for inputting the image recognition command includes the following steps: (1) Defining at least one command input area in the image data captured by the image capturing device; (2) determining a foreground image in the image data to detect state information of the foreground image appearing in the command input area; And (3) storing a control command corresponding to the status information, and extracting the corresponding control command according to the detected status information, and causing the data processing device to perform a functional action by the control command.

In a preferred embodiment, the data processing device is multiplexed with an image presentation device for presenting a specific region in the image data to define the specific region as a display region in step (1), and the step (1) a step of defining at least one command input area within or outside the range of the display area, and the command input area has a function correspondence with the display area to detect state information in the command input area. Preferably, the step (1) above includes the following steps: (1-1) sequentially displaying the first area and the second area of different sizes by the image presenting device; (1-2) the first area The area enclosed by the boundary of the boundary and the second area is defined as a border; and (1-3) defines an area included in the border or outside the border as a display area.

In another preferred embodiment, the step (3) includes: storing a preset posture of the foreground image and a control instruction corresponding to the preset posture, so that the posture of the detected foreground image conforms to the foreground image. When the posture is preset, the control command corresponding to the preset posture is taken out, and the data processing device performs the function action step through the control command.

Compared with the prior art, the present invention defines a display area through the data processing device of the image capturing device, and then uses the image recognition technology to allow the user to directly input control commands into the display area, thereby greatly reducing the command input control system. The cost of installation and the complexity of installation, and the recognition rate can improve the recognition rate, and solve the problems caused by the instruction input technology of the conventional hardware input unit.

The embodiments of the present invention are described by way of specific examples, and those skilled in the art can readily appreciate other advantages and advantages of the present invention. The present invention may be embodied or applied in various other specific embodiments, and various modifications and changes can be made without departing from the spirit and scope of the invention.

Please refer to FIG. 1 , which is a schematic diagram showing an application architecture of a first embodiment of an instruction input system for applying image recognition according to the present invention. In the present embodiment, the command input system for applying image recognition of the present invention is applied to the data processing device 20. The data processing device 20 can be, for example but not limited to, a data processing device such as a personal computer, a notebook computer, or a smart mobile phone. . In addition, the data processing device 20 is overlapped with the image capturing device 30. In the present embodiment, the image capturing device 30 is built in the data processing device 20. In other embodiments of the present invention, the image capturing device 30 can be externally connected to the data processing device 20.

In the above, the image capturing device 30 is configured to convert the captured analog image signal into digital image data, and the converted digital image data is input and/or stored in the data processing device 20 and transmitted through the data. The processing device 20 uses the conventional image processing application to display the captured image on the display unit 21. However, in other embodiments, the image processing performed by the data processing device 20 may only process the image data at the back end. It is not displayed on the display unit 21. In the embodiment, the data processing device 20 is a notebook computer, and the display unit 21 is a display screen of the notebook computer, and the image capturing device 30 is a camera built into the notebook computer, and the captured image is captured. The image will be presented on the display unit 21 of the data processing device 20.

The command input system for applying image recognition of the present invention comprises a setting module 11, a detecting module 12, a storage module 13, and a control module 14.

The setting module 11 is configured to define at least one command input area in the image data captured by the image capturing device 30.

In this embodiment, the setting module 11 can selectively receive the definition information of the command input area through the data processing device 20, and recognize the command input area according to the definition information of the command input area.

The detecting module 12 determines the foreground image from the image data to detect the state information of the foreground image appearing in the command input area. Preferably, the status information may be the attitude, the change of the foreground, the dynamic trajectory and/or the time of the stay of the foreground image. In a preferred embodiment, the setting module 11 can define the command input area according to the specific state information of the foreground image by using an initialization program.

For the specific implementation, the dynamic trajectory is taken as an example. Please refer to FIGS. 2a-2c for displaying the operation of the detection module of the instruction input system for applying image recognition according to the present invention. As shown in FIG. 2a, when the front view image X is detected as an arm or a palm in the command input area A, when the user swings upwards with the arm or the palm, the detecting module 12 detects the image as shown in FIG. 2b. The dynamic trajectory of the foreground image X shown is moving upward from the command input area A. Similarly, if the arm or the palm is swung downward, the detecting module 12 detects that the dynamic trajectory of the foreground image X as shown in FIG. 2c is moving below the command input area A.

In the above, although the embodiment is for detecting the two-dimensional dynamic trajectory of the image in the command input area A, in other embodiments, the three-dimensional dynamic trajectory of the image in the command input area A can also be detected. For example, if the foreground image X is moved in the Z-axis direction, the detection module 12 can detect the zoom-in and zoom-out of the image as its dynamic track.

The storage module 13 is configured to store control instructions corresponding to the status information.

The control module 14 is configured to retrieve a corresponding control command from the storage module 13 according to the status information detected by the detection module 12, so that the data processing device 20 performs a functional action through the control command.

In a specific implementation, the image track of the foreground image X may be pre-stored by the storage module 13, and the image track may be, for example, but not limited to being associated with the "previous page" and "next page" of the control data processing device 20, respectively. The control command for opening and closing the file and/or the application button can be used to define the command input area A by using the system of the present invention when the user plays the brief file through the data processing device 20, as long as the current scene image X appears in the command input. In the area A, the dynamic trajectory of the X-upward or downward-swinging is detected, and the detected dynamic trajectory is compared with the pre-stored image trajectory of the storage module 13 to be self-contained in the storage module 13. The corresponding control command is taken to cause the data processing device 20 to perform a page change function operation of "previous page" or "next page" through the control command.

In another preferred embodiment, taking the dwell time as an example, when the current scene image X staying time is detected to reach 2 seconds or other preset time, the corresponding control command may be pre-stored in the storage module 13 to enable The data processing device 20 performs a function action of clicking a left or right mouse button. In this embodiment, if the dwell time of the foreground image X is detected to reach 2 seconds or other preset time, the operation mode list is opened, and then it is determined whether the foreground image X stays on the display unit 21 Any operation mode option in the screen list reaches 2 seconds or other preset time. If yes, it can be determined that the user wants to execute the operation mode, such as opening and closing the file and/or application command to control the operation of the data processing device 20. . For example, the operation mode can be divided into a general state according to the use state (edit mode, for example: open file / archive / close / cut / copy / paste / delete / screen virtual keyboard function (can directly edit text) / Switching programs/displaying the desktop/dragging window/changing the window size...etc.) and the briefing status (presentation mode, for example: archive/leave/brush function/presentation process video recording/switching to programs outside the presentation...etc.).

It should be noted that the status information detected by the detection module 12 can also preset the dynamic track and the judgment condition of the dwell time. For example, when the data processing device 20 is executing the input "previous page" or "next page" control command, the detection module 12 can be temporarily stopped from detecting the presence of the foreground in the command input area A. Image X for a period of time (for example: 3 seconds) to avoid misjudgment caused by unnecessary movements, such as the reverse inertia of the down/upward swing after the user moves up/down with the arm or palm. The action causes an action to cancel the page change effect with the previous action state.

In another preferred embodiment, the status information detected by the detecting module 12 can be set as the posture of the foreground image X in addition to the determination condition of the dynamic track and/or the dwell time of the foreground image X. For example, the posture of the palm and the finger or the arm's camber, but not limited to this. Specifically, the preset posture of the foreground image X and the control command corresponding to the preset posture may be stored in the storage module 13 so that the posture of the previous scene image X detected by the detection module 12 conforms to the In the preset posture of the foreground image X, the control module 14 extracts a control command corresponding to the preset posture from the storage module 13 to cause the data processing device to perform a functional action through the control command.

The specific posture of the foreground image X stored by the storage module 13 is a single gesture of a specific foreground image or a continuous combination of at least two different gestures according to the foreground detected by the detection module 12. The image X itself has a single or at least two different consecutive combinations, and the associated data of the control command corresponding to the preset posture is directly retrieved from the storage module 14. For example, as shown in FIG. 3a, the specific image frame M of the foreground image X itself can be preset to a different number of gesture changes, so that the image recognized by the detection module 12 conforms to the preset specific image frame M. At that time, the control module 14 retrieves the relative associated control commands from the storage module 13. In addition, as shown in FIGS. 3b and 3c, the continuous combination of different specific image frames M1 and M2 in which the foreground image X represents the split opening and the clamping action is respectively displayed, and different cycles according to the continuous combination may be displayed. The number of times is preset to be associated with different control commands, that is, when the number of times the user opens and closes the number of cycles that meet the continuous combination of the preset specific image frames, the control module 14 extracts the relative association control from the storage module 13 The instruction, for example, can preset the operation of the function selection by the continuous opening and closing twice, and the operation of the mode switching by the preset three times of continuous opening and closing, but not limited thereto, that is, different presets according to the use requirements The number of times of opening and closing is performed for other functions, and in other specific implementations, for example, according to the number of times of opening and closing of the foreground image X in a preset time (for example, two consecutive times), the function of the cursor is called, and then the detection is performed. The measuring unit 12 captures the dynamic trajectory of the foreground image X and correspondingly drags the 〞 cursor to move to a specific operation mode option in the screen list of the display unit 21, so that when the detected foreground image X is in the specific operation When the corresponding image overlap of the mode option reaches 2 seconds or other preset time, the operation corresponding to the preset control command is executed.

In a further preferred embodiment, the setting module 11 of the present invention pre-executes a registration process of the foreground image to obtain the size, direction or direction of the foreground image before the detecting module 12 performs the detecting. The posture of the detection module 14 is improved by detecting the accuracy of the foreground image. As shown in FIG. 2d, the detection module 12 detects the presence of the foreground image in the command input area A and conforms to the preset type. The X can be pre-set through a registration process, that is, the user can temporarily generate the registration area D in the command input area A and move the foreground image X before the detection module 12 performs the detection of the foreground image dynamic track. The image data is moved to overlap with the registration area D to perform a registration process to lock the image data of the foreground image X of a specific arm or palm type, and then the foreground image X dynamic track is detected, for example, In the implementation, the registration process can determine the size of the foreground image X to be operated, so that the detection module 12 can effectively detect, for example, the above-mentioned squat opening, squatting, or other different gesture changes. As a single or at least two different successive specific combinations of foreground images, simultaneously, whereby the registration process may also enhance the gesture command recognition accuracy and efficiency.

Furthermore, in other preferred embodiments, the detection module 12 can also use the illumination of the at least one auxiliary light source to enable the image capturing device 20 to capture the image data, thereby improving the detection module 12. The image capturing device 20 can be combined with a filter device in the above embodiment. Or the reflective/illuminating object is used to improve the accuracy of the foreground image when the image data is captured. For example, the infrared light source is used as an auxiliary light source, and the image capturing device 20 can be used. An infrared filter (Infrared fliter) is installed to filter visible light, and the image is highlighted by the infrared ray irradiation, and then the visible light is filtered to obtain a simple background image without a background, that is, the filter system can be other spectrums. Monochrome or color filter, to improve the accuracy of capturing the dynamic track of the image, without the need for multiple image processing such as back, edge, binarization, etc., making the subject image easier to identify.

In addition, the foreground image X detected in the above embodiment is not limited to a physical object such as an arm or a palm. In other specific implementations, the detected foreground image X is also a physical object that uses a change in brightness as a control signal. According to the change of the status information, the corresponding control command is extracted from the storage module 13 according to the status information, so that the data processing device performs a functional action through the control command. For example, the foreground image X corresponds to The physical object is a reflective device (for example, a reflective bracelet) and is illuminated with the illumination device to make the subject image easier to identify. In other embodiments, the physical object corresponding to the foreground image X is a light-emitting device or a lighting device (for example, a high-brightness light-emitting diode), and the state information of the brightness change is directly generated, but is not limited thereto.

Please refer to FIG. 4 , which is a flowchart of a first embodiment of an instruction input method for applying image recognition according to the present invention. As shown in the figure, the image input command input method of the present invention is applied to the foregoing application image. An instruction input system (as shown in FIG. 1), wherein the command input system for applying image recognition is applied to the data processing device 20, and the data processing device 20 is overlapped with the image capturing device 30, and the present invention The instruction input method for applying image recognition first performs step S10.

In step S10, at least one command input area A is defined in the image data captured by the image capturing device 30. Next, the process proceeds to step S11.

In step S11, the foreground image is determined in the image data to detect state information of the foreground image appearing in the command input area A. Preferably, the status information may be the attitude, the change of the foreground, the dynamic trajectory and/or the time of the stay of the foreground image X. Next, the process proceeds to step S12.

In step S12, a control command corresponding to the status information is stored to retrieve the corresponding control command according to the detected status information, and the data processing device is caused to perform a functional action by the control command.

Please refer to FIG. 5, which is a schematic diagram showing an application architecture of a second embodiment of an instruction input system for applying image recognition according to the present invention.

As shown in the figure, the basic structure of the first embodiment is the same as that of the first embodiment. The difference is that the data processing device 20 in this embodiment is multiplexed with an image presentation device 40. The image presentation device 40 can be, for example, a projection. machine. In a specific implementation, the image captured by the image capturing device 30 can optionally include a specific region in which the image rendering device 40 presents all or part of the image. For example, if the image presentation device 40 is a projector, the image generated by the image presentation device 40 is presented on a projection screen or a wall or the like, and a specific area for presenting the image is formed on the object.

In this embodiment, the command input system for image recognition of the present invention includes a setting module 11, a detecting module 12, a storage module 13, a control module 14, a vibration compensation module 15, and an interference detecting module 16 And the interference cancellation module 17.

In addition to defining at least one command input area A in the image data captured by the image capturing device 30, the setting module 11 further presents a specific area in the image data, and the setting module 11 displays the specific area. Defined as display area S. Preferably, the setting module 11 is configured to recognize the color, gray scale degree, color gradient homogeneity, front and back image difference, specific object, specific pattern, specific type and/or edge detection manner. Display area. Specifically, in the image data, the setting module 11 can recognize the same color, gray level, color gradient, front and back images, specific objects, specific patterns, regions formed by specific patterns, and can be different colors. The degree of grayscale, color gradient, front and back images, specific objects, specific patterns, and areas formed by specific patterns are distinguished. The area thus formed can be used as the display area S. In addition, the display area S can be defined by image edge detection technology.

Please refer to FIG. 6a to FIG. 6D, which are schematic diagrams showing the operation of the setting module of the second embodiment of the image input command input system of the present invention. The image presenting device 40 sequentially displays the first regions of different sizes. a and the second area b, the area enclosed by the boundary of the first area a and the boundary of the second area b by the setting module 11 is defined as a frame c, and the frame or frame of the frame c is defined The area included is display area B. As shown in FIG. 6a, the display area S is set to be the entire range of the image data captured by the image capturing device 30, and is displayed on the display unit 21, and the detection module 12 recognizes the image capturing device 30. The specific area in the captured image data. Next, as shown in FIG. 6b, the setting module 12 defines a specific area as the first area a. Next, as shown in FIG. 6c, the setting module 12 scales the first area a to define the second area b. It should be noted that in other embodiments, the first region a may be enlarged by equal magnification to define the second region b. Thereafter, as shown in FIG. 6d, the area defined by the boundary between the boundary of the first area a and the second area b is the frame c, and the area within the frame c is the display area B, specifically, the frame c The four apex angles are defined by four positioning points, but not limited thereto, that is, the positioning points corresponding to the number of vertices can be defined for the non-quadrilateral linear frame c, and further, the nonlinearity can be further The curved surface of the frame c outside the apex angle corresponds to an increase of the positioning point to solve the projection error caused by the nonlinear frame c, that is, non-linear distortion warping caused by projection, and the same The method can obtain more positioning points to define a more accurate display area B. Preferably, the setting module 12 is multiplexed to frame the frame c in a specific color such as blue or red, and displayed on the data processing device. 20 on.

In other embodiments of the invention, it can be applied in an environment having a plurality of data processing devices. The data processing device 20 for displaying the specific color frame of the frame c and the data processing device 20 with the image capturing device 30 may be different data processing devices, and in other specific implementations and the foregoing The first embodiment is similar and therefore will not be described here.

In another preferred embodiment, the detection area of the detection module 12 for detecting the status information of the foreground image X is not limited to the display area B. As shown in FIG. 6e, the setting module is At least one command input area E is defined in the range of the display area B or outside the range, and the command input area E has a function correspondence relationship with the display area B, so that the detection module 12 detects the command input. Status information in area E. In detail, the command input area E can be preset to the corresponding relationship between the gesture change of the different numbers of the foreground image X and the command input area E as shown in FIG. 3a, so as to be limited by the detection module 12 in the instruction. When the image recognized by the input area E meets the preset specific image frame M, the control module 14 retrieves the relative associated control command from the storage module 13 to cause the data processing device to perform a functional action through the control command, that is, Limiting the foregoing foreground image X can only detect the state information of the foreground image X in the command input area E, for example, evoking the cursor function in the command input area E with a specific gesture, so that when the cursor function is called The detection of the dynamic trajectory of the foreground image X is performed.

In still another preferred embodiment, the present invention can be used by the control module 14 to preset the execution order of the control commands corresponding to the display area B or the command input area E, and to function in different operating environments such as the foregoing general mode or briefing mode. Requirements to achieve the effect of layered control. For example, the specific foreground image X of a single or multiple consecutive combinations/cycles can be defined as the instruction a, the special function menu, the instruction b, and the preset instruction b is executed as the instruction. After the execution of a, it is determined whether the relative associated instruction b is retrieved from the storage module 13, that is, when the cursor function is called, the special function menu function can be called again, in particular, special The function menu can be, for example, a functional action of a touch operation that can be performed by a virtual button, a virtual keyboard, or other physical input device (eg, a mouse, a keyboard).

For details, please refer to FIGS. 7a-7c, which are schematic diagrams showing the operation of the detection module of the second embodiment of the command input system for applying image recognition according to the present invention. As shown in Fig. 3a, the foreground image X (in the case of an arm or a palm) is present in the display area B. Then, when the arm or the palm is swung upward, the foreground image X moves to the upper side of the display area B as shown in FIG. 3b, and the detecting module 12 detects that the dynamic track moves toward the upper side of the display area B. Similarly, when the arm or palm is swung downward, the foreground image X will move below the display area B as shown in FIG. 3c, and the detection module 12 will detect that its dynamic trajectory is toward the display area B. Move below.

It should be noted that, in this embodiment, after the setting module 12 frames the frame c in a specific color, the detecting module 13 can detect the state information of the image appearing in the display area B, and in other The specific embodiment is similar to the foregoing first embodiment, and therefore will not be described herein.

As described above, when the user swings the arm or the palm in the display area B and causes the foreground image X to generate an upward or downward dynamic trajectory as shown in the above-mentioned 7b or 7c, the control module 14 can be based on The up or down dynamic track of the foreground image X detected in the display area B is extracted from the storage module 13 and is relatively associated with the command, that is, "previous page" or "next page", on and off. The file and/or application button commands cause the data processing device 20 to operate in accordance with the "previous page" or "next page", opening and closing of the file and/or application control commands.

In addition, in the technical aspect of the background image cancellation, the difference between the embodiment and the first embodiment is that the detection module 12 can eliminate the variation of the background image by using the auxiliary light source, and can also utilize the interference detection. The detecting module 16 determines the interference area in the display area B, so that the detecting module 12 stops detecting the foreground image X in the interference area, for example, in the display area B as shown in FIG. 6e. At least one command input area E is set as an interference area to prevent the detection module 12 from erroneously operating the data processing unit 20 due to the detection of an unexpected foreground image.

In the above preferred embodiment, the interference cancellation module 17 is configured to compare the predicted variation content in the display area B with the actual variation content in the display area B, so as to enable The detection module 12 determines the foreground image X according to the result of the comparison, and improves the accuracy of the determination by the detection module 12. For example, the interference cancellation module 17 can obtain the image to be presented (ie, the background image) by the data processing device 20 to dynamically perform background elimination on the image presented by the image presentation device 40, for example, the data processing device 20 The specific image segment to be background-removed is set in the projected content, so that when the image presentation device 40 presents the specific image segment, the background segment is dynamically removed for the specific image segment to avoid the image rendering device 40. The projected background image causes unintended actions by the data processing device 20.

Furthermore, referring to the command input system for applying image recognition shown in FIG. 8, the vibration compensating module 15 in the data processing device 20 is used to avoid the vibration of the data processing device 20 or the image presenting device 40 due to external force. The image capturing device 30 captures an image difference or an excessive amount of image difference in the image detecting area. As shown in FIG. 8, the vibration compensating module 15 sets five positioning points C1, C2, C3, C4, and C5 on the display area B in the frame c defined in the above-mentioned 6th figure, wherein the positioning point C1 -C4 is disposed at four corners of the display area B, and the positioning point C5 is disposed between the positioning point C1 and the positioning point C3 of the upper edge of the display area B. Then, the storage module 13 records the original coordinates of each of the positioning points. The coordinate display of the positioning points is performed on each screen displayed by the data processing device 20. In this embodiment, the user blocks the two points at the same time (for example, the positioning points C1 and C2 are blocked by the left side). Therefore, it is only necessary to take the three positioning points with the smallest moving distance to calculate the moving vector. When the average offset of the three positioning points is too large, it is judged to be a vibration. At this time, the vibration compensation module 15 performs a screen correction opposite to the vibration direction.

Please refer to FIG. 9, which is a flowchart for showing a second embodiment of the instruction input method for applying image recognition according to the present invention.

In step S20, a specific area in the image data captured by the image capturing device is identified. In this step, when the image data is input and/or stored in the data processing device, a specific region in the image data captured by the image capturing device is recognized to define the specific region as the display region. Next, the process proceeds to step S21.

In step S21, the first region and the second region of different sizes are sequentially presented by the image rendering device. In this step, the setting module defines a specific area as the first area, and then the setting module scales down the first area to define the second area. It should be noted that in other embodiments, the first region may be enlarged by equal magnification to define the second region. Then it proceeds to step S22.

In step S22, the area enclosed by the boundary of the first area and the boundary of the second area is defined as a frame, and an area included in the frame or outside the frame is defined as a display area. Next, the process proceeds to step S23.

In step S23, status information of the foreground image in the display area is detected. In this step, the detection module determines the foreground image from the image data to detect the state information of the foreground image appearing in the display area. Preferably, the status information may be the attitude, the change of the foreground, the dynamic trajectory and/or the time of the foreground image, and in another embodiment, after the setting module frames the border with a specific color, the Detector The measurement module can detect the status information of the image appearing in the display area, and the other specific implementation changes are similar to the foregoing first embodiment, and thus are not described herein. Next, the process proceeds to step S24.

In step S24, a control command corresponding to the status information is stored, and the corresponding control command is retrieved according to the detected status information, and the data processing device is caused to perform a functional action by the control command.

In summary, the command input system and method for applying image recognition of the present invention have the following advantages:

(1) Low construction cost and convenient installation. The invention can use the image capturing device and the data processing device to perform image recognition, and does not need to purchase other equipment, thereby reducing the cost of constructing the command input system and having the characteristics of simple installation.

(2) The recognition rate is high. The invention utilizes an auxiliary recognition technology such as background elimination or vibration correction to perform foreground image recognition, and can greatly improve the recognition rate, thereby solving the problems caused by the instruction input technology of the conventional hardware input unit.

The above embodiments are merely illustrative of the principles of the invention and its advantages, and are not intended to limit the invention. Modifications and variations of the above-described embodiments can be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention should be as set forth in the scope of the claims described below.

11. . . Setting module

12. . . Detection module

13. . . Storage module

14. . . Control module

15. . . Vibration compensation module

16. . . Interference detection module

17. . . Interference cancellation module

20. . . Data processing device

twenty one. . . Display unit

30. . . Image capture device

40‧‧‧Image presentation device

S, B‧‧‧ display area

D‧‧‧Registration area

A, E‧‧‧ command input area

A‧‧‧first area

b‧‧‧Second area

c‧‧‧Border

X‧‧‧ foreground image

M, M1, M2‧‧‧ specific image screen

C1, C2, C3, C4, C5‧‧‧ anchor points

S10~S12‧‧‧Steps

S20~S24‧‧‧Steps

1 is a schematic diagram of an application architecture of a first embodiment of an instruction input system for applying image recognition according to the present invention;

2a to 2d are diagrams showing the operation of the detection module of the first embodiment of the image input command input system of the present invention;

3A to 3c are diagrams showing an operation of the first embodiment of the image input command input system of the present invention for detecting a preset attitude image of a foreground image;

4 is a flow chart of a first embodiment of an instruction input method for applying image recognition according to the present invention;

5 is a schematic diagram of an application architecture of a second embodiment of an instruction input system for applying image recognition according to the present invention;

6a to 6e are diagrams showing the operation of the setting module of the second embodiment of the command input system for applying image recognition according to the present invention;

7A to 7c are diagrams showing the operation of the detecting module of the second embodiment of the command input system for applying image recognition according to the present invention;

Figure 8 is a schematic diagram showing the setting of the screen compensation detection in the second embodiment of the command input system for applying image recognition according to the present invention;

Figure 9 is a flow chart showing a second embodiment of the method of inputting an image for applying image recognition according to the present invention.

11. . . Setting module

12. . . Detection module

13. . . Storage module

14. . . Control module

20. . . Data processing device

twenty one. . . Display unit

30. . . Image capture device

Claims (38)

An instruction input system for applying image recognition is applied to a data processing device, wherein the data processing device is overlapped with an image capturing device, and the command input system for applying the image recognition comprises: a setting module for using the image Determining at least one command input area and a display area in the image data captured by the device; the detecting module determines the foreground image from the image data to detect the state of the foreground image appearing in the command input area The information storage module is configured to store control commands corresponding to the status information; the control module is configured to extract corresponding control from the storage module according to the status information detected by the detection module. And the vibration compensation module is configured to display a plurality of display positioning points in the display area to record the original coordinates of the plurality of display positioning points, wherein the data processing device performs a function operation; And comparing with the coordinates of the image positioning points of the plurality of display positioning points in the image data captured by the image capturing device, when the average When it is determined that excessive shift shock, the shake compensation module to a plurality of image coordinates of the anchor points is corrected. The instruction input system for applying image recognition as described in claim 1, wherein the status information is a posture, a change in brightness, a dynamic trajectory, and/or a dwell time of the foreground image. The instruction input system for applying image recognition according to claim 2, wherein the dynamic track comprises a two-dimensional or three-dimensional dynamic track of the foreground image in the command input area. The instruction input system for applying image recognition according to claim 1, wherein the setting module defines an instruction input area according to specific state information of the foreground image by using an initialization program. An instruction input system for applying image recognition as described in claim 2, wherein the storage module stores a preset posture of the foreground image and a control command corresponding to the preset posture for the detection module When the posture of the foreground image is detected to match the preset posture of the foreground image, the control module extracts a control command corresponding to the preset posture from the storage module, so that the data processing device is enabled by the control command Perform functional actions. The instruction input system for applying image recognition according to claim 5, wherein the setting module pre-executes a registration process of the foreground image to obtain the foreground image before the detecting module performs the detecting The size, direction or posture of the detection module improves the accuracy of detecting the foreground image. The command input system for applying image recognition according to claim 5, wherein the preset image of the foreground image is a single gesture of a specific foreground image or a continuous combination of at least two different gestures. The instruction input system for applying image recognition according to the first aspect of the patent application, wherein the detecting module is improved when the image capturing device captures the image data by using at least one auxiliary light source Break the accuracy of the foreground image. The image input system for applying image recognition according to claim 8 , wherein the image capturing device is combined with a filtering device or a reflective/light emitting object to improve the detecting mode when capturing the image data. The group judges the accuracy of the foreground image. The image input system for applying image recognition according to claim 1, wherein the data processing device is multiplexed with an image presenting device for presenting a specific area in the image data, and the setting module is This specific area is defined as the display area. The command input system for applying image recognition according to claim 10, wherein the setting module defines at least one command input area within or outside the range of the display area, and the command input area and the display The area has a function correspondence relationship, so that the detection module detects status information in the instruction input area. The instruction input system for applying image recognition according to claim 10, wherein the setting module is configured to be based on color, gray scale degree, homogeneity of color gradient, front and back image difference, specific object, specific The display area is defined by the pattern, the specific pattern, and/or the edge detection method. The image input device of claim 10, wherein the image presenting device sequentially displays the first region and the second region of different sizes, so that the first region is The area enclosed by the boundary of the boundary and the second area is defined as a border, and the area included in the border or outside the border is defined as the display. region. The instruction input system for applying image recognition according to claim 13 , wherein the setting module is multiplexed to define a corresponding top angle of the linear frame as an anchor point. The instruction input system for applying image recognition according to claim 13 , wherein the setting module is multiplexed to define a corresponding apex angle of the non-linear frame as an locating point and a surface outside the apex angle Corresponding to increase the anchor point. The instruction input system for applying image recognition according to claim 13 , wherein the setting module is multiplexed to frame the frame in a specific color and displayed on the data processing device. For example, the command input system for applying image recognition according to claim 1 of the patent application includes an interference detecting module for determining an interference region in the display area, so that the detecting module is in the interference region. Stop detecting the foreground image. For example, the instruction input system for applying image recognition according to claim 1 of the patent application includes an interference cancellation module, which compares the predicted change content in the display area with the actual change content in the display area, and is used to compare The detection module determines the foreground image according to the result of the comparison, and improves the accuracy of the detection module. The instruction input system for applying image recognition according to claim 1, wherein the physical object corresponding to the foreground image is a reflective device. For example, the application of image recognition command input as described in claim 1 Into the system, wherein the physical object corresponding to the foreground image is a light emitting device or a lighting device. An instruction input method for applying image recognition is applied to a data processing device, wherein the data processing device is overlapped with an image capturing device, and the command input method for applying the image recognition comprises the following steps: (1) the image capturing device Defining at least one command input area and a display area in the captured image data; (2) determining a foreground image in the image data to detect state information of the foreground image appearing in the command input area; and (3) And storing a control command corresponding to the status information, according to the detected status information, extracting the corresponding control command, and causing the data processing device to perform a functional action through the control command, and having a plurality of Displaying an anchor point to record the original coordinate of the plurality of display anchor points, and comparing the coordinates of the image data points captured by the image capturing device corresponding to the plurality of image positioning points of the display positioning point, when the average offset When the amount is too large, it is judged as vibration, and the coordinates of the plurality of image positioning points are corrected by the vibration compensation module. The method for inputting an image recognition method according to claim 21, wherein the status information is a posture, a change in brightness, a dynamic trajectory, and/or a dwell time of the foreground image. The method for inputting an image recognition method according to claim 22, wherein the dynamic track comprises a two-dimensional or three-dimensional dynamic track of the foreground image in the command input area. The instruction input method for applying image recognition according to claim 21, wherein the step (1) defines an instruction input area according to specific state information of the foreground image by using an initialization program. The method for inputting an image recognition method according to claim 22, wherein the step (3) comprises: storing a preset posture of the foreground image and a control instruction corresponding to the preset posture, so as to detect When the posture of the foreground image conforms to the preset posture of the foreground image, the control command corresponding to the preset posture is taken out, and the data processing device performs the function operation step through the control command. For example, the instruction input method for applying image recognition according to claim 25 is included before the execution step (2), and a registration process of the foreground image is performed in advance to obtain the size, direction or posture of the foreground image. A step of. The instruction input method for applying image recognition according to claim 26, wherein the preset posture of the foreground image is a single gesture of a specific foreground image or a continuous combination of at least two different gestures. The method for inputting an image recognition method according to claim 21, wherein the step (1) comprises: using the illumination of the at least one auxiliary light source to cause the image capturing device to capture the image data, thereby improving the step ( 2) Determine the accuracy of the foreground image. The method for inputting an image recognition method according to claim 21, wherein the step (1) comprises: causing the image capturing device to be combined with a filtering device or a reflective/light emitting object to capture the image data. When the detection module is determined to determine the accuracy of the foreground image. The method for inputting an image recognition method according to claim 21, wherein the data processing device is multiplexed with an image presentation device for presenting a specific region in the image data for the step (1) This specific area is defined as the display area. The instruction input method for applying image recognition according to claim 30, wherein the step (1) is included in the range of the display area or outside the range to define at least one instruction input area, and the instruction input area A step of functioning with the display area to detect status information in the command input area. The method for inputting an image recognition application according to claim 30, wherein the step (1) includes the color, the degree of gray scale, the homogeneity of the color gradient, the difference between the front and back images, the specific object, and the specific The pattern, the specific pattern, and/or the edge detection method define the steps of the display area. The instruction input method for applying image recognition according to claim 30, wherein the step (1) further comprises the following steps: (1-1) sequentially displaying the first region of different sizes by the image presentation device; a second region; (1-2) defining a region surrounded by a boundary between the boundary of the first region and the second region as a frame; and (1-3) defining an area included in the frame or outside the frame as The display area. An instruction input method for applying image recognition according to claim 33, wherein the step (1-2) includes the frame that is linear The corresponding apex angle is defined as the step of locating the point. An instruction input method for applying image recognition according to claim 33, wherein the step (1-2) comprises: defining a corresponding top angle of the non-linear frame as an anchor point, and outside the top corner The surface corresponds to the step of adding an anchor point. The method for inputting an image for applying image recognition according to claim 33, wherein the step (1-3) comprises the step of framing the frame in a specific color and displaying the image on the data processing device. The method for inputting an image for applying image recognition according to claim 21, further comprising the step of determining an interference region in the display region to stop detecting the foreground image in the interference region. The method for inputting an image for applying image recognition according to claim 21, further comprising the step of comparing the predicted change content in the display area with the actual change content in the display area, according to the comparison. As a result, the foreground image is judged, and the accuracy of the detection module is improved.