JPH0981307A - Equipment controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate the burden of a user in an operation input mode by recognizing the key display that is estimated by the movement of a mobile object in a mobile object area and reading the equipment control data related to the command corresponding to the key display out of a storage means to control an equipment. SOLUTION: An on board equipment controller 10 controls an on board equipment 30 and has a mobile object extraction part 12 which extracts the screen area (mobile object area) of a mobile object out of the image pickup signals of a camera 11 which photographs the mobile object. An action recognition part 13 decides whether the extracted mobile object area is identical with a human hand screen area and also recognizes the actions of fingers in images. A memory part 14 stores a menu screen including toe key display corresponding to the command related to the control of the equipment 30. A screen synthesis part 15 puts the mobile object area that is extracted at the part 12 on the menu screen stored in the part 14. Then a command input extraction part 16 extracts the key display that is designated by the finger actions out of a screen that is synthesized at the part 15.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device control apparatus having a command input function for inputting commands relating to control of, for example, in-vehicle devices.

[0002]

2. Description of the Related Art Conventionally, as such a device control device, there is a device for controlling a device such as an in-vehicle device such as a room air conditioner or a car audio system. The in-vehicle device is controlled by directly touching and operating the operation key corresponding to each command.

However, according to such a device control device, the function of the on-vehicle device has become complicated and the number of commands relating to the function has increased in recent years. However, it has been difficult to secure a key space corresponding to these operation keys.

In order to overcome such a situation, in recent years, a device control device having a touch panel function has been commercialized.

The device control device having the touch panel function displays pseudo keys corresponding to each command on the display screen of the display unit, and operates by directly touching the pseudo keys on the display screen to operate the pseudo keys. The in-vehicle device is controlled based on a command corresponding to the pseudo key.

According to the device control device having the touch panel function, even if the function of the in-vehicle device is complicated and the number of commands related to the function is increased, the number of pseudo keys on the display screen is increased. That is, it is only necessary to increase the number of keys on the program, and it is not necessary to secure the key space as described above.

[0007]

However, according to the above-mentioned conventional equipment control device, the user directly touches and operates the pseudo key corresponding to each command displayed on the display screen. If the user's posture collapses during operation and the line of sight deviates, and if the device control device is applied to an in-vehicle device, the posture collapse and line-of-sight deviation during operation may interfere with driving. Therefore, there is a problem that the user is burdened during the operation.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a device control device which can surely eliminate the burden on the user at the time of operation input. .

[0009]

In order to achieve the above object, a device control apparatus according to a first aspect of the present invention is a first storage for storing a menu screen having a key display corresponding to a command related to device control. Means, a second storage means for storing device control data relating to a command corresponding to the key display, a display means for displaying the menu screen stored in the first storage means, an image of the moving body, and the movement. An image pickup means for obtaining an image pickup signal relating to the body, a moving body extraction means for extracting a moving body area from the image pickup signal, a combination of the extracted moving body area and the menu screen stored in the first storage means are combined. Screen synthesizing means for generating a synthesized screen by means of the moving body, moving body movement recognizing means for recognizing the movement of the moving body in the moving body area and the key display designated by the movement, and the recognized key table. Reading from said second memory means a device control data relating to a command corresponding to, and having a controller for controlling the device based on the read-out device control data.

That is, the device control apparatus according to claim 1 is a device control apparatus having a command input function for inputting a command relating to the control of the vehicle-mounted device, for example, a moving body that executes the input operation, for example, A camera or the like as an image pickup means for picking up an image of a person's hand or the like is provided, and the screen area of the hand, which is a moving body area, is extracted by the moving body extraction means from the image pickup signal picked up by the camera The screen area of the hand is superimposed on the menu screen stored in the first storage means by the screen combining means, and the screen area of the hand on the menu screen, mainly the screen area of the fingertip of the hand, is image-recognized. As a result, the movement of the fingertip and the key display designated by the movement are recognized by the moving body movement recognition means, and the device control data relating to the command corresponding to the recognized key display is stored in the second storage means. Read, and the control of the vehicle equipment based on the read-out device control data as performed by the control unit.

The key display designated by the operation of the fingertip is a key corresponding to the position of the screen area of the fingertip on the menu screen when the screen area of the fingertip is overlaid on the menu screen. It is a display.

Therefore, according to the device control apparatus of the first aspect, the operation of the fingertip, that is, the gesture is recognized as an image without directly touching the pseudo key or the operation key of the touch panel function as in the conventional case, and the image is detected. The command related to the device control is input by the recognized fingertip gesture, that is, the desired command is input only by the fingertip gesture without causing the posture collapse and the deviation of the line of sight during the operation. Therefore, the burden on the user during the input operation can be reliably eliminated.

The device control apparatus according to the second aspect is characterized in that the control means causes the display means to display and output the composite screen generated by the screen composite means.

Therefore, according to the device control apparatus of the second aspect, the fingertip and the menu screen recognized by the moving body motion recognition means are displayed in accordance with the display means, so that the fingertip of the user is displayed. It is possible to recognize which position is on the screen of the display means.

Further, in the equipment control device according to claim 3, the control means according to claim 1 or 2 is on the menu screen which is displaying the key display recognized by the moving body motion recognition means on the display means. It is characterized in that it is identified and displayed in.

Therefore, according to the device control apparatus of the third aspect, the key display recognized by the moving body motion recognition means, that is, the key display designated by the motion of the fingertip is different from the other key display. Differently, for example, the marks and different colors are used for identification display, so that the input operation relating to the command designation can be effectively recognized.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the device control apparatus of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of an in-vehicle device control device showing an embodiment of the device control device of the present invention.

In FIG. 1, the on-vehicle equipment control device 10 is
A camera 11 that controls the vehicle-mounted device 30 and is an image pickup unit that obtains an image pickup signal by picking up an image of the moving body, and a screen area of the moving body from the image pickup signal of the camera 11 (hereinafter, simply referred to as a moving body area). ) Is detected by the moving body extracting unit 12 as a moving body extracting unit, and whether or not the moving body area extracted by the moving body extracting unit 12 is a screen area of a human hand. An operation recognition unit 13 for recognizing an operation of a fingertip in the above, a memory unit 14 as a first storage unit for storing a menu screen having a key display corresponding to a command related to the control of the vehicle-mounted device 30, and the memory unit. 1
On the menu screen stored in No. 4, the moving body extraction unit 1
2. A screen synthesizing unit 15 that is a screen synthesizing unit that superimposes the moving body regions extracted by 2 and a command input that extracts the key display designated by the operation of the finger from the synthetic screen synthesized by the screen synthesizing unit 15. An extraction unit 16, a display unit 17 that is a display unit that displays and outputs the combined screen combined by the screen combining unit 15, an alarm sound generation unit 18 that generates an alarm sound, and a speaker that outputs the alarm sound by voice. 19 and a control unit 20 which is a control means for controlling the device control apparatus 10 as a whole. The moving body motion recognition means according to claim 1 includes the motion recognition unit 13 and the command input extraction unit 1.
This corresponds to 6.

Inside the control unit 20, device control data relating to commands corresponding to each key display on the menu screen is stored.

The display unit 17 is, for example, a head-up display in which the movement of the line of sight is small even during driving.

Next, the operation of the in-vehicle device control apparatus 10 will be described. FIG. 2 is a flowchart showing the processing operation of the control unit 20 in the command input processing of the device control apparatus 10.

The command input processing is to capture a gesture of a person's hand, mainly a fingertip, with the camera 11, image-recognize the captured gesture of the fingertip, and synthesize a screen specified by the gesture of the fingertip. This is a process of executing control of the in-vehicle device 30 related to a command corresponding to the above key display. That is, the command input process is to input a command only by the gesture of the fingertip without directly touching the operation key.

The control unit 20 shown in FIG.
The image pickup signal obtained from step S11 is input (step S11), and the moving body extraction processing shown in FIG. The moving body extraction process is a process in which the moving body extraction unit 12 extracts a moving body region from an image pickup signal from the camera 11.

After the moving object extraction processing in step S12 is completed,
It is determined whether or not the moving body detection flag (hereinafter, simply referred to as the first flag) indicating that the moving body region is extracted in the image pickup signal is "1" (step S13).

When the first flag is "1", the fingertip detection flag (hereinafter, simply referred to as the first area) indicating that the extracted moving body area is the screen area of the hand and the screen area of the fingertip is detected. It is determined whether or not "2 flag") is "1" (step S14). Incidentally, the moving body extraction unit 1
2 extracts only the moving body region 50a from the image pickup signal as shown in FIG. 7A, and extracts the moving body region 50a.
Is supplied to the screen synthesis unit 15 and the motion recognition unit 13, respectively. Further, the screen compositing unit 15 stores the moving body region 50a in the memory unit 14 shown in FIG.
The image content shown in FIG. 7C is displayed and output on the display unit 17 as a composite screen by superimposing it on the menu screen 50b shown in FIG. On the screen being displayed on the display unit 17, if the moving body moves sequentially from the bottom to the top, the images to be superimposed are sequentially superimposed from the bottom to the movement direction.

If the second flag is "1" in step S14, the finger closing action detection flag (hereinafter simply referred to as the third flag) indicating that the finger closing action of the fingertip is detected is ". It is determined whether or not it is "1" (step S15).

If the third flag is "1", whether or not the finger opening action detection flag (hereinafter, simply referred to as a fourth flag) indicating that the finger opening action of the fingertip is detected is "1". It is determined (step S16).

If the fourth flag is "1", the key input corresponding to the position of the fingertip overlaid on the composite screen composited by the screen compositing unit 15, that is, the menu screen is input as the command. The device control data extracted by the extraction unit 16 and related to the command corresponding to the extracted key display is read, and each in-vehicle device 30 is controlled based on the read device control data (step S17). Then, it is determined whether or not the monitoring operation related to the command input is completed (step S18). Note that the display unit 17 also displays the key display and the fingertip (hand) as a combined screen.

If the monitoring operation is finished in step S18, the command input process is finished.

If the first flag is not "1" in step S13, the process proceeds to step S18. Also,
If the second flag is not "1" in step S14, the finger detection process shown in FIG. 4 described later is executed (step S1).
9), after performing the process, the process proceeds to step S18.
In the finger detection process, the moving body region extracted by the moving body extraction process by the motion recognition unit 13 is the screen region of the human hand, and the screen region of the fingertip of the hand is detected. Processing.

If the third flag is not "1" in step S15, a finger closing action detection process shown in FIG. 5 described later is executed (step S20), and after the execution of the process, the process proceeds to step S18. The finger close action detection process is a process of detecting an action of closing the fingertip in the motion recognition unit 13, that is, an action of prompting the start of key display designation by the fingertip on the menu screen.

If the fourth flag is not "1" in step S16, a finger opening action detection process shown in FIG. 6 described later is executed (step S21), and after the execution of the process, the process proceeds to step S18. The finger opening action detection process is a process of detecting an action of opening the fingertip by the motion recognition unit 13, that is, an action of designating a key display by the fingertip on the menu screen.

If the monitoring operation by the command input processing is not ended in step S18, step S18 is executed.
Go to 11.

In the command input process, the user first executes the action of opening the fingertip in order to recognize that the moving body imaged by the camera 11 is a human hand, and then The action is to close the fingertip to prompt the start of the key display designation, and then execute the action to open the fingertip to perform the key display designation.

Next, the moving body extraction processing in step S12 will be described. FIG. 3 is a flowchart showing the processing operation of the moving body extraction unit 12 in the moving body extraction processing.

In the moving body extraction unit 12 shown in FIG.
Inter-frame difference processing is performed to obtain a difference for each pixel from two pieces of frame data from image pickup signals (screens) sequentially obtained from the camera 11 (step S31), and the difference data obtained by the inter-frame difference processing is set to 2 A moving body determination process is performed to determine whether or not there is a moving body region in the image pickup signal by binarization (step S32), and whether or not the moving body region is in the image pickup signal based on the processing result. Is determined (step S33). As a method of discriminating the screen area of the moving body in step S33, whether or not the number of pixels of the binarized difference data is equal to or larger than a predetermined threshold is used as a discrimination reference, and the number of pixels is a predetermined threshold. If the value is greater than or equal to the value, it is determined that there is a moving body area in the image pickup signal.

If there is a moving body area in the image pickup signal in step S33, the first flag indicating that the moving body area is detected in the image pickup signal is set to "1" (step S3).
4), based on the difference data, a moving body extraction process for extracting a moving body region from the image pickup signal is executed (step S35). In step S35, the moving object region is extracted from the image pickup signal by executing the edge extraction process, but the edge extraction is performed by the differential operation.

After the processing of step S35, the moving body update processing for updating the moving body area previously extracted to the moving body area extracted this time is executed (step S36), and the moving body area is monitored on the screen. A moving body screen dropout determination process is performed to determine whether the moving body area is out of the area, that is, whether or not the moving body region is out of the image pickup signal (step S
37), based on the processing result, it is determined whether or not the screen of the moving body region has fallen out of the image pickup signal (step S38).

If the moving object region is out of the screen, the first, second, third and fourth flags are reset (step S39), the moving object extracting process is terminated, and the process is shown in FIG. Control goes to step S13.

If there is no moving object area in the image pickup signal in step S33, it is determined whether the first flag is "1" (step S40). If the first flag is "1", the process proceeds to step S35.

If the screen omission has not occurred in step S38, or if the first flag is not "1" in step S40, the moving object extraction processing is terminated and step S13 shown in FIG. Move to.

Next, the finger detection process of step S19 shown in FIG. 2 will be described. FIG. 4 is a flowchart showing the processing operation of the operation recognition unit 13 in the finger detection processing.

The motion recognition section 13 shown in FIG. 4 detects the tip of the moving body area extracted by the moving body extracting section 12 (step S51), and the moving body area is detected as shown in FIG.
It is determined whether or not it is within the designated frame area 17a shown in (a) (step S52). If the moving body area is within the designated frame area 17a, thinning processing is performed to determine whether the moving body area is the screen area of a human hand (step S53).

Note that the thinning processing is, as shown in FIG. 8, performed by performing a certain amount of reduction processing in the horizontal direction on the moving object region with respect to the image pickup screen of the image pickup signal, and performing the reduction processing. Whether or not the leading end portion of the moving body region that has been thinned is used as a determination criterion, and if the leading end portion is thinned,
This is a process of calculating the length and width of the moving object area in the screen area of the fingertip by considering the moving object area as the screen area of the hand and the tip portion as the screen area of the fingertip.

After the processing of step S53, it is determined whether or not the calculated length of the screen area of the fingertip is equal to or more than a predetermined value set in advance (step S54). If the length of the screen area of the fingertip is greater than or equal to a predetermined value, it is determined whether the width of the screen area of the fingertip is less than or equal to a predetermined value set in advance (step S55). If the width of the screen area of the fingertip is less than or equal to a predetermined value, it is determined that the tip portion is the screen area of the fingertip, and the screen area of the determined fingertip is within the designated frame area 17a. The second flag shown is set to "1" (step S56), the detection processing of the finger is ended, and the process proceeds to step S18 shown in FIG.

The control unit 20 controls the motion recognition unit 1
If it is determined in step S56 that the screen area of the fingertip is within the designated frame area 17a via step 3, an alarm sound is generated by the alarm sound generation unit 18, and the generated alarm sound is transmitted via the speaker 19. To output audio. That is, by listening to the notification sound, the user can recognize that the fingertip is within the designated frame area 17a without visually observing the screen.

If the moving object area is not within the designated frame area 17a in step S52, or in step S54.
If the length of the screen area at the tip of the moving body area is not greater than or equal to a predetermined value, or if the width of the screen area at the tip of the moving body area is not less than or equal to the predetermined value in step S55, the designation is performed. It is determined that there is no screen area of the fingertip in the frame area 17a, the detection processing of the finger is ended, and the process proceeds to step S18 shown in FIG.

Next, the finger close action detection processing of step S20 shown in FIG. 2 will be described. FIG. 5 is a flowchart showing the processing operation of the operation recognition unit 13 in the finger close action detection processing.

In the motion recognition unit 13 shown in FIG. 5, the moving body area, that is, the screen area of the hand is the designated frame area 17.
It is detected that it is within a (step S61), and it is determined whether or not there is a screen area of the fingertip in the designated frame area 17a (step S62). If there is no screen area of the fingertip in the designated frame area 17a, the third flag indicating that the action of closing the fingertip, that is, the action of prompting the start of key display designation by the fingertip on the menu screen is detected is "1". Then (step S63), the finger close action detection process is terminated, and the process proceeds to step S18 shown in FIG.

When the control section 20 detects through the motion recognition section 13 that the screen area of the hand is present in the designated frame area 17a but the screen area of the fingertip is not present in step S62, the control section 20 determines A notification sound is generated in the notification sound generation unit 18, and the generated notification sound is output as voice through the speaker 19. By listening to the notification sound, the user can recognize an action that prompts the start of key display designation without visually observing the screen.

If there is no moving object area screen area in the designated frame area 17a in step S62, the finger close action detection processing is terminated and step S shown in FIG.
Go to 18.

Next, the finger opening action detection processing of step S21 shown in FIG. 2 will be described. FIG. 6 is a flowchart showing the processing operation of the operation recognition unit 13 in the finger opening action detection processing.

In the motion recognition unit 13 shown in FIG. 6, the moving body area, that is, the screen area of the hand is the designated frame area 17.
It is detected that it is within a (step S71), and it is determined whether or not there is a screen area of the fingertip in the designated frame area 17a (step S72). If there is a screen area of the fingertip in the designated frame area 17a, the fourth flag indicating that the action of opening the fingertip, that is, the action of designating the key display by the fingertip on the menu screen is detected is set to "1" ( (Step S73), the finger opening action is ended, and the process proceeds to step S18 shown in FIG.

If there is no screen area of the fingertip in the designated frame area 17a in step S72, the finger opening action detection process is terminated and the process proceeds to step S18 shown in FIG.

When the control section 20 detects in the step S72 that there is a screen area of the fingertip in the designated frame area 17a through the motion recognition section 13, the notification sound generation section 18 is notified. Is generated and the notification sound is output as voice through the speaker 19. By listening to the notification sound, the user can recognize that the key display has been designated without observing the screen.

That is, the in-vehicle device control apparatus 10 according to the above-described embodiment extracts the moving body region from the image pickup signal picked up by the camera 11 by the moving body extraction unit 12 and displays the extracted moving body region in the menu. Overlay on the screen,
When it is determined whether or not the moving body region is the screen region of the hand and the fingertip, and when it is determined that it is the screen region of the fingertip, the fingertip and the action of the fingertip on the display screen displayed on the display unit 17 Image recognition is performed, and the device control data corresponding to the command related to the key display corresponding to the position of the screen area of the fingertip on the display screen being displayed on the display unit 17 is read by the operation of the fingertip recognized by the image, The in-vehicle device 30 based on the read device control data
Was controlled.

Therefore, according to the vehicle-mounted device control apparatus 10 according to the above-described embodiment, the gesture of the hand or the fingertip is recognized by the image without directly touching the pseudo key or the operation key of the conventional touch panel function, and the image is recognized. The command related to the device control is input by the recognized fingertip gesture, that is, the desired command is input only by the fingertip gesture without causing the posture collapse and the deviation of the line of sight during the operation. Therefore, the burden on the user during the input operation can be reliably eliminated, and the driving safety can be improved.

According to the above embodiment, the fingertip is also displayed at the key display position designated by the fingertip on the menu screen of the composite screen being displayed on the display unit 17. The user can recognize which key display has been designated on the screen, and the operability can be remarkably improved.

Further, according to the above embodiment, when the moving body imaged by the camera unit 11 reaches the key input possible area, the notification sound generation unit 18 generates a notification sound,
Since the generated notification sound is output by the speaker 19, the user can recognize that his / her fingertip is in the key input possible area without visually observing the display screen.

In the above embodiment, the key display designated by the fingertip and the fingertip are combined and displayed on the menu screen being displayed on the display unit 17, so that the designated key display is displayed. Although the different key display is displayed differently from the other key display, it is possible that the position of the designated key display is difficult to determine due to the blurring of the fingertip.

Therefore, in order to overcome such a situation, as shown in FIG. 9, the color and brightness of the designated key display 200 on the displayed menu screen are different from those of the other key displays 100. However, in this case, the designated key display 200 becomes clear and the user's input operation can be further improved.

Further, on the displayed menu screen, the designated key display may be a mark display such as a cursor display. In this case, the moving body extraction unit 12 displays the moving body. The position information may be extracted and a mark such as a cursor may be displayed at the position where the moving body is extracted.

Further, according to the above embodiment, the moving body area extracted from the image pickup signal picked up by the camera section 11 is displayed on the menu screen being displayed on the display section 17. If the moving body region is not within the image pickup signal, it is determined that the moving body is not in the key input possible region, and the display screen being displayed on the display unit 17 is extinguished as shown in FIG. When the body region is in the image pickup signal, as shown in FIG. 10B, it is assumed that the moving body is in the key input possible region, and other than the designated frame region 17a on the display screen being displayed on the display unit 17. The screen area may be lit and displayed. In this case, the user can recognize on the screen whether his or her hand is in the vicinity of the key input possible area, and the input operation is very easy.

Further, as shown in FIG.
On the menu screen being displayed in, for example, the designated frame area 17a,
An intermediate screen area 17b surrounding the designated frame area 17a and an outer frame screen area 17 surrounding the intermediate screen area 17b.
When the moving body moves on the outer frame screen area 17c being displayed on the display unit 17, the outer frame screen area 17c is displayed in orange,
When the moving body has moved into the intermediate screen area 17b, the intermediate screen area 17b is displayed in yellow, and when the moving body has moved into the designated frame area 17a, the designated frame area 17a is displayed in green. It may be displayed, and in this case, it is possible to successively recognize which key input area the user's hand is in during a fumbling operation, and it is possible to further improve operability. Further, by moving the moving body to each of these areas 17a, 17b, 17c, not only the display color may be changed but also the volume and sound quality of the notification sound may be changed.

Further, in the above embodiment, the menu screen is displayed on the display unit 17 even when the key input is not used during driving, but the menu screen on the display unit 17 is displayed when the key input is not used. May be turned off and the menu screen may be displayed on the display unit 17 only when the moving body is discriminated in step S33 shown in FIG. 3. In this case, the menu currently displayed is displayed. It is possible to prevent the screen from obstructing the field of vision during driving and reduce unnecessary power consumption when the key input is not used.

Further, as the equipment control device in the above embodiment, the on-vehicle equipment control device for controlling the on-vehicle equipment has been described as an example. However, as shown in FIG. 12, a camera 42 is provided on the personal computer 41. Therefore, it goes without saying that the pointer 43 may be controlled based on the gesture of the fingertip imaged by the camera 42 instead of the mouse for moving the pointer 43 of the personal computer 41.

[0067]

According to the device control device of the first aspect of the present invention configured as described above, the gesture of the fingertip can be performed without directly touching the pseudo key or the operation key of the touch panel function unlike the conventional case. Image recognition is performed, and a command related to the device control is input by the gesture of the fingertip recognized in the image, that is, the gesture of the fingertip is required without causing the posture collapse and the deviation of the line of sight during the operation. Since the command is input, it is possible to reliably eliminate the burden on the user during the input operation.

Further, according to the device control apparatus of the second aspect, since the fingertip and the menu screen recognized by the moving body motion recognition means are displayed on the display means as a composite screen, The position of the fingertip can be recognized on the screen of the display means.

According to the device control apparatus of the third aspect, the key display recognized by the moving body motion recognition means, that is, the key display designated by the motion of the fingertip is different from other key displays. Differently, for example, the marks and different colors are used for identification display, so that the input operation relating to the command designation can be effectively recognized.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an in-vehicle device control device showing an embodiment of a device control device of the present invention.

FIG. 2 is a flowchart showing a processing operation of a control unit in a command input processing of this embodiment.

FIG. 3 is a flowchart showing a processing operation of a moving body extraction unit in a moving body extraction process of this embodiment.

FIG. 4 is a flowchart showing the processing operation of the operation recognition unit in the finger detection processing of the present embodiment.

FIG. 5 is a flowchart showing the processing operation of the operation recognition unit in the finger close action detection processing of the present embodiment.

FIG. 6 is a flowchart showing the processing operation of the operation recognition unit in the finger opening action detection processing of the present embodiment.

FIG. 7 is an explanatory diagram schematically showing a composite screen related to the screen composition unit of the present embodiment. a) Explanatory diagram showing the moving body region extracted from the image pickup signal b) Explanatory diagram showing the menu screen stored in the memory section c) Explanatory diagram showing the composite screen in which the moving body region is superimposed on the menu screen

FIG. 8 is an explanatory diagram schematically showing a thinning process performed by a motion recognition unit according to the present embodiment. a) Explanatory diagram showing a moving body region before thinning processing b) Explanatory diagram showing a moving body region after thinning processing

FIG. 9 is an explanatory diagram showing a menu screen being displayed on a display unit according to another embodiment.

FIG. 10 is an explanatory view briefly showing a menu screen being displayed on a display unit in another embodiment. a) Explanatory diagram showing the display state of the display screen when there is no moving body in the image pickup signal b) Explanatory diagram showing the display state of the display screen when there is a moving body in the image pickup signal

FIG. 11 is an explanatory view briefly showing a menu screen being displayed on a display unit in another embodiment.

FIG. 12 is an external view showing an embodiment in which the device control apparatus of the present invention is applied to a personal computer or the like.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 In-vehicle equipment control device (equipment control device) 11 Camera part (imaging means) 12 Moving body extraction part (moving body extraction means) 13 Motion recognition part (moving body motion recognition means) 14 Memory part (first storage means) 15 Screen synthesis unit (screen synthesis unit) 16 Command input extraction unit (moving body motion recognition unit) 17 Display unit (display unit) 20 Control unit (control unit, second storage unit) 30 Vehicle-mounted device (device)

Claims (3)

[Claims] 1. A first storage means for storing a menu screen having a key display corresponding to a command related to device control, and a second storage means for storing device control data related to a command corresponding to the key display. Display means for displaying the menu screen stored in the first storage means, image pickup means for picking up an image of a moving body to obtain an image pickup signal relating to the moving body, and moving body extraction for extracting a moving body region from the image pickup signal Means, screen synthesizing means for synthesizing the extracted moving body area and the menu screen stored in the first storage means to generate a synthesized screen, and an operation of the moving body in the moving body area and the operation. The moving body motion recognition means for recognizing the designated key display and the device control data relating to the command corresponding to the recognized key display are read out from the second storage means and read out. Device control apparatus characterized by a control means for controlling the apparatus on the basis of the device control data. 2. The device control apparatus according to claim 1, wherein the control unit causes the display unit to display and output the combined screen generated by the screen combining unit. 3. The control means causes the key display recognized by the moving body motion recognition means to be identified and displayed on a menu screen being displayed on the display means. Equipment control device.