JPH11101935A - Close observation state detector and control device of electronic equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to immediately photograph an object of interest. SOLUTION: A pupil image pick-up device 20 picks up a pupil of an eyeball 16. A pupil area calculation device 22 calculates the pupil area from an output image of the pupil image of the pupil image pick-up device 20, and impresses the calculation result on a pupil area correcting device 24. A photo detector 18 detects incident light quantity on the pupil 16. The pupil area correcting device 24 removes a change in the pupil area due to the brightness from the calculation result of the pupil area calculation device 22 referring to an output of the photo detector 18 and a pupil area correction table 26, and supplies the result to a controller 28. The controller 28 decides whether the pupil is expanded or contracted, from the corrected pupil area from the pupil area correction device 24, and makes a camera part 12 record the object image on a recording medium 13 when the pupil is expanded.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze state detecting device and a control device for an electronic device, and more particularly, to a device for detecting a gaze state by a change in pupil area, and an electronic device by a change in pupil area. To the control device.

[0002]

2. Description of the Related Art Conventional silver halide film still cameras,
A still camera using an electronic or magnetic recording medium and a video camera that captures a moving image using a magnetic tape, an optical disk, a solid-state memory, a magneto-optical disk, a magnetic disk, or the like as a recording medium are used to transfer images (and audio). A shutter button or a record button was used to record.

[0003]

However, as is often the case with still cameras, unless the user is sufficiently proficient in operating the device, the user often misses the opportunity to record at the moment of interest. Had become.

Accordingly, an object of the present invention is to provide a gaze state detecting device that can more easily detect a gaze state of a user.

Another object of the present invention is to provide a device capable of operating, that is, controlling, an electronic device in real time based on a result of detecting a gaze state.

[0006]

A gaze state detecting apparatus according to the present invention comprises a pupil photographing means for photographing a pupil, a light detecting means for monitoring the amount of light incident on the pupil, and a pupil based on an output of the pupil photographing means. Pupil area calculation means for calculating the area,
According to the output of the light detecting means, the pupil area correcting means for removing the influence of the amount of light incident on the pupil from the pupil area calculated by the pupil area calculating means, and the change in the pupil area output from the pupil area correcting means. And determining means for determining.

A control apparatus for an electronic device according to the present invention includes a pupil photographing means for photographing a pupil, a light detecting means for monitoring the amount of light incident on the pupil, and a pupil for calculating a pupil area from an output of the pupil photographing means. Area calculation means, pupil area correction means for removing the influence of the amount of light incident on the pupil from the pupil area calculated by the pupil area calculation means according to the output of the light detection means, and output from the pupil area correction means It is characterized by comprising control means for judging a change in the pupil area and controlling the electronic device.

[0008] The diameter of the pupil changes depending on the interest and impression. The change in the pupil can be measured in a non-contact manner, and by correcting the measurement result in accordance with the pupil incident light amount, the influence of the brightness can be removed and the change in the pupil area due to a purely mental factor can be detected. By controlling an electronic device, for example, an imaging device based on the result, it is possible to prevent a user from missing a recording opportunity at a moment when the user is interested.

Further, even in a situation in which the operation by the limbs is inconvenient, the electronic apparatus can be controlled by the pupil, and the operation means or method can be diversified.

[0010]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a schematic block diagram showing an embodiment of the present invention applied to an image pickup apparatus. In the present embodiment, the half mirror 10 reflects part of the light from the subject and transmits the rest, and supplies the reflected light to the camera unit 12 and the transmitted light to the half mirror 14. The camera unit 12 is, for example, a silver halide film camera, a still video camera, a digital still camera, or the like, and includes a recording medium (a silver halide film, a magnetic recording medium, a solid-state memory, and magneto-optical recording) for recording a captured image. Medium 13). The half mirror 14 reflects a part of the light from the half mirror 10 and transmits the rest of the light. The transmitted light enters the eyeball 16 of the photographer looking through the viewfinder, and the reflected light is the light for detecting the amount of light. The light enters the detector 18. Half mirror 14
Is transmitted as proof light to the eyeball 16, and the pupil image of the eyeball 16 is reflected by the half mirror 14 and enters the pupil image capturing device 20.

The pupil image photographing device 20 includes a half mirror 1
The pupil of the eyeball 16 is photographed via 4. The pupil area calculation device 22 calculates the pupil area from the output image of the pupil image capturing device 20 and applies the calculation result to the pupil area correction device 24. The output of the photodetector 18 is also applied to the pupil area correction device 24. The pupil area correction device 24 corrects the calculation result of the pupil area calculation device 22, that is, the pupil area with reference to the output of the photodetector 18 and the pupil area correction table 26, and supplies the result to the control device 28. The control device 28 determines whether the pupil is enlarged or reduced from the corrected pupil area from the pupil area correction device 24, and controls the recording operation of the camera unit 12 based on the determination result.

FIG. 2 shows an operation flowchart of this embodiment. When the user points the camera toward the subject and looks through the finder (S1), the pupil image capturing device 20 captures a pupil image of the user (S2). Pupil area calculation device 22
Performs predetermined image processing on the output image of the pupil image capturing device 20 to calculate a pupil area (S3). The pupil area obtained here reflects factors of the brightness of the subject and mental factors (interest and interest of the user). The pupil area correction device 24 outputs the output of the photodetector 18 and the pupil area correction table 2
6, the influence of the subject luminance included in the pupil area output from the pupil area calculation device 22 is removed. In other words, the pupil area correction device 24 detects a change in the pupil area due to a mental factor from the pupil area output from the pupil area calculation device 22.

FIG. 3 shows a change in the diameter of the pupil with respect to the luminance of the subject. The vertical axis indicates the diameter of the pupil, and the horizontal axis indicates the subject luminance. Such a correspondence is determined by the pupil area correction table 26.
As a table. Since the pupil area corresponding to the luminance is known from the subject luminance indicated by the output of the photodetector 18, the difference between the pupil area and the current pupil area (the calculation result of the pupil area calculation device 22) is determined by the pupil area based on the mental factor. Will show change. FIG. 4 shows an example of a correction result obtained by the pupil area correction device 24. The horizontal axis represents time, and the vertical axis represents the pupil area due to mental factors, that is, the difference between the pupil area due to subject brightness and the current pupil area.

The control device 28 compares the outputs of the pupil area correction device 24 in a time series, and determines whether the user's pupil is expanding or contracting (S5). If the pupil area is enlarged due to a mental factor (for example, after time t1 in FIG. 4), it can be determined that the user's interest or interest is in the subject. The photographing (recording) of the subject is instructed, and the camera unit 12 records the subject image on the recording medium 13 in response to the instruction. If there is no enlargement of the pupil area, no subject image is recorded.

In FIG. 2, the subject image is recorded by enlarging the pupil and the processing is terminated. If the pupil is not enlarged, the processing is ended without recording. It goes without saying that the recording of the subject image may be executed at any time when the enlargement of the pupil is detected.

FIG. 5 is a schematic block diagram of a second embodiment of the present invention applied to a terminal device for a video conference system. In place of the half mirror 10 for the camera unit 12, a display 30 for displaying an image from a communication partner and a self-portrait is arranged. The same components as those in FIG. 1 are denoted by the same reference numerals.

FIG. 6 shows an operation flowchart of the embodiment shown in FIG. When the user looks at the screen of the display 30 (S21), the pupil of the user is photographed (S22) and the pupil area is calculated (S2), as in the embodiment shown in FIG.
3), by correcting the pupil area based on the pupil incident light amount, a change in the pupil area due to a mental factor is detected (S24).

The control device 32 determines whether the pupil is enlarged or reduced due to mental factors based on the output of the pupil area correction device 24 (S25). When the pupil area is enlarged (S25), since the user is interested in the image currently displayed on the display 30, the image currently displayed on the display 30 is stored in the image recording unit 3
4 is instructed to start a recording operation (S26). The image recording device 34 uses a magnetic tape, a magnetic disk, a magneto-optical disk, an optical disk, a solid-state memory, or the like as a recording medium. Conversely, if the pupil has a tendency to shrink (S28), the user's interest has diminished, and the image recording device 34 is instructed to stop the recording operation (S29).

Until the end operation is performed by the user (S2
7) Repeat the above.

It is clear that the embodiment shown in FIG. 5 is also applicable when listening to a television image. In this case, the display 30 is a television receiver,
The image recording device 34 is, for example, a VTR.

In the above embodiment, the electronic device is controlled by the change of the pupil area. However, the electronic device may be controlled by the change speed or acceleration of the pupil area.

[0023]

As can be easily understood from the above description, according to the present invention, the presence or absence or the degree of the user's interest is detected from the pupil area, and the electronic device or the like is controlled based on the detection result. Real-time control that does not miss the moment when the user feels interested.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention applied to an imaging apparatus.

FIG. 2 is an operation flowchart of the embodiment shown in FIG. 1;

FIG. 3 is a graph of luminance versus pupil diameter.

FIG. 4 is an example of a change in pupil area due to a mental factor.

FIG. 5 is a schematic block diagram of a second embodiment of the present invention applied to a terminal for a video conference system.

FIG. 6 is an operation flowchart of the embodiment shown in FIG. 5;

[Description of Signs] 10: half mirror 12: camera unit 13: recording medium 14: half mirror 16: eyeball 18: photodetector 20: pupil image photographing device 22: pupil area calculation device 24: pupil area correction device 26: pupil Area correction table 28: Control device 30: Display 32: Control device 34: Image recording device

Claims (3)

[Claims] 1. A pupil photographing means for photographing a pupil, a light detecting means for monitoring an amount of light incident on the pupil, a pupil area calculating means for calculating a pupil area from an output of the pupil photographing means, and the light detecting means Pupil area correction means for removing the influence of the amount of light incident on the pupil from the pupil area calculated by the pupil area calculation means, and determination means for determining a change in the pupil area output from the pupil area correction means. A gaze state detecting device, comprising: 2. A pupil photographing means for photographing a pupil, a light detecting means for monitoring an amount of light incident on the pupil, a pupil area calculating means for calculating a pupil area from an output of the pupil photographing means, and the light detecting means. Pupil area correction means for removing the influence of the amount of light incident on the pupil from the pupil area calculated by the pupil area calculation means, and determining a change in the pupil area output from the pupil area correction means. A control device for an electronic device, comprising: control means for controlling the device. 3. The electronic device is an imaging device.
A control device for an electronic device according to claim 1.