JP6927389B2 - Ophthalmic equipment - Google Patents

Description

The present disclosure relates to an ophthalmic apparatus for examining an eye.

As conventional ophthalmic devices, for example, an ocular refractive power measuring device, a corneal curvature measuring device, an intraocular pressure measuring device, a fundus camera, OCT, SLO and the like are known. In these ophthalmic devices, it is common to move the optometry part up, down, left, right, front and back with respect to the eye to be inspected by operating an operating member such as a joystick, and align the optometry part with respect to the eye to be inspected at a predetermined position. (See Patent Document 1).

Further, in a conventional ophthalmic apparatus, an ophthalmic apparatus that photographs the face of a subject and performs alignment in a wide range has been proposed (see Patent Document 2).

JP 2013-066760 Japanese Patent Application Laid-Open No. 10-216089

However, the current commercially available ophthalmic apparatus involves the operation of the examiner by the operating member until the eye to be inspected is found, and an apparatus capable of alignment in a wide range has not been realized. That is, there may be various considerations in the realization of a device capable of alignment in a wide range. In view of these, the present inventors have devised a prototype of a device capable of a wide range of alignment.

In view of the above problems, it is a technical subject of the present disclosure to provide an ophthalmic apparatus capable of preferably performing alignment in a wide range.

In order to solve the above problems, the present disclosure is characterized by having the following configurations.

(1)
An ophthalmic device that inspects the eye to be inspected
An optometry unit including an eye examination means for inspecting the eye to be inspected and a face photographing means for taking a photographed image including the face of the subject.
An adjusting means for moving the optometry portion relative to the eye to be inspected,
A position detection sensor for detecting the position of the optometry means and
An arithmetic processing means for setting an area of interest based on the position of the optometry unit moved by the adjusting means on the captured image in order to detect the eye to be inspected and processing an image signal in the area of interest.
The arithmetic processing means changes the set position of the region of interest in the captured image according to the position of the optometry means detected by the position detection sensor, and captures the image based on the changed image signal in the region of interest. It is characterized in that the position of the eye to be inspected in the image is detected, and the movement of the eye inspecting means by the adjusting means is controlled based on the detection result.

It is the schematic which shows the appearance of this Example. It is a block diagram which shows the control system of this Example. It is the schematic which shows the optical system of this Example. It is a figure which shows the control operation at the time of measurement of this Example. It is a figure which shows the control operation of the full-auto alignment of this Example. It is a figure which shows an example of auto gain control. It is a figure which shows the switching example to the position of ROI according to the position of the eye examination part. It is a figure which shows an example of the case where the face illumination is changed according to the position of the eye examination part. It is a figure which shows an example of the three-dimensional region where the left and right eyes are considered to exist in a three-dimensional real space. It is a figure which shows an example for finding the place where the vertices of a solid of interest area in a real space are projected on a photographed image. It is a figure for demonstrating the distortion of a camera image. It is a figure which shows an example of the case where the interest area ROI based on the three-dimensional position of an eye examination part is set in the photographed image of a face photographing part.

<Embodiment>
Hereinafter, embodiments of the ophthalmic apparatus according to the present disclosure will be described. The ophthalmic apparatus of this embodiment examines, for example, the eye to be inspected. The ophthalmic apparatus may be, for example, an optical power measuring device, a corneal curvature measuring device, a corneal shape measuring device, an intraocular pressure measuring device, an axial length measuring device, a fundus camera, an OCT, an SLO, an ultrasonic optometry device, or the like. ..

The ophthalmic apparatus may include an eye examination unit (also referred to as an apparatus main body), an adjustment unit, and an arithmetic processing unit. The optometry unit may include, for example, an optometry system and a facial imaging unit. The optometry system may include a configuration for inspecting the eye to be inspected, and may be, for example, an inspection optical system, an ultrasonic measurement system, or the like. The optometry system may be built in the optometry section or may be provided on the outer surface of the housing of the optometry section.

The face photographing unit may be provided for taking a photographed image including the face of the subject, and may be provided with a photographing optical system including an image pickup element, or may be provided with another photographing system. good. The face photographing unit may be built in the eye examination unit or may be provided on the outer surface of the housing of the eye examination unit. The face photographing unit may be, for example, a face photographing unit capable of photographing the face of the subject including at least both eyes, or a face photographing unit capable of capturing a wide range of one eye of the subject. good. The face photographing unit may also be used as an anterior eye part photographing unit for photographing the anterior eye portion of the eye to be inspected, or may be provided separately from the anterior eye portion photographing unit.

The adjusting unit may be used as a moving unit for relatively moving the optometry unit with respect to the eye to be inspected. May be moved. The adjusting unit may adjust the relative position between the eye to be inspected and the eye examination unit in at least one direction. Of course, it may be adjusted three-dimensionally. For example, the adjusting unit moves the optometry unit in the left-right (X) direction, the up-down (Y) direction, and the front-back (Z) direction with respect to the eye to be inspected.

The arithmetic processing unit may process the captured image, and may be, for example, a processor. The photographed image may be, for example, a photographed image photographed by the face photographing unit or a photographed image taken by the anterior eye part photographing unit. The arithmetic processing unit may be, for example, an arithmetic control unit for processing a captured image to control an apparatus, or an image processing unit for processing a captured image. The arithmetic processing unit may be provided on the photographing unit side. The arithmetic processing unit may be constructed by combining the arithmetic control unit and the image processing unit.

The arithmetic processing unit may control the device by processing the image signal of the captured image by, for example, the photographing unit (for example, the face photographing unit and the anterior eye part photographing unit). The control of the device may be, for example, control of imaging conditions by the imaging unit, or movement control (alignment adjustment) of the eye examination unit with respect to the eye to be inspected.

<Setting of the region of interest (see, for example, FIGS. 6 and 12)>
The arithmetic processing unit may set an area of interest in the captured image captured by the face photographing unit and process the image signal in the set area of interest. In this case, the arithmetic processing unit may set an area of interest to detect at least a part of the face of the subject, for example, may set an area of interest for detecting the eye to be examined. Areas of interest may be set to detect at least a portion of the skin on the subject's face. In this case, an arbitrary portion of the entire face may be set as the region of interest, or a predetermined region of the entire face (for example, the eyes) may be set as the region of interest.

For example, the arithmetic processing unit may process the image signal in the region of interest to detect the eye to be inspected, or process the image signal in the region of interest to determine an evaluation value for controlling the imaging conditions of the face photographing unit. You may get it. The evaluation value in this case may be, for example, an evaluation value for evaluating the brightness of the face of the subject.

For example, the arithmetic processing unit may control the device based on the image signal in the region of interest. In this case, the arithmetic processing unit may control the shooting conditions by the face photographing unit based on the image signal in the region of interest (see, for example, FIG. 6), or the eye examination by the adjusting unit based on the image signal in the region of interest. The movement of the unit may be controlled (see, for example, FIG. 12). The control of the shooting conditions may be, for example, auto gain control of the face shooting unit or automatic exposure time control. The arithmetic processing unit controls the shooting conditions by the face photographing unit based on the image signal in the region of interest, and controls the movement of the eye examination unit by the adjusting unit based on the captured image acquired under the controlled shooting conditions. You may. Further, the arithmetic processing unit may automatically control the illumination conditions of the face illumination optical system based on the image signal in the region of interest.

In setting the region of interest, the arithmetic processing unit may, for example, set the region of interest in the captured image based on the position of the eye examination unit moved by the adjustment unit. In this case, the position of the region of interest on the captured image corresponding to the position of the eye examination unit may be stored in advance in the storage unit, and the arithmetic processing unit is based on the position of the region of interest stored in advance in the storage unit. You may set the area of interest. In this case, the region of interest may be set on the captured image based on the positional relationship between the subject supported by the face support portion and the eye examination portion.

Here, by setting the region of interest based on the position of the eye examination portion, for example, at least a part of the face of the subject can be easily identified, and the processing for the image signal of at least a part of the face is smoothly performed. be able to. When detecting at least a part of the face based on the image taken from the face photographing unit, it may be difficult to detect due to the influence of noise light or the like. Here, at least a part of the face can be easily detected by processing the image signal in the region of interest set in consideration of the positional relationship between the eye examination unit and the subject.

Since the formation position of the face image (at least a part of the face image) in the photographed image varies depending on the individual difference of the face of the subject, the region of interest may be set in consideration of these. Further, when the image signal in the region of interest is preferentially processed and at least a part of the face is not detected, the detection process may be performed on the entire captured image.

Of course, when the region of interest is set for at least a part of the subject's face in the captured image and the image signal in the region of interest is processed, the arithmetic processing unit does not use the position information of the eye examination unit and does not use the position information of the eye examination unit. At least a part may be specified by image processing.

For example, the arithmetic processing unit controls the imaging conditions by the face imaging unit based on the image signal in the region of interest specified by the image processing, and processes the captured image acquired after controlling the imaging conditions to detect the eye to be inspected. You may. Thereby, for example, the eye can be detected in a state where the contrast between the subject's skin and the eye is adjusted. In this case, for example, the entire face of the subject, or at least a part of the subject's face in which the region of interest may be set on at least a part of the skin of the subject's face, is, for example, the subject's face. It may be specified by using the brightness difference between the face and the background, pattern patching, and the like.

<Change the position of the region of interest according to the position of the eye examination part (see, for example, FIG. 7)>
When the region of interest is set based on the position of the eye examination unit, for example, the arithmetic processing unit may change the position of the region of interest in the captured image according to the change in the position of the eye examination unit. As a result, even if the positional relationship between the subject's face and the eye-examined portion changes due to a change in the position of the eye-examined portion, at least a part of the subject's face can be reliably detected.

When the subject's face is properly supported by the face support, the left and right centers of the subject's face are positioned around the center position of the device, and as a result, the subject's face is positioned and in the left and right eyes. Is also positioned.

The position of the face in the captured image changes depending on the position of the face photographing portion with respect to the face. For example, when the face photographing portion is positioned in front of the face, the face of the subject is formed in a certain region including the imaging center (for example, the position of the optical axis) in the captured image. On the other hand, when the face photographing unit is arranged on the right side with respect to the face (as viewed from the device), the face is formed in the region on the left side with respect to the imaging center. Further, when the face photographing portion is arranged on the left side with respect to the face, a region on the right side with respect to the imaging center is formed. Of course, even when the face photographing portion moves up and down with respect to the face, the formation position of the subject's face changes in the vertical direction. The change in the positional relationship is the same for the positions of the left and right eyes in the captured image.

Since the position of the face photographing unit has a paired relationship with the position of the eye examination unit, the arithmetic processing unit sets the region of interest in the photographed image based on the position information of the eye examination unit, thereby performing the face of the subject. At least a part of can be reliably detected.

In this case, the correspondence between the position of the region of interest on the captured image and the position of the eye examination unit may be stored in the storage unit in advance. For example, the position of the region of interest corresponding to each position of the eye examination unit may be stored in advance, or the position of the region of interest corresponding to the reference position of the eye examination unit is stored in advance and the eye examination unit from the reference position. The movement amount of the region of interest corresponding to the movement distance of the above may be stored in advance.

The arithmetic processing unit acquires the position information of the eye examination unit based on the detection signal from the position detection sensor for detecting the position of the eye examination unit, and sets the region of interest corresponding to the acquired position information. You may. The position detection sensor may have various changes, but for example, it may be a potentiometer provided on a base for holding the eye examination unit so as to be movable, an encoder provided on the motor unit of the drive unit, or the like. It may be.

When changing the position of the region of interest according to the position of the eye examination unit, for example, the arithmetic processing unit shifts the region of interest on the captured image in the horizontal direction according to the change in the position of the eye examination unit in the left-right direction. You may do so. As a result, at least a part of the face can be smoothly detected even when the eye examination portion is moved to the left or right.

Further, for example, the arithmetic processing unit may shift the region of interest on the captured image according to the change in the three-dimensional position of the eye examination unit. As a result, at least a part of the face can be smoothly detected even when the eye examination portion is moved three-dimensionally.

<Setting of the region of interest corresponding to the predetermined position (see, for example, FIG. 6)>
When the area of interest is set based on the position of the eye-examination unit, for example, the arithmetic processing unit may set the area of interest corresponding to a predetermined position of the eye-examination unit set in advance in the captured image. As a result, when processing the image signal from the face photographing unit in the state where the eye examination unit is arranged at a predetermined position, at least a part of the face is specified, so that at least a part of the subject's face is specified. Can be reliably detected.

At the stage of taking a picture by the face photographing unit, the arithmetic processing unit arranges the eye examination unit at a predetermined position set in advance, for example. The predetermined position may be the initial position. The predetermined position may be stored in the storage unit. In this case, a position where the examination axis (for example, the optical axis of the optometry optical system) is arranged on the side closer to either the left or right eye may be set as a predetermined position for quick alignment with the eye to be inspected. Of course, the predetermined position may be a position where the inspection axis is arranged between the left and right eyes.

When the eye examination unit is arranged at a predetermined position, the arithmetic processing unit may set an area of interest corresponding to the predetermined position of the eye examination unit and process an image signal in the set area of interest. The position of the region of interest corresponding to the predetermined position may be set to, for example, a position where at least a part of the face of the subject is formed in the captured image when the eye examination portion is arranged at the predetermined position.

<Auto gain control (see, for example, FIG. 7)>
When controlling the shooting conditions by the face photographing unit based on the image signal in the region of interest, for example, the arithmetic processing unit may perform auto gain control of the face photographing unit based on the image signal in the region of interest. Here, the brightness of the face image is adjusted to an allowable level by performing the auto gain control based on the image signal in the region of interest set based on the position of the eye examination unit. As a result, the position of the face is specified and the contrast between the subject's face and the eyes is adjusted even if there is an influence of ambient light or the like. As a result, for example, eye detection in a captured image can be reliably performed, and smooth alignment with respect to the eye to be inspected becomes possible.

In order to accurately adjust the contrast between the skin of the subject's face and the eyes, it corresponds to the position of the eye examination part so that the image area where at least a part of the skin of the face is formed is set as the area of interest. The area of interest may be set. Further, in order to prevent the area of interest from being set to areas other than the face image (for example, background, hair, etc.), the image area in which the central portion of the subject's face is formed is set as the area of interest. The region of interest may be set according to the position.

The position, size, etc. of the region of interest in the captured image can be appropriately set within an adjustable range of the auto gain related to the face. The size of the region of interest is preferably at least smaller than the image region of the average human face image (because the background portion is not included). Here, the region of interest may be set for auto gain control on the face image. The shape of the region of interest may be, for example, a rectangle, a circle, or the like, but the shape for extracting a specific image region is not particularly limited.

When arranging the eye examination unit at a predetermined position, the arithmetic processing unit sets an area of interest corresponding to the predetermined position of the eye examination unit, analyzes the image signal in the area of interest, and performs auto gain control. May be good. Further, the arithmetic processing unit may change the position of the region of interest according to the position of the eye examination unit, analyze the image signal in the changed region of interest, and perform auto gain control.

At the time of auto gain, the image signal in the analysis region may be analyzed and an evaluation value for performing auto gain may be acquired. The evaluation value for auto gain may be the average brightness in the region of interest, the cumulative luminance value of each pixel in the region of interest, or the maximum luminance value in the region of interest. Further, as the evaluation value, the contrast of the image in the region of interest, the histogram, or the like may be used.

In the auto gain control, for example, an auto gain function provided in an image sensor (for example, a CMOS camera) provided in the face photographing unit may be used. In this case, for example, the control unit of the device sets the position information of the region of interest with respect to the arithmetic processing unit provided in the image sensor, so that the auto gain is based on the image signal in the set region of interest. Control is done. Further, when the image sensor itself is not provided with the auto gain function, the image analysis may be performed on the control unit of the apparatus, and the gain may be controlled based on the result of the image analysis. In the above description, the auto gain control of the face photographing unit is performed based on the image signal in the region of interest, but the exposure time of the face photographing unit 90 is automatically set based on the image signal in the region of interest. In the case of control, the above embodiment can be applied.

<Eye position detection (see, for example, FIG. 12)>
For example, the arithmetic processing unit may detect the position of the eye in the captured image based on the image signal in the region of interest set based on the position of the eye examination unit. As a result, since the position of the eye to be inspected can be specified by using the position of the eye-examined portion, for example, the eye can be reliably detected in the captured image, and smooth alignment with respect to the eye to be inspected becomes possible. As a specific method for eye detection, an index formed in the eye (for example, anterior segmental reflection image) may be detected, or a characteristic part of the eye (for example, pupil, iris, sclera, blood vessel, etc.) may be detected. ) May be detected. Further, the region of interest may be set as a search range for searching the eye in the captured image.

In order to accurately detect the eye in the captured image, the region of interest may be set corresponding to the position of the eye examination portion so that the image region in which the eye to be examined is formed is set as the region of interest. When the captured image includes the left and right eyes, the region of interest is set according to the position of the eye examination portion so that the image region in which at least one of the left and right eyes of the subject is formed is set as the region of interest. You may. A different region of interest may be set for each of the left and right eyes, or a region of interest including the left and right eyes may be set.

The position, size, and the like of the region of interest in the captured image can be appropriately set in consideration of the individual difference in the interpupillary distance and the individual difference in the vertical position of the eye due to the difference in the size of the face. Further, in order to detect the eyes in the region of interest, the region of interest may be set to a size that includes the entire facial image.

When the eye examination unit is arranged at a predetermined position, the arithmetic processing unit sets an area of interest corresponding to the predetermined position of the eye examination unit, analyzes the image signal in the area of interest, and performs eye detection. good. By setting the region of interest corresponding to the predetermined position, the influence of the image region (for example, background, hair, etc.) that causes noise is reduced, and the position of the eye on the face is detected.

Further, the arithmetic processing unit may change the position of the region of interest according to the position of the eye examination unit, analyze the image signal in the changed region of interest, and perform eye detection. As a result, regardless of the position of the eye examination portion, the influence of the image region that causes noise is reduced, and the position of the eye on the face is detected.

When the eye position is detected in the captured image, the arithmetic processing unit may automatically align the eye examination unit with respect to the eye to be inspected by controlling the drive unit based on the detection result of the eye position. ..

Even if the arithmetic processing unit performs auto gain control based on the image signal in the region of interest and detects the position of the eye based on the image signal in the region of interest set in the captured image adjusted by the auto gain. good. In this case, the region of interest for eye position detection may differ from the region of interest for autogain at least in position or size. As a result, various controls can be performed with high accuracy. Of course, the present invention is not limited to this, and the region of interest of the same position and size may be used for auto gain and eye position detection.

<Transformation form>
In the above description, the imaging conditions of the face imaging unit are controlled according to the position of the eye examination unit, but the present invention is not limited to this, and illumination by the face illumination optical system is performed according to the position of the eye examination unit. The conditions may be controlled (see FIG. 8).
<Example>
The ophthalmic apparatus according to the present disclosure will be described with reference to the drawings. In the following description, an ophthalmic refractive force measuring device will be described as an example of an ophthalmic device, but a corneal curvature measuring device, a corneal shape measuring device, an intraocular pressure measuring device, an axial length measuring device, a fundus camera, and an OCT (optical coherence). It can also be applied to other ophthalmic devices such as tomography) and SLO (Scanning Laser Ophthalmoscope).

The ophthalmic apparatus of this embodiment objectively measures, for example, the refractive power of the eye to be inspected. For example, the ophthalmic apparatus of this embodiment may be an apparatus that performs measurement for each eye or may be an apparatus that performs measurement for both eyes at the same time (binocular vision). The ophthalmic apparatus mainly includes, for example, an eye examination unit, an imaging unit, a drive unit, and a control unit.
<Appearance>
The appearance of the ophthalmic apparatus will be described with reference to FIG. As shown in FIG. 1, the ophthalmic apparatus 1 of this embodiment mainly includes an eye examination unit 2, a face photographing unit 90, and a driving unit 4. The eye examination unit 2 inspects the eye to be inspected. The eye examination unit 2 may include, for example, an optical system for measuring the optical power of the eye to be inspected, the curvature of the cornea, the intraocular pressure, and the like. Further, the eye examination unit 2 may be provided with an optical system or the like for photographing the anterior eye portion, the fundus, etc. of the eye to be inspected. In this embodiment, the eye examination unit 2 for measuring the refractive power will be described as an example. The face photographing unit 90, for example, photographs the face of the eye to be inspected. The face photographing unit 90 photographs a face including at least one of the left and right eyes to be inspected, for example. The drive unit 4 moves, for example, the eye examination unit 2 and the imaging unit 3 in the vertical, horizontal, front-back directions (three-dimensional directions) with respect to the base 5.

Further, the ophthalmic apparatus 1 of this embodiment may include, for example, a housing 6, a display unit 7, an operation unit 8, a face support unit 9, and the like. For example, the housing 6 houses an eye examination unit 2, a face photographing unit 90, a driving unit 4, and the like. The display unit 7 displays, for example, an observation image of the eye to be inspected, a measurement result, and the like. The display unit 7 may be provided integrally with the device 1, or may be provided separately from the device, for example. The ophthalmic apparatus 1 may include an operation unit 8. The operation unit 8 is used for various settings of the device 1 and an operation at the start of measurement. Various operation instructions by the inspector are input to the operation unit 8. For example, the operation unit 8 may be various human interfaces such as a touch panel, a joystick, a mouse, a keyboard, a trackball, and a button. The face support portion 9 may include, for example, a forehead pad 10 and a chin rest 11. The jaw base 11 may be moved in the vertical direction by driving the jaw base driving unit 12.
<Control system>
As shown in FIG. 2, the present device 1 includes a control unit 70. The control unit 70 controls various controls of the present device 1. The control unit 70 includes, for example, a general CPU (Central Processing Unit) 71, a ROM 72, a RAM 73, and the like. For example, the ROM 72 stores an ophthalmic device control program for controlling the ophthalmic device, initial values, and the like. For example, RAM temporarily stores various types of information. The control unit 70 is connected to an eye examination unit 2, a face photographing unit 90, a drive unit 4, a display unit 7, an operation unit 8, a chin rest drive unit 12, a storage unit (for example, a non-volatile memory) 74, and the like. The storage unit 74 is, for example, a non-transient storage medium capable of holding the stored contents even when the power supply is cut off. For example, a hard disk drive, a detachable USB flash memory, or the like can be used as the storage unit 74. Further, a position detection sensor 75 for detecting the position of the eye examination unit 2 is connected to the control unit 70.
<Eye examination part>
The eye examination unit 2 measures, inspects, photographs, and the like the eye to be inspected. The eye examination unit 2 may include, for example, a measurement optical system for measuring the refractive power of the eye to be inspected. For example, as shown in FIG. 3, the eye examination unit 2 includes a measurement optical system 20, a fixation target display optical system 40, an index projection optical system 50, and an observation optical system (imaging optical system) 60. You may.

The measurement optical system 20 includes a projection optical system (projection optical system) 20a and a light receiving optical system 20b. The projection optical system 20a projects a luminous flux onto the fundus Ef through the pupil of the eye to be inspected. Further, the light receiving optical system 20b extracts the reflected light flux (fundus reflected light) from the fundus Ef through the peripheral portion of the pupil in a ring shape, and captures a ring-shaped fundus reflection image mainly used for measuring the refractive power.

The projection optical system 20a has a measurement light source 21, a relay lens 22, a hall mirror 23, and an objective lens 24 on the optical axis L1. The light source 21 projects a spot-shaped light source image from the relay lens 22 onto the fundus Ef via the objective lens 24 and the central portion of the pupil. The light source 21 is moved in the optical axis L1 direction by the moving mechanism 33. The hall mirror 23 is provided with an opening through which the light flux from the light source 21 passes through the relay lens 22. The hole mirror 23 is arranged at a position optically conjugate with the pupil of the eye to be inspected.

The light receiving optical system 20b shares the hall mirror 23 and the objective lens 24 with the projection optical system 20a. Further, the light receiving optical system 20b includes a relay lens 26 and a total reflection mirror 27. Further, the light receiving optical system 20b has a light receiving diaphragm 28, a collimator lens 29, a ring lens 30, and an image pickup element 32 on the optical axis L2 in the reflection direction of the hall mirror 23. A two-dimensional light receiving element such as an area CCD can be used as the image sensor 32. The light receiving diaphragm 28, the collimator lens 29, the ring lens 30, and the image pickup element 32 are moved integrally with the measurement light source 21 of the projection optical system 20a in the optical axis L2 direction by the moving mechanism 33. When the light source 21 is arranged at a position optically conjugate with the fundus Ef by the moving mechanism 33, the light receiving diaphragm 28 and the image sensor 32 are also arranged at a position optically conjugate with the fundus Ef.

The ring lens 30 is an optical element for shaping the fundus reflected light guided from the objective lens 24 via the collimator lens 29 into a ring shape. The ring lens 30 has a ring-shaped lens portion and a light-shielding portion. When the light receiving diaphragm 28 and the image sensor 32 are arranged at positions optically conjugate with the fundus Ef, the ring lens 30 is arranged at a position optically conjugate with the pupil of the eye to be inspected. The image sensor 32 receives the ring-shaped fundus reflected light (hereinafter referred to as a ring image) via the ring lens 30. The image sensor 32 outputs the image information of the received ring image to the CPU 71. As a result, the CPU 71 displays the ring image on the display unit 7, calculates the refractive power based on the ring image, and the like.

Further, as shown in FIG. 3, in this embodiment, the dichroic mirror 39 is arranged between the objective lens 24 and the eye to be inspected. The dichroic mirror 39 transmits the light emitted from the light source 21 and the fundus reflected light corresponding to the light from the light source 21. Further, the dichroic mirror 39 guides the light flux from the fixation target display optical system 40, which will be described later, to the eye to be inspected. Further, the dichroic mirror 39 reflects the anterior-eye reflected light of the light from the index projection optical system 50 described later, and guides the anterior-eye reflected light to the observation optical system 60.

As shown in FIG. 3, the index projection optical system 50 is arranged in front of the eye to be inspected. The index projection optical system 50 mainly projects an index used for alignment of the optical system with respect to the eye to be inspected to the anterior eye portion. In this case, the index projection optical system 50 projects an index used for alignment in at least either the XY direction or the Z direction of the optical system with respect to the eye to be inspected to the anterior eye portion. It should be noted that the alignment detection may be performed by detecting the feature portion in the anterior eye portion image without using the index projection optical system 50. Of course, the alignment may be detected by using the index detection and the feature region detection in combination.

The index projection optical system 50 includes, for example, a ring index projection unit 51 and an index projection unit 52. The ring index projection unit 51 projects diffused light onto the cornea of the subject's eye E, and projects a ring index (so-called Mayerling). The ring index projection unit 51 is also used as anterior-eye portion illumination for illuminating the anterior-eye portion of the subject's eye E in the ophthalmic apparatus 1 of the present embodiment. The index projection unit 52 projects parallel light onto the cornea of the eye to be inspected and projects an infinity index.

The optotype presenting optical system 40 is provided on the optical axis L4 in the reflection direction of the light source 41, the fixation target 42, the relay lens 43, and the reflection mirror 46. The fixation target 42 is used to fix the eye to be inspected when measuring the objective refractive power. For example, the fixation target 42 is illuminated by the light source 41 and is presented to the eye to be inspected.

The light source 41 and the fixation target 42 are integrally moved in the direction of the optical axis L4 by the drive mechanism 48. The presentation position (presentation distance) of the fixation target may be changed by moving the light source 41 and the fixation target 42. As a result, the refractive power can be measured by applying cloud fog to the eye to be inspected.

The front eye photographing optical system 60 includes an image pickup lens 61 and an image pickup element 62 on the optical axis L3 in the reflection direction of the half mirror 63. The image sensor 62 is arranged at a position optically conjugate with the anterior segment of the eye to be inspected. The image sensor 62 images the anterior eye portion illuminated by the ring index projection unit 51. The output from the image sensor 62 is input to the CPU 71. As a result, the anterior segment image Ia of the eye to be inspected captured by the image sensor 62 is displayed on the display unit 7 (see FIG. 2). Further, in the image sensor 62, an alignment index image (a ring index and an infinity index in this embodiment) formed on the cornea of the eye to be inspected is imaged by the index projection optical system 50. As a result, the CPU 71 can detect the alignment index image based on the image pickup result of the image pickup element 62. Further, the CPU 71 can determine the suitability of the alignment state based on the position where the alignment index image is detected.
<Face photography section>
The face photographing unit 90 is, for example, an optical system for photographing a face including at least one of the left and right eyes to be inspected. For example, as shown in FIG. 3, the face photographing unit 90 of this embodiment mainly includes, for example, an image pickup device 91 and an image pickup lens 92. A visible light cut filter may be arranged in the optical path of the face photographing unit 90 to limit noise light due to visible light.

The face photographing unit 90 of this embodiment is moved together with the eye examination unit 2 by the driving unit 4. Of course, the face photographing unit 90 may be fixed to the base 5 and may not move, for example.

The photographing lens 92 may be, for example, a wide-angle lens. The wide-angle lens is, for example, a fisheye lens, a conical lens, or the like. By providing the wide-angle lens, the face photographing unit 90 can photograph the face of the subject with a wide angle of view. The angle of view is, for example, 87 ° or more. As a result, the face photographing unit 90 can easily take a picture of both eyes of the subject.
<Face lighting optical system>
The face illumination optical system 80 illuminates the face of the eye to be inspected. The face illumination optical system 80 includes, for example, an illumination light source 81. For the face illumination optical system 80, for example, a light source having low directivity is used. The face illumination optical system 80 may be a face illumination optical system that illuminates the subject's face with infrared light. Here, by cutting the visible light in the face photographing unit 90, the reflected light from the face by the infrared light is detected by the face photographing unit 90. In this case, noise light due to infrared light may be incident on the face photographing unit 90. Therefore, by setting the auto gain and the ROI of interest for eye detection in the captured image, the influence of infrared light noise Can be reduced.
<Control method>
Hereinafter, the control operation of the present device 1 will be described. The apparatus 1 automatically aligns the eye examination unit 2 with the eye to be inspected (fully automatic), for example, in order to inspect the eye to be inspected.

The flowchart of the fully automatic measurement is shown in FIG. For example, the control unit 70 performs full-auto alignment after measuring, and aligns the eye E to be inspected with the eye examination unit 2. After that, the eye is measured, the eye examination unit 2 is moved to the other eye, and full auto alignment is performed again. When the alignment is completed, the control unit 70 measures the eye to be inspected and ends the process. Hereinafter, each step will be described.

{Step S100: Jaw stand adjustment}
In step S100, the control unit 70 adjusts the chin rest. Details of the jaw stand adjustment will be described later. When the jaw stand adjustment is completed, the control unit 70 shifts to step S200.

{Step S200: Fully automatic alignment (1)}
In step S200, the control unit 70 performs full-auto alignment with one of the left and right eyes to be inspected. For example, the control unit 70 detects the eye of the subject from the face image taken by the face photographing unit 90, and moves the eye examination unit 2 in that direction. At this time, the control unit 70 may detect the alignment index projected by the index projection optical system 50 from the anterior eye portion image of the eye to be inspected taken by the anterior eye portion observation optical system 60. When rough alignment is performed based on the information of the eye to be inspected detected from the face image and the alignment index is detected from the anterior eye image, the control unit 70 performs fine alignment based on the alignment index. For example, the control unit 70 moves the eye examination unit 2 so that the position of the alignment index becomes a predetermined position, and completes the alignment.

{Step S300: Measurement (1)}
In step S300, the control unit 70 inspects the eye to be inspected. For example, the control unit 70 irradiates the fundus of the eye to be inspected with the measurement light, and measures the optical refractive power of the eye to be inspected based on the detection result of the measurement light reflected by the fundus.

{Step S400: Left / right eye switching}
In step S400, the control unit 70 switches the measurement target eye. For example, the control unit 70 moves the eye examination unit 2 from the eye for which the examination has been completed in step S300 to the other eye.

{Step S500: Fully automatic alignment (2)}
In step S500, the control unit 70 performs full-auto alignment on the eye to be inspected for which the inspection has not been completed, in the same manner as in step S200.

{Step S600: Measurement (2)}
In step S600, the control unit 70 inspects the other eye to be inspected.

<From measurement preparation to measurement>
Subsequently, the control from the measurement preparation in step S100 to the measurement in step S300 will be described in more detail with reference to FIG. For example, the control unit 70 detects both eyes of the subject from the face image taken by the face photographing unit 90.

{Step S110: Face image analysis}
In step S110, the control unit 70 positions the eye examination unit 2 at a preset initial position. In this case, for example, the control unit 70 may position the eye examination unit 2 at the initial position stored in the storage unit 74. Further, the control unit 70 may position the eye examination unit 2 at the initial position by detecting that the eye examination unit 2 has reached the initial position by the position detection sensor 75. The control unit 70 illuminates the face with the face illumination optical system 80, and detects the position of at least one of the left and right eyes to be inspected based on the image pickup signal from the face photographing unit 90.

{Step S120: Anterior eye image analysis}
In step S120, the control unit 70 analyzes the image of the anterior segment of the eye based on the imaging signal from the anterior eye imaging optical system 60, and performs detection processing of the eye to be inspected. For example, the control unit 70 analyzes the image of the anterior eye portion and performs detection processing of an index or a characteristic portion of the eye.

{Step S130: Optometry detection determination}
In step S130, the control unit 70 determines whether or not the eye to be inspected could be detected based on the image pickup signal from the anterior eye photographing optical system 60 in the detection process of step S120. When the eye to be inspected is detected, the control unit 70 skips the jaw stand adjustment in step S140 and shifts to the position adjustment of the eye examination unit 2 by the anterior eye imaging optical system 60. If the eye to be inspected is not detected, the process proceeds to step S140.

{Step S140: Jaw stand adjustment}
In step S140, the control unit 70 controls, for example, the jaw pedestal drive unit 12 to adjust the height of the jaw pedestal. The control unit 70 controls the jaw pedestal drive unit 12 based on the analysis result of step S110, and adjusts the height of the jaw pedestal. In this case, the control unit may control the jaw pedestal drive unit 12 so that the eye to be inspected is arranged within the movable range of the eye examination unit 2 and adjust the height of the jaw pedestal. If it is not necessary to adjust the chin rest, the process may proceed to step S180.

{Step S150: Second face image analysis}
When the height adjustment of the jaw stand is completed, the control unit 70 detects at least one of the left and right eyes to be inspected based on the image pickup signal from the face photographing unit 90 in step S150. ..

{Step S160: Second anterior eye image analysis}
In step S120, the control unit 70 analyzes the image of the anterior segment of the eye based on the imaging signal from the anterior eye imaging optical system 60, and performs detection processing of the eye to be inspected.

{Step S170: Second eye detection determination}
In step S170, the control unit 70 determines whether or not the eye to be inspected could be detected in the detection process of step S160. When the eye to be inspected is detected, the control unit 70 skips the position adjustment of the eye examination unit 2 by the face photographing unit 90 and shifts to the position adjustment of the eye examination unit 2 by the anterior eye photographing optical system 60. If the eye to be inspected is not detected, the process proceeds to step S180.

{Step S180: Position adjustment by the face photographing section}
In step S180, the control unit 70 controls the drive unit 4 based on, for example, an image pickup signal from the face photographing unit 90, and adjusts the position of the eye examination unit 2. The control unit 70 detects the position of the eye based on the image pickup signal from the face photographing unit 90. The control unit 70 controls the drive unit 4 based on the position detection result and adjusts the three-dimensional position of the eye examination unit 2. In this case, the control unit may control the drive unit 4 so that the eye to be inspected is arranged within the range that can be photographed by the anterior eye imaging optical system 60, and may adjust the position of the eye examination unit 2.

{Step S190: Position adjustment by front eye imaging optical system 60}
In step S190, the control unit 70 controls the drive unit 4 based on the image pickup signal from the anterior eye photographing optical system 60, and adjusts the position of the eye examination unit 2. The control unit 70 detects the position of the eye to be inspected based on the image pickup signal from the anterior eye photographing optical system 60. The control unit 70 controls the drive unit 4 based on the position detection result and adjusts the position of the eye examination unit 2. In this case, the control unit may control the drive unit 4 so that the eye to be inspected is arranged within the alignment allowable range, and may adjust the three-dimensional position of the eye examination unit 2.

<Setting of area of interest ROI in auto gain control>
Next, an example of performing auto gain control of the face photographing unit 90 in the face image analysis in step S110 will be described. FIG. 6 is a diagram showing an example of auto gain control. The control unit 70 sets the region of interest ROI based on the position of the eye examination unit arranged at the initial position in the captured image of the face photographing unit 90. In this case, the region of interest ROI is the region of interest set to perform auto gain control.

For example, when the subject supported by the chin rest 11 is photographed at the initial position, an image region that is likely to be a facial region is preset as the region of interest ROI. Here, the gain is automatically adjusted so that the brightness in the region of interest ROI becomes the target value.

For example, the control unit performs auto gain control so that the brightness of the ROI becomes the target value Th1. More specifically, the control unit may perform auto gain control so that the cumulative luminance value of each pixel in the ROI approaches an arbitrary target value Th1. When the cumulative luminance value is less than the target value, If the gain is increased and the cumulative brightness exceeds the target value, the gain may be decreased.

As a result, the contrast between the subject's face and eyes is properly adjusted in the captured image taken by the face photographing unit 90, which tends to collect ambient light due to its wide angle, and subsequent eye detection can be performed smoothly. can.

Even after the auto gain control is performed at the initial position, the auto gain control may be executed (for example, step 150, step 180, etc.) or may be terminated. Auto gain control may be performed at all times.

After the auto gain control is performed at the initial position, the control unit may move the region of interest ROI according to the position of the eye examination unit 2. FIG. 7 is a diagram showing an example of switching to the ROI position according to the position of the eye examination unit 2. In this embodiment, a plurality of different regions of interest ROIs are set according to the left and right positions of the eye examination unit 2. In this case, the amount of movement of the eye examination unit 2 in the left-right direction and the amount of movement of the region of interest ROI in the captured image may have a paired relationship.

The size of the region of interest ROI in the captured image may be appropriately set according to the purpose of image processing. In this embodiment, the purpose is to properly adjust the brightness of the face to perform good eye detection, and the size is set so that at least a part of the skin of the subject is formed (position avoiding the eyes). Is preferable). In this case, a boundary corresponding to the region of interest ROI may be set in each of the vertical and horizontal directions, or a boundary corresponding to the region of interest ROI may be set only in the horizontal direction.

The correspondence between the position of the eye examination unit 2 and the set position of the region of interest ROI may be stored in the storage unit 74 in advance, and the control unit 70 uses the correspondence relationship stored in the storage unit 74 to examine the eye. The region of interest ROI may be set according to the position of the part 2. The correspondence between the position of the eye examination unit 2 and the setting position of the ROI may be obtained by, for example, a simulation considering the positions of the eye examination unit 2 and the face photographing unit 90 in the eye examination unit 2. Further, even if the face of the subject is actually photographed by the face photographing unit 90 while being supported by the chin rest 11, the correspondence between the position of the eye examination unit 2 and the set position of the region of interest ROI can be obtained. good.

In FIG. 7, four different ROIs are set according to the left and right positions, but the ROI is not limited to this, and two or three types of ROIs may be set, and five or more types of ROIs are set. You may. Of course, the region of interest ROI may be set linearly according to the position of the eye examination unit 2.

The control unit 70 changes the set position of the region of interest ROI in the captured image based on the position information of the eye examination unit 2 from the position detection sensor 75, and controls the auto gain based on the image signal in the changed region of interest ROI. May be done. As a result, the region of interest ROI is set at an appropriate position regardless of the position of the eye examination unit 2, and auto gain can be appropriately performed.

Further, the control unit 70 can adjust the gain according to the formation position of the face in the captured image by setting the region of interest ROI according to the position of the eye examination unit 2 and performing the auto gain control. It is even more advantageous. That is, the illumination state by the face illumination differs depending on the position of the eye examination unit 2, and it may be necessary to adjust the captured image. Therefore, by using the region of interest ROI set according to the position of the eye examination unit 2 as described above and performing the auto gain again, it is possible to respond to changes in the lighting state, and the eye using the face photographing unit 90 can be dealt with. Alignment by detection can be performed well. Further, by changing the position of the area of interest ROI according to the position of the eye examination unit 2, the brightness of the face becomes a constant target level regardless of the position of the eye examination unit 2 regardless of the brightness of the external lighting or the like. It is adjusted to, and it is possible to reliably detect the eye and the like.

In the above description, the position of the region of interest ROI is changed according to the position of the eye examination unit 2 to perform auto gain control, but the present invention is not limited to this, and the position of the eye examination unit 2 is not limited to this. , The lighting state by face lighting may be changed.

FIG. 8 is a diagram showing an example of changing the face illumination according to the position of the eye examination unit 2. For example, when the eye examination unit 2 moves to the left eye side, the face illuminations 1 to 3 are turned on, the face illuminations 4 to 6 are turned off, and when the eye examination unit 2 moves to the right eye side, the face illumination unit 2 is turned off. , Face lighting 4 to 6 may be turned on and face lighting 1 to 3 may be turned off. Further, when the eye examination unit 2 is arranged at the center position, the face illuminations 1 to 6 may be turned on. Further, the lighting / extinguishing of a plurality of illumination light sources is not limited, and the amount of illumination light of the light sources may be adjusted. Further, the arrangement configuration of the face lighting is not limited to the above, and various transformations are possible. In addition, the amount of light may be controlled independently for each of the face illuminations 1 to 6 according to the position of the eye examination unit 2.

<Setting of area of interest ROI in eye detection>
Next, an example of performing eye detection in the face image analysis in step S110, step 150, step 180, etc. will be described.

The control unit 70 sets the region of interest ROI based on the position of the eye examination unit to the captured image of the face photographing unit 90. In this case, the region of interest ROI is the region of interest set for performing eye detection and is set as the image search range in eye detection.

In this embodiment, the eye is detected from the face photographing unit 90. In eye detection, if the search range on the captured image can be limited, for example, the detection process can be speeded up and erroneous detection (detecting different objects as eyes) can be reduced. In this case, since the position of the eye changes depending on the position of the eye examination unit 2, the region of interest (image search range) ROI is set based on the three-dimensional position information of the eye examination unit 2, and the eye is searched in the region of interest ROI. do.

When the face is photographed with the face placed on the chin rest 11, the range in which the eye can exist in the three-dimensional real space is limited to some extent. For example, although there are individual differences in the interpupillary distance and the size of the face, they exist within a certain range. Therefore, it is determined which region the real space region in which the eye can exist is on the captured image of the face photographing unit 90, and the region is set as the region to be searched.

In the following, the setting of the region of interest in the three-dimensional real space (the region where the eye is considered to exist), the method of converting the region of interest in the real space into the coordinates on the captured image, and the distortion of the lens of the face photographing unit 90 are considered. The method of correcting the result of the coordinate transformation and the method of obtaining the region of interest (the region where the eye is considered to exist) on the image from the result of the correction of the coordinate transformation will be described.

Here, in a three-dimensional real space, when the subject puts his / her face on the chin rest 11, a three-dimensional region in which the right eye is considered to exist is set (see FIG. 9). This solid is a solid composed of points R1 to R8, and R1 = (XR1, YR1, ZR1), ..., R8 = (XR8, YR8, ZR8), respectively. Similarly, a rectangular parallelepiped consisting of points L1 to L8 where the left eye is considered to be present is set.

For example, in the left-right direction, the position of the surface of R1-R5-R8-R4 is the position of the right eye assuming the eye to be inspected (for example, PD = 90 mm) having the largest interpupillary distance, for example, the center position of the device. It is set at a position 45 mm away from. The position of the surface of R2-R6-R7-R3 is the position of the right eye assuming the eye to be inspected (for example, PD = 30 mm) having the smallest interpupillary distance, for example, 15 mm away from the center position of the device. Set to position. The center position may be, for example, the center position of the drive range in the left-right direction of the drive unit, or the position of the left-right symmetric axis of the device main body (for example, the base, the face support portion, etc.).

Further, in the vertical direction, the position of the surface of R1-R2-R3-R4 is the upper limit of the movable range of the eye examination unit 2, and is set to a position 32 mm above the position of the eye level marker, for example. .. The position of the surface of R5-R6-R7-R8 is the lower limit of the movable range of the eye examination unit 2, and is set, for example, at a position 32 mm above the position of the eye level marker.

Further, in the Z direction, the position of the surface of R3-R7-R8-R4 is a predetermined amount (for example, for example) on the device side with respect to the average eye position when the subject is supported by the face support portion 9. 20 mm) It is set to the shifted position. Further, the position of the surface of R2-R6-R5-R1 is a predetermined amount (for example, 20 mm) on the subject side with respect to the average eye position when the subject is supported by the face support portion 9. It is set to the shifted position. Since the same method can be used for the left eye, the description thereof will be omitted.

It is determined where on the captured image the vertices of the solid of the region of interest in the real space set as described above are projected (see FIG. 10). The method of obtaining the ROI of the region of interest of the right eye in the captured image will be described. First, it is assumed that R1 to R8 are projected onto r1 to r8 on the camera, respectively. R1 = (xr1, yr1), ..., r8 = (xr8, yr8), respectively. The r1 to r8 are obtained.

When the pinhole camera model is applied, the equation between R1 = (XR1, YR1, ZR1) and r1 = (xr1, yr1) is calculated from the relational expression in which the three-dimensional coordinates are projected onto the two-dimensional coordinates on the captured image. The relationship (1) holds.

は、顔撮影部９０のカメラパラメータ行列である。カメラパラメータは、顔撮影部９０の焦点位置、光学中心を表す内部パラメータと、顔撮影部９０の向き、位置を表す外部パラメータである。事前のキャリブレーション（校正）により計測された値と、位置検出センサ７５から検出される検眼部２の３次元的な位置情報より、c11〜c34の値が得られる。式(1)を解くと、式(2)となる。

Is a camera parameter matrix of the face photographing unit 90. The camera parameters are an internal parameter representing the focal position and optical center of the face photographing unit 90, and an external parameter representing the orientation and position of the face photographing unit 90. The values c11 to c34 can be obtained from the values measured by the preliminary calibration and the three-dimensional position information of the eye examination unit 2 detected by the position detection sensor 75. Solving Eq. (1) yields Eq. (2).

Similarly, the coordinates where L1 to L8 are projected on the camera, l1 to l8, are obtained.

In this embodiment, the captured image of the face photographing unit 90 is distorted (see FIG. 11). In this embodiment, eye detection is performed on the original image (without distortion correction). Of course, distortion correction may be performed. If the effect of distortion is small, it is not always necessary to deal with distortion in image processing.

The obtained r1 to r8 are coordinates that do not consider distortion. Therefore, the coordinates r1 to r8 without distortion are converted to the coordinates r'1 to r'8 in consideration of distortion. An example of converting r1 (xr1, yr1) to r'1 (x'r1, y'r1) will be described. The relationship between r1 and r'1 is expressed by equation (3).

However, (xc, yc) is the optical center coordinates on the image, k1 to k3 are the distortion coefficients, and m is the radius from the optical center on the image.

である。

Is.

R'1 can be obtained from Eq. (3). Similarly, r'2 to r'8 are obtained. Similarly, the coordinates l1 to l8 without distortion are converted to the coordinates l'1 to l'8 considering the distortion.

The ROI of interest of the right eye on the captured image is within the smallest rectangular region including all r'1 to r'8 (see the rectangular ROI of FIG. 12). Similarly, the region of interest ROI of the left eye is within the smallest rectangular region that includes all l'1 to l'8.

Using the above method, the control unit 70 sets the region of interest ROI based on the position of the eye examination unit 2. The control unit 70 processes the image signal of the set interest region ROI to search for the eye included in the interest region ROI, and performs eye detection.

<Example of transformation>
<Parallel between face photography and anterior eye photography>
The control unit 70 may perform eye detection by the face photographing unit 90 even after the eye is detected by the anterior eye photographing optical system 60. For example, if the subject's face moves after the eye is detected by the anterior eye imaging optical system 60, the eye may not be detected in the captured image of the anterior eye imaging optical system 60, and the position of the eye may not be detected.

The control unit 70 may perform eye detection by the face photographing unit 90 in parallel even after the eye detection by the anterior eye photographing optical system 60. The eye detection by the face photographing unit 90 may be performed at all times in parallel with the eye detection by the image from the anterior eye photographing optical system 60, or may be performed alternately.

As a result, even after the eye is detected by the anterior eye photographing optical system 60, the position of the eye can be detected in a wide range by the face photographing unit 90, and it is not necessary to return to the initial position again, and smooth alignment can be performed. can. In this case, the control unit 70 may return to the eye detection by the face photographing unit 90 when the eye is not detected by the anterior eye photographing optical system 60.

1 Ophthalmology equipment 2 Eye examination unit 4 Drive unit 5 Base 6 Housing 70 Control unit 71 CPU
72 ROM
73 RAM
90 Face shooting department

Claims (1)

An ophthalmic device that inspects the eye to be inspected
An optometry unit including an eye examination means for inspecting the eye to be inspected and a face photographing means for taking a photographed image including the face of the subject.
An adjusting means for moving the optometry portion relative to the eye to be inspected,
A position detection sensor for detecting the position of the optometry means and
An arithmetic processing means for setting an area of interest based on the position of the optometry unit moved by the adjusting means on the captured image in order to detect the eye to be inspected and processing an image signal in the area of interest.
The arithmetic processing means changes the set position of the region of interest in the captured image according to the position of the optometry means detected by the position detection sensor, and captures the image based on the changed image signal in the region of interest. An ophthalmologic apparatus comprising detecting the position of an eye to be inspected in an image and controlling the movement of the eye-inspecting means by the adjusting means based on the detection result.

Images