JP2021110778A - Imaging device and control method therefor - Google Patents

Abstract

To generate an imaged image in which blurs are suppressed.SOLUTION: An imaging device comprises: an imaging optical system constituted so as to be capable of forming into an image the light of visible ray and infrared ray regions; adjustment means for adjusting a focus position by the imaging optical system; image processing means for converting the optical image of visible ray and infrared ray regions formed on an imaging element by the imaging optical system into an image signal; determination means for determining the presence of blurs in the image indicated by the image signal; and control means for executing a prescribed process for suppressing blurs in the image indicated by the image signal. The determination means determines the presence of blurs on the basis of the image signal obtained after the adjustment of a focus position by the adjustment means, and the control means executes the prescribed process when it is determined that there is a blur.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup apparatus and a control method thereof.

Conventionally, there has been known a technique of controlling the depth of field by adjusting the focus position and the aperture to make the blur less noticeable (see, for example, Patent Document 1).

JP-A-2017-3749

The present invention has been made in view of such a problem, and an object of the present invention is to make it possible to generate an captured image in which blurring is suppressed.

In order to solve the above-mentioned problems, the image pickup apparatus according to the present invention has the following configurations. That is, the image pickup device
An imaging optical system configured to be able to image light in the visible light region and infrared light region,
An adjustment means for adjusting the focus position by the imaging optical system and
An image processing means for converting an optical image in a visible light region and an infrared light region imaged on an image sensor by the image pickup optical system into an image signal.
A determination means for determining the presence or absence of blur in the image indicated by the image signal, and
A control means for executing a predetermined process for suppressing blurring in the image indicated by the image signal, and
Have,
The determination means determines the presence or absence of blur based on the image signal obtained after the adjustment of the focus position by the adjustment means.
The control means executes the predetermined process when it is determined by the determination means that there is a blur.

It is a block diagram which shows the functional structure of the image pickup apparatus which concerns on 1st Embodiment. It is a flowchart of blur control in a captured image. It is a figure explaining the method of determining the presence or absence of blur in a captured image. It is a figure explaining the switching control of a shooting mode. It is a figure which shows the correspondence of the shooting mode according to the surrounding environment. It is a detailed flowchart of the blur suppression process in 2nd Embodiment. It is a figure which shows exemplary the axial chromatic aberration of a lens, and the spectral sensitivity of an image sensor.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

(First Embodiment)
When using an image pickup device that uses a lens, the focus position (focus plane) is generally different between visible light and infrared light (there is a deviation in the optical axis direction). Therefore, when the visible light region and the infrared light region are simultaneously imaged (when the image is taken in the night mode), blurring occurs. Therefore, in the first embodiment, a mode for suppressing blurring in a captured image by using a shooting mode called a digital night mode will be described.

<Device configuration>
FIG. 1 is a block diagram showing a functional configuration of the image pickup apparatus according to the first embodiment. The image pickup apparatus includes an optical system control unit 105 that controls the image pickup optical system 101 and the image pickup optical system 101, an infrared control unit 106 that controls insertion and removal of the infrared cut filter 102 and the infrared cut filter 102, an image pickup element 103, and image processing. Includes part 104. Further, the image pickup apparatus includes a system controller 107 that controls the above-mentioned functional units in a centralized manner.

The subject image obtained through the image pickup optical system 101 passes through a predetermined region into which the infrared cut filter 102 is inserted and removed, and is imaged on the image pickup element 103. Although the imaging optical system 101 is represented as one lens in FIG. 1, it usually includes a plurality of lenses and also includes a mechanical structure for performing aperture, zoom, and focus control. The optical system control unit 105 controls the aperture, zoom, and focus of the imaging optical system 101 by controlling the operation of this mechanical structure. Further, the imaging optical system 101 is configured to be capable of forming light in a visible light region and an infrared light region.

The infrared cut filter 102 is a filter that attenuates infrared light and transmits visible light among the light obtained through the imaging optical system 101. The infrared cut filter 102 is configured to be removable in a predetermined region located between the image pickup optical system 101 and the image pickup element 103. The infrared control unit 106 controls the insertion / removal of the infrared cut filter 102 into a predetermined region. Here, the predetermined region is between the image pickup optical system 101 and the image pickup element 103, but it may be any one that crosses the optical path of the image pickup optical system 101. For example, it may be the front surface or the middle of the imaging optical system 101.

The image sensor 103 is an image sensor (CMOS, CCD, etc.) including a plurality of photoelectric conversion elements for converting an image (optical image) formed on the surface (on the image sensor) of the image sensor 103 into an image signal. .. As will be described in detail later, in order to be able to acquire a color image of visible light, for example, an image sensor 103 having a Bayer-arranged RGB color filter is used. As the image sensor 103, it is also possible to use an image sensor in which each pixel can output an RGB component value.

The image processing unit 104 generates image data after performing predetermined image processing such as gamma correction on the image signal input from the image sensor 103. For example, JPEG-encoded image data is generated.

The system controller 107 includes a CPU that controls the system and a memory that stores signals, and sets control instructions, parameters, and the like for each of the above blocks. The memory is used as a storage area for various purposes such as a control instruction instructed for each block, a set parameter, a focus position, and an image. Further, in order to perform focus control, the system controller 107 acquires a focus evaluation value from the image processing unit 104 and sends a command to the optical system control unit 105 based on the acquired focus evaluation value. The focus evaluation value is a value used for performing so-called autofocus (AF), which automatically focuses. AF includes things such as contrast AF and phase difference AF. Contrast AF is a method of creating a signal from the contrast value of an image and focusing by using the high frequency component thereof. On the other hand, phase-difference AF is a method of dividing an image incident from a lens into two and focusing based on the imaging interval.

<Shooting mode>
As described above, the image sensor 103 is, for example, a color image sensor having a Bayer array RGB color filter. The image processing unit 104 is configured to be able to selectively output a color image (RGB component value) and a grayscale image (Y component value) based on the image signal input from the image sensor 103.

In particular, the imaging device according to the first embodiment is configured to be able to use a shooting mode called a digital night mode as a shooting mode in addition to the day mode and the night mode generally used in a surveillance camera. Specifically, these three shooting modes are defined as follows.
Day mode: A mode in which an infrared cut filter 102 is inserted between the image pickup optical system 101 and the image pickup element 103, and a color image signal is output from the image processing unit 104.
Night mode: A mode in which the infrared cut filter 102 is removed from between the image pickup optical system 101 and the image pickup element 103, and a grayscale image signal is output from the image processing unit 104.
Digital night mode: A mode in which an infrared cut filter 102 is inserted between the image pickup optical system 101 and the image pickup element 103, and a grayscale image signal is output from the image processing unit 104.

<Operation of the device>
FIG. 2 is a flowchart of blur control in the captured image. Specifically, it is a process for making blurring inconspicuous when two wavelength regions of visible light and infrared light are simultaneously imaged. Therefore, as a matter of course, this process is performed in a state where the infrared cut filter is removed (night mode).

In step S201, the system controller 107 controls the optical system control unit 105 and the image processing unit 104 to perform the AF operation. That is, control is performed so that the optimum focus position is set for at least one (generally, the visible light region) of the subject image in the visible light region or the infrared light region. In step S202, the system controller 107 acquires, for example, the maximum value Cmax of the contrast value C in the captured image (or in a predetermined region inside the captured image) as the focus evaluation value.

Here, the contrast value C is given by, for example, the following mathematical formula (1) with respect to an arbitrary region. In addition, Imax is the maximum value of the brightness in the region, and Imin is the minimum value of the brightness in the region.
C = (Imax-Imin) / (Imax + Imin) ... (1)

In step S203, the system controller 107 determines the presence or absence of blur using the focus evaluation value. Specifically, the focus evaluation value (Cmax) is compared with the threshold value (Th) stored in advance, and if it is equal to or less than the threshold value, it is determined that blurring has occurred, and the process proceeds to S204. On the other hand, if the focus evaluation value is larger than the threshold value, it is determined that no blurring has occurred and the process is terminated. That is, it is considered that the focus evaluation value is low because the fine pattern is lost due to the blur, and as a result, the maximum value Cmax of the contrast value is lowered.

For the sake of simplicity, an example of determining the presence or absence of blur using one threshold value Th has been described, but when the above-mentioned determination method is used, the subject has no texture (pattern, etc.) / is not clear. In addition, there is a possibility that it will be judged incorrectly. That is, it may be determined that the subject is out of focus even though it is in focus. Therefore, in practice, it is preferable to use a determination method that can deal with such a problem.

For example, it can be dealt with by setting two threshold values (Th ₁ and Th ₂ ) and using them for determining the presence or absence of blur (here, Th ₁ <Th ₂ ). Specifically, when the contrast value at the optimum focus position set in the AF operation (S201) is Th ₁ or more and Th ₂ or less, it is determined that blurring has occurred, and blurring suppression processing (S204) is performed. To decide. Then, when the contrast value is _{less than Th 1,} it is determined that the subject is in a low contrast state due to the lack of texture. If the contrast value is _{larger than Th 2,} it is simply determined that no blurring has occurred. That is, when the contrast value is less than _{Th 1 or larger than Th 2} , it is determined that the blur suppression process (S204) is not performed.

Generally, the contrast value is about 0.7 when blurring occurs, and about 0.3 when the subject has no texture. Therefore, Th ₁ may be set as a value of 0.2 to 0.4, and Th ₂ may be set as a value of 0.6 to 0.8.

The determination of the presence or absence of blur is not limited to the above-mentioned method, and for example, a contrast value for all focus positions may be acquired and the determination may be made based on the outline shape thereof. Specifically, a method that utilizes a change in the contrast value depending on the presence or absence of blurring may be used.

FIG. 3 is a diagram illustrating a method of determining the presence or absence of blur in a captured image. The graphs of FIGS. 3 (a) to 3 (c) show changes in the contrast value with respect to the focus position, the horizontal axis is the focus position, and the vertical axis is the contrast value. In particular, FIG. 3A shows a case where the subject has a texture and no blur (a predetermined amount or less), and FIG. 3B shows a case where the subject has a texture and a blur (greater than a predetermined amount). 3 (c) indicates the case where the subject has no texture.

As can be understood from the graphs of FIGS. 3A and 3B, when there is no blur, the difference in contrast value between when the subject is out of focus and the optimum focus position is large, and when there is blur, this is the case. The difference becomes smaller. Further, as can be understood from the graph of FIG. 3C, when the subject has no texture, the contrast value does not change much regardless of the focus position.

Therefore, it is possible to determine the presence or absence of blurring based on the difference between the maximum value and the minimum value of the focus evaluation value at each focus position during the focus position adjustment in the AF operation (S201). Specifically, when the difference between the maximum value and the minimum value is Th ₃ or more and Th ₄ or less, it is determined that blurring has occurred (Th ₃ <Th ₄ ). Then, when it is less than _{Th 3 or larger than Th 4} , it is determined that no blurring has occurred. This determination method has an advantage that the presence / absence of blurring can be determined (S202 to S203) at the same time as the AF operation (S201), and the presence / absence of blurring can be determined even when the optimum focus position is unknown. ..

In step S204, the system controller 107 executes the blur suppression process. Specifically, the infrared control unit 106 is controlled to insert the infrared cut filter 102 into a predetermined region, and the operation mode is changed to the digital night mode.

Compared with the day mode, the digital night mode has lower color noise, so that the low illuminance performance (image quality in a low illuminance environment) is improved. Further, in the digital night mode, as compared with the night mode, the light incident on the image sensor 103 is reduced (infrared light component is cut), so that the noise is increased (S / N is lowered). However, since the light incident on the image sensor 103 is limited to the visible region, the blur of the subject image in the infrared light region does not exist (or is inconspicuous).

FIG. 4 is a diagram illustrating switching control of the shooting mode. The switching control is executed by the system controller 107 after shifting to the digital night mode by the blur suppression process (S204).

The determination of the operation mode change is made based on the change in the amount of light in the surrounding environment. For example, when operating in the digital night mode (S401), the average brightness value B of the image indicated by the image data output from the image processing unit 104 is compared with a predetermined brightness threshold value, and based on the comparison result. Change the operation mode. The image signal output from the image pickup device 103 may be used instead of the image data output from the image processing unit 104.

Specifically, in S402, the average luminance value B of the image and Th _H are compared, and if the average luminance value B of the image _{exceeds Th H} , the mode is changed to the day mode. That is, since the surrounding environment has changed to a sufficiently bright environment, it is judged that it is preferable to output a color image.

Then, in S403, the average luminance value B of the image and Th _L are compared, and when the average luminance value B of the image _{is lower than Th L} , the mode is changed to the night mode. That is, since the surrounding environment has changed to a darker environment, it is judged that it is preferable to output an image captured with less noise that allows blurring due to night mode than an image captured with increased noise in digital night mode. .. It is preferable that Th _H > Th _L , and _{the values of Th H} and Th _L are stored in the memory in advance.

FIG. 5 is a diagram showing correspondence of shooting modes according to the surrounding environment. Here, an example is shown in which the contrast value C of the image is considered in addition to the average brightness value B of the image. The average luminance value B is the average luminance of a plurality of pixels included in an arbitrary region, and is calculated by the image processing unit 104. In FIG. 5, when C> Th and Th _L <B <Th _H , it is described as "arbitrary", but for example, the user can arbitrarily set it according to the use case and the parameter to be emphasized. It is shown that. Further, each threshold value may be arbitrarily set by the user.

As described above, according to the first embodiment, when there is a blur in the captured image obtained by imaging the wavelength regions of visible light and infrared light, the operation is shifted to the digital night mode and the blur is suppressed. Output the captured image. As a result, it is possible to obtain an captured image in which blurring is suppressed, although noise is slightly increased as compared with the captured image obtained by the night mode.

In the above description, the amount of light in the surrounding environment is determined based on the captured image obtained when the image is captured in the night mode, and the mode is changed to the optimum shooting mode. However, the judgment of the surrounding environment is not limited to this form. For example, an optical sensor (not shown) that measures the infrared light illuminance of ambient light is separately provided, and the presence or absence of blur is determined based on the output of the optical sensor and the image signal in the day mode, and the mode is directly changed to the optimum mode. It may be configured.

Further, since the higher the S / N in the acquired image or the smaller the dispersion value, the brighter the image, the determination corresponding to S402 and S403 may be performed based on these values. Here, the dispersion value is the dispersion of the pixel signal values, and the S / N is a ratio in which the signal is the pixel value and the noise is the dispersion value.

(Second Embodiment)
In the second embodiment, a mode in which blurring in the captured image is suppressed by controlling the amount of infrared light that illuminates the subject will be described.

<Device configuration>
The imaging device according to the second embodiment is different from the first embodiment in that the infrared control unit 106 further controls an illumination device (not shown) that irradiates infrared light in the photographing direction (that is, the subject). Specifically, the infrared control unit 106 is configured to be able to adjust the output of infrared light projected by the lighting device.

<Operation of the device>
The blur control in the captured image in the second embodiment is the same as that in the first embodiment (FIG. 2). However, as described above, the details of the blur suppression process (S204) are different from those of the first embodiment.

FIG. 6 is a detailed flowchart of the blur suppression process (S204) in the second embodiment. As described in the first embodiment, when one of the visible light region and the infrared light region is dominant, blurring in the captured image in the night mode is suppressed, and a clear captured image is acquired. Therefore, in the process of FIG. 6, the output of infrared light by the lighting device is adjusted so that one of the visible light region and the infrared light region becomes dominant.

In step S601, the system controller 107 instructs the infrared control unit 106 to maximize the output of infrared light from the lighting device. The following S602 to S605 are processes for changing the spectral ratio of ambient light by controlling the output of infrared light by the lighting device.

In step S602, the system controller 107 performs an AF operation to adjust the focus position. In step S603, the system controller 107 determines the presence or absence of blur as in S203. If it is determined that blurring has occurred, the process proceeds to S604, and if it is determined that blurring has not occurred, the process ends.

In step S604, the system controller 107 determines whether or not the S / N is equal to or higher than a predetermined threshold value. If it is equal to or higher than the predetermined threshold value, it is considered that the image deterioration due to blur is greater than the image deterioration due to noise. Execute the process. On the other hand, if it is less than the threshold value, the image deterioration due to noise is considered to be greater than the image deterioration due to blur, so the process is terminated. It is desirable that the above-mentioned predetermined threshold value is set in advance according to the illuminance environment in which the image deterioration due to noise is more conspicuous than the image deterioration due to blur.

In the above description, the method of maximizing the infrared light output of the lighting device and then gradually reducing it has been described, but the present invention is not limited to this, and a method of minimizing the infrared light output and gradually increasing the infrared light output may be used. In this case, it is desirable that the minimum infrared light output is an output that is at least sufficient for imaging (for example, an AF operation is possible).

Further, the infrared light output may be set to a non-maximum output, the AF operation in S602 may be performed, and the illumination output may be increased or decreased when there is a blur. In this case, the focus position after the focus adjustment before changing the infrared light output is referred to. Then, when the difference between the focus position and the focus position of the illumination wavelength is less than a predetermined difference, the infrared light output is strengthened, and when the difference is more than a predetermined difference, the infrared light output is weakened. , Blur can be suppressed quickly. The predetermined difference at this time may be any value, and may be, for example, the depth of focus according to the aperture value at the time of shooting.

As described above, according to the second embodiment, it is possible to suppress blurring in the captured image in the night mode by controlling the amount of infrared light that illuminates the subject. It is also effective when the infrared illumination is turned on when a clear image is desired and blurring occurs as a result.

(Third Embodiment)
In the third embodiment, a mode in which blurring in the captured image is suppressed by adjusting the RGB composition ratio when generating the luminance (Y) signal in the grayscale image will be described. Since the device configuration is the same as that of the first embodiment (FIG. 1), detailed description thereof will be omitted.

<Operation of the device>
The blur control in the captured image in the third embodiment is the same as that in the first embodiment (FIG. 2). However, as described above, the details of the blur suppression process (S204) are different from those of the first embodiment.

FIG. 7 is a diagram exemplifying the axial chromatic aberration of the lens and the spectral sensitivity of the image sensor 103 (RGB color image sensor). FIG. 7A exemplifies the characteristics of the axial chromatic aberration of the lens, and it can be understood that the larger the difference from the reference wavelength (for example, 550 nm), the larger the magnitude (aberration amount) of the aberration. On the other hand, FIG. 7B exemplifies the spectral characteristics of the R pixel, G pixel, and B pixel of the image sensor 103. Here, the R pixel, the G pixel, and the B pixel mean, for example, the pixels corresponding to the R, G, and B color filters in the image sensor 103. When an image sensor capable of outputting an RGB component value is used for each pixel, it means each RGB component in each pixel.

As can be understood from FIG. 7B, in the visible light wavelength region (for example, 350 to 750 nm), the sensitivity characteristics differ greatly depending on the pixel type (R pixel, G pixel, B pixel). On the other hand, in the wavelength region of infrared light (for example, from 800 nm), the sensitivity characteristics depending on the pixel type are almost the same.

Since the magnitude of aberration is related to the size of the scattered circle, it is expected that blurring will be suppressed by relatively strengthening the signal of the color pixel with small aberration and relatively weakening the signal of the color pixel with large aberration. can. Therefore, the signal is emphasized / suppressed by adjusting the RGB composition ratio when creating the Y signal value.

When it is determined that no blurring has occurred, a grayscale image which is a luminance image is generated by using a general conversion formula (for example, Y = 0.3R + 0.6G + 0.1B). On the other hand, when it is determined that blurring has occurred, a grayscale image, which is a luminance image, is generated using an RGB ratio that can suitably suppress blurring calculated in advance. These processes are performed by the image processing unit 104. The RGB ratio capable of preferably suppressing blur may be stored in advance in the memory in the system controller 107. Such an RGB ratio is calculated based on the aberration information of the lens and the spectral characteristics of the image sensor.

For example, when a lens and an image sensor having the characteristics shown in FIG. 7 are used, the difference from blue light (B: around 450 nm) is small when the amount of aberration of infrared light is used as a reference, while green light (B: around 450 nm) is used. The difference in the amount of aberration from G: around 550 nm) is relatively large. Therefore, as the RGB ratio capable of suppressing blurring, a ratio in which the signal value of the B pixel is emphasized and the signal value of the G pixel is suppressed is set. That is, the degree of emphasis on the pixel signal obtained by the B pixel is set to be relatively larger than the degree of emphasis on the pixel signal obtained by the G pixel as compared with the above-mentioned general conversion formula. Specifically, B / R and B / G are set to be 1 or more.

By the way, in a general Bayer array color filter, blocks of 2 × 2 pixels of R, G, G, and B are arranged in a grid pattern. That is, the resolution is higher when the G pixel is used than when the R pixel or the B pixel is used. When such a color filter is used, if the diameter of the blur is 2 pixels or less, it is preferable to emphasize the G pixel regardless of the aberration characteristics.

In the above description, it has been described that the RGB ratio is calculated in advance based on the aberration information of the lens and the spectral characteristics of the image sensor. However, the method is not limited to this method, and the RGB ratio may be adaptively determined based on the captured image. For example, three images composed of only R pixels, only G pixels, and only B pixels are generated (demosaic processing), and the contrast values of each are calculated. Since the contrast value corresponds to the magnitude of the blur, the blur can be suppressed by using the ratio of the contrast values of the three images as it is for the RGB ratio for Y signal generation. When this processing is performed, it is not necessary to store the RGB ratio in advance, which is useful when, for example, an image pickup device of a type in which the lens can be exchanged is used.

As described above, according to the third embodiment, it is possible to suppress blurring in the captured image in the night mode by adjusting the RGB ratio when generating the grayscale image which is the luminance image.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

101 Imaging optical system; 102 Infrared cut filter; 103 Image sensor; 104 Image processing unit; 105 Optical system control unit; 106 Infrared control unit; 107 System controller

Claims (12)

An imaging optical system configured to be able to image light in the visible light region and infrared light region,
An adjustment means for adjusting the focus position by the imaging optical system and
An image processing means for converting an optical image in a visible light region and an infrared light region imaged on an image sensor by the image pickup optical system into an image signal.
A determination means for determining the presence or absence of blur in the image indicated by the image signal, and
A control means for executing a predetermined process for suppressing blurring in the image indicated by the image signal, and
Have,
The determination means determines the presence or absence of blur based on the image signal obtained after the adjustment of the focus position by the adjustment means.
The control means is an image pickup apparatus that executes the predetermined process when it is determined by the determination means that there is a blur. The adjusting means adjusts the focus position so that the contrast value in the image indicated by the image signal obtained by the image processing means is maximized.
The determination means determines that there is no blur if the contrast value in the image indicated by the image signal obtained after adjusting the focus position by the adjustment means is larger than a predetermined threshold value, and is equal to or less than the predetermined threshold value. The imaging device according to claim 1, wherein it is determined that there is blurring. The adjusting means adjusts the focus position so that the contrast value in the image indicated by the image signal obtained by the image processing means is maximized.
The determination means determines that there is blur when the contrast value in the image indicated by the image signal obtained after adjusting the focus position by the adjustment means is equal to or greater than the first threshold value and equal to or less than the second threshold value. The imaging device according to claim 1, wherein it is determined that there is no blur when the value is less than the first threshold value or larger than the second threshold value. The determination means according to any one of claims 1 to 3, further comprising determining the presence or absence of blur based on the contrast value at each focus position obtained during the adjustment of the focus position by the adjusting means. Imaging device. Further, it has an insertion / extraction means for inserting / removing an infrared cut filter into the optical path of the imaging optical system.
The control means is characterized in that, when it is determined by the determination means that there is a blur, the insertion / extraction means is controlled as the predetermined process to insert the infrared cut filter into the optical path of the imaging optical system. The imaging device according to any one of claims 1 to 4. The image sensor is a color image sensor.
The image pickup apparatus has a first mode in which the infrared cut filter is inserted into the optical path of the imaging optical system and outputs a color image signal from the image processing means, and the infrared cut filter is inserted from the optical path of the imaging optical system. A second mode of extracting and outputting a gray scale image signal from the image processing means, and a third mode of inserting the infrared cut filter into the optical path of the imaging optical system and outputting the gray scale image signal from the image processing means. The imaging device according to claim 5, wherein the mode and the mode can be switched. When the determination means determines that there is a blur, the control means has an average luminance value of an image indicated by an image signal output from the image processing means equal to or greater than a first luminance threshold value and a second luminance value. If it is below the threshold value, the image pickup device is changed to the third mode, and if the average brightness value of the image exceeds the second brightness threshold value, the image pickup device is changed to the first mode. The image pickup apparatus according to claim 6, wherein the image pickup apparatus is changed to the second mode when the average luminance value of the image is lower than the first luminance threshold value. It also has a lighting means that irradiates infrared light in the shooting direction.
When the determination means determines that there is a blur, the control means is dominated by one component of the visible light region and the infrared light region in the subject existing in the shooting direction as the predetermined process. The imaging apparatus according to any one of claims 1 to 4, wherein the output of infrared light by the lighting means is adjusted. The claim is characterized in that each time the output of infrared light by the lighting means is adjusted, the adjusting means adjusts the focus position, and the determining means determines the presence or absence of blur in the image indicated by the image signal. 8. The imaging apparatus according to 8. The image sensor is an RGB color image sensor.
The image processing means is configured to be capable of outputting a grayscale image which is a luminance image obtained by synthesizing pixel signals obtained by R pixels, G pixels, and B pixels included in the image pickup element.
The control means controls the image processing means so as to generate a luminance image by synthesizing at the first RGB ratio when it is determined by the determination means that there is no blur, and the determination means determines that there is blur. When this is done, the image processing means is controlled so as to generate a luminance image by synthesizing at a second RGB ratio.
The second RGB ratio is set so that the degree of emphasis on the pixel signal obtained by the B pixel is relatively larger than the degree of emphasis on the pixel signal obtained by the G pixel as compared with the first RGB ratio. The imaging device according to any one of claims 1 to 4, wherein the image pickup apparatus is characterized by the above. 10. The control means according to claim 10, wherein the second RGB ratio is determined based on the ratio of the contrast values of the image using only the R pixel, the image using only the G pixel, and the image using only the B pixel. Imaging device. It is a control method of an image pickup apparatus having an image pickup optical system configured to be able to form light in a visible light region and an infrared light region.
The adjustment step of adjusting the focus position by the imaging optical system and
An image processing step of converting an optical image in a visible light region and an infrared light region imaged on an image sensor by the image pickup optical system into an image signal, and an image processing step.
A determination step for determining the presence or absence of blur in the image indicated by the image signal, and
A control step of executing a predetermined process for suppressing blurring in the image indicated by the image signal, and
Including
In the determination step, the presence or absence of blur is determined based on the image signal obtained after the focus position is adjusted by the adjustment step.
The control step is a control method characterized in that when it is determined by the determination step that there is a blur, the predetermined process is executed.

Images