JP6857006B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup apparatus including an optical lens and a solid-state image pickup device.

Generally, a digital camera having an optical lens and a solid-state image sensor, and having an image pickup device that converts an image of a photographed subject into image data and converts the image into an electronic signal is known. This imaging device is not only used in digital cameras, but is also incorporated as a camera module in mobile devices such as smartphones and tablets. The image quality of the image data captured by such an imaging device is deteriorated due to optical aberration, which causes the light rays from the point light source to not converge to one point and causes optical blurring. The point spread function (PSF) expresses the spread of light rays corresponding to this blur. This PSF differs depending on the position of the region of the image (distance from the center of the image: image height).

For blurring due to the above-mentioned optical aberration, for example, the following processing is performed. That is, it is common that an edge enhancement filter process (Laplacian filter or the like) is performed on an image deteriorated due to optical characteristics such as optical aberration of a lens. However, if the edge enhancement filter processing is strengthened, unnecessary overshoots and undershoots are likely to occur around the edges. In order to obtain a high-quality image over the periphery of the image, correction by PSF is required. For example, a technique has been proposed in which PSF is measured for each camera module and arithmetic processing is performed using this as correction data to correct individual differences for each image region and each camera module (for example, patent documents). 1).

Japanese Unexamined Patent Publication No. 2010-177918

On the other hand, in recent years, the zoom function has been emphasized especially in camera modules used in mobile devices such as smartphones. The optical zoom mechanism requires a lens moving mechanism in the lens unit, and the module size becomes large and expensive, and there is a risk that the optical zoom mechanism becomes vulnerable to a drop impact. From this point of view, mobile devices place importance on an image enlargement function by image processing, a so-called digital zoom function. In general digital zoom, processing such as interpolation by nearest neighbor interpolation method, linear interpolation method, third-order convolution interpolation method, edge enhancement filter processing, etc. is performed, but in edge enhancement filter processing, near the edge as described above. Overshoot and undershoot may occur in the image, resulting in an unnatural image. In addition, linear interpolation and cubic convolution interpolation basically cannot restore high frequency components lost during sampling. Therefore, it is necessary to perform a process of restoring high frequency components in addition to the interpolation process. From the above, in an image pickup apparatus that frequently uses digital zoom, it is necessary to perform processing for image deterioration due to optical characteristics and processing for image deterioration due to enlargement processing, respectively. Therefore, the amount of restoration processing becomes large. In addition, when the magnification is changed by using the digital zoom while shooting a moving image with a high frame rate, the arithmetic processing unit of the imaging device is required to have high processing power, or the magnification of the frame rate or digital zoom is changed. There is a risk that operations etc. will be restricted.

The present invention has been made in view of the above circumstances, and efficiently corrects image quality deterioration due to the optical characteristics of the lens in the image pickup apparatus and image quality deterioration due to image enlargement processing during zooming, and even a digitally zoomed image is natural. It is an object of the present invention to provide an image pickup apparatus capable of obtaining a high-resolution image.

In order to achieve the above object, the image pickup device of the present invention is an image pickup device including an optical lens and a solid-state image sensor, and the optical lens in each region of an image output by the solid-state image sensor and divided into a plurality of regions. A storage means for storing correction data calculated based on both a point spreading function according to the above and a blur function at the time of image enlargement by digital zooming of the image, and the correction data is stored from the storage means for each image to be photographed. Each of the images taken by extracting and using the extracted correction data is provided with an arithmetic processing means for performing arithmetic processing for correcting the deterioration of the image based on the point spreading function and the blur function. It is a feature.

According to such a configuration, in an image pickup apparatus in which digital zoom is performed, deterioration of the captured image due to the optical characteristics (optical aberration, etc.) of the optical lens and deterioration of the captured image based on the image enlargement processing by the digital zoom can be determined. These two types of deterioration can be efficiently corrected by using unified correction data.
That is, the digitally zoomed image is efficiently corrected by using the correction data corresponding to both the point spread function related to the deterioration of the image due to the optical characteristics of the optical lens and the blur function related to the deterioration of the image due to the enlargement processing. Can be done.

In the configuration of the present invention, in the correction data, the point spread function and the blur function are converted into frequency domains by Fourier transform, and then multiplied by each other to obtain the reciprocal of the multiplication result, and the reciprocal obtained is inverse Fourier. It is preferable that the conversion is performed to convert it into a deconvolution filter in the real space region and stored in the storage means.

According to such a configuration, an image efficiently digitally zoomed using correction data that combines a point spread function corresponding to image deterioration due to the optical characteristics of an optical lens and a blur function corresponding to image deterioration due to enlargement processing. Can be corrected.

Further, in the above-described configuration of the present invention, an edge detecting means for detecting the edge intensity in the captured image is provided.
It is preferable that the arithmetic processing means adjusts the strength of the correction by the arithmetic processing according to the edge intensity of each position of the image detected by the edge detecting means.

According to such a configuration, when the correction based on the above-mentioned point spread function and the blur function is performed, the edge intensity is low, that is, the change due to the difference in the position of the light intensity is small, and the light intensity is flat. By performing the same correction as for the portion having high edge strength in the region on the image in such a state, it is possible to prevent the noise from increasing on the contrary. The difference in the strength of the correction may be, for example, not performing the correction when the edge strength is lower than the predetermined value, but performing the correction when the edge strength is higher than the predetermined value. Further, the upper limit value and the lower limit value are set for the edge strength, the correction is not performed when the edge strength is the lower limit value, the above correction is performed when the edge strength is equal to or higher than the upper limit value, and when the edge strength is between the upper limit value and the lower limit value, The above correction may be weakened. That is, the edge strength may be divided into a plurality of stages, and the correction strength may be set so as to be inversely proportional to the strength of the edge strength.

Further, in the configuration of the present invention, a plurality of photographing units including the lens and the solid-state image sensor having different focal lengths, and a switching means for switching the output of the image from the plurality of photographing units depending on the zoom magnification at the time of photographing. With
The storage means stores the correction data corresponding to each photographing unit, and stores the correction data.
It is preferable that the arithmetic processing means performs arithmetic processing for correcting the image output from the imaging unit by using the correction data corresponding to each imaging unit.

According to such a configuration, there are a plurality of shooting units having different focal lengths, and by switching the images output from these shooting units, a plurality of shooting units can be changed so that the interchangeable lenses are exchanged. By combining image switching from the shooting unit and digital zoom, even if the performance approaches the optical zoom, the image can be corrected efficiently by correcting with the above-mentioned correction data corresponding to each shooting unit. Can be done.

According to the present invention, the point spread function, which is one of the optical characteristics of the lens used in the image pickup apparatus, efficiently reduces the resolution deterioration for each image region and the blurring generated as a result of the image enlargement processing by the zoom operation. It is possible to obtain a high-quality image more easily.

It is a block diagram which shows the image pickup apparatus of 1st Embodiment of this invention. It is a block diagram which shows the image correction calculation part of the image pickup apparatus. It is a figure which shows a part of the ideal image (A) of the resolution chart. It is a figure which shows the deteriorated image (B) obtained by photographing a part of the resolution chart. It is a graph which compares the brightness change of the image (A) and the image (B). It is a figure which shows the value which differentiated the brightness change of the image (B) with respect to the image (A). It is a figure which shows a part of the ideal image (A) of the resolution chart. It is a figure which shows the image (B) which is deteriorated by photographing and enlarging a part of the resolution chart. It is a graph which compares the brightness change of the image (A) and the image (B). It is a figure which shows the value which differentiated the brightness change of the image (B) with respect to the image (A). It is a block diagram which shows the image pickup apparatus of the 2nd Embodiment of this invention. It is a figure which shows the image taken with one lens. It is a figure which shows the restored image of the image of FIG. 6A. It is a figure which shows the image taken with the other lens. It is a figure which shows the restored image of the image of FIG. 6C.

Hereinafter, embodiments of the present invention will be described.
(First Embodiment)
First, the first embodiment of the present invention will be described.
As shown in FIG. 1, in the image sensor according to the first embodiment of the present invention, an optical lens 1 for forming an image of light from a subject and an image of the subject are formed on a light receiving surface by the optical lens 1. When this is done, the solid-state image sensor 2 that converts light into signal charge, generates image data, and outputs an image signal indicating this image data, and the image signal output from the solid-state image sensor 2 are demosaic processed (inside). A demosaic unit 3 that performs insertion processing), an enlargement processing unit 4 that magnifies the demosaic processed image data as a zoom, a YC separation unit 5 that converts an RGB signal indicating image data into a brightness signal and a color difference signal, and image data. It is provided with a correction data holding unit 6 which is a storage circuit (memory) for holding correction data for restoring the data.

Further, the image pickup apparatus includes an image correction calculation unit 7 that performs a correction calculation using the correction data stored in the correction data holding unit 6 for the above-mentioned luminance signal, and a color difference signal in the corrected luminance signal. A composition that combines and outputs a timing adjusting unit 8 that matches the timing, a noise reducing unit 9 that reduces unnecessary noise generated by processing in the image correction calculation unit 7, and a corrected luminance signal and a delayed color difference signal. A unit 10 is provided.

The image correction calculation unit 7 described above includes an image data holding unit 11 which is a storage unit for holding N × N image data required for the image correction calculation, a deconvolution calculation unit 12 which performs deconvolution filter processing, and the deconvolution calculation unit 12. Pixels from the output values of the edge detection unit 13 that detects the edge strength of the image data, the brightness difference calculation unit 14 that calculates the brightness difference in the N × N image data, and the deconvolution calculation unit 12 and the edge detection unit 13. An addition unit 15 for calculating a value and a color matrix adjustment unit 16 for adjusting a color are provided.

The optical lens 1 may be a lens group including one lens or a plurality of lenses. The solid-state image sensor 2 may use, for example, a CCD, CMOS, or the like. Further, the solid-state image sensor 2 is provided with a color filter, generates a signal charge corresponding to each color of the color filter, and digitizes the value of the signal charge through a built-in analog-digital conversion circuit to obtain image data. It has become.

The demosaic unit 3 performs demosaic processing of image data output from the solid-state image sensor 2, and calculates pixel values of colors other than the color of each pixel of the color filter according to the data of surrounding pixels (interpolation: inside). For example, the blue and green values in the pixels where the color filter is red are calculated, the blue and red values are calculated in the pixels where the color filter is green, and the red in the pixels where the color filter is blue. , The green value is calculated.

The enlargement processing unit 4 enlarges the image data demosaic-processed by the demosaic unit 3 at an arbitrary magnification according to the zoom command a based on the user's operation or the like. In that case, the number of pixels is increased by interpolation. If the zoom command a does not instruct the enlargement processing unit 4, for example, the enlargement processing unit 4 outputs the input image as it is.
The YC separation unit 5 converts RGB data as image data output from the enlargement processing unit 4 into luminance and color difference data, and utilizes the fact that the human eye is sensitive to luminance to obtain luminance (Y). This is for performing the correction process described later only.

The correction data holding unit 6 is a storage circuit (memory) for holding correction data for restoring the obtained image data, and is based on the zoom command a and the position signal b indicating the position on the image data. The retained correction data is switched and output. Here, the zoom command a includes an enlargement ratio when zooming the image data. Further, the position signal b indicates, for example, the position on the image data by coordinates or the like, and indicates the position on the image data before zooming. When zooming, when the base point of enlargement of the original image is always the same position of the original image, the position serving as the base point may be used as the origin of the coordinates, or the position may be represented by the distance from the base point.

As described later, the correction data is a combination of optical correction data that restores deterioration of image data due to optical characteristics such as optical aberration of the lens and magnification correction data that restores deterioration of image data due to enlargement processing as a digital zoom. The correction data, which differs depending on the position on the original image data and the enlargement ratio, is stored in the correction data holding unit 6. When the enlargement ratios are different, the correction data corresponding to the enlargement ratio may be calculated from the reference correction data with the parameters corresponding to the enlargement ratio.

The image correction calculation unit 7 reads the correction data from the correction data holding unit 6 at the same time as reading the luminance signal of the image data from the YC separation unit 5, and corrects the image data based on the content thereof. The details of the image correction calculation unit 7 will be described later.
The timing adjustment unit 8 performs delay processing in order to match the timing of the color difference signal output from the YC separation unit 5 with the image correction calculation unit 7.

The noise reduction unit 9 reduces unnecessary noise generated by the processing in the image correction calculation unit 7, and a median filter, an epsilon filter, a bilateral filter and the like are appropriately used, but the brightness information (absolute value of brightness) of peripheral pixels (absolute value of brightness) is used. , Brightness difference, etc.) and the filter strength may be changed adaptively. Such noise reduction processing can be omitted in some cases.
The compositing unit 10 synthesizes the restored luminance signal output from the noise reducing unit 9 and the color difference signal output from the timing adjusting unit 8 and converts them into an image format as required, such as RGB, YUV, and RAW. Then, the image data is output as the final output from the image device.

As shown in FIG. 2, the image data holding unit 11 in the image correction calculation unit 7 stores the brightness signal of the image data from the YC separation unit 5, and is an image of N × N pixels required for the image correction calculation. Data is stored in each pixel.
The deconvolution calculation unit 12 multiplies the N × N brightness signal input from the image data holding unit 11 and the correction data read from the correction data holding unit 6 for each element, and further adds the above-mentioned N. The pixel value (output value d of the deconvolution calculation unit 12) at the center position in the image data of × N is obtained. In this embodiment, N = 9.

Or falling edge of edge detection section 13 uses the center portion M × M of the data of the image data holding unit 11, the image, the vertical, to calculate the horizontal edge strength, which output value e of the edge detection unit 13 It is supposed to be. A Sobel filter with M = 3 is used for edge detection, and the squared average in the vertical and horizontal directions is calculated. The edge strength may be obtained by using another algorithm.
From the output values of the deconvolution calculation unit 12 and the edge detection unit 13, the addition unit 15 calculates the pixel value of the pixel of interest by the following algorithm.

here,
d: Output value of deconvolution calculation unit 12 (corrected pixel value)
e: Output value of edge detection unit 13
I: Pixel value of interest (pixel value before correction)
th_H, th_L: Edge strength threshold α: Mixing ratio of deconvolution calculation result and original image The parameters such as th_H, th_L, α can be determined as appropriate, and can be determined at the time of design, trial production, product manufacturing, etc. In the above, it is appropriately determined while looking at the image, and in some cases, it may be set to be adaptively changeable depending on the shooting scene.
The processing in the addition unit 15 is performed after the image correction processing in the deconvolution calculation unit 12 described later, and is for changing the correction strength according to the output value e of the edge detection unit 13. is there. As described above, the upper and lower threshold values th_H and th_L are set in the output value e of the edge detection unit 13. When the output value e of the edge strength is equal to or less than the lower threshold value th_L, the output value from the addition unit 15 is set as the pixel value I of the pixel of interest in the image data before correction. That is, the pixel value of the image data is not corrected in the pixel in which the output value e of the edge strength in each pixel of the image data is equal to or less than the lower threshold value th_L.
When the output value e of the edge strength is equal to or higher than the upper threshold value th_H, the output value from the addition unit 15 is set to the pixel value of the pixel of interest in the corrected image data, that is, the output value d of the deconvolution calculation unit 12. That is, the pixel value is corrected in the pixel in which the output value e of the edge strength in each pixel of the image data is equal to or higher than the upper threshold value th_H.
When the output value e of the edge strength is between the upper threshold value th_H and the lower threshold value th_L, the output value d from the addition unit 15 is the pixel value I of the pixel of interest of the image data before correction and after correction. The value is a combination of the output value d of the deconvolution calculation unit 12 and the deconvolution calculation unit 12. That is, in the pixel in which the output value e of the edge intensity in each pixel of the image data is between the lower threshold value th_L and the upper threshold value th_H, the pixel value is the pixel value I before correction and the pixel value after correction. The value should be between.

The edge strength threshold is set to one, and when the threshold is equal to or higher than the threshold, the output value from the addition unit 15 is set to the pixel value of the pixel of interest of the corrected image data, that is, the output value d of the deconvolution calculation unit 12, and the edge. When the intensity output value e is equal to or less than the threshold value, the output value from the addition unit 15 may be set as the pixel value I of the pixel of interest in the image data before correction. Further, the threshold value may be set to 3 or more, and the pixel value obtained by synthesizing the pixel value before correction and the pixel value after correction in a plurality of stages may be applied between the pixel value before correction and the pixel value after correction. .. In this way, in the part where the edge strength is low, the correction is not performed or the correction strength is lowered, so that the correction is performed in the part where the change due to the difference in the brightness position is small, and the change in brightness is rather large. It prevents noise from being generated.

The output from the addition unit 15 (image correction calculation unit 7) is combined with the color difference signal from the YC separation unit 5 by the composition unit 10 and returned to the RGB signal. After that, the color matrix adjusting unit 16 adjusts the colors to obtain the final image data.

Next, the creation of image correction data using the point spread function (PSF) of the lens and the blur function by enlargement will be described using a conceptual diagram.
FIG. 3A is a subject obtained by enlarging a part of the resolution chart to be the subject, in other words, an ideal image (A) of the subject without deterioration. FIG. 3B is an image (B) in which the same portion is photographed by an imaging device and is not processed at all, but is deteriorated. FIG. 3C corresponds to the case where the images (A) and (B) are scanned as shown by the arrows in FIGS. 3A and 3B, and the light in the ideal image (A) when the original subject is photographed. The change in the intensity (luminance) of the above depending on the position on the arrow and the change in the intensity (luminance) of the light in the enlarged image (B) depending on the position on the arrow are shown one-dimensionally for each pixel. The horizontal axis indicates each pixel on the arrow in pixel units, and the vertical axis is brightness (intensity), but the intensities are standardized so as to have the same scale. FIG. 3D is a derivative of the brightness change of the image (B) of FIG. 3B with respect to the image (A) of FIG. 3A, corresponds to the point spread function of the lens, and is spread and distributed on the light receiving surface of the solid-state image sensor 2. .. That is, it corresponds to the convolution of FIG. 3D in the real space with the change in brightness of FIG. 3 (A), and this is the image (B) of FIG. 3B which is an actually captured image.
To restore the obtained image (B) to the ideal image (A), Fourier transform the brightness change of the image B shown in FIGS. 3D and 3C and expand it into the spatial frequency domain, and the value shown in FIG. 3D. It is generally known that the inverse Fourier transform should be performed after dividing by.

On the other hand, when enlarging an image by zooming, for example, if the size of the image is enlarged while keeping the original number of pixels, the area of one pixel is enlarged. "In this case, even if the image is enlarged, the resolution of the image appears to be low because the number of pixels is the same. Therefore, when the number of pixels is increased by interpolation, the enlarged image and the original image are made the same size. By comparison, the pixels of the image enlarged by interpolation become smaller than the pixel size of the original image.
As shown in FIG. 4, it is necessary to increase the spatial frequency because the pixel size of the enlarged image is relatively smaller than that of the original image. However, in the generally used simple interpolation processing, the original sampling is performed. The frequency has dropped, and the resolution appears to be degraded.
Here, FIGS. 4A and 4B show an image that becomes a part of the resolution chart, FIG. 4A shows an image (A) which is an ideal original image before enlargement, and FIG. 4B shows an image after enlargement. The image (B) which is the image of is shown. 4 (C) shows the change in brightness (light intensity) when the images (A) and (B) are scanned as shown by the arrows in FIGS. 4A and 4B, and the horizontal axis is the image ( The pixels at each position on the arrows A) and (B) are shown, and the vertical axis shows the brightness.

The deterioration of the resolution is caused by the gradual change in brightness as shown in FIG. 4C, and the blur function as shown in FIG. 4D is convoluted in the image (A) of FIG. 4A, which is enlarged in FIG. 4B. Image (B), which means that the resolution has deteriorated. Therefore, in the same manner as the correction of the resolution deterioration by the point spread function of the lens described above, the value of the brightness change of the image (B) of FIG. 4B and the value of the blur function of FIG. 4D are Fourier transformed and divided by the value of the blur function of FIG. 4D. If it is returned by the inverse Fourier transform, an enlarged image having substantially the same resolution as the image (A) can be obtained.

The formula is expressed as follows.
G: Luminance distribution of the subject, I: Luminance distribution of the captured image, P: Point spread function
I = G * P (1)
F (I) = F (G * P) = F (G) × F (P) (2)
G = F ^-1 (F (I) / F (P)) = I * F ^-1 (1 / F (P)) (3)
The symbol * represents the convolution integral, and F () and F ^-1 () represent the Fourier transform and the inverse Fourier transform, respectively. Generally, F ^-1 (1 / F (P)) is called a deconvolution filter.

Further, when the image is enlarged, it is expressed as follows.
I'= I * B = G * P * B (4)
I': Enlarged image, B: Blurred function due to enlargement
To restore G from I', do the following.
G = I * F ^-1 (1 / F (P)) * F ^-1 (1 / F (B)) (5)
Since the deconvolution filter is usually expressed by a matrix of (2N + 1) × (2N + 1), the arithmetic processing of one pixel requires multiplication processing and addition processing (2N + 1) × (2N + 1) times. Become.

At the time of image enlargement, it is necessary to sequentially perform the above-mentioned arithmetic processing twice, and further, in order to obtain the result of the first-stage convolution integration, it is necessary to wait for at least the next N columns of image data input. In the present invention, the number of calculations can be reduced by modifying the equation (5) as follows and calculating and holding the correction data (deconvolution filter) according to the magnification in advance.
= I * F ^-1 (1 / (F (P) × 1 / F (B))) = I * F ^-1 (1 / (F (P) × F (B))) (6)

As the point spread function of the lens, an optical design value, an actually measured value after assembling the imaging device, or a replacement with an appropriate function based on these can be appropriately adopted. Suitable functions are Gaussian functions, Lorenz functions, suitable polynomials, and so on. When using the optical design value and the measured value, the optimum value can be selected by appropriately correcting the spread width while observing the image quality.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
As shown in FIG. 5, the image pickup device according to the second embodiment of the present invention is the image pickup device of the first embodiment shown in FIG. 1, in addition to the optical lens 1 and the solid-state image sensor 2. In addition to adding the optical lens 1'and the solid-state image sensor 2', a switching circuit 17 for switching between the output signal from the solid-state image sensor 2 and the output signal from the solid-state image sensor 2'is added. An imaging unit is composed of an optical lens 1 and a solid-state image sensor 2, and an optical lens 1'and a solid-state image sensor 2', respectively.
The optical lens 1 and the optical lens 1'have different viewing angles or focal lengths, and have different optical magnifications. For example, if the focal length of the optical lens 1'is 1: 2 with respect to the optical lens 1, it corresponds to a 2x zoom optically, and if the zoom ratio Z of the entire image sensor is 1 ≧ Z> 2, it is optical. The switching circuit 17 switches so that the signal is output from the solid-state image sensor 2 provided with the lens 1 and the signal is output from the solid-state image sensor 2'equipped with the optical lens 1'when Z ≧ 2. Other configurations are the same as those of the image pickup apparatus of the first embodiment.

The solid-state image sensors 2 and 2'may have the same specifications or different specifications, but in the latter case, it is necessary to adjust the specifications such as adjusting the zoom ratio.
Also in the second embodiment, the above-described processing performed in the first embodiment is performed on the image data captured by each photographing unit. Therefore, in the imaging device of the second embodiment, for example, by switching the photographing unit that outputs an image or switching the signal of the image output from each of the two imaging units, the number of imaging units is increased stepwise. The image magnification can be changed optically, and when the magnification is continuously changed using the digital zoom, correction based on the point spread function based on the optical characteristics and the blur function by the enlargement process as described above. By creating and storing the data, it is possible to efficiently restore the deterioration of the image due to the optical characteristics and the enlargement processing by using this correction data.

FIG. 6 shows a part of an image obtained by the image pickup apparatus according to the present invention.
FIG. 6A is a normal image obtained by combining the optical lens 1 and the solid-state image sensor 2, FIG. 6B is a corrected image obtained by correcting the normal image by the correction method according to the first embodiment, and FIGS. 6C and 6D have a zoom ratio of 2. It is an image on the optical lens 1'side in the case of, and similarly, a normal image and a corrected image are shown.
In either case, it can be seen that good image quality is obtained by the present invention.

In the image pickup apparatus of each of the above-described embodiments, even if the arithmetic processing portion for processing the output signal from the solid-state image pickup element 2 (2') is inside the camera module serving as the image pickup apparatus, it is outside the camera module. In this case, the image sensor may be composed of a camera module and an external arithmetic processing portion thereof. Further, the arithmetic processing part may be a dedicated circuit, a programmed general-purpose circuit, or a combination of the dedicated circuit and the general-purpose circuit. Further, when the imaging device is incorporated in an electronic device having a general-purpose arithmetic processing device such as a smartphone or tablet, all or a part of the above-mentioned arithmetic processing portion is operated based on an application (program) on the electronic device side. It may be an arithmetic processing unit on the electronic device side.

1 Optical lens 2 Solid-state image sensor 6 Correction data holding unit (storage means)
7 Image correction calculation unit (calculation processing means)
13 Edge detection unit (edge detection means)
17 Switching circuit (switching means)

Claims (4)

In an image pickup device including an optical lens and a solid-state image sensor, when the point spreading function of each region of the image output by the solid-state image sensor and divided into a plurality of regions by the optical lens and the image enlargement by digital zooming of the image are performed. A storage means for storing correction data calculated based on both the blur function and the correction data extracted from the storage means for each image to be imaged, and the image was taken using the extracted correction data. An image pickup apparatus comprising: for each image, an arithmetic processing means for performing arithmetic processing for correcting deterioration of the image based on the point spreading function and the blur function. In the correction data, the point spread function and the blur function are each converted into a frequency domain by Fourier transform, then multiplied by each other to obtain the reciprocal of the multiplication result, and the reciprocal obtained is subjected to inverse Fourier transform to perform the inverse Fourier transform in the real space domain. The imaging device according to claim 1, wherein the image pickup device is converted into the decomplication filter of the above and stored in the storage means. An edge detecting means for detecting the edge intensity in the captured image is provided.
The calculation processing means according to claim 1 or 2, wherein the calculation processing means adjusts the strength of correction by the calculation processing according to the edge strength of each position of the image detected by the edge detection means. Imaging device. A plurality of photographing units including the lens and the solid-state image sensor having different focal lengths, and a switching means for switching the output of the image from the plurality of photographing units according to the zoom magnification at the time of photographing.
The storage means stores the correction data corresponding to each photographing unit, and stores the correction data.
The calculation processing means according to any one of claims 1 to 3, wherein the calculation processing means performs calculation processing for correcting the image output from the shooting unit by using the correction data corresponding to each shooting unit. The imaging apparatus described.

Images