JP2008017109A - Portable imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable imaging device capable of reducing an influence of a hand shake. <P>SOLUTION: The extent of the hand shake is evaluated by calculating the average (average square error) of square errors of pixel values between frame application parts after and before correction processing is performed with a hand shake correcting function. Namely, average square errors generated when the correction processing is performed by using a plurality of evaluation functions prepared in an evaluation function set part 13 as the hand shake correcting function are calculated respectively, and an evaluation function having the largest calculated average square error is selected and held as an evaluation function having an optimum parameter set. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a portable imaging device that captures a still image, and more particularly to a technique for reducing the influence of camera shake.

  In the conventional camera shake correction technology, there are many examples in which the camera shake correction mechanism is configured by an optical method. For example, in the technique described in Patent Document 1, the direction of camera shake is detected, and the optical axis of the lens mechanism is aligned in conjunction with the direction of camera shake.

  There is also an example in which a specific still image (image frame) is selected from a plurality of still images constituting a moving image, and camera shake correction is performed by an electronic method. For example, in the technique described in Patent Document 2, an image frame that is less affected by camera shake is selected as an optimal still image from moving images based on various evaluation functions.

Japanese Patent No. 342440 JP 2003-259291 A

  However, Patent Document 1 using an optical technique has a problem that it is difficult to reduce the size, and thus it is difficult to apply to a portable imaging device.

  Further, Patent Document 2 relates to a conversion device from a moving image to a still image, and does not intend to perform camera shake correction. Therefore, camera shake correction processing is not performed on a selected still image. For this reason, there is a problem that the influence of camera shake on the obtained still image becomes large in an apparatus that is likely to cause camera shake, such as a portable imaging apparatus.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide a portable imaging device that can reduce the influence of camera shake.

  The portable imaging device according to the present invention includes an imaging unit that continuously captures a plurality of still images as moving images, and a selection that selects an optimal still image that has the least influence of camera shake from the moving images captured by the imaging unit. And a camera shake correction unit that performs camera shake correction on the optimum still image selected by the selection unit.

  The portable imaging device according to the present invention includes an imaging unit that continuously captures a plurality of still images as moving images, and a selection that selects an optimal still image that has the least influence of camera shake from the moving images captured by the imaging unit. And a camera shake correction unit that performs camera shake correction on the optimum still image selected by the selection unit. Therefore, the influence of camera shake can be reduced.

<Embodiment 1>
FIG. 1 is a block diagram illustrating a schematic configuration of a camera shake correction image processing system (hereinafter simply referred to as a camera shake correction system) 100 that can be mounted on the portable imaging apparatus according to the first embodiment. In FIG. 1, the flow of image data is indicated by a solid line, and the flow of control signals is indicated by a dotted line.

  The image stabilization system 100 includes a moving image data imaging unit 11 (imaging means) that captures image data (hereinafter referred to as moving image data) of a real-time moving image (a plurality of still images captured continuously) during operation. ) Continues to acquire video data. In the acquired moving image data, the starting point at which the shutter is opened and the image data of a still image (hereinafter referred to as still image data, an image frame, or simply a frame) is started is the imaging timing determination unit 12 (timing determination). Means). When the start point is determined by the imaging timing determination unit 12, the camera shake correction system 100 determines an end point at which the shutter is closed and the imaging is ended after a relatively short predetermined time (for example, 1 sec) from the start point, This predetermined time from the start point to the end point is defined as an imaging time range (imaging timing). The camera shake correction system 100 selects one optimum still image that has the smallest effect of camera shake within this imaging time range (the one that can obtain the best image quality by performing correction processing in the camera shake correction processing unit 16). Then, the optimum still image is subjected to correction processing and output. The end point (that is, the imaging time range) may be freely set according to the user's purpose.

  The moving image data imaging unit 11 is connected to an evaluation function setting unit 13 for identifying a frame having the smallest influence of camera shake from moving image data input by the moving image data imaging unit 11. The evaluation function setting unit 13 analyzes the moving image data captured by the moving image data imaging unit 11 within a predetermined imaging time range in units of frames, and selects a frame with the smallest influence of camera shake from these as an imaging candidate frame. An evaluation function is provided.

  Further, an interface unit 14 is connected to the moving image data imaging unit 11. As described above, when the imaging time range from the start point to the end point is freely set according to the user's purpose, the imaging time range designation signal input from the user is input after the shutter is opened and imaging is started. It is input to the moving image data imaging unit 11 via the unit 14.

  The evaluation function application unit 15 performs evaluation calculation for each frame included in the moving image data captured within the imaging time range.

  Next, a configuration example of the evaluation function setting unit 13 and the evaluation function application unit 15 (selection unit) will be described. In the evaluation function set unit 13, a plurality of camera shake correction functions (camera shake correction function sets) capable of setting a plurality of parameters (parameter sets) relating to camera shake are prepared in advance. The present invention is characterized in that these camera shake correction function sets for correction are applied in the evaluation function application unit 15 as an evaluation function set for evaluating and selecting still image data. Note that the evaluation function application unit 15 may apply the evaluation function set only to a partial region of the image if the processing time is too long if the evaluation function set is applied to the entire image in each frame. The same effect is obtained (hereinafter, an area to which the evaluation function set is applied is referred to as a frame application unit).

  Here, parameters that can be set in the evaluation function (that is, the camera shake correction function) in the evaluation function setting unit 13 are not only scalar signals indicating the degree of camera shake, but also the direction of camera shake, the strength of camera shake, or the size of camera shake. Information on (length in image) can also be configured.

  FIG. 2 is a diagram illustrating an evaluation function (filter) when the evaluation and correction of camera shake are limited to only one dimension. 2A shows a filter (evaluation function) A, FIG. 2B shows a filter (evaluation function) B, and FIG. 2C shows a filter (evaluation function) C. Each of the filters A to C is a one-dimensional filter including nine weighting coefficients, and the degree of camera shake increases in the order of the filters A to C. By using a one-dimensional filter as shown in FIG. 2, camera shake correction is possible even in an apparatus having only a line memory, as will be described below with reference to FIG.

  FIG. 3 is a block diagram showing an example embodying the present invention. In the line memory 31, image data is input in units of lines, and a plurality of lines (here, three lines from the (N-2) th line to the Nth line) are temporarily stored. The evaluation function application unit 15 reads out pixel data of a part of the image (frame application unit) in the line memory 31 and applies the evaluation function. Then, an optimum evaluation function is selected using a method described later, and notified to the camera shake correction processing unit 16 (camera shake correcting means). The camera shake correction processing unit 16 performs the camera shake correction process on the pixel data directly input from the line memory 31 using the optimal evaluation function (that is, the optimal camera shake correction function) notified from the evaluation function application unit 15. Then, pixel data is output in units of columns.

  In the above description, the case where the camera shake processing is performed one-dimensionally has been described. However, the present invention is not limited to this, and in an apparatus that can configure a large frame buffer, the evaluation function and the frame application unit are set in a two-dimensional manner. It is possible to perform more effective correction for the direction of camera shake. In other words, various types of camera shake correction programs operating in the camera shake correction processing unit 16 may be recorded in advance on a recording medium, and desired ones may be read into the portable imaging device.

FIG. 4 is an image diagram illustrating a method for evaluating the degree of camera shake. The evaluation of the degree of camera shake is the average of the square error of the pixel values (mean square error) between the frame application unit after performing the correction process by the evaluation function, ie, the camera shake correction function, and the frame application unit before performing the correction process. This is done by calculating That is, as shown in the following equation (1), the mean square error when the correction process is performed using a plurality of evaluation functions prepared in the evaluation function set unit 13 as a camera shake correction function is calculated. The evaluation function having the largest calculated mean square error (in this case, the evaluation function C) is selected and held as an evaluation function having an optimal parameter set. As shown in Expression (1), the evaluation function C includes the pixel position (vector amount) x of the image, the number of pixels n in the region, and the luminance value I (x) of the image before the camera shake correction process is performed. And the luminance value I ^k (x) of the image after the camera shake correction process (applying the filter k) is selected.

  For example, an original image is an image A, an image A is a blurred image obtained by camera shake, an image B, and correction images generated by applying the correction function X to the images A and B are images A ′ and B ′. Then, it can be considered that the higher the correction capability of the correction function X, the smaller the mean square error of (AB ′). However, since the mean square error of (AB ′) cannot be calculated (image A is not known), in the present invention, the mean square of (BB ′) is substituted for (AB ′). The error is calculated (in other words, the image B is compared with the image B ′ instead of the image A). That is, the present invention is characterized in that a mean square error in a frame is used as a camera shake evaluation value of the frame, and it is determined that an evaluation function having the largest camera shake evaluation value is optimal.

  As these evaluation functions (filters), it is more effective to use those having a sum of weighting coefficients of 1, as shown in FIG. For example, if the sum of the weighting coefficients is 0, the DC component of the luminance value of the image before application of the evaluation function is eliminated and the edge extraction is performed, but the sum of the weighting coefficients is 1 is used. As a result, the DC component of the luminance value of the image before the evaluation function application can be held, so that a more natural image can be obtained. As a result of the experiment, when a correction function X having a sum of weighting coefficients of 1 (in other words, a filter of a type that emphasizes edges) is used, the vicinity of the correction function X that minimizes (AB). It is known that (BB ′) tends to be maximum. Qualitatively, it is considered that the higher the ability to correct blur, the farther the image B ′ is from the image B. However, the correction function X is not limited to the above result, and is limited to a correction function for correcting blur and variations thereof.

  Further, as described above, these evaluation functions (filters) need to function as camera shake correction functions.

  The evaluation function C in which the mean square error has a maximum value as shown in the equation (1) is used as a camera shake correction function when the camera shake correction processing unit 16 performs the camera shake correction process.

  If the number of evaluation functions prepared in the evaluation function set unit 13 increases, the processing time becomes long. Therefore, when a large number of evaluation functions cannot be prepared, the following method may be used. In other words, in addition to the evaluation function having the largest mean square error, the evaluation function having the second largest mean square error is used, and parameter interpolation is performed between these evaluation functions to actually process (evaluate). None of the parameters may be estimated and used as an optimal camera shake correction function. As a result, the processing time can be shortened.

  Note that the above parameter interpolation may be used even when a sufficiently large number of evaluation functions can be prepared in the evaluation function set unit 13. Thereby, it is possible to improve the accuracy of camera shake correction.

  In the above description, the case where the mean square error is calculated and used as the camera shake evaluation value between the frame application unit after performing the correction process and the frame application unit before performing the correction process has been described. Not only the mean square error between parts, but also the closeness of data between frame application parts, such as the sum of absolute values of differences between frame application parts, the inner product between frame application parts, the correlation coefficient between frame application parts, etc. Other measures that represent may be used.

  As described above, the evaluation function application unit 15 in FIG. 1 has an optimum result output by applying the evaluation function to each frame within the imaging time range (in the above example, the mean square error). The optimum still image is selected by selecting the frame corresponding to the one having the largest image capture time). At this time, all frames captured within the imaging time range are stored in a memory buffer (illustrated in the camera shake correction system 100). Not stored).

  If the memory buffer is not sufficient, the frame having the best evaluation value is selected from the frames in the memory buffer when the last frame is evaluated. If the memory buffer is configured to have only one image evaluation result, the last frame is selected.

  1 uses the optimum camera shake correction function (evaluation function) selected by the evaluation function application unit 15 for the imaging candidate frame selected by the evaluation function application unit 15 to perform the camera shake correction process. Apply. When performing parameter interpolation as described above, an image stabilization function (evaluation function) in which optimum parameters are estimated and set is used. Note that, as described above, these parameters may include information on the strength of camera shake, the direction of camera shake, or the size of camera shake (length in the image).

  In addition, as described above with reference to FIG. 2, when a large frame buffer cannot be configured in the memory mounted on the imaging device of the portable imaging apparatus, the filter (camera shake correction function, that is, the evaluation function) is set only in the vertical direction or It is also possible to use a one-dimensional filter only in the horizontal direction.

  The imaging candidate frame whose camera shake is corrected by the camera shake correction processing unit 16 is displayed on the imaging frame display unit 17 in FIG.

  As described above, the portable imaging device according to the present embodiment selects, from the moving image data captured by the moving image data imaging unit 11, the optimum still image with the least influence of camera shake as the imaging candidate frame in the evaluation function application unit 15. The camera shake correction processing unit 16 performs camera shake correction. Therefore, compared with the conventional apparatus described in Patent Document 2 and the like, there is an effect that the influence of camera shake can be reduced.

1 is a block diagram illustrating a schematic configuration of a camera shake correction image processing system that can be mounted on a portable imaging device according to Embodiment 1. FIG. 6 is a diagram illustrating an evaluation function when camera shake evaluation and correction are limited to only one dimension in the portable imaging apparatus according to Embodiment 1. FIG. 1 is a configuration diagram illustrating an example that embodies a portable imaging device according to Embodiment 1. FIG. FIG. 3 is an image diagram illustrating an evaluation method of a degree of camera shake in the portable imaging device according to the first embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Moving image data imaging part, 12 Imaging timing determination part, 13 Evaluation function set part, 14 Interface part, 15 Evaluation function application part, 16 Hand shake correction process part, 17 Imaging frame display part, 31 Line memory, 100 Camera shake correction system

Claims (5)

Imaging means for continuously capturing a plurality of still images as moving images;
Selecting means for selecting an optimum still image with the least influence of camera shake from the moving image imaged by the imaging means;
A portable imaging apparatus comprising: an image stabilization unit that performs image stabilization on the optimum still image selected by the selection unit. The portable imaging device according to claim 1,
A portable imaging apparatus further comprising timing determining means for determining timing for starting imaging in the imaging means. The portable imaging device according to claim 1 or 2, wherein
The selection unit is a portable imaging device that evaluates the influence of the camera shake based on the strength of the camera shake, the direction of the camera shake, or the magnitude of the camera shake. The portable imaging device according to claim 3,
The camera shake correction unit is a portable imaging device that performs the camera shake correction based on the evaluation by the selection unit. A portable imaging device according to any one of claims 1 to 4,
The camera shake correction means is realized by using a correction program, and the correction program can select a desired one from a plurality of recording media on which a plurality of the correction programs are recorded.

Images