JP5130171B2 - Image signal processing apparatus and image signal processing method - Google Patents

Description

  The present invention relates to a technique for increasing the resolution of an image.

  Recent television receivers have become larger in screen size and do not display image signals input from broadcasting, communication, storage media, etc., as they are, but display them by increasing the number of pixels in the horizontal and vertical directions by digital signal processing. It is generally done. In this case, the resolution cannot be increased only by increasing the number of pixels by a generally known low-pass filter using a sinc function, a spline function, or the like.

  Therefore, Non-Patent Document 1 discloses a super-resolution technique for increasing the resolution by combining a plurality of input image frames (hereinafter abbreviated as frames) into one frame.

Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar “Fast and Robust Multiframe Super Resolution”, IEEE Transactions on Image Processing, VOL. 13, NO. 10, October 2004

  However, the high resolution processing by the technique described in Non-Patent Document 1 acquires the imaging conditions such as the blurring condition and the influence of motion at the time of image capturing, and the high resolution processing is performed using the imaging conditions. I do.

  Then, when only the image signal is acquired without acquiring the imaging condition and the resolution is increased, the imaging condition cannot be used, and the resolution increasing process based on an erroneous condition may be performed. . At this time, there is a problem in that high-resolution processing is applied even to an area that is not focused at the time of imaging, and the resolution is increased and the perspective of the image is lost.

  The present invention has been made in view of the above problems, and an object of the present invention is to increase the resolution of an image while maintaining the perspective of the image for achieving higher resolution of the image.

  The embodiment of the present invention may be configured as described in the claims, for example.

  According to the present invention, it is possible to improve the resolution of an image more suitably.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, an image signal processing apparatus and an image signal processing method thereof according to Embodiment 1 of the present invention will be described.

FIG. 1 shows an image signal processing apparatus (100) according to Embodiment 1 of the present invention. First, the input image (101) input to the image signal processing device (100) is input to the blur estimation unit (102), and the blur estimation unit (102) performs image blur for each region in the input image. Estimate and calculate the degree of blur.
Next, the parameter determination unit (103) determines a parameter to be used for the high resolution processing based on the blur level estimated by the blur level estimation unit (102). Further, the parameter determined by the parameter determination unit (103) is input to the resolution enhancement processing unit (104), and the resolution enhancement processing unit (104) increases the resolution of the input image using this parameter. Perform high resolution processing. Since the high resolution processing in the high resolution processing unit (104) may use the processing described in Non-Patent Document 1, for example, the description thereof is omitted.

  Here, as a specific example of the blur estimation process in the blur estimation unit (102), there is a method for estimating the blur by calculating the texture amount of the image. The texture amount of the image can be calculated by, for example, a method of calculating the variance of the image from surrounding pixels. In the portion where the amount of texture calculated in this way is large, the image is clear, that is, the region where the degree of blur is small, and on the contrary, in the portion where the amount of texture is small, the region where the image is blurred, ie, where the degree of blur is large. It can be estimated that this is a region. The blur estimation process may be performed for each partial region in the screen or may be performed for each pixel.

  Here, an example of the blur estimation result of the blur estimation process of the blur estimation unit (102) will be described with reference to FIG.

  In the example of FIG. 2, for example, the subject in the area A is a person, and the area A has a focus during imaging. The subject in the region B is a vase behind the person, and the focus of the region B during imaging is shifted from the region A. Furthermore, it is assumed that the subject in the area C is a background that is further behind the person and the vase, and the focus at the time of imaging is shifted from the areas A and B in the area C.

  When the image of FIG. 2 as described above is used as an input image, according to the blur level estimation processing in the blur level estimation unit (102), the region A is estimated to have a low blur level (clear image), and the region B Is estimated to be medium blur (an image that is more blurred than region A, an image that is less clear than region A), and region C is large (an image that is more blurry than region B and an image that is less clear than region B). Can be estimated.

  In the above example, in the blur estimation process of the blur estimation unit (102), the texture amount is used to estimate the blur amount of the image. For example, the edge information is calculated and the calculated edge information is used. It may be estimated by other methods such as determining whether an image is clear or blurred.

  Next, as a specific example of the parameter determination process of the parameter determination unit (103), there is a method using a blur recovery transfer function as a parameter.

  That is, when the processing described in Non-Patent Document 1 is used for the resolution enhancement processing in the resolution enhancement processing unit (104), a transfer function for recovering image blurring that occurs during imaging is required as a parameter. . Generally, this transfer function is a low-pass filter coefficient. When the low-pass filter coefficient is set to be strong in the low-pass filter process, the blur recovery effect of the high-resolution process is strong. Conversely, when the low-pass filter process is set to be weak, the blur recovery effect of the high-resolution process is weak.

  The parameter determination unit (103) according to the first embodiment changes the low-pass filter coefficient for each partial region in the image according to the blur level calculated by the blur level estimation unit (102). This makes it possible to change the strength of the low-pass filter in the high resolution processing of the high resolution processing unit (104) for each partial region in the image, and this occurs during imaging while maintaining the perspective of the image. Image blur can be recovered and the image can be increased in resolution.

  When the blur level estimation process of the blur level estimation unit (102) is a blur level estimation process by calculating the texture amount, the parameter determination unit (103) calculates the texture amount calculated by the blur level estimation unit (102). A region with a large amount of noise sets a low-pass filter strongly, and a region with a small amount of texture sets a weak low-pass filter.

  In the above description, the blur recovery transfer function is given as an example of the parameter used for the resolution enhancement processing in the resolution enhancement processing unit (104) .However, the parameter for controlling the image blur recovery effect in the other resolution enhancement processing is described. It may be used.

  Note that the blur level estimation processing may be performed for each partial region in the screen or may be performed for each pixel.

  Finally, the high resolution processing unit (104) performs high resolution processing of the input image using the parameters calculated by the parameter determination unit (103). The high resolution processing may be performed for each partial region in the screen or may be performed for each pixel.

  When the parameter determined by the parameter determination unit (103) is a low-pass filter coefficient, the resolution enhancement processing unit (104) uses the low-pass filter coefficient determined by the parameter determination unit (103). High resolution processing including processing is performed.

  Next, FIG. 5 shows a processing flow of image processing by the image signal processing apparatus (100) described above. In the image signal processing device (100), the blur level estimation unit (102) calculates and estimates the blur level of the image for each partial region or one pixel in the screen of the input image (step 501). Next, the parameter determination unit (103) performs the resolution enhancement processing unit (104) for each partial region or one pixel of the input image based on the blur level calculated and estimated by the blur level estimation unit (102). Parameters used for the high resolution processing are determined (step 502). Finally, the resolution enhancement processing unit (104) performs a process for increasing the resolution of the input image using the parameters determined by the parameter determination unit (103) (step 503).

  According to the image signal processing apparatus according to the first embodiment described above, a partial area or pixel that should be improved in blur and a partial area or pixel that should be left out of blur are determined from the degree of blur of the input image. Therefore, it is possible to perform high resolution processing corresponding to the degree of blurring on the input image.

  That is, according to the image signal processing apparatus according to the first embodiment described above, the blurring degree is estimated for each partial region or one pixel of the input image, and the high resolution processing is changed based on the estimated blurring degree. By doing so, it is possible to perform high resolution processing while maintaining the perspective of the image. Therefore, it is possible to improve the resolution of the image more suitably.

FIG. 3 shows an image display apparatus according to Embodiment 2 of the present invention. The image display apparatus according to the present embodiment is an image display apparatus configured to include an image signal processing unit that performs the image signal processing described in the first embodiment.
In the figure, an image display device (300) includes, for example, an input unit (301) for inputting a broadcast signal, image content, image content, etc. via a broadcast wave including a television signal, a network, etc. 301), a recording / playback unit (302) for recording or playing back the content input from the content, a content storage unit (303) for recording the content by the recording / playback unit (302), and an image signal played back by the recording / playback unit (302). The image signal processing unit (304) for performing the image signal processing described in the first embodiment, the display unit (305) for displaying the image signal processed by the image signal processing unit (304), and recording / playback An audio output unit (306) for outputting an audio signal reproduced by the unit (302), a control unit (307) for controlling each component of the image display device (300), and a user operating the image display device (300) A user interface unit (308) for performing
The detailed configuration and operation of the image signal processing unit (304) are the same as those of the image signal processing device (100) described in the first embodiment, and thus description thereof is omitted.
Since the image display device (300) includes the image signal processing unit (304) that is the image signal processing device described in the first embodiment, the image signal input to the input unit (301) The resolution can be increased and displayed on the display unit (305) while being held.

  Further, when reproducing the image content stored in the content storage unit (303), it is possible to increase the resolution while displaying the perspective of the image and display it on the display unit (305).

  Further, the image signal processing unit (304) may be included in the recording / playback unit (302), and the above-described image signal processing may be performed during recording. In this case, since it is not necessary to perform the above-described image signal processing during reproduction, the processing load during reproduction can be reduced.

  According to the image display device according to the second embodiment of the present invention described above, the image display device having the effects shown in the first embodiment can be realized. Therefore, the image can be displayed with a higher resolution while maintaining the perspective of the image.

FIG. 4 shows a recording / playback apparatus according to Embodiment 3 of the present invention. The recording / playback apparatus according to the present embodiment is a recording / playback apparatus configured to include an image signal processing unit that performs the image signal processing of the first embodiment.
In the figure, a recording / playback apparatus (400) includes, for example, an input unit (401) for inputting a broadcast signal, image content, image content, etc. via a broadcast wave including a television signal, a network, etc. A recording / playback unit (402) that records or plays back the content input from the unit (401), a content storage unit (403) that records the content by the recording / playback unit (402), and a recording / playback unit (402) that plays back An image signal processing unit (404) that performs image signal processing described in the first embodiment on the image signal, and an image output that outputs the image signal processed by the image signal processing unit (404) to another device or the like Unit (405), an audio output unit (406) for outputting the audio signal reproduced by the recording / reproducing unit (402) to other devices, and a control unit (407) for controlling each component of the recording / reproducing device (400) ) And a user interface for the user to operate the recording / playback device (400) (408) comprises a like.
The recording / playback apparatus (400) includes the image signal processing unit (404) which is the image signal processing apparatus described in the first embodiment, so that the image signal with higher resolution while maintaining the perspective of the image can be used as another image signal. It can be output to a device or the like.

Further, the image signal processing unit (404) may be included in the recording / playback unit (402) to perform the above-described image signal processing during recording. In this case, since it is not necessary to perform the above-described image signal processing during reproduction, the processing load during reproduction can be reduced.
According to the recording / reproducing apparatus according to the third embodiment of the present invention described above, the recording / reproducing apparatus having the effects shown in the first embodiment can be realized. Therefore, it is possible to record, reproduce or output the image with a high resolution while maintaining the perspective of the image.

It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 2 which concerns on this invention. It is explanatory drawing of Example 3 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention.

Explanation of symbols

101 Input image
102 Blur level estimation unit
103 Parameter determination section
104 High resolution processing section
105 Output image
201 Image texture amount calculator
202 Transfer function determination unit for blur recovery
300 Image display device
301 Input section
302 Recording / playback unit
303 Content storage
304 Image signal processor
305 Display
306 Audio output section
307 Control unit
308 User interface
400 Recording / playback device
401 input section
402 Recording / playback unit
403 content storage
404 Image signal processor
405 Image image output unit
406 Audio output unit
407 Control unit
408 User interface

Claims (4)

A blur estimation unit that calculates blur for each partial region or pixel of the input image;
A parameter determining unit that determines a low-pass filter coefficient used for high resolution processing using the low-pass filter coefficient for the partial region or the pixel as a parameter for the processing intensity , based on the blur level calculated by the blur level estimation unit;
Based on the parameter determination unit has determined parameters, image signal processing apparatus, characterized in that it comprises a high-resolution processing unit for performing high-resolution processing on the input image for each of the partial regions each or said pixels. In claim 1,
The blur level estimation unit images the signal processor that you characterized for calculating the texture of the image for each partial region or per pixel of the input image as the blur level. A blur estimation step for calculating blur for each partial area or pixel of the input image;
A parameter determining step for determining a parameter coefficient to be used for low-pass filter processing included in the high resolution processing for the partial region or the pixel based on the blurring degree calculated in the blurring degree estimation step;
An image signal processing method comprising: a resolution enhancement processing step of performing resolution enhancement processing on the input image for each partial region or for each pixel based on the parameter determined in the parameter determination step. In claim 3,
An image signal processing method, wherein, in the blur level estimation step, a texture amount of an image is calculated as the blur level for each partial region or pixel of the input image.

Images