JP2010057001A - Image processor, method, and image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that since movement blur included in a video signal is one-dimensional anisotropic blur where the generation direction of blur follows the moving direction, unnecessary sharpening may be performed even to an outline orthogonal to a moving direction in a method for performing frequency conversion uniformly on a screen by using a filtering technology. <P>SOLUTION: An image processor includes: a period decision means (14) for detecting a movement blur period based on the decision result about the continuity of tone change about a video signal; and a correction means (3) for correcting a video signal (d2) based on the detection result of the movement blur period detected by the blur period decision means (14). The correction means (3) corrects a video signal (d2) only in the movement blur period (bf) detected by the blur period decision means (14). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method, and an image display apparatus.

  The video signal received by the image display device is obtained by quantizing the total amount of light received from the subject during the frame accumulation time (for example, 1/60 seconds) by the light receiving unit of the camera, and arranging them in the pixel order determined by the standard. It is. When the light receiving unit of the camera and the subject move relative to each other, blurring (hereinafter referred to as motion blur) is generated in the contour portion of the subject depending on the frame accumulation time and the relative speed between the camera and the subject.

  As a method for improving blur included in an image, for example, a method using an enlargement / reduction circuit as disclosed in Patent Document 1 is disclosed. According to this method, the rising and falling of the contour portion of the video can be made steep by using a filtering technique without adding overshoot or undershoot.

  Further, Patent Document 2 discloses a blurring function decombination method using motion vector detection. However, this method has an enormous amount of calculation and is difficult to implement in an actual circuit.

JP 2002-16820 A Japanese Patent No. 3251127

Image blur can be classified into isotropic blur and anisotropic blur.
The isotropic blur caused by defocusing at the time of imaging has a narrow blur width in general, and the filtering technique disclosed in Patent Document 1 is effective as a method for improving blur.
On the other hand, in motion blur, the direction in which blur occurs is one-dimensional anisotropic blur along the motion direction, and the blur width varies greatly depending on the relative speed between the camera and the subject.
Applying the method of uniformly converting the screen frequency using the conventional filtering technique disclosed in Patent Document 1 to anisotropic blur causes unnecessary sharpening even for the contour orthogonal to the motion direction. There was a problem of being done.

  Also, when the filter is optimized so that motion blur with a wide blur width can be corrected, filtering is performed even on an image having a gradual change in luminance such as gradation, which is different from the image originally intended to be displayed. There was a risk of erroneous conversion to images.

  Furthermore, when the moving direction of a subject is specified using a motion vector detection method as disclosed in Patent Document 2 and adaptive filtering is performed, the detection circuit scale increases, which is difficult in terms of cost. There was a problem.

  The present invention has been made in view of the above problems, and is an image processing apparatus capable of detecting motion blur included in a video signal and reducing the detected motion blur without increasing the circuit scale. The purpose is to provide.

The image processing apparatus of the present invention
A first video signal that is not frame-delayed with respect to the input video signal; a second video signal that is delayed by a predetermined number of frames with respect to the input video signal; and the second video signal A delay means for generating a third video signal delayed by the predetermined number of frames;
A first low-pass filter for removing a high-frequency component of the first video signal and outputting a first filtered video signal;
A second low-pass filter for removing a high-frequency component of the second video signal and outputting a second filtered video signal;
A third low-pass filter that removes high-frequency components of the third video signal and outputs a third filtered video signal;
First difference detection means for detecting a gradation difference between the first filtered video signal and the second filtered video signal;
Second difference detection means for detecting a gradation difference between the second filtered video signal and the third filtered video signal;
Third difference detection means for detecting a gradation difference between the third filtered video signal and the first filtered video signal;
Differentiating means for detecting an inter-pixel change in the video signal output by the third difference detecting means;
Contour shape determining means for determining continuity of gradation change in the second video signal;
The gradation difference detected by the first difference detection means, the gradation difference detected by the second difference detection means, the detection result of the inter-pixel change detected by the differentiation means, A blur period determination unit that detects a motion blur period based on a determination result about the continuity of gradation change of the second video signal detected by the contour shape determination unit;
Correction means for correcting the second video signal based on the detection result of the motion blur period by the blur period determination means;
The blur correction unit corrects the second video signal only during the motion blur period detected by the blur period determination unit.

  According to the present invention, it is possible to reduce the motion blur width included in the input video signal without performing unnecessary sharpening processing on the contour orthogonal to the motion direction.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of an image display apparatus according to the present invention. The illustrated image display device 81 includes a delay unit 1, a blur detection unit 2, a blur correction unit 3, and an image display unit 4. The delay unit 1, the blur detection unit 2, and the blur correction unit 3 constitute an image processing apparatus.
The video signal input to the image display device 81 is supplied to the delay unit 1. The delay unit 1 performs frame delay of the input signal using a frame memory, and outputs a plurality of frame-delayed signals d1, d2, and d3 to the blur detection unit 2 and the blur correction unit 3.

  The blur detection unit 2 detects a motion blur region included in the video from the video signals d1, d2, and d3 of a plurality of different frames output from the delay unit 2, and outputs a motion blur detection flag bf.

  The blur correction unit 3 converts the video signal d2 output from the delay unit 1 based on the motion blur detection flag bf detected by the blur detection unit 2, and outputs the video signal d2 to the image display unit 4.

  FIG. 2 is a block diagram illustrating an example of the delay unit 1. The illustrated delay unit 1 includes a frame memory control unit 5 and a frame memory 6. The frame memory 6 has a capacity capable of storing at least two frames of input video signals.

  The frame memory control unit 5 writes the input video signal and reads the stored video signal according to the memory address generated based on the synchronization signal included in the input video signal d0. Frame video signals d1, d2, and d3 are generated.

  The video signal d1 has no delay with respect to the input video signal d0, and is also called a current frame video signal. The video signal d2 is delayed by one frame with respect to the video signal d1, and is also called a one-frame delayed video signal. The video signal d3 is delayed by one frame with respect to the video signal d2, that is, delayed by two frames with respect to the video signal d1, and is also called a two-frame delayed video signal. In addition, since the processing is performed on the video signal d2 as described below, the video signal d2 is also referred to as a target frame video signal, the video signal d1 is also referred to as a previous frame video signal, and the video signal d3 is also referred to as a rear frame video signal. is there.

  FIG. 3 is a block diagram illustrating an example of the blur detection unit 2. The illustrated blur detection unit 2 includes low-pass filters (hereinafter referred to as LPF) 7, 8 and 9, difference detection units 11, 12 and 27, a blur period determination unit 14, a differentiation unit 28, and a contour shape determination unit 35. Prepare.

The video signals d1, d2, and d3 output from the delay unit 1 are supplied to the LPFs 7, 8, and 9, respectively. The LPF 7 removes the high frequency component of the video signal d1 output from the delay unit 1, generates a video signal e1, and outputs the video signal e1 to the difference detection units 11 and 27, respectively.
The LPF 8 removes a high-frequency component of the video signal d2 output with a delay of one frame from the delay unit 1, generates a video signal e2, and outputs it to the difference detection units 11 and 12.
The LPF 9 removes the high frequency component of the video signal d3 output from the delay unit 1, generates the video signal e3, and outputs the video signal e3 to the difference detection units 12 and 27.

The difference detection unit 11 detects a difference between the input video signal e1 and the video signal e2, that is, an inter-frame correction difference for each pixel, and outputs a difference correction signal g1 to the blur period determination unit 14. The inter-frame correction difference output from the difference detection unit 11 is a value obtained by correcting the difference between one frame for each pixel as described later.
Similarly, the difference detection unit 12 detects a difference between the input video signal e3 and the video signal e2, that is, an inter-frame correction difference for each pixel, and outputs a difference correction signal g2 to the blur period determination unit 14. The inter-frame correction difference output from the difference detection unit 12 is also a value obtained by correcting the difference between one frame for each pixel as described later.
Similarly, the difference detection unit 27 detects the difference between the input video signal e3 and the video signal e1, that is, the inter-frame correction difference (for each pixel) for each pixel, and outputs the difference correction signal g3 to the differentiation unit 28. To do. The inter-frame correction difference output from the difference detection unit 27 is a value obtained by correcting the difference between two frames for each pixel as described later.

  The differentiating unit 28 obtains a difference between the difference correction signal g3 for each pixel of the difference correction signal g3 and the difference correction signal g3 for the pixel located immediately before the difference correction signal g3, and determines the difference between adjacent pixels f3 as the blur period determination unit 14. Output to. Here, “immediately before” means “immediately before” in the processing order of pixels, and when pixels in each horizontal line are processed in order from left to right as usual, they are located on the left side and adjacent to each other. Pixel.

The video signal d2 output from the delay unit 1 is also input to the contour shape determination unit 35. The contour shape determination unit 35 determines the continuity of the gradation change of the video signal d2, and if it has continuity, the contour shape detection flag ef is set to a first value, for example, “Hi”, and the blur period determination unit. 14 for output.
Here, “when the gradation change has continuity” means a case where the gradation change satisfying a predetermined condition continues for a predetermined pixel period or longer. The predetermined condition is, for example, that the gradation change (difference between adjacent pixels) over one pixel period is within a predetermined range. The predetermined range is, for example, based on the change between the current pixel and the previous pixel as a reference, a value that is larger than the first predetermined value as an upper limit value, and a value that is smaller than the second predetermined value as a lower limit. The range of values. “Continue for a predetermined pixel period or more” means that the same result is obtained for a predetermined number or more of pixels that are continuously processed before the current pixel. Considering that the distance between pixels is constant, the gradation change given by the difference between adjacent pixels can also be called “gradation change rate”. "Is within a predetermined range" can be rephrased as "the gradation change rate is within a predetermined range".

  In the blur period determination unit 14, the contour shape detection flag ef output from the contour shape determination unit 35, the difference correction signals g 1 and g 2 obtained by the difference detection units 11 and 12, and the inter-frame obtained by the differentiation unit 28 Based on the differential result f3, it is determined whether or not motion blur has occurred, that is, whether or not it is a blur period, and a determination result bf is output.

FIG. 4 is a block diagram illustrating an example of the configuration of the contour shape determination unit 35. The illustrated contour shape determination unit 35 includes a differentiation unit 36, a differentiation result storage unit 37, and a continuity determination unit 38.
The video signal d2 output from the delay unit 1 is input to the differentiation unit 36. The differentiating unit 36 obtains a difference (brightness change amount between adjacent pixels) between the video signal d2 for each pixel and the video signal one pixel before that, and outputs the difference as a differentiation result for the pixel.
The differentiation result Pk output from the differentiation unit 36 is supplied to the differentiation result storage unit 37 and the continuity determination unit 38.

The differentiation result storage unit 37 temporarily stores the differentiation result (current differentiation result) Pk output from the differentiation unit 36, and the stored differentiation result Pk includes one pixel period, two pixel periods, three pixel periods, After the 4-pixel period delay, the differential result of 1-pixel delay is read as the differential result of 2-pixel delay, the differential result of 3-pixel delay, and the differential result of 4-pixel delay.
Therefore, at each time point, the differential results DPk1, DPk2, DPk3, DPk4 of 1 pixel delay, 2 pixel delay, 3 pixel delay, and 4 pixel delay are obtained with respect to the current differential result (differential result for the target pixel) Pk. The data is read from the storage unit 37 and input to the continuity determination unit 38 together with the current differentiation result Pk.
The current differential result is a differential result for the target pixel, and the differential results DPk1, DPk2, DPk3, and DPk4 of 1 pixel delay, 2 pixel delay, 3 pixel delay, and 4 pixel delay are located before the target pixel. It can also be said that this is a differential result for a pixel. Here, “previous” means “previous” in the processing order of pixels. As usual, when pixels in each horizontal line are processed in order from left to right, is there.

The continuity determination unit 38 outputs a delayed differentiation result (differentiation result for a pixel located before the target pixel) DPk1 to DPk4 output from the differentiation result storage unit 37 and includes a predetermined differentiation result Pk. By determining whether it is within the range, it is determined whether there is continuity of gradation change. Specifically, the differential result storage is performed between the value obtained by adding the threshold value S8 given in advance and the differentiation result Pk (Pk + S8) and the value obtained by subtracting the threshold value S9 given in advance from the differentiation result Pk (Pk−S9). It is determined whether or not all four consecutive differential results output from the unit 37 are included, and if “included”, that is, (Pk + S8) ≧ DPkx ≧ (Pk−S9) (x = 1, 2, 3, 4) ... (102)
In this case, it is determined that the gradation change (gradient) has continuity, and the continuity determination unit 38 outputs the value of the contour shape detection flag ef as a first value, for example, “1”.

The threshold value S8 and the threshold value S9 may be the same value or different values.
By performing the above processing, the continuity determination unit 38 allows the difference in gradation values (gradation change rate, differential value) between adjacent pixels to be continuously present over a predetermined number of pixels. It is determined whether or not it falls within a predetermined range (range from (Pk−S9) to (Pk + S8)) including the gradation change rate Pk.

  In the above example, the “section of the predetermined number of pixels” is the “section of four pixels”, but is not limited to four pixels, and may be another number of pixels. In order to make “n pixel interval (n is a natural number)”, the differential result storage unit 37 stores the differential results for n or more pixels, and the current differential result Pk is changed from the differential unit 36 to the continuity determination unit 38. The latest n differential results DPk1 to DPkn may be supplied from the differential result storage unit 37 to the continuity determination unit 38.

FIG. 5 is a diagram illustrating an example of the blur correction unit 3. The illustrated blur correction unit 3 includes a contour shape calculation unit 15 and a pixel conversion unit 16.
The contour shape calculation unit 15 calculates a contour shape based on the input motion blur determination result bf, and outputs a conversion control signal j to the pixel conversion unit 16.
The pixel conversion unit 16 converts the video signal d2 based on the conversion control signal j and the input video signals d1 and d3, and outputs the converted video signal k.

Hereinafter, the operation of each component of the image display device 81 will be described in more detail.
The video signal d0 input to the image display device 81 is input to the delay unit 1.
6A to 6F are diagrams for explaining the relationship between the video signal d0 input to the delay unit 1 and the output video signals d1, d2, and d3. In synchronization with the input vertical synchronization signal SYI shown in FIG. 6A, as shown in FIG. 6B, the input video signals d0 of the frames F0, F1, F2, and F3 are sequentially input. The frame memory control unit 5 generates a frame memory write address based on the input vertical synchronization signal SYI, stores the input video signal d0 in the frame memory 6, and outputs the output vertical synchronization signal SYO (input vertical) shown in FIG. In synchronization with the synchronization signal SYI), as shown in FIG. 6 (d), the video signal d1 (frames F0, F1,. F2 and F3 video signals) are output. The frame memory control unit 5 also generates a frame memory read address based on the input vertical synchronization signal, and stores the 1-frame delayed video signal d2 (FIG. 6 (e)) and the 2-frame delayed video signal stored in the frame memory 6. d3 (FIG. 6F) is read and output. As a result, the delay unit 1 outputs three consecutive frames of video signals d1, d2, and d3 simultaneously. That is, at the timing (frame period) when the video signal of the frame F2 is input as the video signal d0, the video signals of the frames F2, F1, and F0 are output as the video signals d1, d2, and d3, and the video signal of the frame F3 is the video. Video signals of frames F3, F2, and F1 are output as video signals d1, d2, and d3 at the timing (frame period) that is input as the signal d0.

  Three consecutive frames of video signals d1, d2, and d3 output from the delay unit 1 are output to the blur detection unit 2 and the blur correction unit 3. The video signals d1, d2, and d3 input to the blur detection unit 2 are input to the LPFs 7, 8, and 9, respectively.

  FIG. 7 shows an example of the video signals d1, d2, and d3. The horizontal direction indicates the pixel position, and the vertical direction indicates the gradation, indicating the contour portion of the image where the gradation changes gently.

  The LPFs 7, 8, and 9 remove high frequency components of the video signal shown in FIG. The removal of high-frequency components in this way causes blurring due to the frame accumulation time to be a problem with a relatively wide transition width, in other words, a relatively fast moving image. This is because it is an unnecessary component for detecting blur caused by the accumulation time.

  FIG. 8 shows an example of a video signal e obtained by removing high frequency components from the video signal shown in FIG. 7 using LPF. The signals e1, e2, e3 output from the LPFs 7, 8, 9 are input to the difference detectors 11, 12, 27.

FIGS. 9A and 9B are diagrams illustrating the relationship between signals e1, e2, and e3 of three consecutive frames.
Normally, since the video signal transmitted at 60 frames / second is obtained by photographing with a frame accumulation time of 1/60 seconds, when the subject is moving, the contour portions Ce3, Ce2, Ce1 are As shown in FIGS. 9A and 9B, observation is continued between frames. Specifically, when the subject is moving, as shown in FIG. 9A, the gradation change of the 1-frame delay signal e2 starts at the pixel position where the gradation change of the 2-frame delay signal e3 ends. Similarly, as shown in FIG. 9B, the gradation of the current frame signal e1 starts to change at the pixel position where the gradation change of the one-frame delay signal e2 has ended.

FIG. 10 is a diagram illustrating an example of the difference detection unit 11. The illustrated difference detection unit 11 includes a difference calculation unit 19 and a difference correction unit 20.
The difference calculation unit 19 calculates a difference between the input signals e1 and e2 and outputs a calculation result de.
The calculation result de is supplied to the difference correction unit 20.
The difference correction unit 20 corrects the difference calculation result de using a threshold value S3 given in advance, and generates and outputs a difference correction signal g1.

FIGS. 11A and 11B are diagrams for explaining the operation of the difference detection unit 11.
FIG. 11A shows the signals e1 and e2 for two frames to be input. FIG. 11B shows the difference calculation result de (= e2-e1). FIG. 11C shows the generated difference correction signal g1. Here, the horizontal axis of each figure indicates the pixel position.
If there is a place where the subject is moving in the two consecutive frames of signals e2 and e1 (FIG. 11 (a)), the difference calculation unit 19 obtains the interframe difference as shown in FIG. 11 (b). As shown, a difference calculation result de having a peak at the pixel position Ee where the contours are continuous between frames is obtained.

The difference calculation result de output from the difference calculation unit 19 is input to the difference amount correction unit 20. In many cases, a noise component is superimposed on a video signal, and a slight difference may be detected when a difference amount is calculated between frames even in a place where the subject is not originally moving. Therefore, the difference amount correction unit 20 performs a process of reducing the magnitude of the difference calculation result de by the magnitude of the threshold value S3. That is, the threshold value S3 is subtracted from the absolute value of the difference calculation result de. As a result, a difference correction signal g1 (FIG. 11C) in which erroneous detection due to noise is reduced is generated and output.
As will be understood from the following description, only the absolute value of the difference becomes a problem in the processing in the blur detection unit 2. Therefore, when obtaining the difference in the difference detection unit 27, which of the two signals is used. It doesn't matter if you subtract. The same applies to the difference detection units 11 and 12.

  The configuration and operation of the difference calculation unit 12 are the same as those of the difference calculation unit 11, but signals e2 and e3 are input instead of the signals e1 and e2, and a signal g2 is output instead of the signal g1.

  Similarly, the configuration and operation of the difference calculation unit 27 are the same as those of the difference calculation unit 11, but signals e1 and e3 are input instead of the signals e1 and e2, and a signal g3 is output instead of the signal g1.

g1, g2, and g3 are represented by the following formulas.
When | e2-e1 |> S3, g1 = | e2-e1 | -S3
Otherwise, g1 = 0
If | e3-e2 |> S3, then g2 = | e3-e2 | -S3
Otherwise, g2 = 0
If | e3-e1 |> S3, then g3 = | e3-e1 | -S3
Otherwise, g3 = 0

12A to 12D are diagrams showing a process of generating the difference correction signal g3 from the signals e1 and e2 input to the difference calculation unit 27 and a process of calculating the differential value f3 from the difference correction signal g3. It is.
The difference calculation unit 27 obtains a difference de (FIG. 12B) between e1 and e3 from the signals e1 and e3 (FIG. 12A), and further generates a difference correction signal g3 (FIG. 12C). To do. The difference correction signal g3 (FIG. 12C) output from the difference calculation unit 27 is input to the differentiation unit 28.
The differentiating unit 28 detects the absolute value of the change amount between adjacent pixels (difference between adjacent pixels) of the difference correction signal g3 output from the difference detection unit 27, and outputs the differential value f3 as a detection result to the blur determination unit 14. To do.

  When the contour portion is blurred due to the movement of the subject, the contour portion Ce2 of the signal of the frame of interest (1-frame delay signal) has a difference between the previous and subsequent frames (difference between the two-frame delay signal e3 and the current frame signal e1). ) Is differentiated, and f3 (FIG. 12D) is almost zero.

Next, the operation of the contour shape determination unit 35 will be described in more detail.
When the subject is moving, anisotropic blur occurs in the contour determined by the motion of the subject.
FIG. 13 is a diagram showing blurring caused by the frame accumulation time when a subject having a contour whose luminance changes stepwise moves at a constant speed.
When there is no so-called isotropic blur caused by aberration or defocusing of the imaging optical system, the change in the gradation value with respect to the position is discontinuous as indicated by the broken line PRs in the contour of the stationary subject (step Shape). If the charge accumulation time of the image sensor is sufficiently short, a video signal having substantially the same contour as indicated by the wavy line PRs can be obtained. However, since the charge accumulation time has a certain length in practice, there is a section in which the gradation value gradually changes according to the moving speed of the contour, which causes blurring. When the contour in which the luminance changes stepwise moves at a constant speed, the gradation value has a constant slope determined by the speed of movement as shown by the solid line PRm, and a gradation value transition section TTm is generated.

FIG. 14 is a diagram showing blurring caused by the frame accumulation time when a subject having an outline whose luminance changes in a step shape is accelerated and decelerated.
When the subject is accelerating / decelerating, strictly speaking, as shown by solid lines PRma and PRmb in FIG. 14, the gradient of the gradation value changes during the gradation transition period.
However, the blurring of the contour due to the movement of the subject has only a moving speed of about several pixels per frame, and therefore the change in the gradient of the gradation transition period due to the acceleration / deceleration movement is small and within the allowable error range. Even if it is ignored and the gradation is changed with a constant slope as shown by the solid line PRmc, there is no practical problem.

15A to 15E are diagrams for explaining the operation of the contour shape determination unit 35. FIG.
FIG. 15A shows an example of the input video signal d2. The differentiation unit 36 differentiates the input video signal d2 and outputs a differentiation result Pk shown in FIG.
The differentiation result storage unit 37 temporarily stores the differentiation result Pk output from the differentiation unit 36. From the storage unit 37, as shown in FIG. The differential result DPk1 of 1 pixel delay, the differential result DPk2 of 2 pixel delay with respect to the input differential result Pk, the differential result DPk3 of 3 pixel delay with respect to the input differential result Pk, and the input differential result Pk On the other hand, the differential result DPk4 of 4 pixel delay is read out.

FIGS. 15D and 15E are diagrams for explaining the operation of the continuity determination unit 38.
The continuity determination unit 38 calculates a result (Pk + S8) obtained by adding a threshold value S8 given in advance to the differentiation result Pk and a result (Pk−S9) obtained by subtracting the threshold value S9 given in advance from the differentiation result Pk. The continuity determination unit 38 further compares the differential results DPk1 to DPk4 output from the differential result storage unit 37 with the calculation results (Pk + S8) and (Pk−S9), and the differential results DPk1 to DPk4 are (Pk + S8) ≧ DPkx ≧ (Pk−S9) (x = 1, 2, 3, 4) (101)
The contour shape detection flag ef indicating the determination result is output. This flag ef is Hi when the condition expressed by the above equation (101) is satisfied.

  The difference correction signals g1 and g2 output from the difference detection units 11 and 12, the differential value f3 output from the differentiation unit 28, and the contour shape detection flag ef output from the contour shape determination unit 35 are sent to the blur period determination unit 14. Entered.

FIG. 16 is a diagram illustrating a configuration of the blur period determination unit 14. The differential value f3 is input to the binarization unit 22. The binarization unit 22 binarizes the differential value f3 based on a predetermined threshold value S4 and outputs a binarized difference correction signal dg3 to the state determination unit 25.
The difference correction signal g1 is input to the binarization unit 23. Similarly to the binarization unit 22, the binarization unit 23 binarizes the input difference correction signal g1 using a predetermined threshold value S5, and uses the binarized difference correction signal dg1 as a state determination unit. To 25.
Similarly, the difference correction signal g2 is input to the binarization unit 24. Similarly to the binarization units 22 and 23, the binarization unit 24 binarizes the input difference correction signal g2 using a predetermined threshold value S6, and sets the binarized difference correction signal dg2 to the state. Output to the determination unit 25.

  The state determination unit 25 generates a contour state flag gs based on the binarized difference correction signals dg1, dg2, and dg3, and outputs the contour state flag gs to the blur determination unit 26. The blur determination unit 26 outputs a blur detection flag bf based on the contour state flag gs and the contour shape detection flag ef.

FIGS. 17A and 17B are diagrams for explaining the operation of the binarization unit 22 and show the relationship between the input signal f3 and the output signal dg3.
The input signal f3 (FIG. 17A) is binarized to “1” when its absolute value is larger than a predetermined threshold value S4 and “0” when it is smaller, and is output to the state determination unit 25. Is done. A signal dg3 obtained as a result of binarization is shown in FIG.
The operations of the binarizing units 23 and 24 are the same as those of the binarizing unit 22, but signals g1 and g2 are input instead of the signal f3, and signals dg1 and dg2 are output instead of the signal dg3.

FIG. 18 is a diagram illustrating an example of the state determination unit 25. The illustrated state determination unit 25 includes a state comparison unit 21 and a state correction unit 29.
The binarized difference correction signals dg1 and dg2 input to the state determination unit 25 are input to the state comparison unit 21. The state comparison unit 21 compares the two input difference correction signals dg1 and dg2, and outputs a state comparison signal gss indicating the result to the state correction unit 29. The state correction unit 29 corrects the state comparison signal gss based on the binarized difference correction signal dg3 and outputs a state correction signal gs.

  FIGS. 19A to 19C are diagrams for explaining the operation of the state comparison unit 21. FIG. Assume that the binary difference correction signals dg1 and dg2 input to the state comparison unit 21 are as shown in FIGS. 19 (a) and 19 (b), respectively. The state comparison unit 21 outputs a state comparison signal gss (FIG. 19C) having three states (A, B, C) according to the states of the two binarized difference correction signals. When the binarized difference correction signals dg1 and dg2 are both Lo, the state comparison signal gss is in the state A. When the binarized difference correction signals dg1 and dg2 are in different states (when dg1 ≠ dg2), the state comparison signal gss is in the state B. When the binarized difference correction signals dg1 and dg2 are both Hi, the state comparison signal gss is in the state C.

20A to 20E are diagrams for explaining the operation of the state correction unit 29. FIG. The state comparison signal gss input from the state comparison unit 21 is a signal having three states A, B, and C as shown in FIG. When the state of the signal gss is C, the state correction unit 29 performs correction according to the state of the binarized difference correction signal dg3 (FIGS. 20B and 20D), and the state correction signal (flag) gs (FIGS. 20C and 20E) is generated.
Specifically, when the signal gss is in the state A, gs is set to the state A (same as gss), and when the signal gss is the state B, gs is set to the state B (same as gss).
On the other hand, if the signal gss is in state C and dg3 = Lo, gs is in state D.
As shown in FIG. 20D, when the signal gss is in state C and dg3 = Hi, gs is in state C (same as gss) as shown in FIG.
As described above, the state correction signal gs output from the state correction unit 29 is a signal having four states.

21A to 21C are diagrams illustrating the operation of the blur determination unit 26. FIG.
The state correction signal gs shown in FIG. 21A output from the state determination unit 25 is a signal having four states A, B, C, and D, and the contour shape detection flag ef is shown in FIG. As shown, it is a binary signal.
The blur determination unit 26 operates on the state B, C, and D signals of the state correction signal gs. When the contour shape detection flag ef is Lo, the signals of the states B, C, and D are changed to the state A. Convert and output. State A is a state that is not subject to blur correction.
On the other hand, when the contour shape detection flag ef is Hi, the signals of the states B, C, and D are output as B, C, and D as they are. Note that the signal of the state A is output as it is regardless of whether the contour shape detection flag ef is Hi or Lo. In this way, the blur determination unit 26 generates and outputs a motion blur detection flag bf (FIG. 21C) having four states A, B, C, and D.

The motion blur detection flag bf having four states A, B, C, and D output from the blur detection unit 2 and the video signals d1, d2, and d3 output from the delay unit 1 are sent to the blur correction unit 3. Entered.
The configuration of the blur correction unit 3 is as described with reference to FIG. 5. Here, the operation of the blur correction unit 3 will be described.

The motion blur detection flag bf input to the blur correction unit 3 is input to the contour shape calculation unit 15.
FIG. 22 is a diagram illustrating an example of the contour shape calculation unit 15. The illustrated contour shape calculation unit 15 includes a pixel counter unit 30, a state D counter unit 31, a center detection unit 32, and a core position determination unit 33. And a converted signal generator 34.
The motion blur detection flag bf is input to the pixel counter unit 30, the state D counter unit 31, and the converted signal generation unit 34.

The pixel counter unit 30 outputs a count value c1 obtained by counting the pixel clock from the start of processing of each line on the screen as data indicating the pixel position on the line.
The state D counter unit 31 uses the count value c2 obtained by counting the pixel clock from the start time to the end time of the state D included in the motion blur detection flag bf as data indicating the width (duration) of the state D. Output.
The pixel clock is a clock generated in order to synchronize processing in each unit in the image processing apparatus, and a signal of one pixel is processed each time the pixel clock is generated.

The count results c1 and c2 output from the pixel counter unit 30 and the state D counter unit 31 are input to the center detection unit 32.
The center detector 32 detects the center position (data indicating) c3 in the state D period from the count results c1 and c2. The detected center position c3 of the state D period is output to the core position determination unit 33. The core position determination unit 33 calculates a core region at the time of blur correction using a threshold value S7 given in advance, and outputs a core region determination flag c4.
The core region determination flag c4 is sent to the converted signal generator 34. The conversion signal generation unit 34 outputs a conversion control signal j based on the input motion blur detection flag bf and the core area determination flag c4.

FIGS. 23A to 23D show an operation of generating the core region determination flag c4 from the motion blur detection flag bf, the pixel counter output c1, and the state D counter output c2.
When the motion blur detection flag bf shown in FIG. 23A is input to the state D counter unit 31, the count value c2 (FIG. 23C) of the state D is counted up to 9, and the maximum count value c2max is 9 It becomes. The center detection unit 32 is based on the maximum count value c2max of the state D and the value c1m (FIG. 23B) of the output c1 of the pixel counter unit 30 when the maximum count value c2max is reached.
c3 = c1m- (c2max-1) / 2
= 12- (9-1) / 2 = 8
Is calculated (by rounding up or down) to obtain the center position c3 of the state D. In the illustrated example, c3 is “8”. The center position (data indicating) c <b> 3 is output to the core position determination unit 33.

  The core position determination unit 33 calculates the core region at the time of blur correction based on the center position c3 of the state D output from the center detection unit 32 and a predetermined threshold value S7, and the core region determination flag c4 (FIG. 23). (D)) is output. For example, c3-S7 to c3 + S7 are set as the core area, and the core area determination flag c4 is set to Hi during that period. For example, it is assumed that the threshold value S7 is “2”.

  Since the center position c3 of the state D output from the center detection unit 32 under the conditions shown in FIGS. 23A to 23C is “8”, the core region determination flag c4 is c1b = c3−S7 = 8. At a pixel position from −2 = 6 to c1e = c3 + S7 = 8 + 2 = 10, c4 (FIG. 23D) becomes Hi.

FIGS. 24A to 24C are diagrams for explaining the operation of the converted signal generation unit 34. The motion blur detection flag bf (FIG. 24A) and the core region determination flag c4 (FIG. 24) that are input. The relationship between (b)) and the output conversion control signal j (FIG. 24C) is shown. In the conversion signal generation unit 34, the state of the input motion blur detection flag bf is converted into the state E and output during the period when the core region determination flag c4 is Hi. On the other hand, during the period when c4 is Lo, bf is output as it is as the conversion control signal j.
That is, the conversion control signal j output from the conversion signal generation unit 34 is a signal having five states A, B, C, D, and E.

  The conversion control signal j output from the contour shape calculation unit 15 is input to the pixel conversion unit 16 together with the video signals d1, d2, and d3. The pixel conversion unit 16 generates a video signal k to be output to the image display unit 4 from the input video signals d1, d2, and d3 based on the state of the conversion control signal j.

FIGS. 25A to 25C show an example of the process of generating the video signal k.
Assume that the video signals d1, d2, and d3 input to the pixel conversion unit 16 are as shown in FIG. 25A, and the conversion control signal j is as shown in FIG. Here, when the conversion control signal j is in the state B or D,
If | d2-d1 |> | d2-d3 |, output d3 as the output video signal k.
If | d2-d1 | ≦ | d2-d3 |, d1 is output as the output video signal k (FIG. 25 (c)). That is, of d3 and d1, a signal having a small difference from d2 is output as an output video signal k.

When the conversion control signal j is other than the states B and D (that is, when the conversion control signal j is in the states A, C, and E), d2 is output as the output video signal k. In this way, the video signal d2 is output as k in the state E (core region located in the central part of the blurring period), and the signal is output in the state D (area other than the core area in the blurring period). A signal having a smaller difference from d2 out of d3 and d1 is output as an output video signal k.
By such processing, it is possible to narrow the transition width of the contour that causes blurring, as compared with the case where the video signal d2 is used for display without performing this processing.

As described above, when the center position c3 of the state D and the predetermined threshold value s7 are used as the core region from c3 to S7 to c3 + S7, and the gradation value gradually changes outside the core region, By substituting the gradation value of that portion with a smaller value or a larger value, as a result, a gradual gradation change can be sharpened. On the other hand, when the gradation change is originally steep (for example, when the gradation changes only within the core region), the gradation change is not sharpened.
As described above, the isotropic blur that is not caused by the movement has a relatively rapid gradation change, and the anisotropic blur that is caused by the movement has a relatively gentle gradation change. By selecting, the gradation change can be sharpened only in the case of the anisotropic blur caused by the movement without sharpening the gradation change with respect to the isotropic blur.

  In the above-described embodiment, processing is performed using the video signal of the previous frame and the video signal of the next frame in addition to the current frame video signal. On the other hand, processing may be performed using a video signal before a predetermined number of frames other than 1 and a video signal after a predetermined number of frames other than 1.

  As described above, by detecting blur based on the contour shape of the input image signal and the difference amount between frames of the contour portion, it is possible to detect inequality blur peculiar to motion blur. It is possible to improve the image quality when displaying moving images by detecting the area where motion blur is included in the video and narrowing the blur width according to the detected motion blur size. It becomes possible.

  Further, since the blur correction is performed only for the detected unequal blur, unnecessary sharpening is not performed on the contour orthogonal to the motion direction. Further, even if an image having a gradual change in luminance such as gradation is mistaken, no filtering process is performed.

  In addition, motion blur is detected without using motion vector detection, and the detected motion blur is corrected only by pixel replacement between frames without using filtering means. become. Furthermore, since the circuit scale is relatively small, an energy saving effect can be obtained.

It is a block diagram which shows the image display apparatus which concerns on Embodiment 1 of this invention. 3 is a block diagram illustrating an example of a delay unit 1. FIG. It is a block diagram which shows an example of the blur detection part 2. 5 is a diagram illustrating an example of a contour shape determination unit 35. FIG. 3 is a diagram illustrating an example of a blur correction unit 3. FIG. (A)-(f) is a figure explaining operation | movement of the delay part 1. FIG. It is the figure which showed the example of the video signal. It is the figure which showed the example of the video signal from which the high frequency component was removed. (A) And (b) is a figure which shows the example of the video signal of 3 continuous frames. It is the figure which showed an example of the difference detection part 11. (A)-(c) is a figure explaining operation | movement of the difference detection part 11. FIG. (A)-(d) is a figure explaining operation | movement of the difference calculating part 27 and the differentiating part 28. FIG. It is the figure which showed the anisotropic blurring in the outline where a brightness | luminance changes in step shape. It is the figure which showed the anisotropic blurring in the outline where a brightness | luminance changes in step shape. It is a figure explaining operation | movement of the outline shape determination part. 3 is a diagram illustrating a configuration of a blur period determination unit 14. FIG. (A) And (b) is a figure explaining operation | movement of the binarization part 22. FIG. 3 is a diagram illustrating an example of a state determination unit 25. FIG. (A)-(c) is a figure explaining operation | movement of the state comparison part 21. FIG. (A)-(e) is a figure explaining operation | movement of the state correction | amendment part 29. FIG. (A)-(c) is a figure explaining operation | movement of the blur determination part 26. FIG. 5 is a diagram illustrating an example of a contour shape calculation unit 15. FIG. (A)-(d) is a figure explaining operation | movement of the outline shape calculation part 15. FIG. (A)-(c) is a figure explaining operation | movement of the conversion signal production | generation part 34. FIG. (A)-(c) is a figure explaining operation | movement of the pixel conversion part 16. FIG.

Explanation of symbols

  1 delay unit, 2 blur detection unit, 3 blur correction unit, 4 image display unit, 5 frame memory control unit, 6 frame memory, 7, 8, 9 low-pass filter, 11, 12 difference detection unit, 14 blur period determination unit, DESCRIPTION OF SYMBOLS 15 Outline shape calculation part, 16 Judgment flag production | generation part, 17 Trinization part, 18 Judgment flag production | generation part, 19 Difference calculation part, 20 Difference correction part, 21 State comparison part, 22, 23, 24 Binarization part, 25 State determination unit, 26 blur determination unit, 27 difference detection unit, 28 differentiation unit, 29 state correction unit, 30 pixel counter unit, 31 state D counter unit, 32 center detection unit, 33 core position determination unit, 34 conversion signal generation unit , 35 contour shape determination unit, 36 differentiation unit, 37 differentiation result storage unit, 38 continuity determination unit, 81 projection Display device.

Claims (10)

A first video signal that is not frame-delayed with respect to the input video signal; a second video signal that is delayed by a predetermined number of frames with respect to the input video signal; and the second video signal A delay means for generating a third video signal delayed by the predetermined number of frames;
A first low-pass filter for removing a high-frequency component of the first video signal and outputting a first filtered video signal;
A second low-pass filter for removing a high-frequency component of the second video signal and outputting a second filtered video signal;
A third low-pass filter that removes high-frequency components of the third video signal and outputs a third filtered video signal;
First difference detection means for detecting a gradation difference between the first filtered video signal and the second filtered video signal;
Second difference detection means for detecting a gradation difference between the second filtered video signal and the third filtered video signal;
Third difference detection means for detecting a gradation difference between the third filtered video signal and the first filtered video signal;
Differentiating means for detecting an inter-pixel change in the video signal output by the third difference detecting means;
Contour shape determining means for determining continuity of gradation change in the second video signal;
The gradation difference detected by the first difference detection means, the gradation difference detected by the second difference detection means, the detection result of the inter-pixel change detected by the differentiation means, A blur period determination unit that detects a motion blur period based on a determination result about the continuity of gradation change of the second video signal detected by the contour shape determination unit;
Correction means for correcting the second video signal based on the detection result of the motion blur period by the blur period determination means;
The image processing apparatus, wherein the blur correction unit corrects the second video signal only during the motion blur period detected by the blur period determination unit.   The delay means includes a frame memory capable of storing a plurality of input video signals for a plurality of frames, generates the first video signal without delaying the input video signal, and generates the first video signal. The second video signal is generated by delaying the signal by the predetermined frame number using the frame memory, and the second video signal is delayed by the predetermined frame number by using the frame memory. The image processing apparatus according to claim 1, wherein a third video signal is generated. The delay means has a frame memory capable of storing two frames,
Generating the first video signal, a second video signal obtained by delaying the first video signal by one frame, and a third video signal obtained by delaying the second video signal by one frame. The image processing apparatus according to claim 2. The blur period determining means includes
The absolute value of the gradation difference output from the first difference detection means is greater than a predetermined value;
The absolute value of the gradation difference output from the second difference detection means is greater than a predetermined value;
The absolute value of the inter-pixel change output from the differentiating means is smaller than a predetermined value;
When it is determined by the contour shape determination means that the gradation change has continuity,
The image processing apparatus according to claim 1, wherein the image processing apparatus determines that the period is a blur period. The contour shape determining means includes
A differential unit that receives the second video signal and obtains a differential result for the pixel of interest by obtaining a difference from the immediately preceding pixel for each pixel of interest;
A differentiation result storage unit for storing a differentiation result detected by the differentiation unit for a predetermined number of pixels;
The differentiation result for each pixel of interest output from the differentiating unit and the differentiation result for a plurality of pixels located before the pixel of interest are received, and all the differentiation results for the plurality of pixels are the attention When the gradation change is within a predetermined range including the differential result for the pixel, it is determined that the gradation change has continuity, and any of the differential results for the plurality of pixels represents the differential result for the target pixel. 5. The image processing apparatus according to claim 4, wherein the gradation change is determined not to be continuous when it is out of a predetermined range. The predetermined range including the differentiation result for the pixel of interest has a second predetermined value in the differentiation result for the pixel of interest from a value obtained by subtracting the first predetermined value from the differentiation result for the pixel of interest. The image processing apparatus according to claim 5, wherein the range is up to the added value.   The blur correction unit selects and outputs one of the first video signal, the second video signal, and the third video signal based on the result determined by the blur period determination unit. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The blur correction means includes
An outline shape calculation unit and a pixel conversion unit;
The contour shape calculation unit
Detecting a core region located in a central portion of the blur period detected by the blur period determination means;
The pixel conversion unit
In the core region, the second video signal is selected and output,
Of the blur period, in a region other than the core region, the first video signal and the third video signal having a smaller difference from the second video signal are output. The image processing apparatus according to claim 7. The image processing apparatus according to any one of claims 1 to 8,
An image display device comprising display means for displaying an image based on the image data output from the image processing device. A first video signal that is not frame-delayed with respect to the input video signal; a second video signal that is delayed by a predetermined number of frames with respect to the input video signal; and the second video signal A delay step of generating a third video signal delayed by the predetermined number of frames;
A first low-pass filtering step of removing a high-frequency component of the first video signal and outputting a first filtered video signal;
A second low-pass filtering step of removing a high-frequency component of the second video signal and outputting a second filtered video signal;
A third low-pass filtering step of removing a high-frequency component of the third video signal and outputting a third filtered video signal;
A first difference detection step of detecting a gradation difference between the first filtered video signal and the second filtered video signal;
A second difference detecting step for detecting a gradation difference between the second filtered video signal and the third filtered video signal;
A third difference detecting step for detecting a gradation difference between the third filtered video signal and the first filtered video signal;
A differentiation step of detecting an inter-pixel change in the video signal output by the third difference detection step;
A contour shape determination step for determining continuity of gradation change in the second video signal;
The gradation difference detected in the first difference detection step, the gradation difference detected in the second difference detection step, the detection result of the inter-pixel change detected in the differentiation step, and A blur period determination step for detecting a motion blur period based on a determination result about the continuity of gradation change of the second video signal detected in the contour shape determination step;
A correction step of correcting the second video signal based on the detection result of the motion blur period in the blur period determination step;
The image processing method, wherein the blur correction step corrects the second video signal only during the motion blur period detected in the blur period determination step.

Images