JP2010261985A - Image processing device, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform over-drive processing with simpler constitution. <P>SOLUTION: In an image generating part 105, an interpolation image interpolated between a first image and a second image is generated, on the basis of the first images constituting a first moving image and of the second images older than the first images. In a switching part 82, a plurality of images constituting the second moving images are sequentially set as an object images, while the images older than the object image among a plurality of images constituting the second moving image are set to reference image. In an over-drive processing part 83, the object image is corrected by over-drive processing on the basis of the reference image. Also, the switching part 82 synchronizes the object image with the reference image and outputs them to the over-drive processing part 83. This invention, for example, ia applicable to a personal computer or the like which displays an image on a hold-type display. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly to an image processing apparatus, an image processing method, and a program that are suitable for use in performing overdrive processing that improves the response speed of a display.

  For example, in a hold-type display represented by an LCD (liquid crystal display) or the like, when a moving image composed of a plurality of frames (or fields) is displayed, motion blur inherent to the hold-type display is displayed in the displayed frame. Can occur.

  Therefore, there is an overdrive process for improving the response speed of the display in order to suppress the deterioration of the image quality due to the motion blur generated in the frame.

  FIG. 1 shows a configuration example of a conventional image processing apparatus 1 that performs overdrive processing for improving the response speed of a display.

  The image processing apparatus 1 includes a gamma correction unit 21, an overdrive processing unit 22, a timing controller 23, and an LCD 24.

  The gamma correction unit 21 performs gamma correction on each frame (RGB signal) constituting the input moving image, and supplies the frame after the gamma correction to the overdrive processing unit 22.

  The overdrive processing unit 22 performs overdrive processing on the frame after the gamma correction from the gamma correction unit 21 and supplies the frame after the overdrive processing to the timing controller 23.

  The overdrive processing unit 22 includes a frame memory 41, a correction value generation unit 42, and an addition unit 43.

  In the overdrive processing unit 22, the frame after the gamma correction from the gamma correction unit 21 is supplied to the frame memory 41, the correction value generation unit 42, and the addition unit 43.

  The frame memory 41 delays the frame after the gamma correction from the gamma correction unit 21 by one frame period and outputs the delayed frame to the correction value generation unit 42. That is, the correction value generation unit 42 outputs the nth frame n after the gamma correction output from the gamma correction unit 21 and the (n−1) th frame n− after the gamma correction output from the frame memory 41. 1 is entered.

  The correction value generation unit 42 incorporates a lookup table 42a. In order to improve the response speed of the LCD 24, the look-up table 42a sets the correction value added to the frame n after the gamma correction by the gamma correction unit 21 to the pixel value of the pixel of the frame n and the corresponding frame n-1. Each pixel value is stored in association with each pixel value.

  The look-up table 42a is generated based on experiments performed in advance. The same applies to the embodiments described later.

  The correction value generation unit 42 includes the pixel value of the pixel constituting the frame n from the gamma correction unit 21 and the pixel value of the pixel constituting the frame n-1 from the frame memory 41 existing at the same position as the pixel. The correction value associated with the combination is read from the built-in lookup table 42 a and supplied to the adding unit 43.

  The adder 43 adds the pixel values of the pixels constituting the frame n from the gamma correction unit 21 to the corresponding correction values from the correction value generation unit 42, and displays the resulting display frame as a timing controller. 23.

  The timing controller 23 supplies the display frame from the adder 43 to the LCD 24 for display at a predetermined timing.

  The LCD 24 displays a display frame from the timing controller 23.

  As described above, the overdrive processing unit 22 of the conventional image processing apparatus 1 uses the frame n supplied from the gamma correction unit 21 and the frame n-1 held in the frame memory 41. Process.

  In addition, there is a memory utilization technique that exclusively uses one frame memory in two processes of an IP conversion process for converting an interlace video signal into a progressive video signal and an overdrive process (for example, (See Patent Document 1).

JP 2006-30352 A

  By the way, in the conventional image processing apparatus 1, the overdrive processing unit 22 needs to provide a frame memory 41 for holding the frame n−1 of the previous frame in order to perform the overdrive process on the frame n. there were.

  Similarly, in order to perform overdrive processing using a frame two frames before the frame n (for example, the frame n-2), a frame memory for holding the frames before the second frame must be further provided. did not become.

  As described above, in the overdrive processing, in order to perform the overdrive processing on the frame n, it is necessary to provide frame memories for the frames used for the overdrive processing.

  The same applies to the conventional memory utilization technology.

  The present invention has been made in view of such circumstances, and enables overdrive processing to be performed with a simpler configuration.

  An image processing apparatus or program according to one aspect of the present invention provides an image processing apparatus that converts a first moving image into a second moving image having a higher frame rate, or a first moving image that has a higher frame rate. A program for functioning as a computer of an image processing apparatus for converting to the second moving image, wherein an image constituting the first moving image is a first image, the first image, and the first image An image constituting the second moving image is an interpolated image interpolated between the first image and the second image based on a second image that is earlier in time than the first image. And a first setting for setting a plurality of images constituting the second moving image as a target image that is a processing target of overdrive processing that sequentially corrects the images to be displayed on the display unit. Means and the second moving image A second setting unit that sets an image that is earlier in time than the target image as a reference image that is referred to for correcting the target image by the overdrive processing among the plurality of images to be formed; An image including overdrive processing means for correcting the target image by the overdrive processing based on a reference image, and output means for outputting the target image and the reference image to the overdrive processing means in synchronization with each other A program for causing a processing apparatus or an image processing apparatus to function.

  The first setting means sequentially sets the interpolation image and the first image as the target image, and the second setting means sets the target among the second image and the interpolation image. An image that is earlier in time than the image can be set as the reference image.

  In the image generating means, based on the first and second images, among the images constituting the second moving image, 1 to n-1 (n is 2 in terms of time) than the second image. The first to (n-1) th extrapolated images representing each of the past images (natural number) are generated, and the second setting means generates the first to (n-1) th extrapolated images, the first Of the two images and the interpolated image, 1 to n past images in time from the target image can be set as the first to nth reference images, respectively.

  The output means synchronously outputs the target image and the first to nth reference images to the overdrive processing means, and the overdrive processing means is based on the first to nth reference images. Thus, the target image can be corrected.

  The first reference image is an image that is one image earlier in time than the target image among the plurality of images constituting the second moving image, and the overdrive processing means includes the Prediction value generation for generating a prediction value representing the gray level of the display unit that is reached when the first reference image corrected by the overdrive processing unit is displayed based on the first to nth reference images Means for generating a correction value to be added to the target image to correct the target image based on the target image and the predicted value; and adding the correction value to the target image Then, an adding means for correcting the target image can be provided.

  Motion vector detecting means for detecting a motion vector based on the first and second images can be further provided, and the image generating means is configured to detect the interpolated image and the first to first based on the motion vector. n-1 extrapolated images can be generated.

  The image generation means can generate a plurality of the interpolation images based on the first and second images.

  An image processing method according to an aspect of the present invention is an image processing method of an image processing apparatus that converts a first moving image into a second moving image having a higher frame rate, and the image processing apparatus includes: Means, a first setting means, a second setting means, an overdrive processing means, and an output means, wherein the image generating means displays an image constituting the first moving image as a first image. And an interpolated image interpolated between the first image and the second image based on the first image and the second image that is temporally past the first image. Overdriving the second moving image as an image constituting the second moving image, wherein the first setting means sequentially corrects the plurality of images constituting the second moving image into images to be displayed on the display unit. A second setting unit configured to set a target image to be processed; Among the plurality of images constituting the second moving image, an image that is temporally past the target image is set as a reference image that is referred to for correcting the target image by the overdrive processing. The overdrive processing unit corrects the target image based on the reference image by the overdrive processing, and the output unit synchronizes the target image and the reference image, and the overdrive processing unit. Is an image processing method including a step of outputting to.

  According to an aspect of the present invention, an image constituting the first moving image is a first image, and the first image and a second image that is earlier in time than the first image. The interpolated image interpolated between the first image and the second image is generated as an image constituting the second moving image, and a plurality of images constituting the second moving image Are set as target images to be processed in the overdrive process for sequentially correcting the images to be displayed on the display unit, and among the plurality of images constituting the second moving image, the time is longer than the target image. In particular, a past image is set as a reference image that is referred to in order to correct the target image by the overdrive processing, and the target image that is output in synchronization with the reference image is based on the reference image. , Compensated by the overdrive process It is.

  According to the present invention, overdrive processing can be performed with a simpler configuration.

It is a block diagram which shows the structural example of the conventional image processing apparatus. It is a block diagram which shows the structural example of the image processing apparatus which is 1st Embodiment. It is a figure which shows the flame | frame which comprises a 1st moving image. It is a figure which shows the 2nd moving image obtained by double speed interpolation. It is a block diagram which shows the 1st structural example of an overdrive process part. It is a flowchart for demonstrating 1st image processing. It is a 1st figure for demonstrating the case where a suitable correction value is not produced | generated. It is a 2nd figure for demonstrating the case where a suitable correction value is not produced | generated. It is a 3rd figure for demonstrating the case where a suitable correction value is not produced | generated. It is a block diagram which shows the structural example of the image processing apparatus which is 2nd Embodiment. It is a figure which shows the flame | frame used for an overdrive process. It is a block diagram which shows the 2nd structural example of an overdrive process part. It is a flowchart for demonstrating 2nd image processing. It is a block diagram which shows the 3rd structural example of an overdrive process part. It is a figure which shows the 2nd moving image obtained by quadruple speed interpolation. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Hereinafter, modes for carrying out the invention (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. First Embodiment 2. FIG. Second Embodiment 3. FIG. Modified example

<1. First Embodiment>
[Configuration Example of Image Processing Device 61]
FIG. 2 shows a configuration example of the image processing apparatus 61 according to the first embodiment.

  Note that the image processing device 61 improves the response speed of display display or the like by overdrive processing while converting the input first moving image to a second moving image having a high frame rate.

  The image processing device 61 includes a reference image generation unit 81, a switching unit 82, an overdrive processing unit 83, a timing controller 84, and an LCD 85.

  The reference image generation unit 81 interpolates (generates) a new frame based on a plurality of frames constituting the input first moving image, and sets the new frame as a frame constituting the second moving image. A plurality of frames constituting the first moving image to be input are supplied to the switching unit 82.

  The reference image generation unit 81 includes a frame memory 101, a frame delay unit 102, a frame memory 103, a motion vector detection unit 104, and a pixel generation unit 105.

  Frames constituting the first moving image to be input are sequentially input to the frame memory 101 and the frame delay unit 102.

  The frame memory 101 holds an input frame.

  The frame delay unit 102 delays the input frame by one frame period, and supplies the frame to the frame memory 103 for storage (holding).

  The frame memory 103 holds a frame delayed from the frame delay unit 102 by one frame period.

  That is, when the frame memory 101 holds the nth frame n among the frames constituting the first moving image to be input, the frame memory 103 is delayed by one frame period, the (n−1) th frame. Frame n-1 will be held.

  The motion vector detection unit 104 reads frame n from the frame memory 101 and frame n−1 from the frame memory 103, respectively. Then, the motion vector detection unit 104 detects, for example, a motion vector representing the motion of the frame n with respect to the frame n−1 by block matching using the read frame n and the frame n−1, and the pixel generation unit 105 To supply.

  The pixel generation unit 105 reads frame n from the frame memory 101 and frame n−1 from the frame memory 103, respectively. Then, the pixel generation unit 105 performs an interpolation process for interpolating (generating) a new frame based on the read frame n and frame n−1 and the motion vector from the motion vector detection unit 104.

  In other words, for example, the pixel generation unit 105 converts the first moving image input to the reference image generating unit 81 into a second moving image having a larger number of frames, in order to convert the frame n−1 and the frame n. In between, an interpolated frame is generated as a new frame.

  Next, an interpolation process performed by the pixel generation unit 105 will be described with reference to FIGS. 3 and 4.

[Frames constituting the first moving image]
FIG. 3 shows frames constituting the first moving image.

  Note that the rectangle shown in FIG. 3 represents the size of the image corresponding to the frame.

Also, the motion vector MV (n-1, n), the pixel 121 _n-1 in the frame n-1 is the starting point, corresponding to the pixels 121 _n-1, the motion vector and ending pixel 121 _n at frame n Show.

  When the frame n is held in the frame memory 101 and the frame n−1 is held in the frame memory 103, the pixel generation unit 105 generates an interpolation frame between the frame n−1 and the frame n.

That is, for example, the pixel generation unit 105 ends the motion vector 1 / 2MV (n-1, n) obtained by halving the motion vector MV (n-1, n) from the motion vector detection unit 104. Is determined as a position on the interpolation frame for interpolating the pixel corresponding to the pixel 121 _n−1 .

Then, the pixel generation unit 105 determines the position on the interpolation frame based on the pixel 121 _n−1 , the pixel 121 _n , the pixel existing around the pixel 121 _n−1 , the pixel existing around the pixel 121 _n , and the like. At 122, the pixels of the interpolation frame are interpolated.

  The pixel generation unit 105 performs interpolation of the pixels of the interpolation frame described above for all the motion vectors detected by the motion vector detection unit 104, and generates an interpolation frame.

[Second video obtained by double-speed interpolation]
Next, FIG. 4 shows frames constituting a second moving image obtained by interpolating the first moving image at double speed (generating one interpolation frame between frames n-1 and n). Is shown.

  In FIG. 4, a frame indicated by a white rectangle represents a frame constituting the first moving image input to the reference image generation unit 81.

  A frame indicated by a black rectangle represents an interpolation frame that is interpolated by the pixel generation unit 105.

  As illustrated in FIG. 4, the pixel generation unit 105 generates an interpolation frame arranged between the frame n−1 and the frame n. Note that the interpolation frame arranged between the frame n-1 and the frame n is the time T / from the time when the frame n is displayed, when the frame constituting the first moving image is displayed every time T. This represents a frame obtained by interpolating the frame n-1 / 2 displayed in the past by 2.

  The pixel generation unit 105 supplies the generated interpolation frame to the switching unit 82.

  In addition, the pixel generation unit 105 controls the frame memory 101 to output the held frame n to the switching unit 82, and also controls the frame 103 to output the held frame n−1 to the switching unit 82. Let

  Returning to FIG. 2, the switching unit 82 sets the frame from the reference image generation unit 81 as the target frame that is the target of the overdrive process and the reference frame that is referenced in the overdrive process, and at the same timing ( The data is supplied to the overdrive processing unit 83 (in synchronization).

  That is, for example, the switching unit 82 supplies the overdrive processing unit 83 with the frame n−1 from the frame memory 103 as a reference frame and the interpolation frame from the pixel generation unit 105 as a target frame at a certain timing.

  Then, at the next timing, the switching unit 82 supplies the overdrive processing unit 83 with the interpolation frame from the pixel generation unit 105 as a reference frame and the frame n from the frame memory 101 as a target frame.

  The overdrive processing unit 83 performs overdrive processing on the target frame from the switching unit 82 while referring to the reference frame from the switching unit 82, and supplies the display frame obtained as a result to the timing controller 84.

  The timing controller 84 supplies the display frame from the overdrive processing unit 83 to the LCD 85 for display at a predetermined timing.

  The LCD 85 displays a display frame from the timing controller 84.

[Configuration Example of Overdrive Processing Unit 83]
Next, FIG. 5 shows a configuration example of the overdrive processing unit 83.

  The overdrive processing unit 83 includes gamma correction units 141 and 142, a correction value generation unit 143, and an addition unit 144.

  The target frame is supplied from the switching unit 82 to the gamma correction unit 141. The gamma correction unit 141 performs gamma correction on the target frame from the switching unit 82 and supplies the target frame after the gamma correction to the correction value generation unit 143 and the addition unit 144.

  A reference frame is supplied from the switching unit 82 to the gamma correction unit 142. The gamma correction unit 142 performs gamma correction on the reference frame from the switching unit 82 and supplies the reference frame after the gamma correction to the correction value generation unit 143.

  The correction value generation unit 143 includes a lookup table 143a. In order to improve the response speed of the LCD 85, the look-up table 143a includes a correction value added to the target frame after the gamma correction by the gamma correction unit 141, a pixel value of the target frame pixel, and a corresponding reference frame pixel. Each pixel value is stored in association with each other.

  The correction value generation unit 143 calculates the pixel value of the pixel that forms the target frame from the gamma correction unit 141 and the pixel value of the pixel that forms the reference frame from the gamma correction unit 142 that exists at the same position as the pixel. The correction value associated with the combination is read from the built-in lookup table 143a and supplied to the adding unit 144.

  The addition unit 144 adds the pixel values of the pixels constituting the target frame from the gamma correction unit 141 to the corresponding correction values from the correction value generation unit 143, and displays the display frame obtained as a result as a timing controller. 84.

[Description of operation by first image processing]
Next, an interpolated frame is generated between the frame n-1 and the frame n based on the frame n-1 and the frame n constituting the input first moving image. First image processing for performing overdrive processing will be described.

  FIG. 6 is a flowchart for explaining the first image processing performed by the image processing apparatus 61. The first image processing is started each time a frame n (n is a natural number of 2 or more) among the frames constituting the first moving image is input to the reference image generation unit 81.

  In step S <b> 1, the frame delay unit 102 supplies the n−1th frame n−1 input last time to the frame memory 103 for storage.

  At this time, the frame memory 101 receives and holds the frame n.

  In step S <b> 2, the motion vector detection unit 104 reads frame n from the frame memory 101 and frame n−1 from the frame memory 103. Then, the motion vector detection unit 104 performs, for example, block matching using the read frame n and the frame n−1, for example, a motion vector MV (n−1, n) representing the motion of the frame n with respect to the frame n−1. Is detected.

  In step S <b> 3, as described with reference to FIG. 3, the pixel generation unit 105 uses each motion vector from the motion vector detection unit 104 to determine a position on the interpolation frame where the pixel is to be interpolated.

  Then, the pixel generation unit 105 interpolates the pixel at the determined position based on the frame n held in the frame memory 101 and the frame n-1 held in the frame 103, and generates an interpolation frame. This is supplied to the switching unit 82.

  In addition, the pixel generation unit 105 controls the frame memory 101 to output the held frame n to the switching unit 82, and also controls the frame 103 to output the held frame n−1 to the switching unit 82. Let

  In step S4, the switching unit 82 sets the interpolation frame from the pixel generation unit 105 and the frame n from the frame memory 101 as a target frame in the order of the interpolation frame and the frame n.

  In step S <b> 5, the switching unit 82 sets, as a reference frame, a frame that is one frame ahead of the target frame for the frames constituting the second moving image.

  That is, for example, when the interpolation frame is set as the target frame, the switching unit 82 sets the frame n−1 that is one frame earlier than the interpolation frame supplied from the frame memory 103 as the reference frame.

  In step S6, the switching unit 82 supplies the target frame and the reference frame to the overdrive processing unit 83 at the same timing.

  In steps S7 to S10, the overdrive processing unit 83 performs overdrive processing based on the target frame and the reference frame.

  That is, in step S7, the gamma correction unit 141 performs gamma correction on the target frame from the switching unit 82, and supplies the target frame after the gamma correction to the correction value generation unit 143 and the addition unit 144.

  In step S <b> 8, the gamma correction unit 142 performs gamma correction on the reference frame from the switching unit 82 and supplies the reference frame after the gamma correction to the correction value generation unit 143.

  In step S9, the correction value generation unit 143 determines the pixel value of the pixel that forms the target frame from the gamma correction unit 141, and the pixel that forms the reference frame from the gamma correction unit 142 that exists at the same position as the pixel. The correction value associated with the combination with the pixel value is read from the built-in lookup table 143a and supplied to the adding unit 144.

  In step S10, the addition unit 144 adds the pixel values of the pixels constituting the target frame from the gamma correction unit 141 to the corresponding correction values from the correction value generation unit 143, and the display frame obtained as a result Is supplied to the timing controller 84.

  In step S11, the timing controller 84 supplies the display frame from the adder 144 to the LCD 85 at a predetermined timing for display.

  In step S12, the switching unit 82 determines whether or not the overdrive process in steps S7 to S10 has been executed with the interpolation frame and the frame n as target frames.

  In step S12, when the switching unit 82 determines that the overdrive process in steps S7 to S10 is not executed with the interpolation frame and the frame n as the target frame, the process returns to step S4.

  In step S4, the switching unit 82 sets the frame n as the target frame, and in step S5, the switching unit 82 sets the interpolation frame as the reference frame, and the subsequent processing is performed.

  In step S12, when the switching unit 82 determines that the overdrive process in steps S7 to S10 has been executed using the interpolation frame and the frame n as the target frame, the process ends.

  As described above, in the first image processing, the switching unit 82 outputs the reference frame to the overdrive processing unit 83 at the same timing as the timing of outputting the target frame to the overdrive processing unit 83. .

  Therefore, the overdrive processing unit 83 does not need to provide a frame memory for holding the reference frame in order to perform the overdrive process on the target frame.

  Therefore, the overdrive processing unit 83 can have a simpler configuration than the overdrive processing unit 22 shown in FIG.

  Further, since the overdrive processing unit 83 does not need to provide a frame memory for holding the reference frame, the cost for providing the frame memory can be reduced.

  By the way, in the overdrive process 83, there is a situation in which an appropriate correction value r is not generated by the correction value generation unit 143 depending on the combination of the pixel value of the pixel constituting the target frame and the pixel value of the pixel constituting the reference frame. Can occur.

  Next, a case where an appropriate correction value r is not generated by the correction value generation unit 143 will be described with reference to FIGS.

  7 to 9 show the relationship between the pixel value obtained by the overdrive processing performed by the overdrive processing unit 83 and the liquid crystal response waveform of the LCD 85 corresponding to the pixel value.

  7 to 9, the horizontal axis represents time, and the vertical axis represents pixel values.

  Periods t0 to t3 indicate periods in which one display frame is displayed on the LCD 85, respectively.

  In the period t0, the overdrive processing unit 83 uses the interpolated frame generated between the frame n-2 and the frame n-1 as a reference frame and the frame n-1 having the pixel value x0 as a target frame. Process.

  In this case, in the correction value generation unit 143, the pixel value (pixel value smaller than the pixel value x0) of the pixel of the interpolation frame generated between the frame n-2 and the frame n-1 and the frame n-1 A correction value r0 associated with the combination with the pixel value x0 of the pixel is generated.

  Then, the addition unit 144 adds the generated correction value r0 and the pixel value x0 of the pixel of the frame n−1, and outputs the resultant addition value (x0 + r0) to the LCD 85 via the timing controller 84. Is done.

  In the period t0, the liquid crystal response waveform of the LCD 85 reaches the pixel value x0 (corresponding to the gradation) of the frame n-1 that is the target frame, just before the end of the period t0.

  In the period t1, the overdrive processing unit 83 performs overdrive processing using the frame n-1 as a reference frame and an interpolation frame having the pixel value x1 as a target frame.

  In this case, in the correction value generation unit 143, the correction value −r1 (r1>) associated with the combination (x0, x1) of the pixel value x0 of the pixel of the frame n−1 and the pixel value x1 of the pixel of the interpolation frame. 0) is generated.

  Then, the addition unit 144 adds the generated correction value −r1 and the pixel value x1 of the pixel of the interpolation frame, and the resultant addition value (x1−r1) is sent to the LCD 85 via the timing controller 84. Is output.

  In the period t1, the liquid crystal response waveform of the LCD 85 reaches the pixel value x1 of the interpolation frame that is the target frame, just before the end of the period t1.

  In the period t2, the overdrive processing unit 83 performs overdrive processing using the interpolation frame as a reference frame and the frame n having the pixel value x2 as a target frame.

  In this case, the correction value generation unit 143 generates the correction value r2 (> 0) associated with the combination (x1, x2) of the pixel value x1 of the pixel in the interpolation frame and the pixel value x2 of the pixel in frame n. Is done.

  Then, the addition unit 144 adds the generated correction value r2 and the pixel value x2 of the pixel in frame n, and outputs the resultant addition value (x2 + r2) to the LCD 85 via the timing controller 84. .

  In the period t2, the liquid crystal response waveform of the LCD 85 reaches the pixel value x2 included in the interpolation frame that is the target frame, just before the end of the period t2.

  In the overdrive processing unit 83, as shown in FIG. 7, the pixel value of the pixel of the target frame (pixel value x1 in the example of FIG. 7) is changed to the pixel value of the pixel of the reference frame (pixel value x0 in the example of FIG. 7). ) Is smaller than the pixel value (pixel value x1 in the example of FIG. 7) of the pixel of the target frame (addition value x1-r1 in the example of FIG. 7).

  Further, in the overdrive processing unit 83, as shown in FIG. 7, the pixel value of the pixel of the target frame (pixel value x2 in the example of FIG. 7) is changed to the pixel value of the pixel of the reference frame (pixel in the example of FIG. 7). When the value is larger than the value x1), a value larger than the pixel value of the pixel of the target frame (pixel value x2 in the example of FIG. 7) (added value x2 + r2 in the example of FIG. 7) is output as the added value.

  Therefore, at the end of the period t0 to t2, the liquid crystal response waveform of the LCD 85 matches the pixel value (corresponding to the gradation) of the target frame.

  Next, an overdrive process performed when the pixel value x0 is a value near the maximum value and the pixel value x1 is the minimum value will be described with reference to FIG.

  In the period t0, the overdrive processing unit 83 uses the interpolated frame generated between the frame n-2 and the frame n-1 as a reference frame, as in the case of FIG. 7, and the frame n having the pixel value x0. Overdrive processing is performed with -1 as the target frame.

  In the period t1, the overdrive processing unit 83 performs overdrive processing using the frame n-1 as a reference frame and an interpolation frame having the pixel value x1 as a target frame.

  In FIG. 8, the pixel value x1 is the minimum value, and the addition unit 144 may set the addition value output from the addition unit 144 to the LCD 85 via the timing controller 84 to a value smaller than the pixel value x1. Can not.

  As described above, in the overdrive processing unit 83, when the pixel value of the pixel of the target frame is smaller than the pixel value of the pixel of the reference frame, as shown in FIG. A value smaller than the pixel value is output.

  Therefore, the liquid crystal response waveform of the LCD 85 coincides with the pixel value (corresponding to the gradation) of the target frame at the end of the period t1.

  However, during the period t1 in FIG. 8, the addition value is not a value smaller than the pixel value x1 of the pixel of the target frame, but the same value x1 as the pixel value x1 of the pixel of the target frame is output to the LCD 85.

  For this reason, in the period t1 of FIG. 8, the liquid crystal response waveform of the LCD 85 does not reach the pixel value x1 of the interpolation frame that is the target frame, even just before the end of the period t1.

  In the period t2, the overdrive processing unit 83 performs overdrive processing using the interpolation frame as a reference frame and the frame n having the pixel value x2 as a target frame.

  In this case, the correction value generation unit 143 generates a correction value r2 associated with a combination (x1, x2) of the pixel value x1 of the pixel in the interpolation frame and the pixel value x2 of the pixel in frame n.

  Then, the addition unit 144 adds the generated correction value r2 and the pixel value x2 of the pixel in frame n, and outputs the resultant addition value (x2 + r2) to the LCD 85 via the timing controller 84. .

  The addition value (x2 + r2) calculated by the adder 144 is the liquid crystal response waveform of the LCD 85 when the liquid crystal response waveform of the LCD 85 is the pixel value x1 just before the end of the period t1, and the pixel value just before the end of the period t2. x2 is reached.

  However, in the period t1, the liquid crystal response waveform of the LCD 85 does not coincide with the pixel value x1 even at the end of the period t1, and is a value x1 ′ larger than the pixel value x1.

  Accordingly, in the period t2, the liquid crystal response waveform of the LCD 85 rises not from the pixel value x1 but from the value x1 ′, and greatly exceeds the pixel value x2 of the frame n that is the target frame at the end of the period t2. .

  In this case, since the liquid crystal response waveform of the LCD 85 is significantly different from the pixel value x2 just before the end of the period t2, motion blur occurs in the display frame displayed on the LCD 85.

  For this reason, as shown in FIG. 9, the overdrive processing unit 83 calculates a value that the liquid crystal response waveform of the LCD 85 reaches just before the end of the period t1 as the predicted value x1 ′, and calculated it during the period t2. It is desirable to generate the correction value from the combination (x1 ′, x2) of the predicted value x1 ′ and the pixel value x2 of the pixel of the frame n that is the target frame.

<2. Second Embodiment>
[Configuration Example of Image Processing Device 161]
Next, FIG. 10 shows a configuration example of the image processing apparatus 161 according to the second embodiment.

  The image processing device 161 converts the input first moving image into a second moving image having a high frame rate, and continuously arranges three frames among the frames constituting the second moving image. The overdrive process using this frame is performed.

  The image processing device 161 includes a reference image generation unit 181, a switching unit 182, an overdrive processing unit 183, a timing controller 184, and an LCD 185.

  The reference image generation unit 181 interpolates a new frame based on a plurality of frames constituting the input first moving image, and inputs the new frame as a frame constituting the second moving image. A plurality of frames constituting the first moving image are supplied to the switching unit 182.

  Note that, in the reference image generation unit 181, portions that are configured in the same manner as the reference image generation unit 81 of FIG.

  That is, the reference image generation unit 181 is configured in the same manner except that the pixel generation unit 201 is provided instead of the pixel generation unit 105.

  The pixel generation unit 201 reads frame n from the frame memory 101 and frame n−1 from the frame memory 103, respectively. Then, the pixel generation unit 201 will be described in detail below with reference to FIG. 11 based on the read frame n and frame n−1 and the motion vector MV (n−1, n) from the motion vector detection unit 104. An interpolation frame and an extrapolation frame to be described are generated.

  FIG. 11 shows frames used by the overdrive processing unit 183 among the frames constituting the second moving image.

  Similar to the pixel generation unit 105, the pixel generation unit 201 generates an interpolation frame between the frame n−1 and the frame n and supplies the interpolation frame to the switching unit 182.

  In addition, the pixel generation unit 201 generates a frame corresponding to the interpolation frame generated between the frame n−2 and the frame n−1 in the overdrive processing unit 183 described later, An extrapolated frame used for the overdrive process of the interpolated frame generated in between is generated and supplied to the switching unit 182.

That is, for example, the pixel generation unit 201 obtains a motion vector −1 / 2MV (n−1, n) obtained by multiplying the motion vector MV (n−1, n) from the motion vector detection unit 104 by −½. Is determined as the position on the extrapolated frame where the pixel corresponding to the pixel 121 _n-1 is interpolated.

In addition, the pixel generation unit 201 performs extrapolation determined based on the pixels 121 _n−1 , the pixels 121 _n , the pixels existing around the pixels 121 _n−1 , the pixels existing around the pixels 121 _n , and the like. The extrapolated frame pixel is interpolated at a position on the frame.

  Then, the pixel generation unit 201 supplies an extrapolated frame generated by interpolation to the switching unit 182.

  Returning to FIG. 10, the switching unit 182 sets the frame from the reference image generation unit 181 as the target frame that is the target of the overdrive process, and the first and second reference frames that are referred to in the overdrive process. , And supplied to the overdrive processing unit 183 at the same timing.

  That is, for example, at a certain timing, the switching unit 182 uses the extrapolated frame from the pixel generation unit 201 as the second reference frame, sets the frame n−1 from the frame memory 103 as the first reference frame, and generates pixels. The interpolation frame from the unit 201 is supplied to the overdrive processing unit 183 as a target frame.

  At the next timing, the switching unit 182 uses the frame n−1 from the frame memory 103 as the second reference frame, the interpolation frame from the pixel generation unit 201 as the first reference frame, The frame n is supplied to the overdrive processing unit 183 as a target frame.

  The overdrive processing unit 183 performs overdrive processing on the target frame from the switching unit 182 while referring to the first and second reference frames from the switching unit 182, and displays the display frame obtained as a result as a timing controller. 184.

  The timing controller 184 supplies the display frame from the overdrive processing unit 183 to the LCD 185 for display at a predetermined timing.

  The LCD 185 displays the display frame from the timing controller 184.

[Configuration Example of Overdrive Processing Unit 183]
Next, FIG. 12 uses the first and second reference frames to generate prediction values in the first reference frame, and refers to the generated prediction values to overdrive the pixel values of the pixels in the target frame. The example of a structure of the overdrive process part 183 which performs a process is shown.

  In the overdrive processing unit 183, the same reference numerals are given to the same components as those of the overdrive processing unit 83 in FIG.

  That is, the overdrive processing unit 183 has the same configuration as the overdrive processing unit 83 in FIG. 5 except that gamma correction units 221 and 222 and a predicted value generation unit 223 are provided instead of the gamma correction unit 142. Has been.

  The gamma correction unit 221 is supplied with the first reference frame from the switching unit 182. The gamma correction unit 221 performs gamma correction on the first reference frame from the switching unit 182 and supplies the first reference frame after the gamma correction to the predicted value generation unit 223.

  The gamma correction unit 222 is supplied with the second reference frame from the switching unit 182. The gamma correction unit 222 performs gamma correction on the second reference frame from the switching unit 182, and supplies the second reference frame after the gamma correction to the predicted value generation unit 223.

  Based on the first reference frame from the gamma correction unit 221 and the second reference frame from the gamma correction unit 222, the prediction value generation unit 223 displays the first reference frame on which overdrive processing has been performed on the LCD 185. A value that the liquid crystal response waveform of the LCD 185 reaches when it is displayed is generated as a predicted value.

  The predicted value generation unit 223 includes a difference value generation unit 241 and an addition unit 242.

  The difference value generation unit 241 includes a lookup table 241a. In the lookup table 241a, from the predicted value (for example, predicted value x1 ′ in FIG. 9) that the liquid crystal response waveform of the LCD 185 reaches when the first reference frame subjected to the overdrive process is displayed on the LCD 185. The difference value obtained by subtracting the pixel value of the pixel of the first reference frame (for example, the pixel value x1 in FIG. 9) is associated with each combination of the pixel values of the first reference frame and the second reference frame. It has been.

  The difference value generation unit 241 is supplied with the first reference frame from the gamma correction unit 221 and the second reference frame from the gamma correction unit 222.

  The difference value generation unit 241 configures the pixel value of the pixel constituting the first reference frame from the gamma correction unit 221 and the second reference frame from the gamma correction unit 222 existing at the same position as the pixel. The difference value associated with the combination with the pixel value of the pixel is read from the built-in lookup table 241a and supplied to the adding unit 242.

  The first reference frame is supplied from the gamma correction unit 221 to the addition unit 242.

  The adding unit 242 adds the pixel values of the pixels constituting the first reference frame from the gamma correction unit 221 to the corresponding difference values from the difference value generation unit 241, and is configured by the predicted values obtained as a result. The predicted frame is supplied to the correction value generation unit 143.

  Note that the correction value generation unit 143 includes the pixel value of the pixel that forms the target frame from the gamma correction unit 141 and the pixel value of the pixel that forms the prediction frame from the addition unit 242 that exists at the same position as the pixel ( The correction value associated with the combination with the predicted value) is read from the built-in lookup table 143a and supplied to the adding unit 144.

[Explanation of operation by second image processing]
Next, based on the frame n-1 and the frame n constituting the input first moving image, the interpolation frame between the frame n-1 and the frame n, and the frame n-2 and the frame n-1 A second image process in which an extrapolated frame is generated during this period and an overdrive process is performed on the interpolated frame and frame n will be described.

  FIG. 13 is a flowchart for explaining the second image processing performed by the image processing apparatus 161. The second image processing is started each time frame n is input to the reference image generation unit 181 among the frames constituting the first moving image.

  In step S31 and step S32, processing similar to that in step S1 and step S2 in FIG. 6 is performed.

  In step S <b> 33, the pixel generation unit 201 reads frame n from the frame memory 101 and frame n−1 from the frame memory 103.

  The pixel generation unit 201 multiplies the motion vector MV (n−1, n) from the motion vector detection unit 104 by −1/2, and obtains a motion vector −1 / 2MV (n−1, n) obtained as a result. The position of the end point is determined as a position for interpolating the pixel (the pixel value) of the extrapolation frame.

  Then, the pixel generation unit 201 interpolates the pixel at the determined position on the extrapolated frame based on the read frame n and frame n-1.

  The pixel generating unit 201 generates an extrapolated frame by interpolating the pixels of the extrapolated frame for all the motion vectors from the motion vector detecting unit 104, and supplies the extrapolated frame to the switching unit 182.

  In step S <b> 34, the pixel generation unit 201 performs processing similar to that in step S <b> 3 in FIG. 6, generates an interpolation frame, and supplies it to the switching unit 182.

  In addition, the pixel generation unit 201 controls the frame memory 101 to output the held frame n to the switching unit 182, and controls the frame 103 to output the held frame n−1 to the switching unit 182. Let

  In step S35, the switching unit 182 sets the interpolation frame from the pixel generation unit 201 and the frame n from the frame memory 101 as the target frame in the order of the interpolation frame and the frame n.

  In step S36, the switching unit 182 sets, as the first reference frame, a frame that is one frame older than the target frame, and a frame that is two frames older than the target frame. Are set in the second reference frame, respectively.

  That is, for example, when the interpolation frame is set as the target frame, the switching unit 182 sets the frame n−1 that is past one frame from the interpolation frame as the first reference frame and the past two frames from the interpolation frame. Are set as second reference frames.

  In step S37, the switching unit 182 supplies the target frame, the first reference frame, and the second reference frame to the overdrive processing unit 183 at the same timing.

  In steps S38 to S44, the overdrive processing unit 183 performs overdrive processing using the target frame, the first reference frame, and the second reference frame.

  In step S <b> 38, the gamma correction unit 141 performs gamma correction on the target frame from the switching unit 182, and supplies the target frame after the gamma correction to the correction value generation unit 143 and the addition unit 144.

  In step S39, the gamma correction unit 221 performs gamma correction on the first reference frame from the switching unit 182, and uses the first reference frame after the gamma correction as the difference value generation unit 241 of the prediction value generation unit 223. And to the adder 242.

  In step S <b> 40, the gamma correction unit 222 performs gamma correction on the second reference frame from the switching unit 182, and uses the second reference frame after the gamma correction as a difference value generation unit 241 of the predicted value generation unit 223. To supply.

  In step S <b> 41, the difference value generation unit 241 includes the pixel value of the pixel that forms the first reference frame from the gamma correction unit 221 and the second reference from the gamma correction unit 222 that is present at the same position as the pixel. The difference value associated with the combination of the pixel values of the pixels constituting the frame is read from the built-in lookup table 241a and supplied to the adding unit 242.

  In step S42, the addition unit 242 adds the pixel values of the pixels constituting the first reference frame from the gamma correction unit 221 with the corresponding difference values from the difference value generation unit 241 and obtains the result. A prediction frame configured by the prediction value is supplied to the correction value generation unit 143.

  In step S42, the predicted frame supplied from the adding unit 242 to the correction value generating unit 143 is reached when the first reference frame is used as the target frame and the processing of steps S38 to S44 is performed and displayed on the LCD 185. This is a frame composed of pixels having pixel values representing gradations to be reproduced.

  In step S43, the correction value generation unit 143 includes the pixel value of the pixel that forms the target frame from the gamma correction unit 141 and the pixel of the pixel that forms the prediction frame from the addition unit 242 that exists at the same position as the pixel. The correction value associated with the combination with the value is read from the built-in lookup table 143a and supplied to the adding unit 144.

  In step S44 and step S45, processing similar to that in step S10 and step S11 in FIG. 6 is performed.

  In step S46, the switching unit 182 determines whether or not the overdrive process in steps S38 to S44 has been executed with the interpolation frame and the frame n as target frames.

  In step S46, when the switching unit 182 determines that the interpolation frame and the frame n are the target frames and the overdrive process in steps S38 to S44 is not executed, the process returns to step S35.

  In step S35, the switching unit 182 sets the frame n as the target frame, and in step S36, the switching unit 182 sets the interpolation frame as the first reference frame and the frame n-1 as the second reference frame. Then, the subsequent processing is performed.

  In step S46, when the switching unit 182 determines that the overdrive process in steps S38 to S44 has been executed with the interpolation frame and the frame n as the target frame, the process ends.

  As described above, in the second image processing, the switching unit 182 converts the first and second reference frames into the overdrive processing unit at the same timing as when the target frame is output to the overdrive processing unit 183. Output to 183.

  Therefore, the overdrive processing unit 183 needs to provide a frame memory for holding the first reference frame and a frame memory for holding the second reference frame in order to perform the overdrive process on the target frame. Absent.

  Therefore, the overdrive processing unit 183 can have a simpler configuration than the overdrive processing unit 22 illustrated in FIG.

  Further, in the second image processing, a prediction frame composed of pixels having pixel values representing a gradation reached when the first reference frame that is one frame before the target frame is displayed on the LCD 185 is predicted. Then, a display frame to be displayed on the LCD 185 is generated based on the predicted frame and the target frame.

  Therefore, in the overdrive processing unit 183, the gradation represented by the pixel value of the pixel of the first reference frame is different from the gradation reached when the first reference frame after the overdrive process is displayed on the LCD 185. Even in this case, it is possible to generate a display frame capable of preventing a situation such as the liquid crystal response waveform going too far as shown in FIG.

  For this reason, it is possible to further reduce the motion blur of the moving image displayed on the LCD 185 as compared with the first image processing.

  In the overdrive processing unit 183, the first reference frame and the second reference frame are used to generate the prediction frame supplied to the correction value generation unit 143.

  However, among each frame constituting the second moving image, in addition to the first reference frame that is past by one frame from the target frame, and the second reference frame that is past two frames from the target frame, It is possible to generate a prediction frame supplied to the correction value generation unit 143 using a third reference frame that is a frame past three frames from the target frame.

  In this case, in the image processing device 161, the pixel generation unit 201 interpolates between the frame n-1 and the frame n and the interpolation that is interpolated between the frame n-2 and the frame n-1. In addition to the extrapolated frame corresponding to the frame, an extrapolated frame corresponding to the frame n-2 is generated and supplied to the switching unit 182.

  Hereinafter, the extrapolation frame corresponding to the interpolation frame interpolated between the frame n-2 and the frame n-1 is referred to as a first extrapolation frame, and the extrapolation frame corresponding to the frame n-2 is the first extrapolation frame. 2 extrapolation frame.

  In the image processing device 161, the switching unit 182 uses the second extrapolated frame from the pixel generation unit 201 as a third reference frame and the first extrapolated frame from the pixel generation unit 201 at a certain timing. The second reference frame is used, the frame n-1 from the frame 103 is used as the first reference frame, and the interpolated frame from the pixel generation unit 201 is used as the target frame and is supplied to the overdrive processing unit 183.

  At the next timing, the switching unit 182 sets the first extrapolated frame from the pixel generation unit 201 as the third reference frame, sets the frame n−1 from the frame memory 103 as the second reference frame, The interpolated frame from the generation unit 201 is used as the first reference frame, and the frame n−1 from the frame memory 101 is used as the target frame and is supplied to the overdrive processing unit 183.

<3. Modification>
[Configuration Example of Overdrive Processing Unit 261]
Next, FIG. 14 illustrates a configuration example of the overdrive processing unit 261 that generates a prediction frame supplied to the correction value generation unit 143 using the first to third reference frames.

  In the overdrive processing unit 261, parts that are configured in the same manner as the overdrive processing unit 183 are denoted by the same reference numerals, and description thereof will be omitted.

  That is, the overdrive processing unit 261 is configured in the same manner as the overdrive processing unit 183 except that a gamma correction unit 281 and a predicted value generation unit 282 are newly provided.

  Note that the first reference frame is supplied to the gamma correction unit 281, the second reference frame is supplied to the gamma correction unit 221, and the third reference frame is supplied to the gamma correction unit 222 from the switching unit 182. .

  The gamma correction unit 281 performs gamma correction on the first reference frame from the switching unit 182, and supplies the first reference frame after the gamma correction to the predicted value generation unit 282.

  The predicted value generation unit 282 includes a difference value generation unit 301 and an addition unit 302.

  The difference value generation unit 301 includes a lookup table 301a. In the lookup table 301a, the pixel value of the pixel of the first reference frame is calculated from the predicted value that the liquid crystal response waveform of the LCD 185 reaches when the first reference frame subjected to the overdrive process is displayed on the LCD 185. The difference value obtained by subtraction is associated with each combination of the pixel value of the pixel of the first reference frame and the prediction value of the prediction frame of the second reference frame.

  The difference value generation unit 301 is supplied with the first reference frame after the gamma correction from the gamma correction unit 281 and the prediction frame in the second reference frame from the prediction value generation unit 223, respectively.

  The difference value generation unit 301 includes a pixel value of a pixel constituting the first reference frame from the gamma correction unit 281 and a second reference frame from the prediction value generation unit 223 that exists at the same position as the pixel. The difference value associated with the combination of the pixel value (predicted value) of the pixels constituting the predicted frame is read from the built-in lookup table 301 a and supplied to the adding unit 302.

  The first reference frame after the gamma correction is supplied from the gamma correction unit 281 to the addition unit 302.

  The adding unit 302 adds the pixel values of the pixels constituting the first reference frame from the gamma correction unit 281 with the corresponding difference values from the difference value generation unit 301. Then, the adding unit 302 supplies the predicted value frame in the first reference frame, which is configured by the predicted value obtained as a result of the addition, to the correction value generating unit 143.

  The correction value generation unit 143 includes the pixel value of the pixel that forms the target frame from the gamma correction unit 141, and the pixel value of the pixel that forms the prediction frame from the addition unit 302 that exists at the same position as the pixel. The correction value associated with the combination is read from the built-in lookup table 143a and supplied to the adding unit 144.

  As described above, in the overdrive processing unit 261, the prediction value generation unit 282 is based on the first reference frame and the prediction frame in the second reference frame, not the first and second reference frames. A prediction frame in the first reference frame supplied to the correction value generation unit 143 is generated.

  Therefore, for example, the liquid crystal response waveform of the LCD 185 in the second reference frame does not have a gradation corresponding to the pixel value of the second reference frame, and thus the liquid crystal response waveform of the LCD 185 in the first reference frame. However, the gradation corresponding to the pixel value of the first reference frame does not become a gradation, so that the liquid crystal response waveform of the LCD 185 does not become a gradation corresponding to the pixel value of the object frame in the object frame. Can be prevented.

  Further, since the first to third reference frames are also input to the overdrive processing unit 261 at the timing when the target frame is input from the switching unit 182, the frame memory for holding the first to third reference frames There is no need to provide a new one.

  In the overdrive processing unit 261, the prediction frame in the first reference frame supplied to the correction value generation unit 143 is generated using the first to third reference frames. You may make it produce | generate using the reference frame of m (m is a natural number of 4 or more).

  As the number of reference frames used to generate the prediction frame supplied to the correction value generation unit 143 increases, the prediction frame supplied to the correction value generation unit 143 becomes more accurate, so that the response speed of the LCD 185 is improved. Therefore, a more appropriate display frame can be generated.

  In the second embodiment described above, as illustrated in FIG. 11, the pixel generation unit 201 performs frame n−2 based on the motion vector MV (n−1, n) from the motion vector detection unit 104. The extrapolated frame is generated between the frame n-1 and the frame n-1, but the present invention is not limited to this.

  That is, for example, the pixel generation unit 201 uses the frame n-2 based on the motion vector MV (n−2, n−1) detected by the motion vector detection unit 104 using the frame n−2 and the frame n−1. An interpolation frame generated between -2 and frame n-1 may be held, and the held interpolation frame may be supplied to the switching unit 82 as an extrapolated frame.

  In this case, the pixel generation unit 201 generates an extrapolated frame (between frame n-2 and frame n-1) based on the motion vector MV (n−1, n) from the motion vector detection unit 104. There is no need to generate the interpolation frame) again.

  In the first embodiment described above, the reference image generation unit 81 converts the first moving image into the second moving image having twice the number of frames, and the second moving image after the overdrive processing is performed. Although the image is output as it is to the LCD 85, it is not limited to this.

  That is, for example, a stereoscopic image may be generated based on the frames constituting the second moving image, and after the overdrive process is performed on the generated stereoscopic image, it may be output to the LCD 85.

  The same can be said for the reference image generation unit 181 in the second embodiment.

  In the first embodiment, the correction value generation unit 143 generates the correction value using the lookup table 143a. For example, the correction value is calculated based on the target frame and the reference frame. The correction value may be generated using a function capable of

  The same can be said for the difference value generation units 241 and 301.

  Furthermore, in the first and second embodiments, one interpolated frame is generated between the frame n-1 and the frame n. However, the present invention is not limited to this. It is possible to generate two or more interpolated frames between n and n.

  Next, referring to FIG. 15, the pixel generation unit 201 interpolates the first moving image at a quadruple speed (generates three interpolation frames between frames n−1 and n), and overruns. It will be described that the drive processing unit 183 performs overdrive processing on the frames constituting the second moving image obtained by the interpolation.

  Note that the pixel generation unit 201 generates one interpolation frame between the frame n−1 and the frame n, but in the description performed with reference to FIG. 15, the frame n−1 and the frame n It is assumed that three interpolation frames are generated between the two.

[Second video obtained by quadruple speed interpolation]
FIG. 15 shows frames constituting a second moving image obtained by interpolating the first moving image at a quadruple speed.

  In FIG. 15, a frame indicated by a white rectangle represents a frame constituting the first moving image input to the reference image generation unit 181.

  A frame indicated by a black rectangle represents an interpolation frame that is interpolated by the pixel generation unit 201.

  Further, the first interpolation frame shown in FIG. 15 is displayed in the past by time T / 4 from the time frame n is displayed when the frames constituting the first moving image are displayed every time T. This represents a frame obtained by interpolating the frame n-1 / 4.

  Further, the second interpolation frame shown in FIG. 15 represents a frame obtained by interpolating the frame n-2 / 4 displayed in the past by the time 2T / 4 from the time when the frame n is displayed.

  Further, the third interpolation frame shown in FIG. 15 represents a frame obtained by interpolating a frame n-3 / 4 displayed in the past by a time 3T / 4 from the time when frame n is displayed.

  Further, the extrapolated frame shown in FIG. 15 represents a frame obtained by interpolating a frame n-5 / 4 displayed in the past for a time 5T / 4 from when frame n is displayed.

  Based on the frame n-1 and the frame n, the pixel generation unit 201 is a first interpolated frame obtained by interpolating the frame n-1 / 4, and a second interpolated frame n-2 / 4. A third interpolation frame obtained by interpolating the interpolation frame and the frame n-3 / 4 is generated.

  Further, the pixel generation unit 201 generates an extrapolated frame (corresponding to the interpolation frame) obtained by interpolating the frame n-5 / 4 based on the frame n-1 and the frame n.

  Then, the pixel generation unit 201 supplies the generated first to third interpolation frames and extrapolation frames to the switching unit 182.

  In addition, the pixel generation unit 201 controls the frame memory 101 to output the held frame n to the switching unit 182, and controls the frame 103 to output the held frame n−1 to the switching unit 182. Let

  At a certain timing, the switching unit 182 sets the extrapolated frame from the pixel generation unit 201 as the second reference frame, sets the frame n−1 from the frame memory 103 as the first reference frame, and outputs from the pixel generation unit 201. The third interpolation frame is supplied to the overdrive processing unit 183 as the target frame.

  At the next timing, the switching unit 182 uses the frame n−1 from the frame memory 103 as the second reference frame, the third interpolation frame from the pixel generation unit 201 as the first reference frame, and generates pixels. The second interpolation frame from the unit 201 is supplied to the overdrive processing unit 183 as a target frame.

  Further, at the next timing, the switching unit 182 sets the third interpolation frame from the pixel generation unit 201 as the second reference frame, sets the second interpolation frame from the pixel generation unit 201 as the first reference frame, The first interpolation frame from the generation unit 201 is supplied to the overdrive processing unit 183 as a target frame.

  Thereafter, at a further next timing, the switching unit 182 sets the second interpolation frame from the pixel generation unit 201 as the second reference frame, and sets the first interpolation frame from the pixel generation unit 201 as the first reference frame. The frame n from the frame memory 101 is supplied to the overdrive processing unit 183 as a target frame.

  The overdrive processing unit 183 performs overdrive processing on the target frame from the switching unit 182 while referring to the first and second reference frames from the switching unit 182, and displays the display frame obtained as a result as a timing controller. 184.

  The timing controller 184 supplies the display frame from the overdrive processing unit 183 to the LCD 185 for display at a predetermined timing.

  In the first and second embodiments, among the frames constituting the first moving image, the frame n-1 and the frame n which are temporally different by one frame are used, and the frame n-1 and the frame n Although an extrapolated frame is generated as an interpolated frame interpolated between n and a frame earlier than the frame n-1, the present invention is not limited to this.

  That is, for example, among the frames constituting the first moving image, a frame nm and a frame n that are temporally different from each other by m frames (m is a natural number of 2 or more) are used. It is also possible to generate an extrapolated frame as an interpolated frame and a frame past the frame nm, and perform the overdrive processing described in the first and second embodiments.

  As the image processing apparatuses 61 and 161, for example, a personal computer that displays an image on a hold-type display can be employed.

  Next, the series of processes described above can be executed by dedicated hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a so-called embedded computer or various programs. For example, it is installed from a recording medium in a general-purpose personal computer.

[Computer configuration example]
FIG. 16 shows a configuration example of a personal computer that executes the above-described series of processing by a program.

  A CPU (Central Processing Unit) 341 executes various processes according to a program stored in a ROM (Read Only Memory) 342 or a storage unit 348. A RAM (Random Access Memory) 343 appropriately stores programs executed by the CPU 341, data, and the like. The CPU 341, the ROM 342, and the RAM 343 are connected to each other via a bus 344.

  An input / output interface 345 is also connected to the CPU 341 via the bus 344. The input / output interface 345 is connected to an input unit 346 including a keyboard, a mouse, and a microphone, and an output unit 347 including a display and a speaker. The CPU 341 executes various processes in response to commands input from the input unit 346. Then, the CPU 341 outputs the processing result to the output unit 347.

  The storage unit 348 connected to the input / output interface 345 includes, for example, a hard disk, and stores programs executed by the CPU 341 and various data. The communication unit 349 communicates with an external device via a network such as the Internet or a local area network.

  Further, a program may be acquired via the communication unit 349 and stored in the storage unit 348.

  The drive 350 connected to the input / output interface 345 drives a removable medium 351 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the programs and data recorded therein. Get etc. The acquired program and data are transferred to and stored in the storage unit 348 as necessary.

  As shown in FIG. 16, a recording medium for recording a program that is installed in a computer and can be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (compact disc-read only). memory), DVD (including digital versatile disc)), magneto-optical disc (including MD (mini-disc)), or removable media 351, which is a package media consisting of semiconductor memory, or the program is temporary or permanent ROM 342 recorded on the hard disk and a hard disk constituting the storage unit 348. Recording of a program on a recording medium is performed using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 349 that is an interface such as a router or a modem as necessary. Is called.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, the present embodiment is not limited to the first and second embodiments described above, and various modifications can be made without departing from the scope of the present invention.

  61 image processing device, 81 reference image generation unit, 82 switching unit, 83 overdrive processing unit, 84 timing controller, 85 LCD, 101 frame memory, 102 frame delay unit, 103 frame memory, 104 motion vector detection unit, 105 pixel generation , 141,142 gamma correction unit, 143 correction value generation unit, 143a lookup table, 144 addition unit, 161 image processing device, 181 reference image generation unit, 182 switching unit, 183 overdrive processing unit, 184 timing controller, 185 LCD, 201 pixel generation unit, 221 and 222 gamma correction unit, 223 prediction value generation unit, 241 difference value generation unit, 241a lookup table, 242 addition unit, 261 overdrive processing unit 281 Gamma correction unit 282 Prediction value generation unit 301 Difference value generation unit 301a Look-up table 302 Addition unit

Claims (9)

In an image processing apparatus for converting a first moving image into a second moving image having a higher frame rate,
An image constituting the first moving image is a first image, and the first image and the second image that is temporally past the first image and the first image, Image generating means for generating an interpolated image interpolated between the second image and an image constituting the second moving image;
First setting means for setting a plurality of images constituting the second moving image as a target image that is a processing target of overdrive processing that sequentially corrects the images to be displayed on a display unit;
Of the plurality of images constituting the second moving image, an image that is earlier in time than the target image is set as a reference image that is referred to for correcting the target image by the overdrive processing. Two setting means;
Overdrive processing means for correcting the target image by the overdrive processing based on the reference image;
And an output unit that outputs the target image and the reference image to the overdrive processing unit in synchronization with each other. The first setting unit sequentially sets the interpolation image and the first image as the target image,
The image processing apparatus according to claim 1, wherein the second setting unit sets, as the reference image, an image that is temporally past the target image among the second image and the interpolated image. The image generating means is based on the first and second images, and among the images constituting the second moving image, the image generating means temporally 1 to n-1 (n is 2) from the second image. (1) to (n-1) extrapolated images representing each of the past images (natural number)
The second setting means is an image that is 1 to n temporally past the target image among the first to n-1 extrapolated images, the second image, and the interpolated image. The image processing apparatus according to claim 2, wherein each of the first to nth reference images is set. The output means synchronizes the target image and the first to n-th reference images and outputs them to the overdrive processing means,
The image processing apparatus according to claim 3, wherein the overdrive processing unit corrects the target image based on the first to nth reference images. The first reference image is an image that is one image earlier in time than the target image among a plurality of images constituting the second moving image,
The overdrive processing means includes
Based on the first to n-th reference images, a prediction value that generates a prediction value that represents the gradation of the display unit that is reached when the first reference image corrected by the overdrive processing unit is displayed. Generating means;
Correction value generating means for generating a correction value to be added to the target image in order to correct the target image based on the target image and the predicted value;
The image processing apparatus according to claim 4, further comprising: adding means for correcting the target image by adding the correction value to the target image. Motion vector detecting means for detecting a motion vector based on the first and second images;
The image processing device according to claim 3, wherein the image generation unit generates the interpolation image and the first to n−1 extrapolated images based on the motion vector. The image processing apparatus according to claim 1, wherein the image generation unit generates a plurality of the interpolated images based on the first and second images. In an image processing method of an image processing apparatus for converting a first moving image into a second moving image having a higher frame rate,
The image processing apparatus includes:
Image generating means;
First setting means;
A second setting means;
Overdrive processing means;
Output means and
The image generating means sets the image constituting the first moving image as a first image, and based on the first image and the second image that is earlier in time than the first image, Generating an interpolated image interpolated between the first image and the second image as an image constituting the second moving image;
The first setting means sets a plurality of images constituting the second moving image as a target image that is a processing target of an overdrive process that sequentially corrects the images to be displayed on the display unit,
The second setting means refers to an image that is temporally past the target image among the plurality of images constituting the second moving image to correct the target image by the overdrive process. Set to the reference image
The overdrive processing means corrects the target image based on the reference image by the overdrive processing,
An image processing method including a step in which the output means outputs the target image and the reference image to the overdrive processing means in synchronization. A computer of an image processing apparatus that converts a first moving image into a second moving image having a higher frame rate,
An image constituting the first moving image is a first image, and the first image and the second image that is temporally past the first image and the first image, Image generating means for generating an interpolated image interpolated between the second image and an image constituting the second moving image;
First setting means for setting a plurality of images constituting the second moving image as a target image that is a processing target of overdrive processing that sequentially corrects the images to be displayed on a display unit;
Of the plurality of images constituting the second moving image, an image that is earlier in time than the target image is set as a reference image that is referred to for correcting the target image by the overdrive processing. Two setting means;
Overdrive processing means for correcting the target image by the overdrive processing based on the reference image;
A program for causing the target image and the reference image to function in synchronization with each other as output means for outputting to the overdrive processing means.

Images