JP2007074415A - Motion vector estimation device and motion vector estimation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion vector estimation device and a motion vector estimation program which are capable of accurately estimating a motion vector even in the case of quick changes in illumination luminous intensity. <P>SOLUTION: The motion vector estimation device includes; a threshold determination means 10 which determines a threshold TH on the basis of luminance of pixels a base frame BF and a predetermined luminance check area SA of a reference frame RF; and a motion vector estimation means 11 which estimates a motion vector on the basis of the number of corresponding pixels in a base block BB within the base frame BF and in the reference block RB within the reference frame RF, in which luminance difference absolute values are equal to or smaller than the threshold TH. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a motion vector estimation device and a motion vector estimation program, and more particularly to a motion vector estimation device and a motion vector estimation program capable of accurately estimating a motion vector even when the illumination illuminance of a subject changes rapidly. .

  In television broadcasting, motion vector detection is required in television system conversion devices, image encoding devices, and the like.

  When converting a moving image produced in the NTSC format by a television conversion device into a moving image that can be broadcast in the PAL format, it is necessary to convert a moving image of 60 frames per second into a moving image of 50 frames per second. .

  That is, when the first frame of the NTSC system is matched with the first frame of the PAL system, the second frame of the PAL system is inserted between the second frame and the third frame of the NTSC system and the third frame of the NTSC system. It is necessary to generate a PAL system third frame between the frame and the fourth frame, and a PAL frame between two NTSC frames by interpolation. A motion vector is required when generating a PAL frame moving image by interpolation.

  In addition, in an image encoding apparatus to which MPEG2 is applied, motion vector detection is required to generate a B picture and a P picture to be generated that transmit only motion vector information.

  That is, a motion vector estimation device is used to detect a motion vector, and the motion vector estimation device constitutes a part of a television system conversion device or an image encoding device.

  For example, as shown in the block diagram of FIG. 16, the image encoding device 9 includes a preprocessing unit 91 that converts a video signal of a current image output from the video camera 90 into an SIF signal, an SIF signal, and a local decoding unit. A motion vector estimation unit 92 that estimates a motion vector based on the SIF signal output from 96, a DCT unit 93 that performs discrete cosine transform (DCT) on an inter-screen pixel difference value output from the motion vector estimation unit 92, and A quantization unit 94 that quantizes the discrete cosine transform result; a modulation modulation unit 95 that performs variable-length encoding of the quantized value; and a local decoding unit 96 that decodes the quantized value into an inter-screen pixel difference value. It is out.

  The image encoding device 9 is generally configured by installing a program in a programmable element such as a DSP or a microprocessor.

  Therefore, the motion vector estimation unit 92 is configured by a programmable element in which a program is installed or a part of the programmable element.

  Conventionally, a motion vector estimation apparatus for accurately and quickly estimating a motion vector has been proposed (see, for example, Patent Document 1).

  FIG. 17 is a block diagram of the motion vector estimation unit 92 disclosed in Patent Document 1. The reference frame memory 921 stores the current image data as reference frame data. The reference frame memory 922 stores image data one frame before the current time as reference frame data.

  In addition, the class code generation unit 923 uses, for example, 1-bit ADRC (Adaptive Dynamic Range Coding) based on the reference frame data stored in the reference frame memory 921 as a feature amount related to the target pixel position of the reference frame. Generate some class code.

  Then, the motion vector calculation unit 924 is based on the base frame data stored in the base frame memory 921, the reference frame data stored in the reference frame memory 922, and the class code generated by the class code generation unit 923. A motion vector is calculated.

That is, the conventional motion vector estimation unit 92 can estimate a motion vector accurately and quickly by calculating a motion vector by a block matching method using a pixel block corresponding to a feature amount.
Japanese Patent Laying-Open No. 2005-63319 (FIG. 1, [0050] to [0052])

  However, the conventional motion vector estimation device estimates motion vectors when the brightness of the illumination light that illuminates the subject changes abruptly, such as flashing flashes in news programs and spotlights flashing in song programs. There is a problem that it cannot be avoided that errors occur.

  The present invention has been made to solve the conventional problems, and is a motion vector estimation device and a motion vector estimation capable of accurately estimating a motion vector even when there is a sudden change in illumination illuminance. The purpose is to provide a program.

  The motion vector estimation apparatus according to the present invention includes threshold determination means for determining a threshold based on the luminance of pixels in a predetermined luminance inspection area of the base frame and the reference frame, the base block in the base frame, and the reference frame And a motion vector estimation means for estimating a motion vector based on the number of pixels whose luminance difference absolute value between corresponding pixels of the reference block is equal to or less than the threshold value.

  With this configuration, a motion vector is estimated in consideration of a change in luminance between the base frame and the reference frame.

  In the motion vector estimation apparatus according to the present invention, the threshold value determining means includes a reference frame dividing means for dividing the reference frame into reference blocks having a predetermined number of pixels, and a reference frame luminance inspection area in the reference frame for each reference block. A reference frame luminance inspection area setting means for setting; a reference frame pixel luminance median value determining means for determining a median value of a reference frame pixel luminance which is a median value of a pixel luminance histogram in the reference frame luminance inspection area; A reference frame luminance inspection region setting means for setting a reference frame luminance inspection region at the same position as the reference frame luminance inspection region in the reference frame, and a reference that is a median value of a pixel luminance histogram in the reference frame luminance inspection region Reference frame pixel luminance median value determining means for determining a frame pixel luminance median value Has a configuration comprising a threshold calculating means for calculating the absolute value of the difference between the reference frame luminance median and the reference frame luminance central value as the threshold value.

  With this configuration, the threshold value is calculated based on the absolute value of the difference between the median values of the luminance histograms of the base frame and the reference frame.

  In the motion vector estimation device of the present invention, the motion vector estimation means includes a motion vector search area setting means for setting a motion vector search area in the reference frame luminance inspection area for each reference block, and the reference block for each reference block. Based on the reference block and the reference block, a search is made for a reference block having a minimum number of pixels having a luminance that is smaller than the threshold value and the absolute value of the difference between the luminance of the pixels in the reference block from a motion vector search region. And a motion vector calculating means for determining a motion vector.

  With this configuration, a motion vector is estimated based on the number of pixels whose absolute value of luminance difference between the base frame and the reference frame is equal to or less than a threshold value.

In the motion vector estimation apparatus of the present invention, the reference frame pixel luminance median value determining means determines the luminance of each pixel in the reference frame luminance inspection area as a difference between the maximum luminance and the minimum luminance of the pixels in the reference frame luminance inspection area. Determining the median value of the reference frame dimensionless pixel luminance, which is the median value of the histogram of the dimensionless pixel luminance divided by
Non-dimensional pixel luminance histogram obtained by dividing the luminance of each pixel in the reference frame luminance inspection area by the difference between the maximum luminance and the minimum luminance of the pixels in the reference frame luminance inspection area The reference frame non-dimensional pixel luminance median value that is the median value is determined.

  With this configuration, the threshold value is calculated based on the absolute value of the difference between the median values of the dimensionless luminance histogram of the base frame and the reference frame.

  In the motion vector estimation device of the present invention, when the motion vector estimation means does not resemble the shape of the luminance histogram of the pixels in the reference frame luminance inspection area and the luminance histogram in the reference luminance inspection area, It has a configuration including motion vector estimation stopping means for canceling the estimation.

  With this configuration, a motion vector is estimated only when the luminance histograms of the base frame and the reference frame are similar.

  In the motion vector estimation apparatus according to the present invention, the threshold value determining means includes a reference frame dividing means for dividing the reference frame into reference blocks having a predetermined number of pixels, and a reference frame luminance inspection area in the reference frame for each reference block. A reference frame luminance inspection area setting means for setting; a reference frame luminance average value determining means for determining a reference frame luminance average value which is an average value of luminance histograms of pixels in the reference frame luminance inspection area; and the reference frame luminance average value determining means in the reference frame. Reference frame luminance inspection region setting means for setting a reference frame luminance inspection region at the same position as the reference frame luminance inspection region in the reference frame, and reference frame luminance that is an average value of luminance histograms of pixels in the reference frame luminance inspection region A reference frame luminance average value determining means for determining an average value; Beam intensity average value and the absolute value of the difference between the reference frame luminance average value has a configuration that includes a threshold calculating means for calculating as said threshold value.

  With this configuration, the threshold value is calculated based on the absolute value of the difference between the average values of the luminance histograms of the base frame and the reference frame.

  The present invention provides a threshold value determining means for determining a threshold value based on the luminance of the pixels in the luminance inspection area of the base frame and the reference frame, so that the motion vector can be accurately obtained even when there is a sudden change in illumination illuminance. It is possible to provide a motion vector estimation device and a motion vector estimation program having an effect that estimation is possible.

Hereinafter, a motion vector estimation apparatus according to an embodiment of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 shows a block diagram of an embodiment of a motion vector estimation apparatus according to the first embodiment of the present invention.

  That is, the motion vector estimation apparatus 1 according to the present invention includes threshold determination means 10 that determines the threshold TH based on the luminance of the pixels in the predetermined luminance inspection area SA of the reference frame BF and the reference frame RF, and the reference frame Motion vector estimation means 11 for estimating a motion vector based on the number of pixels whose luminance difference absolute value between corresponding pixels of the reference block BB in the BF and the reference block RB in the reference frame RF is equal to or less than a threshold value TH. .

  Then, the threshold value determining means 10 includes a reference frame dividing means 101 for dividing the reference frame BF into reference blocks BB having a predetermined number of pixels, and a reference frame for setting a reference frame luminance inspection area BSA in the reference frame BF for each reference block BB. Luminance inspection area setting means 102, reference frame pixel luminance median value determining means 103 for determining a reference frame pixel luminance median value BMV that is the median value of the pixel luminance histogram in the reference frame luminance inspection area BSA, and in the reference frame RF Reference frame luminance inspection area setting means 104 for setting the reference frame luminance inspection area RSA at the same position as the reference frame luminance inspection area BSA in the reference frame BF, and the median value of the pixel luminance histogram in the reference frame luminance inspection area SA Determine a reference frame pixel luminance median value RMB that is It includes a reference frame pixel intensity median determining unit 105, a threshold calculating means 106 which calculates the absolute value of the difference between the reference frame luminance median RVM a reference frame luminance median BMV as the threshold TH.

  Also, the motion vector estimation means 11 includes a motion vector search area setting means 111 that sets a motion vector search area MVA in the reference frame luminance inspection area RSA, and a motion vector search area MVA in the reference block BB for each reference block BB. A motion vector for searching for a reference block RB having a minimum number of pixels having a luminance whose absolute value of the difference from the pixel luminance is smaller than the threshold TH and determining a motion vector MV based on the base block BB and the reference block RB Calculation means 112.

  FIG. 2 is a block diagram showing a hardware configuration of the motion vector estimation apparatus 1 according to the present invention. The motion vector estimation apparatus 1 according to the present invention is configured by a microprocessor 2.

  The microprocessor 2 includes a CPU 20 that executes processing according to a program, a memory 21 that stores the program and processing results, an image signal reading interface (I / F) 22 that reads an image signal, and a motion vector output I that outputs a motion vector. / F23 and peripheral device I / F24 are coupled to each other by a bus 25.

  The peripheral device I / F 24 is connected with a display device 26, a keyboard 27, and a pointing device 28. The display device 26, the keyboard 27, and the pointing device 28 maintain a motion vector estimation device such as program installation and parameter setting. Used to do. Note that a dedicated operation panel for the motion vector estimation device may be connected to the peripheral device I / F 24 so that the motion vector estimation device is maintained by the dedicated operation panel.

  That is, by installing the motion vector estimation program in the microprocessor 2, the microprocessor 2 functions as the motion vector estimation device 1 according to the present invention.

  Next, the operation of the motion vector estimation apparatus 1 according to the present invention will be described with reference to a flowchart of a motion vector estimation program installed in the microprocessor 2.

  FIG. 3 is a flowchart of the first motion vector estimation program executed by the CPU 20 in the first embodiment, and is executed at predetermined time intervals (for example, 30 milliseconds).

  First, the CPU 20 reads the image signal at the current time as reference frame data BF into the reference frame memory constituted by a part of the memory 21 via the image signal read I / F 22 and simultaneously stores a plurality of images already stored in the memory 21. One image frame is read as reference frame data RF into a reference frame memory constituted by a part of the memory 21 (step S30).

  Next, the CPU 20 divides the reference frame data BF into reference blocks BB including a predetermined number of pixels (for example, 8 rows and 8 columns total 64) (step S31).

  Then, the CPU 20 initializes an index i representing the block column number to “1” (step S32), and initializes an index j representing the block row number to “1” (step S33).

  Further, the CPU 20 determines a threshold value TH (step S34) and searches for a motion vector MV (step S35), details of which will be described later. Then, the CPU 20 outputs parameters MB and NB that specify the motion vector MV corresponding to the reference block BB (i, j) calculated in step S35 (step S301).

  The CPU 20 determines whether or not the block row number j has reached the maximum value J (step S36). If the negative determination is made, the CPU 20 increments the block row number j (step S37) and returns to the processing of step S34.

  Conversely, when the CPU 20 determines that the block row number j has reached the maximum value J, the CPU 20 determines whether or not the block column number i has reached the maximum value I (step S38). The block string number i is incremented (step S39), and the process returns to step S33.

  Conversely, when the CPU 20 determines that the block row number i has reached the maximum value I, the program is terminated.

  FIG. 4 is a flowchart of a first threshold value determination routine that the CPU 20 executes in step S34 of the first motion vector estimation program shown in FIG. First, the CPU 20 sets an area of vertical K pixels and horizontal L pixels centering on the reference block BB (i, j) in the reference frame BF as the reference frame luminance inspection area BSA (step S40).

  FIG. 5 is a configuration diagram of a base frame BF and a reference frame RF having 8 blocks in the column direction and 13 blocks in the row direction. The block column number and the block row number (i, j) of the reference block BB are (4, 6), and K = 34 and L = 68.

  Next, the CPU 20 generates a luminance histogram in the luminance inspection area BSA (step S41). That is, the CPU 20 inspects the distribution of pixel values (= luminance) of K × L pixels included in the reference frame luminance inspection area BSA.

  Next, the CPU 20 sets the reference frame luminance inspection area RSA at the same position as the reference frame luminance inspection area BSA in the reference frame BF in the reference frame RF (step S42).

  Next, the CPU 20 generates a histogram of pixel values of K × L pixels included in the reference frame luminance inspection area RSA (step S43).

  Finally, the CPU 20 calculates the threshold value TH (step S44) and ends this routine. Details of the calculation of the threshold value TH will be described later.

  FIG. 6 is a flowchart of a first threshold value calculation routine that the CPU 20 executes in step S44 of the first threshold value determination routine shown in FIG.

  First, the CPU 20 determines a reference frame luminance median value BMV that is an average value of the maximum luminance BLmax and the minimum luminance BLmin in the reference frame luminance inspection area BSA (step S60).

  Next, the CPU 20 determines a reference frame luminance median value RMV that is an average value of the reference frame maximum luminance RBmax and the minimum luminance RBmin in the reference frame luminance inspection area RSA (step S61).

  Finally, the CPU 20 calculates the threshold value TH as the absolute value of the difference between the base frame luminance median value BMV and the reference frame luminance median value RMV (step S62), and ends this routine.

  FIG. 7 is a flowchart of a first motion vector search routine that the CPU 20 executes in step S35 of the first motion vector estimation program shown in FIG.

  First, the CPU 20 is at the same position as the reference block BB (i, j) in the reference frame BF in the reference frame RF, and is a reference block RB that is a block including the same number of pixels as the reference block BB (i, j). A block of M pixels in the vertical direction and N pixels in the horizontal direction centering on (i, j) is set as the motion vector search range MVSA (step S700). Note that the motion vector search range MVSA needs to be set inside the reference frame luminance inspection area RSA. That is, K ≧ M and L ≧ N. This is because it is appropriate to determine the motion vector in the reference frame luminance inspection area in which the luminance is inspected in order to determine the threshold value TH in advance.

  Next, the CPU 20 sets the maximum number of effective pixels Max to “0” (step S701).

  Further, the CPU 20 sets the index m representing the pixel column number of the motion vector search range MVSA to “1” (step S702), and then sets the index n representing the pixel row number of the motion vector search range MVSA to “1”. Setting is performed (step S703).

  Then, the CPU 20 counts the effective pixel number SUM (step S704), which will be described in detail later.

  Next, the CPU 20 determines whether or not the effective pixel number SUM is larger than the effective pixel number maximum value Max (step S705).

  When the CPU 20 determines that the effective pixel number SUM is larger than the effective pixel number maximum value Max, the CPU 20 replaces the effective pixel number maximum value Max with the effective pixel number SUM (step S706), and at the same time, the current motion vector search range pixel sequence The number m is stored in MB, the current motion vector search range pixel row number n is stored in NB (step S707), and the process proceeds to step S708. On the other hand, when the CPU 20 determines that the number of effective pixels SUM is equal to or less than the maximum number of effective pixels Max, the CPU 20 proceeds directly to step S708.

  The CPU 20 determines whether or not the motion vector search range pixel row number n has reached the maximum value N (step S708), and determines that the motion vector search range pixel row number n has not reached the maximum value N Increments the motion vector search range pixel row number n (step S709) and returns to step S704. On the other hand, when determining that the motion vector search range pixel row number n has reached the maximum value N, the CPU 20 determines whether or not the motion vector search range pixel column number m has reached the maximum value M (step S710). ).

  When the CPU 20 determines that the motion vector search range pixel column number m has not reached the maximum value M, the CPU 20 increments the motion vector search range pixel column number m (step S711), and returns to step S703. On the other hand, when it is determined that the motion vector search range pixel column number m has reached the maximum value M, the CPU 20 ends this routine.

  FIG. 8 is a flowchart of the effective pixel number counting routine executed by the CPU 20 in step S704 of the first motion vector search routine shown in FIG. 7. The CPU 20 first initializes the effective pixel number SUM to “0” ( Step S80).

  Next, the CPU 20 includes pixels included in the reference block RB (m, n) of 8 pixels × 8 pixels centered on the pixel (m, n) in the base block BB (i, j) and the motion vector search range MVSA. Column number ii is initialized to “1” (step S81), and row number jj is initialized to “1” (step S82).

  The CPU 20 then compares the pixel value of the pixel BPX (ii, jj) included in the base block BB (i, j) and the pixel value of the pixel RPX (ii, jj) included in the reference block RB (m, n). Is calculated (step S83).

  The CPU 20 determines whether or not the absolute value Δ of the pixel value difference calculated in step S83 is equal to or less than the threshold value TH calculated in the threshold value determination routine (step S84). When it is determined that the threshold value TH is equal to or less than the threshold value TH, the effective pixel number SUM is incremented (step S85), and the process proceeds to step S86.

  On the other hand, when the CPU 20 determines that the absolute value Δ of the pixel value difference is larger than the threshold value TH, the process directly proceeds to step S86.

  The CPU 20 determines whether or not the pixel row number jj has reached the maximum value JJ (step S86). If it is determined that the pixel row number jj has not reached the maximum value JJ, the pixel row number jj is incremented. Then (step S87), the process returns to step S83. On the other hand, when determining that the pixel row number jj has reached the maximum value JJ, the CPU 20 determines whether or not the pixel column number ii has reached the maximum value II (step S88).

  When the CPU 20 determines that the pixel column number ii has not reached the maximum value II, the CPU 20 increments the pixel column number ii (step S89) and returns to step S82. On the other hand, when the CPU 20 determines that the pixel column number ii has reached the maximum value II, this routine is terminated.

  FIG. 9A is a graph for explaining the operation of the motion vector estimation apparatus according to the first embodiment, in which a solid line is a luminance histogram of a reference frame with low illumination, and a broken line is high illumination. Is a luminance histogram of a reference frame. In other words, when the illumination illuminance changes significantly due to flash or the like, theoretically, the brightness histogram at high illuminance is translated from the brightness histogram graph at low illuminance to the high brightness side by the amount of brightness change due to flash. It should be a thing.

  Therefore, the motion vector estimation apparatus according to the first embodiment regards the absolute value of the difference between the median value of the luminance histogram of the base frame and the median value of the luminance histogram of the reference frame as the luminance change amount, and sets the threshold value TH. The motion vector is estimated by searching for a reference block having the largest number of pixels having a pixel value whose absolute value of the difference from the pixel value of the pixel included in the pixel is less than or equal to the threshold.

As described above, according to the motion vector estimation device according to the first embodiment, it is possible to estimate a motion vector in consideration of a change in illumination illuminance.
(Second Embodiment)
In the first embodiment, the motion vector is searched using the luminance value itself, but the motion vector is accurately estimated even when the change in illumination illuminance is extremely large by normalizing the luminance histogram. Is possible.

  FIG. 10 is a flowchart of a second threshold value determination routine executed by the CPU 20 in the second embodiment, and step S45 is performed between step S41 and step S42 of the first threshold value determination routine of FIG. Step S46 is inserted between S43 and S44.

  In step S45, the CPU 20 normalizes the luminance axis of the luminance histogram of the reference frame with the difference between the reference frame maximum luminance BBmax and the minimum luminance BBmin to calculate a normalized reference frame luminance histogram.

  In step S46, the CPU 20 calculates a normalized reference frame luminance histogram by normalizing the luminance axis of the luminance histogram of the reference frame by the difference between the reference frame maximum luminance RBmax and the minimum luminance RBmin.

  The solid line in FIG. 9B is a normalized reference frame luminance histogram, and the broken line is a normalized reference frame luminance histogram. By comparing the normalized reference frame luminance histogram and the normalized reference frame luminance histogram on the same horizontal axis scale, It is possible to accurately estimate the motion vector.

  The processing of the other steps is the same as that of the first threshold value determination routine, and thus description thereof is omitted.

  In the second embodiment, the threshold value calculated in step S62 of the first threshold value calculation routine is calculated by the following equation.

  Further, the pixel value difference in step S83 of the effective pixel number counting routine is calculated by the following equation.

As described above, according to the second embodiment, it is possible to accurately estimate a motion vector even when the change in illumination illuminance is extremely large by normalizing the luminance histogram.
(Third embodiment)
In the motion vector estimation apparatus according to the present invention, the motion vector is estimated based on the similarity degree of the luminance histogram of the reference block and the luminance histogram of the reference block. It is possible to omit the motion vector search of the reference block.

  FIG. 11 is a flowchart of the second motion vector estimation program executed by the CPU 20 in the third embodiment, and step S300 is provided between step S34 and step S35 of the first motion vector estimation program shown in FIG. Have been added.

  In step S300, the CPU 20 determines in advance whether or not the difference between the standard deviation of the luminance histogram of the reference block and the standard deviation of the luminance histogram of the reference block is equal to whether or not the shapes of the luminance histogram of the standard block and the luminance histogram of the reference block are similar. It is determined whether or not the error is equal to or less than a predetermined error ε.

  When the CPU 20 determines that the difference in standard deviation is equal to or less than the error ε, the CPU 20 proceeds to step S35, estimates a motion vector, and outputs a parameter for specifying the motion vector MV in step S301. On the other hand, when the CPU 20 determines that the difference in standard deviation is larger than the error ε, the CPU 20 proceeds directly to step S36 without estimating the motion vector.

  The other processes are the same as those of the first motion vector estimation program, and thus description thereof is omitted.

  FIG. 12 is a flowchart of the third threshold value determination routine executed by the CPU 20 in step S34 of the second motion vector estimation program, and is performed between step S41 and step S42 of the first threshold value determination routine of FIG. In step S47, step S48 is inserted between step S43 and step S44.

  The CPU 20 calculates the standard deviation of the luminance histogram of the reference frame luminance inspection area in step S47, and calculates the standard deviation of the luminance histogram of the reference frame luminance inspection area in step S48.

  Since the process of other steps is the same as the first threshold value determination routine, the description thereof is omitted.

As described above, according to the third embodiment, when the luminance histogram of the base block and the luminance histogram of the reference block are greatly different, the calculation amount is reduced by omitting the motion vector search of the reference block. It becomes possible to reduce.
(Fourth embodiment)
In the first to third embodiments, the threshold value is calculated as an absolute value of the difference between the median value of the luminance histogram of the reference frame and the median value of the luminance histogram of the reference frame. When the distribution is close, an absolute value of a difference between the average value of the luminance histogram of the standard frame and the average value of the luminance histogram of the reference frame may be used as the threshold value.

  FIG. 13 is a flowchart of the second threshold value calculation routine executed by the CPU 20 in step S44 of the second threshold value determination routine shown in FIG. 10 in the fourth embodiment. The CPU 20 indicates that the luminance histogram of the reference frame has a normal distribution. It is determined whether it can be considered (step S63).

  When the CPU 20 determines that it cannot be regarded as a normal distribution, the threshold is set to the median of the luminance histogram of the reference frame and the luminance histogram of the reference frame in steps S60 to S62 as in the first threshold value determination routine. Calculated as the absolute value of the value difference.

  On the other hand, when the CPU 20 determines in step S63 that it can be regarded as a normal distribution, in steps S64 to S66, the threshold is set to the difference between the average value of the luminance histogram of the reference frame and the average value of the luminance histogram of the reference frame. Calculated as an absolute value.

As described above, according to the fourth embodiment, when the luminance histogram of the standard frame is close to a normal distribution, the threshold value is set to the difference between the average value of the luminance histogram of the standard frame and the average value of the luminance histogram of the reference frame. An absolute value can be determined.
(Fifth embodiment)
In the first to fourth embodiments, the evaluation function is the number of pixels whose threshold value is the absolute value of the luminance difference between the pixels included in the base block and the reference block in order to estimate the motion vector. An integrated value of luminance differences between pixels included in the block and the reference block may be used as the evaluation function.

  FIG. 14 is a flowchart of the second motion vector search routine executed by the CPU 20 in step S35 of the motion vector estimation program shown in FIG. 3 in the fifth embodiment, and the first motion vector search routine of FIG. Steps S701, S704, and S705 are replaced with steps S721, S722, and S723.

  That is, in step S721, the CPU 20 sets the minimum luminance difference integrated value Acc to a conceivable maximum value (for example, a luminance difference maximum value × a value equal to or larger than the number of pixels in the reference block RB). When the luminance is represented by 8 bits, that is, 256 gradations and the number of pixels in the reference block RB is 8, Acc = 256 × 8 = 2048 is set.

  In step S722, the CPU 20 calculates the luminance difference integrated value DEL, which will be described in detail later.

  In step S723, the CPU 20 determines whether or not the luminance difference integrated value DEL is less than the minimum luminance difference integrated value Acc, and when determining that the luminance difference integrated value DEL is less than the minimum luminance difference integrated value Acc, In step S706, the luminance difference integrated value DEL is stored in the minimum luminance difference integrated value Acc.

  If the CPU 20 determines that the luminance difference integrated value DEL is greater than or equal to the minimum luminance difference integrated value Acc, the CPU 20 proceeds directly to step S708.

  FIG. 15 is a flowchart of the luminance difference integrated value calculation routine executed by the CPU 20 in step S722 of the second motion vector search routine shown in FIG. 14 in the fifth embodiment, and the effective pixel count shown in FIG. Step S80 and step S85 of the coefficient routine are replaced with step S800 and step S801.

  First, in step S800, the CPU 20 initializes the luminance difference integrated value DEL to “0”.

  In step S801, the CPU 20 updates the luminance difference integrated value DEL with DEL + Δ.

  The other processes are the same as those in the second motion vector search routine, and thus description thereof is omitted.

  As described above, according to the fifth embodiment, it is possible to use an integrated value of luminance differences between pixels included in the base block and the reference block as an evaluation function.

  As described above, the motion vector according to the present invention has an effect that the motion vector can be accurately estimated even when the illumination illuminance fluctuates. It is effective as a device.

Block diagram of motion vector estimation apparatus according to the present invention The block diagram of the hardware constitutions of the motion vector estimation apparatus which concerns on this invention Flowchart of a first motion vector estimation program executed in the first embodiment of the motion vector estimation device according to the present invention. The flowchart of the 1st threshold value determination routine performed in 1st Embodiment of the motion vector estimation apparatus which concerns on this invention. Configuration diagram of base frame and reference frame having 8 blocks in the column direction and 13 blocks in the column direction The flowchart of the 1st threshold value calculation routine performed with 1st Embodiment of the motion vector estimation apparatus which concerns on this invention. Flowchart of the first motion vector search routine executed in the first embodiment of the motion vector estimation apparatus according to the present invention. Flowchart of effective pixel number coefficient routine executed in the first embodiment of the motion vector estimation apparatus according to the present invention. The graph explaining operation | movement of the motion vector estimation apparatus which concerns on this invention The flowchart of the 2nd threshold value determination routine performed with 2nd Embodiment of the motion vector estimation apparatus which concerns on this invention. Flowchart of the second motion vector estimation program executed in the third embodiment of the motion vector estimation device according to the present invention. The flowchart of the 3rd threshold value determination routine performed with 3rd Embodiment of the motion vector estimation apparatus which concerns on this invention. The flowchart of the 2nd threshold value calculation routine performed in 4th Embodiment of the motion vector estimation apparatus which concerns on this invention. Flowchart of the second motion vector search routine executed in the fifth embodiment of the motion vector estimation apparatus according to the present invention. Flowchart of luminance difference integrated value calculation routine executed in the fifth embodiment of the motion vector estimation apparatus according to the present invention. Block diagram of image encoding device Block diagram of a conventional motion vector estimation device

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Motion vector estimation apparatus 10 Threshold value determination means 11 Motion vector estimation means 101 Reference frame division means 102 Reference frame luminance inspection area setting means 103 Reference frame pixel luminance median value determination means 104 Reference frame luminance inspection area setting means 105 Reference frame pixel luminance center Value determining means 106 Threshold value calculating means 111 Motion vector search area setting means 112 Motion vector calculating means

Claims (7)

Threshold determination means for determining a threshold based on the luminance of pixels in a predetermined luminance inspection area of the reference frame and the reference frame;
Motion vector estimation means for estimating a motion vector based on the number of pixels whose luminance difference absolute value between corresponding pixels of the reference block in the reference frame and the reference block in the reference frame is equal to or less than the threshold value Vector estimation device. The threshold value determining means is
Reference frame dividing means for dividing the reference frame into reference blocks having a predetermined number of pixels;
A reference frame luminance inspection area setting means for setting a reference frame luminance inspection area in the reference frame for each reference block;
A reference frame pixel luminance median value determining means for determining a reference frame pixel luminance median value which is a median value of a pixel luminance histogram in the reference frame luminance inspection area;
Reference frame luminance inspection area setting means for setting a reference frame luminance inspection area at the same position as the reference frame luminance inspection area in the reference frame in the reference frame;
Reference frame pixel luminance median value determining means for determining a reference frame pixel luminance median value that is a median value of a pixel luminance histogram in the reference frame luminance inspection area;
The motion vector estimation apparatus according to claim 1, further comprising a threshold value calculation unit that calculates an absolute value of a difference between the base frame luminance median value and the reference frame luminance median value as the threshold value. The motion vector estimation means comprises:
A motion vector search region setting means for setting a motion vector search region in the reference frame luminance inspection region for each of the reference blocks;
For each of the reference blocks, a search is performed for a reference block having a minimum number of pixels having a luminance that is smaller than the threshold value from the motion vector search area and the difference between the luminance of the pixels in the reference block and the reference block. The motion vector estimation device according to claim 2, further comprising: motion vector calculation means for determining a motion vector based on the reference block. The dimensionless pixel luminance histogram obtained by dividing the luminance of each pixel in the reference frame luminance inspection area by the difference between the maximum luminance and the minimum luminance of the pixels in the reference frame luminance inspection area The median of the reference frame dimensionless pixel luminance median is determined,
The non-dimensional pixel luminance histogram obtained by dividing the luminance of each pixel in the reference frame luminance inspection area by the difference between the maximum luminance and the minimum luminance of the pixels in the reference frame luminance inspection area. The motion vector estimation apparatus according to claim 2, wherein the median reference frame dimensionless pixel luminance median value is determined. The motion vector estimation canceling unit stops motion vector estimation when the shape of the luminance histogram of the pixel in the reference frame luminance inspection region and the luminance histogram in the reference luminance inspection region are not similar. The motion vector estimation device according to any one of claims 1 to 4, further comprising: The threshold value determining means is
Reference frame dividing means for dividing the reference frame into reference blocks having a predetermined number of pixels;
A reference frame luminance inspection area setting means for setting a reference frame luminance inspection area in the reference frame for each reference block;
Reference frame luminance average value determining means for determining a reference frame luminance average value which is an average value of luminance histograms of pixels in the reference frame luminance inspection area;
Reference frame luminance inspection area setting means for setting a reference frame luminance inspection area at the same position as the reference frame luminance inspection area in the reference frame in the reference frame;
A reference frame luminance average value determining means for determining a reference frame luminance average value which is an average value of luminance histograms of pixels in the reference frame luminance inspection area;
The motion vector estimation apparatus according to claim 1, further comprising a threshold value calculation unit that calculates an absolute value of a difference between the reference frame luminance average value and the reference frame luminance average value as the threshold value. On the computer,
A threshold value determination process for determining a threshold value based on the luminance of pixels in a predetermined luminance inspection area of the reference frame and the reference frame;
A motion vector estimation process for estimating a motion vector based on the number of pixels whose luminance difference absolute value between corresponding pixels of the reference block in the reference frame and the reference block in the reference frame is equal to or less than the threshold value; Motion vector estimation program.

Images