JP5390794B2 - Motion vector search apparatus and motion vector search method - Google Patents

Description

  The present invention relates to a motion vector search apparatus and a motion vector search method for searching for a motion vector of a moving image in units of blocks.

  2. Description of the Related Art Conventionally, for example, a digital video camera is known as a recording apparatus that captures an image of a subject and compresses and encodes moving image data obtained thereby to a recording medium. In recent years, the recording medium used in such a recording apparatus has changed from a conventional magnetic tape to a disk medium, a semiconductor memory, or the like, which is more advantageous in terms of random accessibility and high-speed access. On the other hand, generally, disk media and semiconductor memories have lower cost performance with respect to storage capacity than magnetic tapes. Therefore, when a disk medium or a semiconductor memory is used as a recording medium, it is necessary to compress and encode moving image data with higher efficiency.

  In addition, from the expectation of high image quality, digital video cameras that handle HD (High Definition) video with a larger amount of information are becoming widespread. From this point of view, more efficient compression coding of moving image data is desired. As a compression encoding method for compressing and encoding moving image data with high efficiency, the MPEG2 method has been used as a standard.

  Furthermore, in recent years, the need for encoding at a lower bit rate for portable terminals with limited recording capacity and further improvement in the recordable time on recording media has increased. Has been developed. One of them is H.264. H.264 | AVC format. H. The H.264 | AVC system requires a large amount of calculation for encoding and decoding compared with conventional encoding systems such as the MPEG2 system and the MPEG4 system, but is known to realize high encoding efficiency. It has been. In the following, H.P. H.264 | AVC format H.264.

H. In H.264, various ideas have been made in order to increase the encoding efficiency. One example is variable block size motion compensation. This macroblock is coded units (coded blocks) further divided to form a macroblock partitions (motion compensation block), Ru der to perform motion compensation in the motion compensation block. H. In H.264, by using variable block size motion compensation, more precise motion compensation is possible as compared with a conventional compression coding method such as the MPEG2 method in which motion compensation is performed with a single block size.

  On the other hand, H. In H.264, the size of an encoded block can be selected by variable block size motion compensation, resulting in an increase in calculation load for determining the block size. That is, it is necessary to perform motion vector search processing for all motion compensation blocks that can be set for a coding block to be coded, and to select an optimal one. Therefore, as a device for determining an appropriate block size as easily as possible, a proposal has been made to reduce the amount of calculation related to block size selection using information when an image to be encoded is captured (for example, a patent). Reference 1).

JP 2006-254370 A

  As mentioned above, H.M. In H.264, in order to subdivide motion compensation blocks by variable block size motion compensation, it is necessary to perform enormous operations to search for motion vectors and determine a method for dividing motion compensation blocks. Therefore, especially when performing encoding processing in real time, such as the above-described digital video camera, a processor capable of extremely high-speed computation is required, which increases the cost of the apparatus and increases power consumption. There was a problem of becoming.

  In the configuration of Patent Document 1, the imaging unit needs to output information at the time of imaging to the encoding unit, and the encoding unit also inputs the information at the time of encoding supplied from the imaging unit. A mechanism for processing is required.

  Therefore, an object of the present invention is to provide a motion vector search apparatus and a motion vector search method that can determine a block division method with high accuracy and perform a motion vector search by a simpler method.

In order to solve the above-described problem, the present invention divides one screen into coding blocks that are coding units, and divides the coding block into motion compensation blocks, which are included in the coding block to be coded. A motion vector search apparatus for searching for a motion vector with reference to at least a past image with respect to the motion compensation block, wherein the encoding block to be encoded is divided by a plurality of different division methods set. , According to the set search accuracy for each of the dividing means for forming a motion compensation block corresponding to the dividing method and the motion compensating block formed by the dividing means by the plurality of different dividing methods. Based on a motion vector search means for searching for a motion vector, and a result of the motion vector search by the search accuracy in the motion vector search means. The division method is selected, and the selected division method and a predetermined division method among the plurality of different division methods are set for the division unit, and more accurate search accuracy is obtained. Setting means for setting as accuracy; setting the dividing method and the search accuracy by the setting means; and searching for the motion vector by the motion vector searching means according to the set dividing method and the search accuracy. Control means for performing control so as to repeat sequentially, and the setting means assigns a group to each of the plurality of motion compensation blocks formed by the plurality of different division methods according to the division method, For each group, the division method is selected based on the motion vector search result and set in the division unit, and the predetermined division is performed. Set law to said dividing means, said control means, if it is determined that the repeating the operation by high accuracy search accuracy than the set by said setting means is performed reaches a preset search accuracy, the motion Based on the search result of the motion vector by the vector search means, one of the division methods is selected from the division method selected based on the search result and the predetermined division method, and the motion by the selected division method is selected The motion vector search apparatus is characterized in that a motion vector obtained as a vector search result is used as a final motion vector.

In addition, the present invention divides one screen into coding blocks that are coding units, divides the coding block into motion compensation blocks, and at least a past of the motion compensation blocks included in the coding block to be coded. A motion vector search method for searching for a motion vector with reference to an image of the image, wherein the dividing means divides the encoding block to be encoded by a plurality of different division methods set, and supports the division method A dividing step for forming the motion compensation block, and a motion vector search means for each of the motion compensation blocks formed in the dividing step by the plurality of different dividing methods according to the set search accuracy. A motion vector search step for searching for a motion vector, and a setting means depend on the search accuracy in the motion vector search step. Selecting the division method based on a motion vector search result, and setting the selected division method and a predetermined division method among the plurality of different division methods for the division step; A setting step of setting a search accuracy with higher accuracy as the search accuracy, and a control unit sets the division method and the search accuracy by the setting step, and the setting method sets the division method and the search accuracy according to the set division method and the search accuracy. And a control step of controlling the motion vector search step so as to sequentially repeat the motion vector search operation, and the control step has a higher accuracy than that set in the setting step by the repeated operation. If it is determined that the high search accuracy has reached the preset search accuracy, the motion vector search step Based on the search result of the motion vector, one of the division methods is selected from the division method selected based on the search result and the predetermined division method, and the search result of the motion vector by the selected division method The motion vector search method is characterized in that a motion vector obtained as a final motion vector is used, wherein the predetermined division method is a method of dividing the encoding block to be encoded into the plurality of different division methods. The motion vector search result performed for each of the motion compensation blocks formed by dividing the search results is accumulated for a plurality of image data, and the accumulated search results are subjected to statistical processing. It is a motion vector search method characterized by being determined .

  Since the present invention has the above-described configuration, it is possible to determine a block division method with high accuracy and perform a motion vector search by a simpler method.

<Configuration common to each embodiment>
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 schematically shows a configuration of an example of a digital video camera 1 to which the present invention can be applied. The imaging unit 10 performs predetermined processing on an imaging system such as an optical system, a CCD that converts light incident through the optical system into an electric signal, and an imaging signal output from the imaging element, and a baseband moving image A signal processing unit that outputs data is included. The baseband moving image data output from the imaging unit 10 is supplied to the encoding unit 11 to which the present invention is applied. H.264 compression / encoding processing is performed.

  The compressed moving image data that has been compression-encoded is output from the encoding unit 11 and supplied to the recording control unit 12. The recording control unit 12 controls a data recording operation on the recording medium 13 using a disk medium or a semiconductor memory. The recording control unit 12 performs error correction encoding processing, recording encoding processing according to the type of the recording medium 13, and the like on the supplied compressed moving image data to generate recording data. The recording control unit 12 writes the recording data to the recording medium 13 in the recording unit of the recording medium 13 via the buffer memory.

  The CPU 20 controls the overall operation of the digital video camera 1. That is, the CPU 20 operates using the RAM 22 as a work memory according to a program stored in advance in the ROM 21, and controls the operations of the imaging unit 10, the encoding unit 11, and the recording control unit 12. The operation unit 23 is provided with a predetermined operation element for a user to operate the digital video camera 1 and a display element for the user to confirm the operation of the digital video camera 1. The CPU 20 can control the operation of the digital video camera 1 in accordance with a user operation on the operation unit 23.

  FIG. 2 shows an exemplary configuration of the encoding unit 11 according to the embodiment of the present invention. The encoding unit 11 compresses the supplied baseband video data by performing orthogonal transform using Hadamard transform and integer precision DCT, and intra-frame prediction encoding using intra-frame prediction encoding and motion compensation. Encode. Hereinafter, the Hadamard transform and orthogonal transform using integer precision DCT are referred to as integer transform, and intra-frame prediction coding and inter-frame prediction coding are referred to as intra coding and inter coding, respectively. The operation in the encoding unit 11 is controlled by the CPU 20 as control means.

  Prediction of a P picture for prediction with one reference frame for a unit of motion compensation (MC block) by inter coding, and prediction with up to two reference frames before and after the MC block in time series A B picture is formed. Also, an I picture is formed by intra coding. Further, in inter-frame predictive encoding, moving image data is encoded as data having a GOP structure in which these I picture, P picture, and B picture are arranged in a predetermined manner.

For example, when the encoding unit 11 forms 1 GOP with 15 frames including one I picture, four P pictures, and 10 B pictures, the frames input to the encoding unit 11 are processed in the following order: A picture type is assigned. The subscript indicates the order of input or display.
B ₁ B ₂ I ₃ B ₄ B ₅ P ₆ B ₇ B ₈ P ₉ B ₁₀ B ₁₁ P ₁₂ B ₁₃ B ₁₄ P ₁₅

Here, since the B picture performs predictive coding using the future picture together with the past picture in time series, the coding is performed by changing the order of the B picture with respect to the I picture and the P picture as follows: Done in order. Note that the B ₁ picture and the B ₂ picture following the I ₃ picture are predictively encoded using the I ₃ picture and the P ₁₅ picture in the immediately preceding GOP.
I ₃ B ₁ B ₂ P ₆ B ₄ B ₅ P ₉ B ₇ B ₈ P ₁₂ B ₁₀ B ₁₁ P ₁₅ B ₁₃ B ₁₄

  Baseband moving image data supplied from the imaging unit 10 is sequentially stored in the frame memory 102 in the frame order along the time series of imaging. From the frame memory 102, moving image data is read in the encoding order for each frame.

  When performing intra coding, image data of a block serving as a coding unit in a frame corresponding to an I picture of moving image data is read from the frame memory 102, and subtracted inputs of the intra prediction unit 105 and the subtractor 107 are read. Supplied at each end. Here, the frame corresponds to one screen based on moving image data. That is, the block of the coding unit is formed by dividing one screen of moving image data into a predetermined number.

  The intra prediction unit 105 applies block matching with the encoding target block to each of a plurality of prediction images generated from the reconstructed image that is supplied from the adder 112 and is located near the encoding target block in the same frame. Against. Then, as a result of block matching, an intra predicted image having the highest correlation is selected from a plurality of predicted images. The intra-predicted image is output to the switch 106 selected on the intra-prediction unit 105 side according to the execution of intra-coding.

  On the other hand, when performing inter coding, image data of a block to be coded is read from the frame memory 102 and supplied to the motion vector search unit 103. At the same time, a reference image is read from the frame memory 102 and supplied to the motion vector search unit 103. The reference image can be selected from a plurality of pictures. The motion vector search unit 103 detects a motion vector from the image data of the block to be encoded and the reference image. At this time, the motion vector search is performed by dividing the encoding target block into macroblocks and partitions by a process described later. The motion vector detected by the motion vector search unit 103 is supplied to the inter-frame motion compensation unit 104.

  The inter-frame motion compensation unit 104 performs motion compensation based on the motion vector supplied from the motion vector search unit 103 and the reference image read from the frame memory 102, and generates an inter prediction image by inter prediction. The inter-predicted image is output to the switch 106 selected on the inter-frame motion compensation unit 104 side according to the execution of the inter coding.

  The output of the switch 106 is supplied to the subtraction input terminal of the subtractor 107 and one input terminal of the adder 112. The subtracter 107 outputs difference information of pixel values between the encoding target block supplied from the frame memory 102 and the inter prediction image or intra prediction image supplied from the switch 106. The difference information is supplied to the integer transform unit 108, subjected to integer transform by Hadamard transform or integer precision DCT, and then quantized by the quantizer 109 based on a predetermined quantization coefficient. The output of the quantization unit 109 is entropy-encoded in the entropy encoding unit 115 and is output from the encoding unit 11.

  The quantization amount in the quantization unit 109 is calculated by the code amount control unit 116 based on the feedback of the code amount generated by the entropy encoding unit 115. Also, the quantized transform coefficient that is the output of the quantization unit 109 is inversely quantized by the inverse quantization unit 110 and further subjected to inverse integer transform processing by the inverse integer transform unit 111 to be decoded. Further, the inter prediction image or the intra prediction image output from the switch 106 is added by the adder 112 to form a reconstructed image. This reconstructed image is supplied to the intra prediction unit 105 and is used to generate the above-described intra predicted image.

  Also, the reconstructed image output from the adder 112 is subjected to encoding distortion reduction processing by the in-loop filter 113 and then stored in the frame memory 102 as a reference image used for inter encoding.

<First Embodiment>
Next, processing in the motion vector search unit 103 according to the first embodiment of the present invention will be described in detail. In the first embodiment, a macroblock / partition division method is obtained by a two-stage process of a coarse search that performs motion vector search with low search accuracy and a dense search that performs search with high search accuracy. . Then, the motion vector corresponding to the obtained macroblock partition is used as the final motion vector search result.

  FIG. 3 is a flowchart showing an example motion vector search process according to the first embodiment. First, in step S <b> 10, image data of a coding block to be coded (hereinafter, coding target block) is read from the frame memory 102 and supplied to the motion vector search unit 103. In addition, image data (reference image data) in the motion vector search range is read from the frame memory 102.

<Coarse search>
The motion vector search unit 103 as setting means reduces the image data of the encoding target block read and supplied from the frame memory 102 and the image data of the motion vector search range at a predetermined reduction rate. . As an example, when the size of the encoding target block is 16 pixels × 16 pixels and the reduction rate is 1/4 both in the horizontal and vertical directions, the encoded target block after reduction has a size of 4 pixels × 4 pixels. Is done. The image data in the motion vector search range is also reduced at the same ratio.

  The reduction processing is performed by thinning out pixels according to the reduction rate in the image data read from the frame memory 102 in the motion vector search unit 103, for example. However, the present invention is not limited to this, and reduction processing may be performed using linear interpolation or the like. Further, the reduction process is not limited to being performed by the motion vector search unit 103, and may be performed, for example, by storing image data reduced in advance at a predetermined reduction rate in the frame memory 102. Further, the image data may be read from the frame memory 102 while thinning out pixels according to the reduction ratio.

  In the following description, in the coarse search, the reduction ratio of the image data is set to 1/4 in both the horizontal and vertical directions, and the search range is ±± with respect to the encoding target block in the horizontal and vertical directions in the reduced image data. The range is 8 pixels.

  When the process for reducing the encoding target block and the image data in the search range is completed in step S10, the process proceeds to step S11. In step S11, the motion vector search unit 103 as a dividing unit sets a motion compensation block (hereinafter referred to as a partition) for the reduced encoding target block. In the first embodiment, the motion vector search unit 103 as the setting unit determines four types of partitions illustrated in FIGS. 4A to 4D in accordance with a loop process in step S13 described later. The setting is sequentially performed on the encoding target block.

  FIG. 4A shows a case where the encoding target block is not divided. In other words, in FIG. 4A, the encoding target block itself is regarded as one partition. FIG. 4B shows a case where the encoding target block is divided into two equal parts in the horizontal direction. FIG. 4C shows a case where the encoding target block is divided into two equal parts in the vertical direction. FIG. 4D shows an example in which the current block is divided into two equal parts in the horizontal and vertical directions and divided into four partitions.

  Hereinafter, due to the number of pixels before reduction, the division method of FIG. 4A is referred to as a partition type 16 × 16, and the division method of FIG. 4B is referred to as a partition type 16 × 8. Similarly, the division method of FIG. 4C is called a partition type 8 × 16, and the division method of FIG. 4D is called a partition type 8 × 8. The naming method of this partitioning method is the same for the following similar cases.

  When a partition is set for the encoding target block in step S11, the process proceeds to step S12. In step S12, the motion vector search unit 103 as a motion vector search means performs a rough search of motion vectors for each set partition using the reduced encoding target block and search range image data. That is, in step S12, the block with the highest correlation is selected for each partition of the encoding target block by block matching with the image data in the search range.

  Then, the motion vector search unit 103 stores the difference between the coordinates of the selected block having the highest correlation and the coordinates of the partition of the encoding target block as the motion vector of the partition. Here, as the coordinates of the block and the partition, the coordinates of one representative point in the block and the partition, for example, the coordinates of the upper left corner are used. In addition, the motion vector search unit 103 stores a value obtained by summing the difference values between corresponding pixels of the block having the highest correlation and the partition of the encoding target block as a pixel difference of the partition.

  When the rough search process in step S12 is completed, the process proceeds to step S13, and it is determined whether or not the rough search has been completed for all partitions set in step S11. If it is determined that the rough search is not completed for the partition, the process returns to step S11, and the rough search is performed for the next partition. For example, the partition types 16 × 16, 16 × 8, 8 × 16, and 8 × 8 shown in FIGS. 4A to 4D are sequentially set, and these partition types 16 × 16, 16 × 8, and 8 × are set. Processing for each of the partitions in 16 and 8 × 8 is performed sequentially.

  If it is determined in step S13 that the rough search has been completed for all partitions, the process proceeds to the next step S14. In step S14, the pixel differences stored for each partition in step S12 described above are summed for each block to be encoded. In other words, in step S14, the pixel differences obtained for each partition are collected into pixel differences for each block to be encoded for each partitioning method. For example, if the partition type is 16 × 8, the pixel difference of the upper partition and the pixel difference of the lower partition are summed. This is done for each of the partition types 16 × 16, 16 × 8, 8 × 16 and 8 × 8.

  In the next step S15, candidates for the partitioning method for fine search are determined. First, the pixel difference in the encoding target block unit for each partition division method obtained in step S14 is compared, and the partition division method with the smallest pixel difference is selected to be the first partition division method. . Secondly, a partition division method having a maximum occurrence probability, which is obtained probabilistically by statistical processing, is referred to as a second partition division method.

  Note that the second partitioning method has, for example, the highest probability that the pixel difference for each block to be encoded is the smallest when the technique for obtaining the first partitioning method is applied to a large number of moving images. Get by selecting a partitioning method that will be higher. For example, the results of obtaining the first partitioning method for a large number of moving images are accumulated, and the accumulated data is statistically analyzed. In this case, the second partitioning method is fixed. When the partition types 16 × 16, 16 × 8, 8 × 16, and 8 × 8 are used, generally, the above-described partition type 16 × 16 has a high probability of being the second partitioning method.

<Dense search>
When candidates for the partitioning method for fine search are determined, the process proceeds to step S16, and the motion vector search unit 103 reads the image data of the block to be encoded and the reference image data from the frame memory 102 again. It is. In the dense search, the search process is performed without reducing the image data of the encoding target block and the reference image data.

  In the next step S17, a motion vector search is performed for each partition set by the first partitioning method. More specifically, the search range is a range of ± 3 pixels both horizontally and vertically with respect to the partition centered on the coordinates indicated by the motion vector of the partition obtained by the rough search described above. Block matching of partitions is performed within this search range, and the block having the highest correlation is selected to obtain a motion vector. Furthermore, a value obtained by summing the difference values between corresponding pixels of the block having the highest correlation and the partition in the encoding target block obtained as a result of block matching is stored as a pixel difference of the partition.

  In the next step S18, for each partition set by the second partitioning method, a motion vector search is performed in the same manner as in step S17 described above. When the motion vector search process based on the second partition division method is completed in step S18, the process proceeds to step S19.

  In step S19, the partition division method is determined, and the motion vector of the encoding target block is determined. Similar to step S14 described above, the pixel differences stored for each partition are summed in units of encoding target blocks for each of the first and second partition division methods. Then, the sum of pixel differences in units of encoding blocks in the first partitioning method is compared with the sum of pixel differences in units of encoding blocks in the second partitioning method. As a result of the comparison, the smaller one of the sums of pixel differences is selected from the first and second partition division methods, and this is used as the partition division method for the encoding target block.

  As for the motion vector, the motion vector corresponding to the selected partitioning method is determined as the motion vector of the coding target block among the motion vectors stored in the above-described step S17 and step S18.

  The operation of the motion vector search unit 103 described above will be described more specifically with reference to FIG. Here, the first partition division method with the smallest pixel difference in the coarse search is the partition type 8 × 16, and the second partition division method with the highest occurrence probability is the partition type 16 × 16 And

Through the processes in steps S11 to S13 in FIG. 3, a rough search is performed for each of the partition types 16 × 16, 16 × 8, 8 × 16, and 8 × 8 (see FIG. 5A). A difference and a motion vector are obtained for each partition. In step S14, a pixel difference for each block to be encoded is obtained. On the basis of the result, in step S15, the first partitioning method (partition type 8 × 16 in this example) that minimizes the pixel difference and the second partitioning method (partition type in this example) having the maximum probability of occurrence. 16 × 16).

  In the fine search, as illustrated in FIG. 5B, for the first and second partitioning methods, a process for obtaining a motion vector and a process for obtaining a pixel difference are performed (step S17 and step S18), respectively. . Of the first and second partition division methods, the one with the smaller sum of pixel differences of the encoding target block is selected as the partition division method for the encoding target block. At the same time, the motion vector corresponding to the selected partitioning method is determined as the motion vector of the encoding target block. In the example of FIG. 5C, the partition type 16 × 16 is selected as the final partitioning method.

  As described above, according to the first embodiment of the present invention, only two partition division methods for searching for motion vectors at the time of dense search are required. This reduces the amount of calculation required for motion vector search.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. The second embodiment of the present invention is a more generalized motion vector search operation according to the first embodiment described above. That is, in the first embodiment described above, the motion vector search is performed in two stages, a coarse search and a fine search. On the other hand, in the second embodiment, the motion vector search is performed in n stages, the search accuracy is increased stepwise, the partition division method is selected, and the motion vector is searched. Therefore, it can be said that the motion vector search operation according to the first embodiment described above is a limited example of the operation according to the second embodiment.

  In the second embodiment, the configuration of the encoding unit 11 described with reference to FIG. 2 can be applied as it is, and only the motion vector search unit 103 is different in processing from the first embodiment. Hereinafter, an example of processing in the motion vector search unit 103 according to the second embodiment will be described with reference to the flowchart of FIG.

In step S30, the accuracy of motion vector search by the motion vector search unit 103 is set. Then, similarly to the case of the first embodiment described above, the motion vector search unit 103 reduces the image data and reference image data of the encoding target block read from the frame memory 102 according to the search accuracy. To do. For example, if the search accuracy is 4 pixel units, the image data reduction rate is 1/4, if the search accuracy is 2 pixel units, the reduction rate is 1/2, and if the search accuracy is 1 pixel unit, the reduction rate. Is 1/1 (not reduced).

  In the next step S31, a partition is set and group setting for the partitioning method is performed. In the second embodiment, the partition division method is divided into groups, and motion vector search processing is performed for each group. Although details will be described later, the grouping can be performed based on the occurrence probability of each partition type.

  In step S32 and step S33, the motion vector search process is performed according to the search accuracy in the same manner as the process of step S12 of the first embodiment described above for each set partition for the partition group to be searched. . That is, block matching with the reference image data is performed for each partition of the encoding target block, and the block with the highest correlation is selected. Then, for the block and the partition, the coordinate difference is stored as a motion vector of the partition, and the value obtained by summing the difference values of the corresponding pixels is stored as the pixel difference of the partition.

  If it is determined in step S33 that the search process for one partition group has been completed, the process proceeds to step S34. In step S34, candidates for the partitioning method in the motion vector search process with the next search accuracy are determined. That is, in step S34, in the same partition group as in step S15 described above, the pixel difference in the encoding target block unit for each partition division method is compared, and the partition division with the smallest pixel difference is performed. Select a method. At the same time, the partitioning method having the maximum occurrence probability obtained by a statistical method is selected.

  The process proceeds to step S35, and it is determined whether or not the motion vector search process for all the partition groups at the search accuracy set in step S30 is completed. If it is determined that the process has not been completed, the process returns to step S31, and the process for the next partition group is performed.

  If it is determined in step S35 that the motion vector search process for all the partition groups in the search accuracy has been completed, the process proceeds to step S36. In step S36, it is determined whether the number of partition groups is 1 and the number of partition division method candidates is 2. If it is determined in step S36 that the number of partition groups is not 1 or the number of candidates for the partitioning method is not 2, the process returns to step S30 to set motion vector search accuracy. Is called. And the process of step S30-step S36 is repeated sequentially until the conditions of step S36 are satisfy | filled.

  On the other hand, if it is determined in step S36 that the number of partition groups is 1 and the number of partition division method candidates is 2, the process proceeds to step S37. In step S37, the pixel differences of the respective partitions are summed up in units of encoding target blocks for each of the two partition division methods. Then, the sum of the pixel differences by the two partition division methods is compared, a partition division method having a small sum of the pixel differences is selected, and the partition division method of the coding target block is determined. Similarly, the motion vector corresponding to the selected partitioning method is determined as the motion vector of the current encoding target block.

  The process of the flowchart in FIG. 6 will be described more specifically with reference to FIGS. 7 and 8. Here, the search accuracy is increased in the order of 4 pixels, 2 pixels, and 1 pixel, and the motion vector search operation is performed in three stages. Further, seven types illustrated in FIG. 7 are prepared as partitioning methods. That is, partition types 8 × 4, 4 × 8, and 4 × 4 are prepared in addition to the partition types 16 × 16, 16 × 8, 8 × 16, and 8 × 8 used in the above embodiment. The partition type 8 × 4 divides each partition of the partition type 8 × 8 into two equal parts in the horizontal direction. The partition type 4 × 8 divides each partition of the partition type 8 × 8 into two equal parts in the vertical direction. The partition type 4 × 4 divides each partition of the partition type 8 × 8 into two equal parts horizontally and vertically.

  In the first step, the accuracy of motion vector search is set in units of 4 pixels in step S30, and the image data in the encoding target block and motion vector search range is reduced at a reduction rate of 1/4. In step S31, group setting for partition setting and partition division method is performed. For example, as illustrated in FIG. 8A, partition types 16 × 16, 16 × 8, and 8 × 16 are set as the first partition group, and partition types 8 × 8, 8 × 4, 4 × 8, and 4 are used. Let x4 be the second partition group.

  Here, the group setting for the partition division method can be performed based on the occurrence probability of each partition division method, for example. That is, it is known that when seven types of partition division methods are prepared as shown in FIG. 7, occurrence probability peaks occur in the partition types 16 × 16 and 8 × 8. The partitioning methods that cause this occurrence probability peak are assigned to different groups. As a result, an opportunity to compare the partitioning method selected based on the occurrence probability with the partitioning method selected based on the pixel difference increases, and the partitioning method selection and motion vector search are performed with higher accuracy. It becomes possible.

  In step S32 and step S33, a motion vector search for each partition group is performed. First, the motion vector search process is performed on the first partition group, and when the process ends, the process proceeds to step S34. Then, by comparing pixel differences in units of encoding blocks for each partitioning method within the first partition group, the first partitioning method that minimizes the pixel difference and the second that has the highest probability of occurrence. Partitioning method is selected. In the example of FIG. 8A, the partition type 8 × 16 is selected as the first partitioning method, and the partition type 16 × 16 is selected as the second partitioning method.

  Similarly, a motion vector search process is performed for the second partition group. In step S34, a third partition division method that minimizes the pixel difference, and a fourth partition division method that maximizes the occurrence probability, Is selected. In the example of FIG. 8A, the partition type 4 × 8 is selected as the third partitioning method, and the partition type 8 × 8 is selected as the fourth partitioning method.

  Since the motion vector search process for all the partition groups in the first stage is completed (step S35), the process proceeds to step S36. At this stage, as illustrated in FIG. 8A, since the number of partition groups is 2 and the number of partition division method candidates is 4, the process returns to step S30. Then, the search accuracy is set in units of two pixels, and the second stage process is started.

  In the second stage process, a partition group is set in step S31. In the first stage of processing, the first to fourth partition division methods are selected as candidates for the partition division method. In step S31, as illustrated in FIG. 8B, a new partition group (referred to as a third partition group) is created from these first to fourth partition division methods.

  In step S32 and step S33, the motion vector search process is performed for the third partition group in the same manner as described above. When the process is completed, the process proceeds to step S34. Then, the fifth partitioning method that minimizes the pixel difference and the sixth partitioning method that maximizes the occurrence probability are selected. In the example of FIG. 8B, the partition type 4 × 8 is selected as the fifth partition group, and the partition type 16 × 16 is selected as the sixth partition group.

  Since the motion vector search process for all the partition groups in the second stage is completed (step S35), the process proceeds to step S36. At this stage, as illustrated in FIG. 8B, since the number of partition groups is 1 and the number of partition division method candidates is 4, the process returns to step S30. Then, the search accuracy is set in units of one pixel, and the third stage process is started.

  In the third stage process, a partition group is set in step S31. In the second stage processing, the fifth and sixth partition division methods are selected as the partition division method candidates. In step S31, as illustrated in FIG. 8C, a new partition group (referred to as a fourth partition group) is created from these fifth and sixth partition division methods.

  In step S32 and step S33, a motion vector search process is performed for the fourth partition group in the same manner as described above. When the process ends, the process proceeds to step S34. Then, the partitioning method that minimizes the pixel difference and the partitioning method that maximizes the occurrence probability are selected. In the example of FIG. 8C, the partition type 4 × 8 is selected as the partition group with the smallest pixel difference, and the partition type 16 × 16 is selected as the partition group with the highest occurrence probability.

  Since the motion vector search process for all the partition groups in the third stage has been completed (step S35), the process proceeds to step S36. At this stage, as illustrated in FIG. 8C, since the number of partition groups is 1 and the number of partition division method candidates is 2, the process proceeds to step S37.

  In step S37, for each of the two partition division methods, the pixel differences of each partition are summed up in units of encoding target blocks, and the smaller one of the pixel differences is selected and used as the partition division method of the encoding target block. decide. Similarly, the motion vector corresponding to the selected partitioning method is determined as the motion vector of the current encoding target block. In the example of FIG. 8D, the partition type 4 × 8 is selected as the final partitioning method.

  As described above, according to the second embodiment of the present invention, even when motion vector search processing is performed in n stages according to search accuracy, the number of partition division methods to be searched is small, and motion vector search can be performed. The required arithmetic processing can be reduced.

  In the above description, the present invention has been described as applied to a digital video camera, but this is not limited to this example. That is, the present invention relates to H.264. Any apparatus that encodes moving image data in accordance with H.264 | AVC can be applied to other types of apparatuses.

<Other embodiments>
Each of the above-described embodiments can be realized by software by a computer of a system or apparatus (or CPU, MPU, etc.).

  Accordingly, the computer program itself supplied to the computer in order to realize the above-described embodiments by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  Note that the computer program for realizing each of the above-described embodiments may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  The computer program for realizing each of the above-described embodiments is supplied to the computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code.

  Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.

  That is, a server apparatus that provides a client computer with a program file for realizing each of the above-described embodiments is also one aspect of the present invention.

  In addition, a storage medium that encrypts and stores a computer program for realizing each of the above-described embodiments is distributed, and key information for decryption is supplied to a user who satisfies a predetermined condition, and the computer has the computer. May be allowed to install. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

  In addition, the computer program for realizing each of the above-described embodiments may use an OS function already running on the computer.

  Further, a part of the computer program for realizing each of the above embodiments may be configured by firmware such as an expansion board mounted on the computer, or may be executed by a CPU provided in the expansion board. Also good.

It is a block diagram which shows roughly the structure of an example of the digital video camera which can apply this invention. It is a block diagram which shows the structure of an example of the encoding part which concerns on embodiment of this invention. It is a flowchart which shows the motion vector search process of an example by the 1st Embodiment of this invention. It is a basic diagram which shows the example of the macroblock partition set with respect to the encoding object block in the 1st Embodiment of this invention. FIG. 6 is a schematic diagram for specifically explaining the operation of the motion vector search unit according to the first embodiment of the present invention. It is a flowchart which shows the motion vector search process of an example by the 2nd Embodiment of this invention. It is a basic diagram which shows the example of the macroblock partition set with respect to the encoding object block in the 2nd Embodiment of this invention. It is an approximate line figure for explaining operation of a motion vector search part by a 2nd embodiment of the present invention more concretely.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital video camera 11 Encoding part 102 Frame memory 103 Motion vector search part 104 Inter-frame motion compensation part 105 Intra prediction part 108 Integer conversion part 109 Quantization part 110 Inverse quantization part 111 Inverse integer conversion part 113 In-loop filter 115 Entropy Encoding unit 116 Code amount control unit

Claims (9)

One screen is divided into coding blocks which are coding units, the coding block is divided into motion compensation blocks, and at least past images are included in the motion compensation block included in the coding block to be coded. A motion vector search device for searching for a motion vector with reference to
A dividing unit configured to divide the encoding block to be encoded by a plurality of different setting division methods to form a motion compensation block corresponding to the division method;
Motion vector search means for searching for the motion vector according to a set search accuracy for each of the motion compensation blocks formed by the dividing means by the plurality of different dividing methods;
Selecting the division method based on the search result of the motion vector based on the search accuracy by the motion vector search means, the selected division method, and a predetermined division method among the plurality of different division methods; And setting means for setting a higher accuracy search accuracy as the search accuracy,
The division means and the search accuracy are set by the setting means, and the motion vector search means is controlled to sequentially repeat the motion vector search operation according to the set division method and the search accuracy. Control means,
The setting means includes
A group is assigned to each of the plurality of motion compensation blocks formed by the plurality of different division methods according to the division method, and the division method is selected for each group based on the result of the motion vector search. And setting the dividing means and setting the predetermined dividing method to the dividing means,
The control means includes
If it is determined that the higher-accuracy search accuracy set by the setting unit has been set to a preset search accuracy by the repeated operation, based on the motion vector search result by the motion vector search unit, One division method is selected from the division method selected based on the search result and the predetermined division method, and a motion vector obtained as a search result of the motion vector by the selected division method is finally obtained. A motion vector search apparatus characterized in that a motion vector is used. The setting means is provided for each group.
According to claim 1, characterized in that setting the dividing means 1 of the predetermined division method and sets the dividing means to select the division method 1 based on the result of the search of the motion vector Motion vector search device. In the predetermined division method, a search result of the motion vector performed on each of the motion compensation blocks formed by dividing the encoding block to be encoded by the plurality of different division methods is obtained as a plurality of image data. stored for the accumulated the search result to the motion vector search apparatus according to claim 1 or claim 2, characterized in that it is determined on the basis of a result of performing a statistical process. One screen is divided into coding blocks which are coding units, the coding block is divided into motion compensation blocks, and at least past images are included in the motion compensation block included in the coding block to be coded. A motion vector search device for searching for a motion vector with reference to
  A dividing unit configured to divide the encoding block to be encoded by a plurality of different setting division methods to form a motion compensation block corresponding to the division method;
  Motion vector search means for searching for the motion vector according to a set search accuracy for each of the motion compensation blocks formed by the dividing means by the plurality of different dividing methods;
  Selecting the division method based on the search result of the motion vector based on the search accuracy by the motion vector search means, the selected division method, and a predetermined division method among the plurality of different division methods; And setting means for setting a higher accuracy search accuracy as the search accuracy,
  The division means and the search accuracy are set by the setting means, and the motion vector search means is controlled to sequentially repeat the motion vector search operation according to the set division method and the search accuracy. Control means and
With
  The control means includes
  If it is determined that the higher-accuracy search accuracy set by the setting unit has been set to a preset search accuracy by the repeated operation, based on the motion vector search result by the motion vector search unit, One division method is selected from the division method selected based on the search result and the predetermined division method, and a motion vector obtained as a search result of the motion vector by the selected division method is finally obtained. A motion vector search device for a simple motion vector,
  In the predetermined division method, a search result of the motion vector performed on each of the motion compensation blocks formed by dividing the encoding block to be encoded by the plurality of different division methods is obtained as a plurality of image data. A motion vector search apparatus characterized in that it is determined on the basis of a result obtained by performing statistical processing on the stored search results. One screen is divided into coding blocks which are coding units, the coding block is divided into motion compensation blocks, and at least past images are included in the motion compensation block included in the coding block to be coded. A motion vector search method for searching for a motion vector with reference,
A dividing step in which a dividing unit divides the encoding block to be encoded by a plurality of different dividing methods set to form a motion compensation block corresponding to the dividing method;
A motion vector search step for searching for the motion vector according to a set search accuracy for each of the motion compensation blocks formed in the division step by the plurality of different division methods;
The setting means selects the division method based on a result of the motion vector search based on the search accuracy in the motion vector search step, and the selected division method and a predetermined one of the plurality of different division methods A setting method for setting the search accuracy with higher accuracy as the search accuracy.
The control means sequentially repeats the operation of setting the division method and the search accuracy in the setting step, and searching for the motion vector in the motion vector search step according to the set division method and the search accuracy. A control step for controlling
The control step includes
If it is determined that the higher-precision search accuracy set in the setting step has reached a preset search accuracy by the repeated operation, based on the motion vector search result in the motion vector search step, One division method is selected from the division method selected based on the search result and the predetermined division method, and a motion vector obtained as a search result of the motion vector by the selected division method is finally obtained. a motion-out vector search method shall be the motion vectors,
In the predetermined division method, a search result of the motion vector performed on each of the motion compensation blocks formed by dividing the encoding block to be encoded by the plurality of different division methods is obtained as a plurality of image data. A motion vector search method characterized in that it is determined based on a result obtained by performing statistical processing on the stored search results . An image processing apparatus that encodes moving image data by intraframe prediction encoding or interframe prediction encoding,
  A frame memory for storing the frame image of the video data;
  A determination unit configured to determine a motion compensation block type and a motion vector used for inter-frame prediction encoding of the image data of the encoding target block read from the frame memory; and the determination unit includes a plurality of types of motion When determining a motion compensation block used for encoding image data of the encoding target block among the compensation blocks,
  Reducing the image data and reference image data of the encoding target block;
  A plurality of types of motion compensation blocks including the specific motion compensation block, including pixel difference values between the block corresponding to the motion compensation block and the reduced reference image data in the image data of the reduced encoding target block. And performing a first determination process using the motion compensation block having the smallest pixel difference value as the first motion compensation block,
  Without reducing the image data and reference image data of the encoding target block,
  A pixel difference value between the first motion compensation block and the reference image data in the image data of the encoding target block, and an occurrence probability that is probabilistically obtained by statistical processing in the image data of the encoding target block. A pixel difference value between the maximum motion compensation block and the reference image data is calculated, and the first motion compensation block and the motion compensation block having the maximum occurrence probability determined by the statistical processing are: An image processing apparatus that performs a second determination process in which a smaller pixel difference value is used as a type of motion compensation block used for interframe predictive encoding of image data of the encoding target block.   The image processing apparatus according to claim 6, wherein the determination unit reduces the image data and the reference image data of the encoding target block at least in a vertical direction to ¼ in the first determination process.   8. The image processing according to claim 6, wherein the determination unit reduces the image data and the reference image data of the block to be encoded to at least a quarter in the horizontal direction in the first determination process. apparatus.   An image processing apparatus that encodes moving image data by intraframe prediction encoding or interframe prediction encoding,
  A determination step of determining a type and a motion vector of a motion compensation block used for interframe predictive encoding of image data of an encoding target block read from a frame memory storing a frame image of moving image data;
  The determining step is to determine a motion compensation block used for encoding image data of the encoding target block among a plurality of types of motion compensation blocks.
  Reducing the image data and reference image data of the encoding target block;
  A plurality of types of motion compensation blocks including the specific motion compensation block, including pixel difference values between the block corresponding to the motion compensation block and the reduced reference image data in the image data of the reduced encoding target block. And performing a first determination process using the motion compensation block having the smallest pixel difference value as the first motion compensation block,
  Without reducing the image data and reference image data of the encoding target block,
  A pixel difference value between the first motion compensation block and the reference image data in the image data of the encoding target block, and an occurrence probability that is probabilistically obtained by statistical processing in the image data of the encoding target block. A pixel difference value between the maximum motion compensation block and the reference image data is calculated, and the first motion compensation block and the motion compensation block having the maximum occurrence probability determined by the statistical processing are: An image processing method, wherein a second determination process is performed in which a smaller pixel difference value is used as a type of motion compensation block used for interframe predictive encoding of image data of the encoding target block.

Images