JP2012142702A - Image processing device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve an accuracy in motion vector detection.SOLUTION: It is determined whether or not an image which is an inspection target of a motion vector has an image including an edge. When it is determined that the image has an edge, a processing for giving preferential treatment to a predicted motion vector (PMV) is performed. When it is determined that the image does not have an edge, the processing for giving preferential treatment to the predicted motion vector (PMV) is not performed, but a usual processing is performed. For giving preferential treatment to the PMV, calculation where a cost function value of the PMV is a value smaller than the cost function value of other motion vectors is performed so that the PMV can be more easily selected than the other motion vectors. The present invention is applicable to an encoder of an AVC (Advanced Video Coding) system.

Description

  The present invention relates to an image processing apparatus, method, and program, and more particularly, to an image processing apparatus, method, and program that can prevent deterioration in image quality during encoding.

  In recent years, image information has been handled as digital data, and at that time, for the purpose of efficient transmission and storage of information, encoding is performed by orthogonal transform such as discrete cosine transform and motion compensation using redundancy unique to image information. An apparatus that employs a method to compress and code an image is becoming widespread. This encoding method includes, for example, MPEG (Moving Picture Experts Group).

  In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image coding system, and is a standard that covers both interlaced scanning images and progressive scanning images, as well as standard resolution images and high-definition images. For example, MPEG2 is currently widely used in a wide range of applications for professional and consumer applications. MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the image coding system, the standard was approved as an international standard as ISO / IEC 14496-2 in December 1998.

  In addition H. There is a standard called 26L (ITU-T Q6 / 16 VCEG). H. 26L is known to realize higher encoding efficiency, although a larger amount of computation is required for encoding and decoding than encoding methods such as MPEG2 and MPEG4. Later, as part of MPEG4 activities, Based on 26L, H. Standardization to achieve higher coding efficiency by incorporating functions not supported by 26L is being carried out as Joint Model of Enhanced-Compression Video Coding. In March 2003, H.C. H.264 and MPEG-4 Part10 (Advanced Video Coding, hereinafter referred to as H.264 / AVC).

H.264 / AVC Software Coordination [online] [December 9, 2010] Search Internet <URL: http://iphome.hhi.de/suehring/tml/>

  As a result of encoding, image quality may deteriorate. FIG. 1 shows an example of image quality deterioration. FIG. 1 is a diagram in which motion vectors detected at the time of encoding are visualized and superimposed on an image using predetermined analysis software. In FIG. 1, a high-rise building is projected, and a motion vector is visualized and superimposed on one image obtained when the high-rise building is photographed by tilting upward from below. In FIG. 1, white dots (lines) indicate motion vectors (however, directions are not shown in FIG. 1). A motion vector is detected as a vector that moves by the same amount in the same direction on an image that is taken by tilting from bottom to top. Therefore, when such a motion vector is visualized, it becomes a line (point) of the same length.

  However, in FIG. 1, a motion vector detected in a direction and length different from those of other motion vectors is detected in a portion surrounded by an ellipse 11, and therefore a line having a length different from that of the other portion exists. is doing. A further enlarged view of a part of the ellipse 11 in FIG. 1 is shown in FIG. Among the motion vectors shown in FIG. 2, for example, the motion vector 21 and the motion vector 22 have different lengths. Many vectors having the same length as the motion vector 22 exist in FIG. 1, and it can be determined that the motion vector 22 is a correctly detected vector. Therefore, it can be determined that the motion vector 21 is an erroneously detected vector. Such erroneous detection of motion vectors causes deterioration of image quality, and it is desired to be improved.

  FIG. 3 shows another example. FIG. 3 is a diagram in which a motion vector is visualized and superimposed on one image of a video in which a camera is fixed along a roadway and a traffic image is taken. Similar to FIG. 1, in FIG. 3, white dots (lines) indicate motion vectors. In FIG. 3, the image is taken with a fixed camera. Therefore, since the roadway is fixed and an image in which the vehicle is moving is captured, a motion vector is not detected from the roadway, and a motion vector is detected from the vehicle.

  However, referring to FIG. 3, there is also a line indicating that a motion vector has been detected on the roadway near the car in the ellipse 31. Referring to FIG. 3, it can be read that the motion vector is detected not only in the ellipse 31 but also in other parts on a stationary object near the moving object. As described above, a motion vector that is not originally detected may be detected in the vicinity of a moving object, and such erroneous detection leads to deterioration of image quality and is improved as in the case described above. It is hoped that.

  The present invention has been made in view of such a situation, and is intended to prevent image quality from being deteriorated by encoding.

  An image processing apparatus according to an aspect of the present invention includes: a determination unit that determines whether or not a continuous edge is detected in an image; and when the determination unit determines that a continuous edge is detected, the predicted motion vector is And a preferential treatment unit that performs preferential treatment over other motion vectors.

  The determination unit may use mode information in the chroma intra mode.

  The determination unit may determine that consecutive edges have been detected when the chroma intra mode is the horizontal prediction mode or the vertical prediction mode.

  The preferential treatment unit may subtract a predetermined value from the cost function value of the predicted motion vector.

  An image processing method according to one aspect of the present invention is an image processing method of an image processing apparatus including a determination unit and a preferential unit, wherein the determination unit determines whether a continuous edge is detected in an image, and the preferential unit The unit includes a step of processing so that the predicted motion vector is favored over other motion vectors when it is determined by the determination unit that a continuous edge is detected.

  A program according to one aspect of the present invention is directed to a computer that controls an image processing apparatus including a determination unit and a preferential unit, wherein the determination unit determines whether a continuous edge is detected in the image, and the preferential unit includes: When the determination unit determines that consecutive edges have been detected, the program is a program for executing a process including a step of processing so that a predicted motion vector is preferentially given over other motion vectors.

  In the image processing apparatus and method, and the program according to one aspect of the present invention, when a continuous edge is detected in an image, the predicted motion vector is processed in preference to other motion vectors.

  According to one aspect of the present invention, it is possible to prevent image quality from being deteriorated by encoding.

It is a figure for demonstrating the motion vector detected. It is a figure for demonstrating the motion vector detected. It is a figure for demonstrating the motion vector detected. It is a figure which shows the structure of one Embodiment of an encoder. It is a figure for demonstrating an interlace. It is a figure for demonstrating a misdetection. It is a figure for demonstrating the motion vector detected. It is a flowchart for demonstrating operation | movement of a motion estimation apparatus. It is a figure for demonstrating the mode of an intra chroma. It is a figure for demonstrating a recording medium.

  Embodiments of the present invention will be described below with reference to the drawings.

[Example of encoder configuration]
FIG. 4 is a block diagram illustrating a configuration example of an embodiment of an AVC (Advanced Video Coding) encoder. 4 includes an A / D conversion device 101, a screen rearrangement buffer 102, an arithmetic device 103, an orthogonal transformation device 104, a quantization device 105, a lossless encoding device 106, a storage buffer 107, an inverse quantization device 108, an inverse quantization device. The apparatus includes an orthogonal transformation device 109, an addition device 110, a deblock filter 111, a frame memory 112, a motion compensation device 113, an intra prediction device 114, a rate control device 115, a motion prediction device 116, and a selection device 117. The encoder of FIG. 4 compresses and encodes the input image by the AVC method.

  Specifically, the A / D conversion apparatus 101 of the encoder performs A / D conversion on an image in frame units input as an input signal, and outputs to the screen rearrangement buffer 102 for storage. The screen rearrangement buffer 102 rearranges the stored frame images in the display order in the order for encoding according to the GOP (Group of Picture) structure.

  The arithmetic device 103 subtracts the predicted image supplied from the selection device 117 from the image read from the screen rearrangement buffer 102 as necessary. The arithmetic device 103 outputs an image obtained as a result of the subtraction to the orthogonal transform device 104 as residual information. When the prediction image is not supplied from the selection device 117, the arithmetic device 103 outputs the image read from the screen rearrangement buffer 102 as it is to the orthogonal transform device 104 as residual information.

  The orthogonal transform device 104 performs orthogonal transform processing on the residual information from the arithmetic device 103 according to the block size, and supplies coefficients obtained as a result of the orthogonal transform processing to the quantization device 105. The quantization device 105 quantizes the coefficient supplied from the orthogonal transformation device 104. The quantized coefficient is input to the lossless encoding device 106. The lossless encoding apparatus 106 acquires information indicating the optimal intra prediction mode (hereinafter referred to as intra prediction mode information) from the intra prediction apparatus 114, information indicating the optimal inter prediction mode (hereinafter referred to as inter prediction mode information), motion Vector information and the like are acquired from the motion prediction device 116.

  The lossless encoding device 106 performs variable length coding (for example, CAVLC (Context-Adaptive Variable Length Coding)), arithmetic coding (for example, CABAC) on the quantized coefficients supplied from the quantization device 105. (Context-Adaptive Binary Arithmetic Coding, etc.) is performed, and the resulting information is used as a compressed image. Further, the lossless encoding apparatus 106 performs lossless encoding on intra prediction mode information, inter prediction mode information, motion vector information, and the like, and uses the resulting information as header information added to the compressed image. The lossless encoding device 106 supplies the compressed image to which the header information obtained as a result of the lossless encoding is added to the accumulation buffer 107 as image compression information, and accumulates it.

  The accumulation buffer 107 temporarily stores the compressed image information supplied from the lossless encoding device 106 and outputs the information to, for example, a recording device or a transmission path (not shown) in the subsequent stage. The quantized coefficient output from the quantizing device 105 is also input to the inverse quantizing device 108, and after being inverse quantized, is supplied to the inverse orthogonal transform device 109.

  The inverse orthogonal transform device 109 performs an inverse orthogonal transform process according to the block size on the coefficient supplied from the inverse quantization device 108, and supplies residual information obtained as a result of the inverse orthogonal transform process to the adder device 110. To do. The adder 110 adds the residual information supplied from the inverse orthogonal transform device 109 to the predicted image supplied from the intra prediction device 114 or the motion compensation device 113 as necessary, and locally decoded image. Get. The adding device 110 supplies the obtained image to the deblocking filter 111 and also supplies it to the intra prediction device 114 as a reference image.

  The deblocking filter 111 removes block distortion by filtering the locally decoded image supplied from the adding device 110. The deblocking filter 111 supplies the image obtained as a result to the frame memory 112 and accumulates it. The image stored in the frame memory 112 is output to the motion compensation device 113 and the motion prediction device 116 as a reference image.

  The motion compensation device 113 performs a compensation process on the reference image supplied from the frame memory 112 based on the motion vector supplied from the motion prediction device 116 and the inter prediction mode information, and generates a prediction image. The motion compensation device 113 supplies the cost function value (details will be described later) supplied from the motion prediction device 116 and the generated predicted image to the selection device 117.

  Note that the cost function value is also called an RD (Rate Distortion) cost function value.For example, either the High Complexity mode or the Low Complexity mode as defined by JM (Joint Model), which is reference software in the AVC method, is used. It is calculated based on this method.

Specifically, when the High Complexity mode is employed as the cost function value calculation method, all the prediction modes that are candidates are subjected to lossless encoding, and are expressed by the following equation (1). A cost function value is calculated for each prediction mode.
Cost = D + λ × R (1)
D is a difference (distortion) between the original image and the decoded image, R is a generated code amount including up to the coefficient of orthogonal transform, and λ is a Lagrange multiplier given as a function of the quantization parameter QP.

On the other hand, when the Low Complexity mode is adopted as a cost function value calculation method, a decoded image is generated and header bits such as information indicating the prediction mode are calculated for all candidate prediction modes. A cost function value function expressed by the following equation (2) is calculated for each prediction mode.
Cost = D + QPtoQuant (QP) × Header_Bit (2)
D is a difference (distortion) between the original image and the decoded image, QPtoQuant is a function (Q value) given as a function of the quantization parameter QP, and Header_Bit is a header bit for the prediction mode.

  In the Low Complexity mode, it is only necessary to generate a decoded image for all the prediction modes, and it is not necessary to perform lossless encoding. Here, the Law Complexity mode is adopted as a cost function value calculation method.

  The intra prediction device 114, based on the image read from the screen rearrangement buffer 102 and the reference image supplied from the addition device 110, all candidate blocks in block units of all candidate block sizes. Intra prediction processing in the intra prediction mode is performed to generate a predicted image.

  Further, the intra prediction apparatus 114 calculates cost function values for all candidate intra prediction modes and block sizes. Then, the intra prediction device 114 determines the combination of the intra prediction mode and the block size that minimizes the cost function value as the optimal intra prediction mode. The intra prediction device 114 supplies the prediction image generated in the optimal intra prediction mode and the corresponding cost function value to the selection device 117. The intra prediction device 114 supplies the intra prediction mode information to the lossless encoding device 106 when the selection device 117 is notified of the selection of the predicted image generated in the optimal intra prediction mode.

  The motion prediction device 116 performs motion prediction in all candidate inter prediction modes based on the image supplied from the screen rearrangement buffer 102 and the reference image supplied from the frame memory 112, and generates a motion vector. . At this time, the motion prediction device 116 calculates cost function values for all candidate inter prediction modes, and determines the inter prediction mode that minimizes the cost function value as the optimal inter prediction mode. Then, the motion prediction device 116 supplies the inter prediction mode information, the corresponding motion vector, and the cost function value to the motion compensation device 113. When the selection device 117 is notified of selection of a predicted image generated in the optimal inter prediction mode, the motion prediction device 116 outputs inter prediction mode information, information on a corresponding motion vector, and the like to the lossless encoding device 106.

  The selection device 117 determines one of the optimal intra prediction mode and the optimal inter prediction mode as the optimal prediction mode based on the cost function values supplied from the intra prediction device 114 and the motion compensation device 113. Then, the selection device 117 supplies the prediction image in the optimal prediction mode to the arithmetic device 103 and the addition device 110. In addition, the selection device 117 notifies the intra prediction device 114 or the motion prediction device 116 of selection of the prediction image in the optimal prediction mode.

  The rate control device 115 controls the quantization operation rate of the quantization device 105 based on the image compression information stored in the storage buffer 107 so that overflow or underflow does not occur.

[About motion vector generation]
Next, generation of motion vectors performed by the motion prediction device 116 will be described. Reference is now made again to FIGS. With reference to FIGS. 1 to 3, it has been described that a motion vector may be erroneously detected. One cause of erroneous detection will be described with reference to FIG. In FIG. 5, original images 201 to 204 indicate the original images. The next image after the original image 201 is the original image 202, the next image after the original image 202 is the original image 203, and the next image after the original image 203 is the original image 204. In the original images 201 to 204, diagonal lines are respectively projected, and the diagonal lines move upward from the original image 201 to the original image 204 with time.

  The original images 211 to 214 are pseudo-illustrated scanning lines when the original images 201 to 204 are converted to interlace. A top field 221 is generated from the original image 211, and a bottom field 222 is generated from the original image 212. Similarly, a top field 223 is generated from the original image 213, and a bottom field 224 is generated from the original image 214. In this way, one field is generated from one original picture. Therefore, in the case of an oblique line as shown in FIG. 5, for example, referring to the top field 221, a part of the oblique line is included in the bottom field 222. A part of is lost.

  When a motion vector is detected by comparing such fields with missing information, for example, the top field 221 and the top field 223 or the bottom field 222 and the bottom field 224, there is a possibility of erroneous detection. A case where a motion vector is detected using the top field 221 and the top field 223 will be further described with reference to FIG.

  FIG. 6A is an enlarged view of a part assuming a state in which the original image 201 and the original image 203 are superimposed and displayed. An oblique line L1 represents an oblique line in the original image 201, and an oblique line L2 represents an oblique line in the original image 203. From the original image 201 to the original image 203, it means that the oblique line L1 moves upward and reaches the position of the oblique line L2. Since such an oblique line has only moved upward, the motion vector MV1 in the upward direction should be detected as shown in FIG. 6A. In other words, it is detected that the point P1 on the oblique line L1 has moved to the point P2 on the oblique line L2, and the motion vector MV1 is detected.

  However, as shown in FIG. 6B, when a motion vector is detected from the top field 221 and the top field 223, there is a possibility that an incorrect motion vector MV2 is detected. FIG. 6B is a partially enlarged view assuming a state in which the top field 221 and the top field 223 are displayed in an overlapping manner. The oblique line L1 in the top field 221 and the oblique line L2 in the top field 223 each include only the portion shown in FIG. 6B.

  Although the diagonal line L1 ′, which is a part of the diagonal line L1, originally moved to a part of the diagonal line L2 included in the bottom field 222, the top field 223 does not include that part, so the motion vector Is detected, an oblique line L2 ′ which is a part of the oblique line L2 included in the top field 223 is used. Therefore, when a motion vector is detected using the diagonal line L1 ′ and the diagonal line L2 ′, it is determined that the point P1 ′ on the diagonal line L1 ′ has moved to the point P2 ′ on the diagonal line L2 ′. The motion vector MV2 may be detected.

  This is considered to be a cause of erroneous detection of motion vectors. The cause given here is one factor and there are other factors. In the above description, the oblique line is taken as an example. However, the detection is not limited to the oblique line, and erroneous detection may occur even in the case of a vertical line or a horizontal line. If erroneous detection is performed regardless of the cause, as described with reference to FIGS. 1 to 3, there is a motion vector that is erroneously detected among the motion vectors that are correctly detected, and therefore the image quality is low. There is a possibility of deterioration.

  Therefore, such erroneous detection is prevented from being performed. A result as shown in FIG. 7 is obtained by the motion prediction device 116 performing processing as described later. FIG. 7 is a diagram further enlarging a part of the ellipse 11 in FIG. 1, and is a diagram of the same location as FIG. 2, but is a diagram in the case of calculating a motion vector by a process different from that in FIG. This will be described in comparison with FIG. In FIG. 2, it can be read from FIG. 2 that the motion vector 21 and the motion vector 22 are motion vectors of different sizes. On the other hand, it can be read from FIG. 7 that the motion vector 21 ′ and the motion vector 22 ′ in FIG. 7 corresponding to the motion vector 21 and the motion vector 22 are the same magnitude.

  In this way, the number of motion vectors that are erroneously detected can be reduced by executing the processing described later. Therefore, it is possible to prevent the image quality from being deteriorated by encoding.

[About processing of motion prediction device 116]
Next, processing of the motion prediction device 116 will be described. FIG. 8 is a flowchart for explaining processing performed by the motion prediction apparatus 116. The motion prediction device 116 performs motion prediction in all candidate inter prediction modes based on the image supplied from the screen rearrangement buffer 102 and the reference image supplied from the frame memory 112, and generates a motion vector. To do.

In step S11, SAD is calculated. SAD is an abbreviation for Sum of Absolute Difference, and is translated as an absolute difference. The SAD is calculated, for example, when block matching processing is performed. Reference image M ₁ is divided into a plurality of reference blocks, for the reference block, the block matching is performed. That is, the position where the reference block is displaced in the comparison image _Mn is searched (searched), and the amount of displacement is detected as a motion vector for the reference block.

In the comparison image M _n , a search range for searching for a reference block is set to a range based on the position of the reference block. Then, within the search range of the comparative image _Mn , a reference block position most similar to the reference block is searched while a reference block having the same size as the reference block is shifted in units of pixels. That is, the correlation value between the standard block and the reference block is used as an evaluation value for evaluating the degree of similarity between the standard block and the reference block, and the correlation value is obtained for the reference block at each position in the search range.

  As the correlation value, for example, SAD is used for all the pixels in the reference block and the reference block, which are absolute values of the difference between the luminance value of the pixel in the reference block and the luminance value of the pixel at the same position in the reference block. In step S11, a coordinate (relative coordinate) that minimizes the correlation value SAD is detected as a motion vector of the reference block. The SAD that is the minimum is also used in the following equation.

In step S12, Cost (hereinafter referred to as a cost function value) is calculated. The calculation of the cost function value is performed based on either the High Complexity mode or the Low Complexity mode as defined by JM (Joint Model) which is reference software in the AVC method as described above. Here, since the Law Complexity mode is adopted as the cost function value calculation method, the cost function value is calculated by Expression (2). Here, Formula (2) is described again.
Cost = D + QPtoQuant (QP) × Header_Bit (2)

D is the difference (distortion) between the original image and the decoded image, and here, the SAD calculated in step S11 is used. QPtoQuant is a function (Q value) given as a function of the quantization parameter QP, and Header_Bit is a header bit for the prediction mode. When D is rewritten by SAD, the following equation (2) ′ is obtained. Further, a value calculated by the following equation (2) ′ is described as a normal cost function value.
Cost = SAD + QPtoQuant (QP) × Header_Bit (2) '

  In step S13, it is determined whether an edge is detected. If it is determined in step S13 that no edge has been detected, the process proceeds to step S14. If it is determined that an edge has been detected, the process proceeds to step S15. In step S14, PMV (predicted motion vector) is set not to be preferential, and the cost function value of PMV is calculated. The cost function value calculated in step S14 is described as a normal cost function value.

  When the PMV is a target macroblock (current macroblock) for which the prediction vector PMV is calculated, the encoded macroblocks A, A, B, which are positioned at the left side, top left, top, and top right of the current macroblock, respectively. Using the B and C motion vectors MVa, MVb, MVc, and MVd, the prediction vector PMV of the current macroblock is calculated. That is, the PMV is calculated by obtaining the median (median value) of the motion vectors MVa, MVb, MVc, and MVd of the encoded macroblocks A, B, C, and D.

On the other hand, if it is determined in step S13 that an edge is detected, the PMV cost function value is calculated if PMV is preferentially set. When PMV preferential treatment is performed, a cost function value is calculated based on the following equation (4).
Cost_PMV = SAD + QPtoQuant (QP) × Header_Bit-PMVoffset (4)

  Expression (4) is an expression for calculating the cost function value by subtracting PMVoffset from the normal cost function value obtained by Expression (2) ′. The value calculated by the equation (4) is described as a PMV preferential cost function value. As will be described later, the cost function value having the smallest value among the normal cost function value and the PMV preferential cost function value is selected. PMV preferential means to subtract the offset value (PMVoffset) so that the PMV preferential cost function value calculated by the equation (4) becomes small so that it can be selected more easily than other cost function values. means.

  When PMV preferential treatment is performed, the PMV preferential processing is executed by subtracting a predetermined value (PMVoffset) from the PMV calculated from the motion vectors detected from the surrounding blocks. The reason for giving PMV preferential treatment when an edge is detected will be described later, and the description of the flowchart of FIG. 8 will be continued first. If PMV preferential treatment is not performed in step S14, or if PMV preferential treatment is performed in step S15, the process proceeds to step S16. In step S16, it is determined whether cost function values have been calculated in all modes. Modes include 16x16, 16x8, 8x16, 8x8, 4x8, 8x4, 4x4, and PMV.

  Note that the processes in steps S13 to S15 may be performed only in the PMV mode because they are related to the PMV mode. For example, a process for determining whether or not the PMV mode is set is provided between step S12 and step S13. When the PMV mode is set, the process proceeds to step S13. When the PMV mode is not set, the process proceeds to step S16. The processing flow may be such that the process proceeds.

  If it is determined in step S16 that cost function values have not been calculated for all modes, the process returns to step S11, and the process proceeds to a mode that has not yet been processed, and the processes in and after step S11 are repeated. It is. On the other hand, if it is determined in step S16 that cost function values have been calculated for all modes, the process proceeds to step S17. In step S17, the cost function value having the smallest value is selected from the calculated normal cost function value and the PMV preferential cost function value, and the mode when the cost function value having the smallest value is calculated is obtained. In step S18, the MV (motion vector) in the determined mode is set as the final motion vector.

  As described above, in the present embodiment, it is determined whether or not a predetermined situation is satisfied, and in the above-described embodiment, it is determined whether or not the situation that an edge is detected in the image is satisfied. When it is determined that the above condition is satisfied, the process is executed such that the predicted motion vector (PMV) is preferentially given over other motion vectors (MV). By executing such processing, the accuracy of the detected motion vector can be improved.

[PMV preferential treatment]
Here, the reason why the PMV mode is preferentially selected when an edge is extracted in step S13 will be described. As described above, the PMV is calculated using the motion vectors of the encoded macroblocks that are located on the left side, top left, top, and top right of the current current macroblock. That is, since PMV is a motion vector obtained from an adjacent block, it is a vector that is easily affected by the adjacent block. Therefore, the PMV is preferably used when it is preferable to align the motion vectors of adjacent blocks.

  For example, in the case of a video as described with reference to FIG. 1 (FIG. 2), by using PMV, it can be aligned with other adjacent motion vectors. For example, the motion vector 21 shown in FIG. The motion vector 22 can be aligned. Since the motion vector 21 and the motion vector 22 are aligned, as a result, as shown in FIG. 7, it is possible to make a state in which there is no erroneously detected vector.

  On the other hand, since PMV is affected by surrounding motion vectors, if used in a situation where it is not desirable to align with surrounding motion vectors, there is a high possibility of erroneous detection of motion vectors. For example, as described with reference to FIG. 3, the motion vector of the roadway is aligned with the motion vector detected by the movement of the vehicle in an image obtained by shooting a car traveling on the roadway with a fixed camera. In spite of the fact that it is preferable that no motion vector is originally detected from the roadway, a false detection that a motion vector is detected also on the roadway will be made. In such a case, it is preferable not to use PMV.

  Thus, there are situations where it is preferable to use PMV and situations where it is not preferred. In the situation as described with reference to FIG. 1, it is preferable to use PMV. In the situation as described with reference to FIG. 3, it is preferable not to use PMV. When it is preferable to use PMV, PMV is preferentially controlled so that PMV can be selected easily. When it is preferable not to use PMV, it is difficult to select PMV without preferentially using PMV. To control. As described above, in the processing in the flowchart of FIG. 8, in step S14, control is performed so that PMV preferential treatment is not performed, and in step S15, control is performed so that PMV preferential treatment is performed.

  Even when PMV is not favored, PMV may be selected as a result, and when PMV is selected, it is determined that it is more appropriate than other motion vectors. There will be no problems. Similarly, even when PMV is given preferential treatment, PMV may not be selected as a result. When PMV is not selected, it is determined that other motion vectors are appropriate. So there is no problem.

  Further, in the present embodiment, when a predetermined condition is satisfied, the PMV is merely preferentially selected so that the PMV is easily selected. When the predetermined condition is satisfied, control is not performed so as to select the PMV. Therefore, even when the predetermined condition is satisfied, when the PMV is not the optimal motion vector, it is possible to control so that the PMV is not selected. By being able to perform such control, it is possible to detect an appropriate motion vector and reduce false detection.

  Moreover, when PMV is preferentially treated as described above, a PMV preferential cost function value is calculated based on Equation (4). In the equation (4), when PMV is preferably selected by appropriately setting PMVoffset (predetermined value) subtracted from the normal cost function value, PMV is selected over other motion vectors. It is possible to control so as to be easily performed. In other words, it is possible to improve the accuracy of the detected motion vector by appropriately setting the value of PMVoffset. PMVoffset may be a fixed value or a variable value. When a variable value is used, the value can be varied depending on a predetermined situation.

  Although it has been described that PMV is favored when a predetermined situation is satisfied, the predetermined situation is whether or not an edge is detected in the above-described embodiment. As described above, an image (video) in which PMV is favored and PMV is preferably selected is, for example, a video as shown in FIG. Referring to FIG. 1 again, a portion that is likely to be erroneously detected is a portion where a straight line exists. The straight line may be erroneously detected in both the vertical direction and the horizontal direction, and may be erroneously detected even in a lattice shape. The reason for this is as explained with reference to FIGS. 5 and 6.

  That is, a motion vector is likely to be erroneously detected at a location where an edge of a geometric pattern is likely to occur. In such a part, the vector is likely to be violated. For example, if it is a diagonal line in the vertical direction during panning, the vector tends to be violated in the vertical direction. In addition, for example, if it is an oblique line in the horizontal direction at the time of tilt, the vector tends to be violated in the left-right direction. The tendency to be violent is a situation in which a motion vector is likely to be erroneously detected, and means that there is a motion vector detected in a direction or size different from that of a correctly detected motion vector.

  Therefore, PMV is preferentially given to an image (image) having a geometric pattern edge. The edge of the geometric pattern is, for example, a line. In the line, a boundary between a part that is a line and a part that is not a line exists as an edge. It can be said that the line is a state where the edges are continuous. When such a continuous edge is detected, it is determined that the vector tends to be violated, and the process described with reference to the flowchart of FIG.

  That is, in the flowchart of FIG. 8, it is determined in step S13 whether an edge is detected (preferably, whether a continuous edge is detected). If it is determined that an edge is detected, in step S15 PMV is preferential. Thus, in the present embodiment, an example has been described in which it is determined whether or not a motion vector tends to be violated by determining whether or not an edge has been detected. The determination of whether or not the motion vector tends to be violent (whether or not there is a possibility of erroneous detection) is not a description indicating that the edge is detected, but other determinations. It may be made to be performed.

  A further description will be given regarding the determination of whether or not an edge has been detected. To determine whether an edge has been detected, information on the mode of the Chroma Intora mode, which is one of the AVC Tools, is used. The AVC Chroma Intora mode has four modes shown in FIG. FIG. 9 is a diagram illustrating an intra prediction mode (Intra_chroma_pred_mode) of a color difference signal. Intra_chroma_pred_mode has four modes, mode 0 to mode 3, as shown in FIG.

  Intra prediction of color difference signals is performed in units of macroblock size. The prediction direction is performed by referring to the pixel values of the adjacent left, upper, and upper left blocks that have been encoded, and the four prediction modes shown in FIG. 9 are defined. Mode 0 (Mode 0) is DC Prediction (average value prediction mode), Mode 1 (Mode 1) is Horizontal Prediction (horizontal prediction mode), and Mode 2 (Mode 2) is Vertical Prediction (vertical prediction). Mode 3 and Mode 3 (Mode 3) is a Plane Prediction (plane prediction mode).

  These modes are modes for macroblock prediction and are not defined as modes for detecting the presence or absence of an edge. However, in the present embodiment, when an edge is detected in step S13 (FIG. 8), it is determined whether an edge is detected by determining which mode of these modes. To. Specifically, in the horizontal prediction mode of Mode 1 (Mode 1) or the vertical prediction mode of Mode 2 (Mode 2), it is determined that an edge has been detected, and PMV is preferentially treated.

  In this case, the process in step S13 is a process for determining whether the macroblock prediction mode is mode 1 or mode 2. And when it is judged that it is mode 1 or mode 2, a process is advanced to step S14 and the process for giving preferential treatment to PMV is performed.

  In this way, by using the mode of macroblock prediction, it is determined whether or not it is an edge, in other words, using the mode of macroblock prediction and whether or not the vector is in a state of rampage. The motion vector detection performance can be improved without adding new hardware or software for detecting an edge. Further, it is possible to improve the motion vector detection performance while suppressing an increase in circuit scale and processing.

  In addition to the above-described embodiment, it is possible to narrow down a specific place by adding GlobalMV indicating the motion vector of the entire screen and the past average InterSAD value to the detection condition. Further, it is possible to further improve by lowering the Q value of the detected block.

  If the motion vector deviates significantly, even if the bit rate is increased, the image quality is noticeable. However, according to the present embodiment, it is possible to prevent the motion vector from deviating greatly. Therefore, the motion vector performance can be improved, and an improvement in image quality can be expected.

[About recording media]
The series of processes described above can be executed by hardware or software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.

  FIG. 10 is a block diagram illustrating an example of a hardware configuration of a computer that executes the above-described series of processing by a program. In a computer, a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, and a RAM (Random Access Memory) 303 are connected to each other by a bus 304. An input / output interface 305 is further connected to the bus 304. An input unit 306, an output unit 307, a storage unit 308, a communication unit 309, and a drive 310 are connected to the input / output interface 305.

  The input unit 306 includes a keyboard, a mouse, a microphone, and the like. The output unit 307 includes a display, a speaker, and the like. The storage unit 308 includes a hard disk, a nonvolatile memory, and the like. The communication unit 309 includes a network interface and the like. The drive 310 drives a removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, the CPU 301 loads the program stored in the storage unit 308 to the RAM 303 via the input / output interface 305 and the bus 304 and executes the program, for example. Is performed.

  The program executed by the computer (CPU 301) can be provided by being recorded on a removable medium 311 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the storage unit 308 via the input / output interface 305 by attaching the removable medium 311 to the drive 310. Further, the program can be received by the communication unit 309 via a wired or wireless transmission medium and installed in the storage unit 308. In addition, the program can be installed in the ROM 302 or the storage unit 308 in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  101 A / D conversion device, 102 screen rearrangement buffer, 103 arithmetic device, 104 orthogonal transform device, 105 quantization device, 106 lossless encoding device, 107 accumulation buffer, 108 inverse quantization device, 109 inverse orthogonal transform device, 110 Adder, 111 deblock filter, 112 frame memory, 113 motion compensation device, 114 intra prediction device, 115 rate control device, 116 motion prediction device, 117 selection device

Claims (6)

A determination unit for determining whether or not a continuous edge is detected in the image;
An image processing apparatus comprising: a preferential unit that performs processing so that a predicted motion vector is preferentially given over other motion vectors when it is determined that a continuous edge is detected by the determination unit. The image processing apparatus according to claim 1, wherein the determination unit uses mode information in a chroma intra mode. The image processing apparatus according to claim 1, wherein the determination unit determines that a continuous edge is detected when the chroma intra mode is a horizontal prediction mode or a vertical prediction mode. The image processing apparatus according to claim 1, wherein the preferential unit subtracts a predetermined value from a cost function value of the predicted motion vector. In an image processing method of an image processing apparatus including a determination unit and a preferential unit,
The determination unit determines whether a continuous edge is detected in the image,
The preferential processing unit includes a step of processing so that a predicted motion vector is preferentially given over other motion vectors when the determination unit determines that a continuous edge is detected. In a computer that controls an image processing apparatus including a determination unit and a preferential unit,
The determination unit determines whether a continuous edge is detected in the image,
The preferential treatment unit is a program for executing processing including a step of processing so that a predicted motion vector is preferentially preferentially over other motion vectors when it is determined that a continuous edge is detected by the determination unit.

Images