JP2014230031A - Image encoding device and image encoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a calculation amount required for encoding in encoding processing of an image.SOLUTION: An image encoding device comprises: an information amount reduction unit that generates a moving image with a reduced information amount from an input moving image; an encoding parameter estimation unit that encodes the moving image with the reduced information amount to estimate an encoding parameter to be used to encode the input moving image; and an encoding unit that encodes the input moving image on the basis of the estimated encoding parameter.

Description

  The present invention relates to an image encoding device and an image encoding program.

  Currently, transmission and storage of ultra-high-definition moving images are common. For example, the information amount of uncompressed HDTV exceeds 1 Gbp / sec. For this reason, a technique for efficiently compressing a high-definition moving image is required. MPEG2, MPEG4, H.264, and the like are standard coding techniques for moving image data. H.264 / AVC (Advanced Video Coding), H.264. H.265 / HEVC (High Efficiency Video Coding).

  H. H.265 / HEVC does not simply divide the screen from the upper left into blocks, which are coding units, as in the conventional coding technology, but newly adopts a hierarchical division structure (CTU: Coding Tree Unit). The block division of a plurality of hierarchies is possible. A coding unit divided into a plurality of blocks is also referred to as a CU (Coding Unit) or a coding block. H. In H.265 / HEVC, an image can be divided into 64 × 64, 32 × 32, 16 × 16, 8 × 8, and the like and encoded.

  H. In H.265 / HEVC, in order to efficiently express a prediction error signal, a CU can be divided into hierarchies and divided into conversion units. The transform unit is also referred to as a TU (Transform Unit) or a transform block.

  FIG. 1A shows a CU. FIG. 1B shows a TU. As shown in FIG. 1, for example, for a 32 × 32 CU, a TU with zero layer division is the same block as a 32 × 32 CU, and a TU with one layer division is a block divided into 16 × 16 It is. In addition, a TU with two hierarchical divisions is a block divided into 8 × 8.

  Since TUs are hierarchically divided according to the size of CUs, the TU sizes in each layer are different depending on the size of CUs.

  FIG. 1C shows a prediction unit. H. In H.265 / HEVC, a CU is divided into prediction units that are a plurality of types of rectangular areas, and prediction processing is performed in each prediction unit. This prediction unit is also referred to as a PU (Prediction Unit) or a prediction block. There are eight types of PU division shapes.

  Next, predictive coding is performed on the divided blocks. In-screen coding (intra coding) encodes an image by performing spatial signal prediction within the screen. This technique is also referred to as intra prediction (intra prediction). Also, inter-screen prediction (inter-coding) searches for similar regions between different frames and encodes an image for prediction. This technique is also referred to as motion compensation prediction (inter prediction).

  For example, H.M. In H.265 / HEVC, the above-described various division shapes are determined by a technique such as RD (Rate-Distortion) optimization.

  RD optimization is to calculate the RD (Rate-Distortion) cost, which is the relationship between bit rate and distortion, compare RD costs calculated with different encoding parameters, and determine the optimal encoding parameter for the target bit rate. This is an optimization method for obtaining a combination.

  A specific CU size and PU size selection method based on RD optimization will be described below.

  First, the encoding system divides the CU of the highest hierarchy into 8 types of PU shapes, calculates the RD cost in each case of intra prediction and inter prediction, and optimal combinations of PU shapes, prediction methods, and prediction modes Ask for.

  Subsequently, the encoding system divides the CU into four, calculates RD costs in the same manner for all four CUs in the lower layer, and obtains an optimal combination of the PU shape, the prediction method, and the prediction mode. The above processing is performed recursively, and the optimum combination of PU shape, prediction method, and prediction mode is obtained up to the deepest CU.

If the optimal combination can be obtained up to the deepest hierarchy, the RD costs of four CUs in the same hierarchy are summed, and the CU and RD cost of one higher hierarchy are compared. If the total RD cost of the lower layer is larger, the division is not performed, and the same processing is performed on the upper layer.
On the other hand, when the total RD cost of the lower layer is smaller, the corresponding CU is divided, and the process proceeds to the next CU. These processes are repeated up to the highest layer, and the division shape and transform size are calculated for all CUs in the CTU (see Non-Patent Document 1).

  As described above, when RD optimization is used to determine various block division shapes for all CUs in one CTU, although the encoding efficiency is high, RD cost calculation is required for all combinations of various encoding parameters. It becomes a huge amount of calculation. When the spatial resolution of the original image is high, processing with a single encoder may not be in time.

  For this reason, a solution is known in which an original image is spatially divided into a plurality of pieces, and an encoder is prepared for the number of divisions, and parallel coding processing is performed. For example, a technique for dividing an image into eight and performing parallel processing with eight encoders has been reported (see Non-Patent Document 2).

K.McCann, B.Bross, W.-J.Han, etc ... "High Efficiency Video Coding (HEVC) Test Model 9 (HM 9) Encoder Description", JCT-VC K1002, Shanghi, China, Oct. 2012 Y.Shishikui, K.Iguchi, S.Sakaida, etc… "High-Performance Video Codec for Super Hi-Vision", Proceedings of the IEEE, Vol. 101 Issue. 1, pp 130-139, 2013

  H. H.264 / AVC and H.264. H.265 / HEVC repeats hierarchical division from a block having a reference size. In this case, for example, RD optimization or the like is used to obtain encoding parameters such as a block size, a division shape, a prediction method, and a prediction mode. However, in the RD optimization, it is necessary to calculate the RD cost for all combinations of encoding parameters. As described above, this processing requires a huge amount of calculation. In particular, a long time calculation is required for an image having a high spatial resolution.

  In addition, when the screen is spatially divided in order to distribute the calculation and a plurality of encoding devices are used in parallel, as many encoders as the number of divisions of the original image are required. In addition, a complicated configuration such as synchronization processing of a plurality of encoders is required. Furthermore, since a plurality of encoders perform encoding processing separately in principle, image quality may deteriorate at the division boundary.

  Accordingly, an object of the present invention is to reduce the amount of calculation required for determining the encoding parameter in the image encoding process.

  An image encoding device according to an aspect of the present invention encodes an information amount reducing unit that generates a moving image with a reduced amount of information from an input moving image, and the moving image with the reduced amount of information The encoding parameter estimation unit for estimating the encoding parameter used for encoding the input moving image, and the encoding of the input moving image based on the estimated encoding parameter And an encoding unit.

  The encoding unit may correct a predetermined encoding parameter among the estimated encoding parameters within a predetermined range.

  Further, the encoding parameter estimation unit is obtained by encoding the moving image with the reduced information amount based on the relationship between the input moving image and the moving image with the reduced information amount. You may have a parameter conversion part which converts an encoding parameter into the said estimated encoding parameter.

  The information amount reduction unit includes a pixel number reduction unit that reduces the number of pixels of each of a plurality of frames included in the input moving image, a bit number reduction unit that reduces the number of bits of pixels in the frame, and It may include at least one of a frame number reduction unit that reduces the number of frames per unit time of the input moving image.

In addition, when the number of pixels of each of the plurality of frames included in the input moving image is reduced by the pixel number reduction unit, the encoding parameter estimation unit is configured to perform a predetermined frame of the input moving image. Among the estimated encoding parameters, the block size and the motion vector length may be calculated using the ratio between the number of pixels in and the number of pixels in the predetermined frame with the number of pixels reduced. Good.
When the number of frames per unit time of the input moving image is reduced by the frame number reduction unit, the encoding parameter estimation unit is obtained from a plurality of frames other than the reduced frame. A motion vector may be calculated among the estimated encoding parameters related to the reduced frame by interpolating the estimated encoding parameters.

  An image encoding program according to another aspect causes a computer to function as any one of the above image encoding apparatuses.

  In one aspect, the present invention can reduce the amount of calculation required for encoding in the process of encoding an image.

H. The figure which showed the division | segmentation of CU, TU, PU in H.265 / HEVC. The block diagram which shows the outline | summary of an Example. The figure which shows the example of an original image and a 1/2 reduction image. The block diagram which shows the detail of an Example. The flowchart which shows operation | movement of an Example. The block diagram which shows the example of a parameter conversion part. The flowchart which shows operation | movement of a parameter conversion part. The figure which shows the example of a slice.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. It should be noted that similar components may be given the same reference numerals in different drawings.

[Example]
In the embodiment, when the encoding unit 200 encodes the original image (moving image) 100 and outputs the bit stream 300, the amount of calculation required for encoding by the encoding unit 200 can be reduced. .

<Outline of configuration>
FIG. 2 is a block diagram showing an outline of the embodiment. The image encoding device 10 includes an encoding unit 200, an information amount reduction unit 400, and an encoding parameter estimation unit 500. An outline of each part will be described below.

  The encoding unit 200 encodes the original image 100 and outputs a bit stream 300. In this specification, as an encoding process of the encoding unit 200, H.264 is used. The encoding process will be described by taking the encoding conforming to H.265 / HEVC as an example, but the encoding process used by the encoding unit 200 is H.264. It is not limited to H.265 / HEVC.

  The information amount reduction unit 400 includes, for example, at least one of a pixel number reduction unit 410, a bit number reduction unit 420, and a frame number reduction unit 430. The information amount reduction unit 400 reduces the information amount of the original image 100, the image 412 in which the information amount is reduced, and the information 411 indicating what is used as the information amount reduction processing, and the coding parameter estimation unit 500. To give.

  The pixel number reduction unit 410 can generate a reduced image by reducing the number of pixels of the original image 100.

  FIG. 3 shows an example in which every other pixel of the original image in FIG. 3B is thinned out to form a 1/2 reduced image as shown in FIG. For example, by thinning out every other pixel, the size of the vertical and horizontal images is halved. The area ratio is 1/4 (the amount of information is also 1/4). The CU 310 of the original image 100 corresponds to the CU 320 in the 1/2 reduced image. In the original image 100, the length of 2N becomes the length of N in the 1/2 reduced image. Further, ¼ of the area of the CU 310 is the area of the CU 320. In FIG. 3, the divided shapes of CU and PU are shown. PU dividing lines are indicated by 321, 322, 311, and 312.

  Returning to FIG. The bit number reduction unit 420 has a function of reducing information held by each pixel of the original image 100. For example, when each pixel has a 10-bit information amount, the lower 2 bits are rounded down to 8 bits. By this processing, an image 412 with a reduced amount of information than the original image 100 is obtained.

  The frame number reduction unit 430 performs processing for reducing the number of frames of the original image (moving image) 100. For example, when the frame number reduction unit 430 reduces every other frame, the frame rate of the original image (moving image) 100 is reduced to ½. By this processing, the information amount per unit time of the original image (moving image) 100 can be reduced to ½.

  Note that the above description is an example, and the rate of reducing the amount of information is not limited to the above example.

  The encoding parameter estimation unit 500 includes a parameter conversion unit 511. The encoding parameter estimation unit 500 receives the image 412 with the information amount reduced from the information amount reduction unit 400. The encoding parameter estimation unit 500 performs processing for encoding the received image.

  Since the image processed by the encoding parameter estimation unit 500 has a smaller amount of information than the original image 100, the image can be encoded with a smaller amount of calculation than when the original image 100 is encoded. And the encoding parameter estimation part 500 can acquire the encoding parameter obtained in the encoding of an image.

Examples of the encoding parameters acquired by the encoding parameter estimation unit 500 include the following.
・ Block division shape (CU, TU, PU)
・ Block division hierarchy (CU, TU)
・ Block size (CU, TU, PU)
・ Prediction methods (for example, inter prediction, intra prediction)
-Prediction mode (motion vector value, prediction direction)
・ Quantization parameter (Qp)
The above encoding parameters are obtained based on the image 412 in which the information amount is reduced. However, since the original image 100 and the image 412 in which the information is reduced are very similar images, Using the encoding parameter obtained based on the image 412 whose amount has been reduced, the encoding parameter obtained when the original image 100 is encoded can be estimated with sufficient accuracy.

  Note that the original image 100 and the image 412 with the reduced amount of information may differ in image size, frame rate, and the like, and therefore, depending on the type of encoding parameter in consideration of the differences. It is desirable to appropriately convert the encoding parameter obtained based on the image 412 in which the information amount is reduced.

  The parameter conversion unit 511 performs a process of appropriately converting the encoding parameter obtained based on the image 412 in which the information amount is reduced according to the type of the encoding parameter. This converted encoding parameter is referred to as an estimated encoding parameter 600. The estimated encoding parameter 600 is given to the encoding unit 200.

  The encoding unit 200 encodes the original image 100 using the estimated encoding parameter 600 and outputs a bit stream 300.

The main conversion process of the parameter conversion unit 511 will be described below.
(1) When the number of pixels of the original image 100 is reduced and a reduced image having a short side of 1 / N is processed by the encoding parameter estimation unit.

  In this case, the encoding parameter can be estimated by multiplying the block division shape and the block size related to the block division hierarchy by N times. H. In a standard in which the block size, such as H.265 / HEVC, is specified discretely by a power of 2, it is desirable that the value of N is also a power of 2.

  Further, for example, when the above-mentioned size multiplied by N exceeds the upper limit size (for example, 64 × 64 in H.265 / HEVC) of the CU size, this upper limit size (for example, 64 × in H.265 / HEVC). 64) is the estimated CU size. When such a situation occurs, it is determined that the image of the corresponding CU in the original image (moving image) 100 is still or the prediction process is easy. For this reason, even if the above processing is executed and the original image is encoded using the block of the upper limit size, it can be determined that the encoding can be performed with the parameter with high accuracy.

  Note that the block size upper limit size in encoding of the encoding parameter estimation unit 500 is set to 1 / N of the standard upper limit size so that the block size multiplied by N as described above does not exceed the standard upper limit size. You may restrict | limit beforehand.

  As the prediction method, the same method as the prediction method of the encoding parameter estimation unit 500 can also be used in the encoding unit 200. For example, if the prediction method of the encoding parameter estimation unit 500 is intra prediction (for example, block a2 in FIG. 3), the prediction method of the encoding unit 200 may also use intra prediction (for example, block a1 in FIG. 3).

  The prediction mode is processed as follows. When the prediction method of the encoding parameter estimation unit 500 is intra prediction, the same direction may be used in the encoding unit 200 as the prediction direction as the prediction mode. When the prediction method of the encoding parameter estimation unit 500 is inter prediction, the encoding unit 200 is set in the same direction as the prediction vector obtained by the encoding parameter estimation unit 500, and the vector distance is N. What is necessary is just to convert to double.

As shown in FIG. 3A, the block division in FIG. 3B is performed by using the encoding unit 200 to determine the CU or PU division shape obtained by the encoding parameter estimation unit 500 when N = 2. In order to use, an example of the result converted by the parameter conversion unit 511 is shown.
(2) When the number of frames of the original image (moving image) 100 is reduced and a moving image having a frame rate of 1 / N is processed by the coding parameter estimation unit.

  In this case, the encoding parameter of the encoding parameter estimation unit 500 regarding the frame that has not been deleted can be used as it is in the encoding unit 200. For the deleted frame, since the encoding parameter estimation unit 500 has not acquired the encoding parameter, for example, from the encoding parameter obtained by the two time frames acquired in the past, The encoding parameters can be predicted by interpolation (extrapolation). For this reason, the encoding parameter estimation method based on the frame reduction is suitable for an input moving image with little image motion.

  A specific example in which every other frame is deleted, the frame rate of the original image (moving image) 100 is reduced to 1/2, and the encoding parameter estimation unit 500 estimates the encoding parameter is shown below. Then, frame numbers 3 and 5 out of frame numbers 2, 3, 4, and 5 are deleted, and the encoding parameters of frame numbers 2 and 4 have already been acquired by the encoding parameter estimation unit 500. Assume that the frame number is 5.

In this case, when the prediction method is inter prediction, for example, the block division of the immediately preceding frame 4 may be adopted as the block division of the frame 5. If the block division of the corresponding frame 2 and the block division of the frame 4 are the same in the block of interest in the frame 5, the motion vector of the frame number 2 of the block of interest is α2, and the motion vector of the frame number 4 is Assuming α4, the motion vector α5 of the block of interest in the deleted current processing frame 5 is
α5 = aα4-bα2 (where a and b are arbitrary coefficients corresponding to the time distance from the processing frame)
Can be estimated.

  If the block division is different between frames 2 and 4, the above interpolation (outside) is performed from the motion vectors of the blocks of frames 2 and 4 that are spatially close to the block of interest in frame 5. Insertion) may be performed. Alternatively, the motion vector of the corresponding block of the frame 4 may be employed as it is.

Similarly, when the prediction method is intra prediction, the same interpolation (extrapolation) is performed. In the standard, a predetermined pattern is defined for the prediction direction of intra prediction (34 directions in the case of H.265 / HEVC). For this reason, a pattern of a predetermined intra prediction direction that is closest to the direction of intra prediction obtained by extrapolation may be specified.
(3) When only the bit number reduction process is performed, the encoding parameter obtained by the encoding parameter estimation unit 500 can be used by the encoding unit 200 as it is. In the case of reducing the number of bits, an effect of reducing resources (memory) used for calculation can be expected.

  The specific processing of the parameter conversion unit 511 has been described above by dividing the case into (1) to (3). For coding parameters other than those described above, the estimated coding parameter 600 obtained by the coding parameter estimation unit 500 can be used as it is by the coding unit 200. In the information amount reduction unit 400, which method is used as the information amount reduction method is designated from the outside, or after each reduction method is tried, the encoding parameter is considered in consideration of the processing capacity of the entire system. You may make it select on the basis of the reduction effect of the processing amount in the estimation part 500 and the encoding part 200. FIG.

  Note that the image encoding device 10 in FIG. 2 can be configured as hardware. It can also be realized as a program for operating a computer.

  Since both the encoding parameter estimation unit 500 and the encoding unit 200 perform the same encoding process, both may be shared by the same hardware. In this case, in the first process, the image 412 with the reduced amount of information is processed to obtain an estimated coding parameter 600, and in the second process, the estimated coding parameter 600 is used to encode the original image. May also be performed. Even when the image coding apparatus 10 is realized by a computer and a program, one computer processes both the coding parameter estimation unit 500 and the coding unit 200 in series in time. Also good.

  In addition, when the encoding parameter estimation part 500 and the encoding part 200 are implement | achieved by separate hardware, both can perform a process like a pipeline.

<Detailed configuration>
FIG. 4 is a block diagram showing the image encoding device 10 shown in FIG. 2 in more detail. Elements having the same functions as those in FIG. 2 are denoted by the same reference numerals. Elements already described in FIG. 2 will be omitted in the following description. Also, the encoding parameter estimation unit 500 and the encoding unit 200 can use the same components except for the parameter conversion unit 511. For this reason, in the following description, the component of the encoding part 200 is demonstrated.

  The encoding control unit 201 receives the estimated encoding parameter 600 from the encoding parameter estimation unit 500 and uses it for encoding the original image 100. Alternatively, the encoding control unit 201 further performs RD optimization or the like for some parameters to obtain appropriate encoding parameters.

  The screen dividing unit 220 performs block division and outputs block division boundary information to the loop filter unit 290.

  The prediction error signal generation unit 225 obtains block data in which the encoding target image of the input moving image data is divided into blocks such as 32 × 32, 16 × 16, and 8 × 8 pixels, for example. The prediction error signal generation unit 225 generates a prediction error signal based on the block data and the output of the intra prediction unit 260 or the motion compensation prediction unit 270 obtained via the prediction method switching unit 210. The prediction error signal generation unit 225 outputs the generated prediction error signal to the transform / quantization unit 230.

  The transform / quantization unit 230 performs orthogonal transform processing on the input prediction error signal and quantizes it. The transform / quantization unit 230 reduces the code amount of the output signal by this processing. The transform / quantization unit 230 outputs this output signal to the entropy coding unit 240 and the inverse quantization / inverse transform unit 250. The transform / quantization unit 230 may output the Qp value of the quantization parameter to the loop filter unit 290.

  The entropy encoding unit 240 entropy-encodes and outputs the output signal from the transform / quantization unit 230, the motion vector information output from the motion detection unit 280, the filter coefficient from the loop filter unit 290, and the like. Entropy coding is a method of assigning variable-length codes according to the appearance frequency of symbols.

  The inverse quantization / inverse transform unit 250 dequantizes the output signal from the transform / quantization unit 230 and performs inverse orthogonal transform. By performing decoding processing by the transform / quantization unit 230, a signal having the same level as the prediction error signal before encoding is obtained.

  The decoded image generation unit 255 includes the block data of the image predicted in the screen by the intra prediction unit 260 or the image subjected to motion compensation by the motion compensation prediction unit 270, and the prediction error signal decoded by the inverse quantization / inverse conversion unit. And add. The decoded image generation unit 255 outputs the decoded image block data generated by the addition to the loop filter unit 290.

  The loop filter unit 290 is, for example, an ALF (Adaptive Loop Filter) or a deblocking filter. The loop filter unit 290 holds the filter processing result as a reference image.

  The loop filter unit 290 outputs the stored reference image to the intra prediction unit 260, the motion compensation prediction unit 270, and the motion detection unit 280.

  The intra prediction unit 260 generates block data of the predicted image from the already-encoded reference pixels for the processing target block of the encoding target image. The intra prediction unit 260 performs prediction using a plurality of prediction directions and determines an optimal prediction direction. As for the prediction direction, the difference value is output to the entropy encoding unit 240 in order to include the difference value with the prediction direction of the encoded block in the bitstream.

  The motion compensation prediction unit 270 performs motion compensation on the reference image data with the motion vector provided from the motion detection unit 280. Thereby, block data as a motion-compensated reference image is generated. For the motion vector, the difference value with the motion vector (predicted vector) of the encoded block is output to the entropy encoding unit 240 in order to include the difference value in the bitstream.

  The motion detection unit 280 obtains a motion vector using the block data in the encoding target image and the reference image.

  The motion detection unit 280 outputs the obtained motion vector to the motion compensation prediction unit 270, and outputs motion vector information including information indicating the reference image to the entropy encoding unit 240.

  The block data output from the intra prediction unit 260 and the motion compensation prediction unit 270 are input to the prediction method switching unit 210.

  The prediction method switching unit 210 selects one of the block data acquired from the intra prediction unit 260 and the motion compensation prediction unit 270 as a predicted image. The selected prediction image is output to the prediction error signal generation unit 225.

  The constituent elements included in the encoding parameter estimation unit 500 can be the same constituent elements as the constituent elements of the encoding unit 200 described above, except for the parameter conversion unit 511, and thus the description thereof is omitted.

  Note that the configurations of the encoding unit 200 and the encoding parameter estimation unit 500 illustrated in FIG. 4 are examples, and the configurations may be combined or the configurations may be changed as appropriate.

<Operation>
Next, the operation of the image encoding device 10 in the embodiment will be described.

  FIG. 5 is a flowchart illustrating an example of the operation of the image encoding device 10.

  In step 502, the information amount reduction unit 400 reduces the information amount of the input original image 100. The image 412 whose information amount has been reduced is given to the encoding parameter estimation unit 500. A specific operation for reducing the information amount of the original image 100 will be described with reference to FIG.

  In step 504, the encoding parameter estimation unit 500 encodes the image 412 in which the information amount is reduced, acquires the encoding parameter, and performs necessary conversion to estimate the encoding parameter. The estimated encoding parameter 600 is given to the encoding unit 200.

  In step 506, the encoding unit 200 encodes the original image 100 using the given estimated encoding parameter 600 and outputs a bitstream 300.

  FIG. 5B is a flowchart showing an operation for reducing the information amount of the original image 100.

  In step 591, the information amount reduction unit 400 checks whether there is a designation to reduce the number of pixels. If this check is “Yes” (if reduction of the number of pixels has been specified), the process proceeds to Step 592. If this check is “No”, the process proceeds to Step 593.

  In step 592, the information amount reduction unit 400 performs an operation of reducing the number of pixels of the original image 100. For example, when pixels are reduced every other pixel, an image obtained by reducing the original image 100 to ½ is obtained (the area ratio becomes ¼).

  Note that the reduction in the number of pixels may be performed by simply thinning out the pixels.

  In step 593, the information amount reduction unit 400 checks whether there is a designation to reduce the number of bits. If this check is “Yes” (if bit number reduction is specified), the process proceeds to step 594. If this check is “No”, the process proceeds to Step 595.

  In step 594, the information amount reduction unit 400 reduces the number of bits of pixel information of the original image 100. For example, the information amount of the original image 100 can be reduced by setting the number of bits of one pixel to 8 bits with respect to the original image 100 expressing one pixel with 10 bits. The number of bits can be reduced, for example, by simply truncating the lower bits.

  In step 595, the information amount reduction unit 400 checks whether there is a designation to reduce the number of frames. If this check is “Yes” (if frame number reduction is specified), the process proceeds to Step 596. If this check is “No”, the process proceeds to Step 597.

  In step 596, processing for reducing the number of frames of the original image (moving image) 100 is performed. For example, a process of thinning out every other frame may be performed. For example, frames having even frame numbers may be thinned out. By performing this processing, the frame rate of the original image (moving image) 100 can be reduced.

  In step 597, the information amount reduction unit 400 returns an error. In this case, since it means that the amount of information is not reduced, a predetermined error process may be executed.

  As described above, at least one information reduction process among a plurality of information amount reduction means is performed. A plurality of information reduction processes may be combined. In addition, the information reduction process of a present Example is an illustration, Information reduction processes other than this illustration may be made.

  Note that which process is to be performed can be determined in advance. Alternatively, which information amount reduction process is to be performed may be determined as a default. In addition, as a default, a mode in which each information amount reduction method is tried and a reduction method with the lowest RD cost is selected can be set.

<Regarding Parameter Conversion Unit 511>
The parameter conversion unit 511 has already been described with reference to FIG. Hereinafter, an example of the configuration and operation will be described with reference to FIGS. 6 and 7.

<Configuration Example of Parameter Conversion Unit 511>
FIG. 6 shows an example of the configuration of the parameter conversion unit 511. The parameter conversion unit 511 includes a parameter storage unit 512, a switching control unit 513, a parameter magnification changing unit 514, and a parameter interpolation unit 515.

  The parameter storage unit 512 acquires the encoding parameter obtained when the encoding parameter estimation unit 500 encoded the image 412 with the reduced information amount from the encoding control unit 501 shown in FIG. Save to. Since the acquired encoding parameter is a value obtained by encoding the image 412 in which the amount of information is reduced, necessary conversion is performed so that the encoding unit 200 can use the estimated encoding parameter 600. Provided to the encoding unit 200.

  When the information amount reduction unit 400 reduces the number of frames, it is desirable to obtain an estimated coding parameter 600 corresponding to the reduced frame by interpolation (extrapolation). The processing relating to this point has already been described. In order to perform interpolation (extrapolation), it is desirable that at least the encoding parameters of two frames encoded in the past are stored in the parameter storage unit 512.

  If the number of frames has not been reduced, it is only necessary to store the encoding parameters of the currently processed frame. Alternatively, the parameter storage unit 512 only needs to store at least the encoding parameter corresponding to the currently processed independent slice segment, which is the unit of encoding.

  A slice is defined as a set of units called slice segments. The slice segment includes an independent slice segment that can encode all encoded data independently, an independent slice segment in the same slice, and a dependent slice segment that has a dependency on decoding processing. Thus, the independent slice segment is a processing unit of encoded data. Therefore, the parameter conversion unit 511 only needs to be able to generate the estimated coding parameter 600 and provide it to the coding unit 200 in units of independent slice segments that are coding processing units.

  FIG. 8 shows that the frame 700 is divided into independent slice segments 710, 720, and 730. The independent slice segment 710 starts from the top CTU 711. The independent slice segment 720 starts from the top CTU 721. The independent slice segment 730 starts from the top CTU 731.

  Returning to FIG. The parameter magnification changing unit 514 functions, for example, when the reduction of the number of pixels is specified and the reduced image of the original image 100 is processed by the encoding parameter estimation unit 500. For example, when a reduced image reduced to ½ is processed in the encoding parameter estimation unit 500, the block size is doubled in principle to obtain the estimated encoding parameter 600. Also, when the encoding parameter estimation unit 500 prediction method is inter prediction, the direction is the same as the prediction vector obtained by the encoding parameter estimation unit 500, and the parameter magnification changing unit 514 doubles the vector distance. Can be converted to.

  The parameter interpolation unit 515 performs interpolation (extrapolation) in order to obtain the estimated coding parameter 600 of the deleted frame when the process of reducing the number of frames is performed. A specific example of the interpolation (extrapolation) processing has already been described.

  As shown in FIG. 4, the switching control unit 513 receives information 411 indicating what is used as information amount reduction processing from the information amount reduction unit 400. Based on this information 411, the switching control unit 513 gives the encoding parameter from the parameter storage unit 512 to the parameter magnification changing unit 514 and the parameter interpolation unit 515 as necessary.

  The parameter conversion unit 511 outputs the estimated coding parameter 600 obtained in this way.

<Operation of Parameter Conversion Unit 511>
FIG. 7 shows a flowchart of the operation of the parameter conversion unit 511.

  In step 602, the parameter conversion unit 511 checks whether a reduction in the number of pixels is designated. If this check is “Yes” (if the number of pixels has been reduced), the process proceeds to Step 604. If this check is “No”, the process proceeds to Step 606.

  In step 604, the parameter conversion unit 511 converts a predetermined parameter using a predetermined magnification. The specific contents of this conversion are as already described.

  In step 606, the parameter conversion unit 511 checks whether or not the reduction of the number of frames has been specified. If this check is “Yes” (if the number of frames has been reduced), the process proceeds to Step 608. If this check is “No”, the process ends.

  In step 608, the parameter conversion unit 511 performs interpolation (extrapolation) using the already acquired encoding parameter in order to obtain the estimated encoding parameter 600 of the deleted frame.

  Through the above processing, the encoding parameter acquired by the encoding parameter estimation unit 500 can be appropriately converted, and the estimated encoding parameter 600 can be obtained. When only the bit number reduction process is performed, the encoding parameter obtained by the encoding parameter estimation unit 500 can be used as it is by the encoding unit 200. The estimated encoding parameter 600 is given to the encoding unit 200.

<Use of Estimated Coding Parameter 600 in Encoding Unit 200>
As described above, the encoding unit 200 can encode the original image 100 using the estimated encoding parameter 600 obtained by the encoding parameter estimation unit 500. In this case, the encoding unit 200 encodes the original image 100 and obtains the bitstream 300 by omitting the above-described comprehensive calculation such as RD optimization by using the estimated encoding parameter 600 as it is. Can do. Since the encoding parameter estimation unit 500 can obtain the estimated encoding parameter 600 by encoding the reduced image, the processing efficiency of the image encoding device 10 can be improved.

  If the processing capability of the encoding unit 200 is sufficient, the estimated encoding parameter 600 in the encoding unit 200 may be partially used. For the remaining parameters, the encoding unit 200 may re-determine encoding parameters using an optimization method such as RD optimization when the original image 100 is encoded.

  For example, an example in the case of re-determining the CTU division shape from the estimated encoding parameter 600 by RD optimization is shown below.

  When the encoding unit 200 re-determines the CTU division shape, first, the RD cost is calculated when each CU is encoded using the value of the estimated encoding parameter 600.

  Thereafter, each CU is divided. The encoding unit 200 tries to encode all combinations of the encoding parameters in the lower layer, and calculates the RD cost (total value). The one with the smaller RD cost is selected compared to before the division. This is repeated up to the hierarchy permitted by the encoding standard (4 hierarchies in H.265 / HEVC) to obtain the CTU division shape.

  In this case, for example, the range to be divided into lower layers may be limited to one or two layers in the lower layer direction.

  Further, when considering the division of a CU into lower layers, the same prediction method and prediction mode for calculating the RD cost may be used as the estimated coding parameter 600.

  For example, for a CU that is inter-predicted with the estimated coding parameter 600 of the reduced image, the encoding unit 200 calculates the RD cost only for inter prediction even when considering division into lower layers. May be. Even when intra prediction is used in the estimated encoding parameter 600, the encoding unit 200 may similarly use intra prediction.

  In the above-described case, the encoding unit 200 has advanced the process to the lower layer, but may proceed to the upper layer.

  As described above, whether to use inter prediction or intra prediction is determined by the encoding unit 200 using the same method as the prediction method estimated by the estimated encoding parameter 600. It is not necessary to perform the process for.

  In addition, when the estimated coding parameter 600 of the block of interest is inter prediction and the motion vectors are merged, the motion of the block around the block of interest is the same. The motion vector of the estimated encoding parameter 600 may be used as it is. Merging means a mode in which the amount of motion vector information in the processing target block is reduced by indicating that the motion vector of the processing target block is the same as that of the adjacent block.

  As described above, the encoding unit 200 has to calculate the estimated encoding parameter 600 by the encoding parameter estimation unit 500 using the image with the reduced amount of information with respect to the original image 100. The number of CTUs can be reduced. Therefore, the image encoding device 10 can estimate the encoding parameter at high speed. As a result, the overall processing time of the image encoding apparatus 10 for encoding can be shortened.

  Also, the encoding unit 200 can re-determine the encoding parameter from the estimated encoding parameter 600, thereby minimizing image quality degradation that may be caused by using the estimated encoding parameter 600.

  In addition, since high-speed encoding is possible as compared with a parallel encoding apparatus using simple screen division, it is not necessary to divide a screen and perform parallel processing using a lot of hardware for the encoding process. Accordingly, there is no need for processing such as synchronization processing of a plurality of encoders or countermeasures for image quality deterioration at the screen division boundary. Also, the number of encoders can be reduced.

  The information of the original image 100 includes, for example, information on luminance information (Y) and color difference signals (Cb and Cr). It goes without saying that the technique described in this specification can be applied to each of these pieces of information.

<Implementation by program>
All or part of the above-described embodiments can be implemented by a program. This program can be stored in a portable recording medium. A portable recording medium refers to a non-transitory storage medium. Examples of portable recording media include magnetic recording media, optical discs, magneto-optical recording media, and nonvolatile memories.

  All or a part of the embodiments of the present invention can be implemented by reading a program stored in a portable recording medium and executing it by a processor.

  The embodiments of the present invention have been described in detail with reference to the drawings. It should be noted that the above-described embodiments are for understanding the invention and are not intended to limit the scope of the present invention. In the operation of the embodiment, the order of processing may be changed or skipped as long as there is no contradiction. Alternatively, a plurality of processes may be executed simultaneously. Needless to say, these embodiments are also included in the technical scope of the invention described in the claims.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS, a virtual machine monitor VMM, Needless to say, the present invention also includes a case where a program such as firmware or BIOS performs part or all of the actual processing and the functions of the embodiments are realized by the processing.

  In addition, the constituent elements of the embodiments of the present invention may be realized by a plurality of physically separated hardware. Also, each component of the various embodiments of the invention may be implemented by running on one or more computers.

10 Image Encoding Device 100 Original Image (Moving Image)
200 Encoding unit 201, 501 Encoding control unit 210, 510 Prediction method switching unit 220, 520 Screen division unit 225, 525 Prediction error signal generation unit 230, 530 Transform / quantization unit 240, 540 Entropy encoding unit 250, 550 Inverse quantization / inverse transform unit 255, 555 Decoded image generation unit 260, 560 Intra prediction unit 270, 570 Motion compensation prediction unit 280, 580 Motion detection unit 290, 590 Loop filter unit 300 Bit stream 400 Information amount reduction unit 410 Number of pixels Reduction unit 411 Information indicating what is used as information amount reduction processing 412 Image 420 with reduced information amount Bit number reduction unit 430 Frame number reduction unit 511 Parameter conversion unit 600 Estimated coding parameter

Claims (7)

An information amount reduction unit that generates a moving image with a reduced amount of information from the input moving image;
An encoding parameter estimator configured to estimate an encoding parameter used for encoding the input moving image by encoding the moving image with the reduced information amount;
An encoding unit that encodes the input moving image based on the estimated encoding parameters;
An image encoding device having:   The information amount reduction unit includes a pixel number reduction unit that reduces the number of pixels of each of a plurality of frames included in the input moving image, a bit number reduction unit that reduces the number of bits of pixels of the frame, and the input The image coding apparatus according to claim 1, further comprising at least one frame number reduction unit that reduces the number of frames per unit time of the moving image.   The encoding parameter estimator is configured to perform encoding obtained by encoding the moving image with the reduced information amount based on the relationship between the input moving image and the moving image with the reduced information amount. The image coding apparatus according to claim 2, further comprising: a parameter conversion unit that converts a parameter into the estimated coding parameter. When the number of pixels of each of the plurality of frames included in the input moving image is reduced by the pixel number reduction unit,
The parameter conversion unit uses a ratio between the number of pixels in the predetermined frame of the input moving image and the number of pixels of the predetermined frame in which the number of pixels is reduced to determine the size of the block and the length of the motion vector. Convert
The image encoding device according to claim 3. When the number of frames per unit time of the input moving image is reduced by the frame number reduction unit,
The parameter conversion unit calculates a motion vector of the reduced frame by extrapolating motion vectors obtained from a plurality of frames other than the reduced frame;
The image encoding device according to claim 3.   The image encoding device according to claim 1, wherein the encoding unit corrects a predetermined encoding parameter of the estimated encoding parameters within a predetermined range.   An image encoding program for causing a computer to function as the image encoding apparatus according to any one of claims 1 to 6.

Images