JPH07298258A - Image coding/decoding method - Google Patents

Abstract

PURPOSE:To attain real time processing by dividing an image to be coded at a sender side into plural small images, applying high efficiency coding in parallel to each small image, sending the coded image, allowing a receiver side to decode and synthesize coded data with respect to the plural small images to obtain the original decoded image thereby reducing the processing speed per picture elements. CONSTITUTION:A sender side uses an image division section 403 to divide a coding object image signal 402 into n-sets of small images 404. The signal of each small image 404 is given to an information compression section 405, in which coding processing is implemented and a coded output 406 is given to a multiplexer section 407, where the data are multiplexed into one coded output data 408 and sent to a transmission line 410. A receiver side uses a multiplexer/ demultiplexer section 413 to demultiplex the received coded data 412 into coded data 414 corresponding to n-sets of small images. Each decoded small image 416 decoded by each information decoding section 415 is given to an image synthesis section 417, in which the size of the image is restored to the original coded object image signal and the result is outputted to an output terminal 419 as a decoded signal 418.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION This invention is applicable to high-definition (HD
For high-efficiency coding of images with a high sampling rate such as TV (high density television),
The present invention relates to an image encoding / decoding method that divides an image to be encoded into a plurality of standard TV size small images and performs parallel processing to reduce the processing speed per pixel and enable real-time processing.

[0002]

2. Description of the Related Art FIG. 9 is a block diagram of a typical moving picture coding method for applying a discrete cosine transform (DCT) to a motion compensation prediction error.

In this moving image coding method, as shown in FIG. 9, an image 102 input from an input terminal 101 is used.
Is the motion vector detection unit 103,
The motion vector 105 is obtained from the decoded image of one frame before stored in No. 4, and the motion compensation unit 106 performs motion compensation according to the motion vector 105 to generate the predicted image 107. In the subtractor 108,
The difference between the input image 102 and the predicted image 107 is calculated, and the prediction error 109 is obtained. The prediction error 109 is calculated by the discrete cosine transform unit 110 for each small block (DCT block).
After being input to the input terminal and subjected to the discrete cosine transform (DCT), the quantization unit 111 quantizes the DCT coefficient. The quantization step used at this time is the buffer memory 11
It is controlled by feeding back from 3. Quantized D
A code is assigned to the CT coefficient by the code assigning unit 112 and sent to the buffer memory 113. The occupied state of the buffer memory is sent to the quantization step calculation unit 114, the quantization step 115 of the next DCT block is determined, and is fed back to the quantization unit 111. The data 116 stored in the buffer memory 113 is output to the output terminal 117 as final encoded data.

On the other hand, the quantized DCT coefficient is inversely quantized by an inverse quantizer 118, then inversely DCTed by an inverse discrete cosine transform unit 119, and then added with a predicted image 107 by an adder 120. It becomes the locally decoded signal 121 and is stored in the frame memory 104 for prediction of the next frame.

Usually, in such a high-efficiency encoding, processing is performed in a pipeline manner according to the order of input signals. That is, as shown in FIG. 10, first, the DCT block at the upper left of the screen is coded first, and when it is finished, the DCT block next to the right is coded next. When the coding of the uppermost DCT block is completed, the lower DCT block is similarly processed sequentially from the left side.

If the number of pixels of the entire screen of one frame is N and the time of one frame is T, by performing processing at t = T / N seconds per pixel, real-time encoding becomes possible. For example, 720 horizontal pixels, 480 vertical pixels, and frame rate 29.9 according to ITU-R Recommendation 601-3.
In the case of a standard TV signal of 7 Hz, t = (1 / 29.9
7) / (720 × 480) = 97 (nsec).

[0007]

However, in the above-mentioned conventional method, since the input signal is processed in a pipeline manner, for example, a signal sampled at a high frequency such as HDTV is highly efficient coded. If so, it may be difficult to operate the hardware in real time. Consider, for example, the case of handling an HDTV signal that has twice the number of pixels in a horizontal / vertical direction as in a standard TV. According to pipeline encoding processing, one pixel is encoded in a standard TV. It is necessary to perform it in about 25 nsec which is 1/4 of the above. When dealing with larger images,
There is a problem that real-time processing becomes difficult because processing is required in a shorter time.

Further, the quantization step when quantizing the DCT coefficient is feedback-controlled by the occupancy of the buffer memory, but the buffer occupancy at the time when the coding of the immediately preceding DCT block is finished is used. In order to determine the quantization step of the DCT block to be quantized, a series of three kinds of processing, that is, quantization, code allocation, and determination of the quantization step of the next DCT block from the buffer occupancy, is performed for one DCT block. Must finish in time.

However, when the number of pixels to be processed increases per unit time, the processing cannot be completed in the time of one DCT block, and the feedback is not the buffer state at the time when the processing of the immediately preceding DCT block is finished,
In this situation, the quantization step determined by the buffer state before the 2DCT block or the 3DCT block must be used. That is, there is a problem in that the control is delayed and the decoded image is adversely affected.

The present invention has been made to solve this problem, and an object of the present invention is to reduce the processing speed per pixel in the case of highly efficient encoding of an image having a high sampling rate. To provide a technology that enables real-time processing.

The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

[0012]

The outline of the representative one of the inventions disclosed in the present application will be briefly described as follows.

(1) An image encoding / decoding method, in which a transmitting side divides an image to be encoded into a plurality of small images, and high efficiency coding is performed in parallel on each of the divided small images. It is characterized in that the data is encoded and transmitted, and the receiving side decodes the encoded data for the transmitted plurality of small images and then synthesizes them to obtain a composite image of the original image.

(2) In the image coding / decoding method according to (1), the image to be coded is subjected to sub-sampling at a predetermined ratio in the horizontal and vertical directions to obtain one small image. After obtaining multiple small images by shifting the phase when performing subsampling,
It is characterized in that high efficiency coding is performed in parallel on each of the obtained small images.

(3) The image coding / decoding method according to the above (2), characterized in that a dividing method for obtaining a small image is selected according to the property of the signal.

[0016]

According to the above-mentioned means, it is possible to perform the high-efficiency coding processing in parallel for each of the divided small images, and when dividing the image into n small images, the number of times is n times as large as the encoding of one pixel. You can take the time. Therefore, if the screen size is very large and pipeline processing is performed, even when the number of pixels to be processed per second is large, it is possible to perform coding in real time.

Further, when a small image is created, according to (1) in the section of the above-mentioned means, if the image is spatially divided and divided,
The encoding process can be performed while maintaining the maximum correlation between pixels.

According to item (2) of the above-mentioned means, when a small image is created by pixel interleave division, each small image becomes an almost similar image and has statistically similar properties. It is divided and the encoding process is performed.

Further, according to (3) in the section of the means, the dividing method of (1) is locally applied depending on the characteristics of the image.
By selecting the division method of (2) above, if the spatial division improves the coding efficiency, the spatial division is performed and then the encoding process is performed. If not, the pixel interleaving is performed. After the division, the encoding process will be performed.

[0020]

Embodiments of the present invention will be described below in detail with reference to the drawings.

Example 1 FIG. 1 shows Example 1 according to the present invention.
11 is a block diagram for explaining the principle of the small image division type image encoding method, 11 is an image to be encoded, 12 is a divided small image signal, 13 is an encoding / decoding process, and 14 is a decoded image. , 15 are decoding target images.

As shown in FIG. 1, the principle of the small image division type image encoding method of the present embodiment is that, on the encoding side (transmission side), one screen of the encoding target image 11 is divided into a plurality of screens. Small image 12
And each small image 12 is encoded in parallel by an encoding / decoding process 13. The encoded data of each encoded small image 12 is encoded by the decoding side (reception side).
In the decoding process 13, the decoded small image 14 is obtained,
After that, the signals that are respectively placed at corresponding positions of the image to be coded and combined are the decoded signals, and the image to be decoded 1
It becomes 5.

FIG. 2 is a block diagram showing the functional arrangement of an apparatus for implementing the small image division type image coding method according to the first embodiment. 401 is an input terminal, 402 is an image signal to be encoded, and 403 is an image. A division unit, 404 is a small image, 405 is an information compression unit, 406 is an encoded output, 407 is a multiplexing unit, 4
08 is encoded output data, 409 is an output terminal, 410 is a transmission line, 411 is a receiving terminal, 412 is encoded data, 4
13 is a demultiplexing unit, 414 is encoded data, 415 is an information decoding unit, 416 is a decoded small image, 417 is an image combining unit,
Reference numeral 418 is a decoded signal and 419 is an output terminal.

Next, the operation of the apparatus for implementing the small image division type image coding method according to the first embodiment will be described with reference to FIG. As shown in FIG. 2, first, on the transmission side, the encoding target image signal 402 input from the input terminal 401 is input to the image dividing unit 403.
In, the image is divided into n small images 404. The signals of the respective divided small images 404 are respectively information compression units 405.
Is input to and encoded processing is performed. Encoded output 406
Are input to the multiplexing unit 407, combined into one encoded output data 408, and then transmitted to the transmission line 410 via the output terminal 409.

On the other hand, on the receiving side, the demultiplexing unit 413 demultiplexes the coded data 412 received from the transmission line 410 at the receiving terminal 411 into coded data 414 corresponding to n small images. Each of the separated encoded data 414 is input to the information decoding unit 415 and subjected to a decoding process. Each decoded small image 416 that has been decoded is input to the image combining unit 417 and is returned to the size of the original image signal to be encoded, and the decoded signal 418 is output as an output terminal 419.
Is output to.

As a method of dividing the screen, as shown in FIG.
As shown in (b) and (c), it is possible to use various space division methods that spatially divide the image.

As a method of converting into a small image, an image to be encoded is sub-sampled in a horizontal / vertical direction at a predetermined ratio to obtain one small image. The sub-sampling ratio is determined by how many small images (small screens) are divided. Then, a configuration is obtained in which a plurality of small images are obtained by shifting the phase when performing sub-sampling (see FIG. 4: Example of small image constructing method by pixel interleave division). That is, the pixels are interleaved to obtain a small image.

Further, it is possible to select a division method for obtaining a small image, depending on the nature of the signal, for each scene or for each predetermined area, either spatial division or pixel interleave division so as to be adapted to the target image. .

Example 2 HD of Example 2 according to the present invention
A case will be described in which a TV signal is divided into four standard TV size small images and processed, and discrete cosine transform is used as an information compression method.

FIG. 5 is a block diagram showing the functional arrangement of an apparatus for implementing the image coding / decoding method according to the second embodiment of the present invention. 701 is an input terminal, 702 is an HDTV signal, and 7 is an HDTV signal.
Reference numeral 03 is an image dividing unit, 704 to 707 are standard TV size small image images, 708 to 711 are discrete cosine transform units, 7
12 to 715 are quantization units, 716 to 719 are code assignment units, 720 to 723 are buffer memory units, and 724 to 7
Reference numeral 27 is an encoded output, 728 is a multiplexer, 729 is encoded output data, and 730 is an output terminal.

The operation of the apparatus for implementing the image coding / decoding method of the second embodiment will be described with reference to FIG. First, the input terminal 7
The HDTV signal 702 input from 01 is input to the image dividing unit 703 as four standard TV size small images 704-
It is divided into 707.

Each standard T thus obtained
Subsequent processing is performed in parallel on the V size small images 704 to 707. The divided standard TV size small images 704 to 707 are respectively divided into discrete cosine transform units 708 to 708.
After being inputted to 711 and subjected to discrete cosine transform, they are inputted to the quantizers 712 to 715. Quantizer 712
In 715, the discrete cosine transform coefficient is quantized, the code is assigned in the code assigning units 716 to 719, and then sent to the buffer memory units 720 to 723.

In the buffer memory units 720 to 723, feedback control is applied to the quantizing units 712 to 715 so that the outputs thereof have a predetermined information amount. Encoded outputs 724-727 for each small image
Are input to the multiplexing unit 728, combined into one, and then output to the transmission line via the output terminal 730.

As a division method, spatial division as shown in FIG. 6A or pixel interleave division as shown in FIG. 6B is adopted. In the spatial division shown in FIG. 6A, the small image is divided into “square” shapes so that the size of the small image is half the horizontal and vertical sizes of the HDTV image. , Small image 3 and small image 4.

On the other hand, in the pixel interleave division shown in FIG. 6B, first, a small image 1 is obtained by subsampling every other pixel at a ratio of 2: 1 in the horizontal and vertical directions. When the small image 1 is obtained, the sub-sampling phase is shifted by 1 pixel in the horizontal direction and sub-sampled to obtain the small image 2, and by shifting 1 pixel in the vertical direction and sub-sampled, the small image 3 is obtained. Is obtained, and a small image 4 is obtained by subsampling by shifting by one pixel in the horizontal and vertical directions.

Example 3 HD of Example 3 according to the present invention
Described below is the case of switching the space division and the pixel interleave division when dividing the TV signal into four standard TV size small images.

FIG. 7 is a block diagram showing the functional arrangement of an apparatus for carrying out the image coding / decoding method according to the third embodiment of the present invention. 901 is an input terminal, 902 is an HDTV signal, and 9 is.
Reference numeral 03 is an extra-small amount calculation unit, 904 is a switching signal, 905 is an image division unit, and 906 to 909 are standard TV size small images.
Reference numerals 910 to 913 denote discrete cosine transform units, and 914 to 917.
Is a quantizer, 918 to 921 are code assigners, 922
~ 925 is a buffer memory unit, 926 to 929 are encoded outputs, 930 is a multiplexing unit, 931 is encoded output data,
932 is an output terminal.

The operation of the apparatus for implementing the image coding / decoding method of the third embodiment will be described with reference to FIG. First, the input terminal 9
From the HDTV signal 902 input from 01, the feature of the screen is calculated by the extra-small amount calculation unit 903, and the switching signal 904
Is required. Based on the obtained switching signal 904,
The image division unit 905 divides into four standard TV size small images 906 to 909. Subsequent processing is performed in parallel for each of the standard TV size small images 906 to 909 obtained in this way. The divided small images 906 to 909 of the standard TV size are input to the discrete cosine transform units 910 to 913, respectively, subjected to the discrete cosine transform, and then input to the quantization units 914 to 917. In the quantizing units 914 to 917, the discrete cosine transform coefficients are quantized, the codes are assigned in the code assigning units 918 to 921, and then the buffer memory units 922 to 922.
Sent to 25. In each of the buffer memory units 922 to 925, feedback control is applied to the quantizing units 914 to 917 so that the output thereof has a predetermined information amount. The coded outputs 926 to 929 corresponding to the small images 906 to 909 are input to the multiplexing unit 930, combined into one coded output data 931 and then transmitted to the transmission path via the output terminal 932.

The switching signal may be set as follows, for example. As shown in FIG. 8A, when the change is large for each pixel, the power of the high frequency component of the HDTV signal becomes very large, and the pixel interleave division can provide a high frequency for a small image. The component can be made small and the encoding can be facilitated. However, as shown in FIG. 8B, when the pixel interleave division is performed in the case where there is a smooth change, the correlation between adjacent pixels in the obtained small image becomes small, and the coding efficiency decreases. , It is more advantageous to perform spatial division. Therefore, for example, in the trace amount calculation unit 903, the frequency spectrum of the HDTV signal is calculated, and when the power of the high frequency component is less than or equal to a certain threshold value, the signal for selecting space division has the power of the high frequency component If it is equal to or more than the threshold value, a signal for selecting pixel interleave division may be output as a switching signal.

In the above, the embodiment has been described for the case of dividing into four, but the case of dividing into a plurality can be considered in exactly the same manner. Further, the coding method can be applied not only to the intra-frame DCT described in the above embodiment but also to a general method such as motion compensation inter-frame DCT coding.

Although the present invention has been specifically described based on the embodiments, the present invention is not limited to the above embodiments, and it goes without saying that various modifications can be made without departing from the scope of the invention.

[0042]

The effects obtained by the typical ones of the inventions disclosed in the present application will be briefly described as follows.

(1) Since high-efficiency encoding processing can be performed in parallel for each of the divided small images, when dividing into n small images, it takes n times as long to encode one pixel. be able to. Therefore, if the screen size is very large and pipeline processing is performed, even when the number of pixels to be processed per second is large, it is possible to perform coding in real time.

(2) When a small image is created, if it is spatially divided and divided into a plurality of small images, the inter-pixel correlation can be used to the maximum extent, so that the deterioration of the coding efficiency is minimized. I can do it.

(3) When a small image is created by the pixel interleaving method, the divided small images have almost the same image and have statistically similar properties. The degree of difficulty is not different. Therefore, good coding control can be performed by simply allocating the information amount allocated to the whole so as to be even.

(4) Switching suitable for coding is made possible by switching between a partitioning method for locally partitioning into a plurality of small images by spatially partitioning it into a plurality of small images according to the characteristics of the image and a partitioning method by the pixel interleaving method. , The coding efficiency can be improved.

(5) According to the above (1) to (4), the HDT
Even in the case of a large moving image having a size of V or more, considering the encoding efficiency, the ease of encoding control, etc.,
Coding and decoding can be performed in real time.

[Brief description of drawings]

FIG. 1 is a block diagram for explaining the principle of a small image division type image coding method according to a first embodiment of the present invention.

FIG. 2 is a block configuration diagram showing a functional configuration of an implementation device of a small image division type image encoding method according to the first embodiment.

FIG. 3 is a diagram showing an example of a small image forming method by space division according to the first embodiment.

FIG. 4 is a diagram showing an example of a small image constructing method by pixel interleave division according to the first embodiment.

FIG. 5 is a block configuration diagram showing a functional configuration of an implementation device of an image encoding / decoding method according to a second embodiment of the present invention.

FIG. 6 is a diagram illustrating a method of dividing a small image into small images according to the second embodiment.

FIG. 7 is a block configuration diagram showing a functional configuration of an implementation device of an image encoding / decoding method according to a third embodiment of the present invention.

FIG. 8 is a diagram illustrating a method of switching a small image forming method according to the third embodiment.

[Fig. 9] Fig. 9 is a diagram illustrating a functional configuration of an apparatus for performing an image coding method by a conventional motion-compensated prediction discrete cosine transform.

FIG. 10 is a diagram showing an encoding processing order by a conventional pipeline processing.

[Explanation of symbols]

11 ... Encoding target image, 12 ... Divided small image, 13
Is an encoding / decoding process, 14 ... Decoded image, 15 ... Decoding target image, 101 ... Input terminal, 102 ... Input image signal, 1
03 ... motion vector detection unit, 104 ... frame memory,
105 ... Motion vector 106 ... Motion compensation unit 107 ...
Predicted image, 108 ... Subtractor, 109 ... Prediction error, 110
... discrete cosine transform section, 111 ... quantization section, 112 ... code allocation section, 113 ... buffer memory, 114 ... quantization step calculation section, 115 ... quantization step, 116 ...
Coded data 117 ... Output terminal 118 ... Inverse quantization unit 119 ... Inverse discrete cosine transform unit 120 ... Adder,
121 ... Local decoded signal, 401 ... Input terminal, 402 ... Encoding target image signal, 403 ... Image dividing unit, 404 ... Divided n small images, 405 ... Information compressing unit, 406 ... Encoding output, 407 ... Multiplexing unit, 408 ... Encoded output data, 409 ... Output terminal, 410 ... Transmission line, 411 ... Reception terminal, 412 ... Encoded data, 413 ... Demultiplexing unit, 4
14 ... Encoded data corresponding to each small image, 415 ... Information decoding unit, 416 ... Decoded n small images, 417 ... Image combining unit, 418 ... Decoded image signal, 419 ... Output terminal,
701 ... Input terminal, 702 ... Input HDTV signal, 703
... Image division unit, 704 to 707 ... Divided standard TV size small image, 708 to 711 ... Discrete cosine transform unit, 7
12-715 ... Quantization unit, 716-719 ... Code assignment unit, 720-723 ... Buffer memory, 724-72
7 ... Encoding output for small image, 728 ... Multiplexing unit, 7
29 ... Encoded data for HDTV signal, 730 ... Encoded data output terminal, 901 ... Input terminal, 902 ... Input HDTV signal, 903 ... Extra small amount calculation section, 904 ... Switching signal, 905 ... Image division section, 906-909 ... Split standard TV size small image, 910-913 ... Discrete cosine transform section, 914-917 ... Quantization section, 918-921
... code assigning unit, 922 to 925 ... buffer memory,
926 to 929 ... Coded output for small image, 930 ...
Multiplexer, 931 ... Coded data for HDTV signal, 932 ... Coded data output terminal.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁶ Identification number Internal reference number for FI Technical indication H03M 7/30 A 8842-5J H04N 7/015 G06F 15/66 330 D H04N 7/00 A

Claims (3)

[Claims] 1. A transmission side divides an image to be encoded into a plurality of small images, then performs high-efficiency encoding in parallel on each of the divided small images and transmits the small images.
An image coding / decoding method characterized by decoding the transmitted coded data for a plurality of small images and then synthesizing them to obtain a decoded image of the original image. 2. A plurality of images to be encoded are obtained by performing subsampling in a horizontal / vertical direction at a predetermined ratio to obtain one small image, and shifting the phase at the time of subsampling. 2. The image coding / decoding method according to claim 1, wherein, after obtaining the small images, the high efficiency coding is performed in parallel for each of the obtained small images. 3. The image coding / decoding method according to claim 2, wherein the division method for obtaining the small image is selected according to the property of the signal.