JP2002369220A - Extended image coding method, extended image decoding method, extended image coder, extended image decoder and extended image recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a constitution of an extended coded image transmitter for transmitting image data having upper hierarchical structure, without changing the transmission format of compressed coded signals coded to transmit according to the standard system main profile. SOLUTION: For generating coded bit streams based on the international coding standard, difference image signals for obtaining image data having an upper hierarchical structure are coded and inserted in insertion-permitted user data recording areas to transmit the data. A usual decoder decodes image signals conforming to the main profile coding standard, while a decoder capable of decoding difference image signals transmitted over user data areas usually adds decoded difference image signals to coded images to obtain image signals on the upper hierarchical layer.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the generation or decoding of a compression-encoded signal having a hierarchical property, and more particularly to a method of generating a higher-layer differential signal while maintaining compatibility with compression-encoded data introduced on the market. Encoding method of an extended image for generating data as the same compression encoded data, decoding method of the extended image, extended image encoding device equipped with the method, extended image decoding device, and encoded data thereof The present invention relates to an extended image recording medium on which an image is recorded.

[0002]

2. Description of the Related Art MPEG (moving) is a technique for encoding moving picture signals such as television signals with high efficiency.
picture experts group) The standard is digital broadcasting, DV
D (Digital versatile Disc), digital tape recorders, and widely used as signals transmitted in communication networks.

[0003] The MPEG was introduced in 1988 by the ISO / IE.
C (International Organization for Standardization
/ International Electrotechnical Commission) J
TC1 / SC2 (Joint Technical Committee 1 / Sub
committee 2 It is an organization that considers the video coding standards established by the International Standards Organization / International Electrotechnical Commission Technical Committee 1 / Specialized Subcommittee 2).

[0004] The SC2 is currently a moving picture as SC29,
In addition, activities for establishing standards relating to encoding of audio signals and the like have been continued, and international standards established by MPEG people are commonly referred to as MPEG.

[0005] The first established MPEG1 (MPEG Phase 1) is a coding standard for moving picture signals accompanied by audio signals and intended for storage media recorded at a transmission rate of about 1.5 Mbps. JPEG (Joint Photographic Coding Experts Grou
p) and ISDN (Integrated services digital netwo)
H.rk) for the purpose of video compression for video conferences and videophone low transfer rates. 261 (CCITT SGXV, current IT
This is a coding standard that uses the basic technology of UT SG15).

[0006] Thus, MPEG1 was introduced in August 1993.
The disc was enacted in January as ISO / IEC 11172, and many discs encoded and recorded according to the MPEG1 standard have been commercialized.

[0007] MPEG2 (MPEG Phase 2), which was subsequently enacted, is intended to be used as a general-purpose standard in order to support various applications such as communication and broadcasting.
In November 2004, ISO / IEC 13818 and "H.2
62 ”.

[0008] The encoding by MPEG1 and MPEG2 is composed of a plurality of encoding techniques. These techniques are used to convert a "frame" image constituting a moving image into a block of 16 x 16 pixels called a "macroblock". And, for each macroblock unit, obtain a motion amount called a “motion vector” between a reference image and a coded image separated by a predetermined number of frames in the future or the past in time, and calculate the motion amount. A motion compensated prediction technique for encoding a coded picture from a reference picture based on a reference picture, and a DCT (Discrete Cosine) technique which is one of orthogonal transform techniques for an error signal of the motion compensated prediction or the coded picture itself. Transform: Discrete cosine transform) is used to convert image information into frequency information, so that only visually significant information can be obtained from the converted frequency domain information. The technique is defined based on two elemental techniques of image encoding, that is, a “transform encoding” technique for performing compression encoding by using a technique.

There are three modes in the motion compensation prediction in the case of prediction from both the past, the future, and the past and the future, and the three modes are 16 pixels × 1.
Switching is possible for each macroblock consisting of data of six pixels.

The prediction direction is determined by the picture type given to the frame of the input image, that is, I (Intra-code).
d), P (Predictive-coded), and B (Bidirectional
ly predictive-coded) are defined.

The I-picture is a picture to be coded without performing motion prediction, while the P-picture has two modes of prediction from the past and coding without performing prediction. Has four MCs (Motion Comp.) That perform intra-frame prediction without performing prediction from the future, prediction from the past, prediction from both the past and future directions, and prediction.
ensation) mode.

The motion compensation performed using these future or past images is performed by performing pattern matching on a motion area for each macroblock and performing half-pel (one pixel distance).
/ 2) A motion vector is obtained with accuracy, and an image signal supplied is encoded based on an image obtained by moving a future or past reference image position in the vector direction in accordance with the obtained motion vector amount. .

There are a horizontal direction and a vertical direction in the direction of the motion vector thus obtained, and the vector information is transmitted together with the MC mode as additional information of the macro block.

[0014] Of the picture data thus produced, three types of pictures, I, P and B, are I pictures.
The pictures are arranged and transmitted in a predetermined order starting from the picture,
A set of pictures (frame images) from the I picture to a picture before the next I picture is represented by GOP (Group O).
f Picture), and in encoding performed on ordinary storage media, etc., a GOP
Is configured.

[0015] The I picture and the P and B pictures coded as motion compensated pictures are DCT (discrete).
te cosine transform), that is, an integral transform using a cosine function as an integral kernel is performed as an orthogonal transform that discretely transforms into a finite space.

In the orthogonal transformation, the macro block is divided into 8 pixels ×
The DCT block is divided into four DCT blocks relating to the luminance signal of eight pixels, BY which is obtained by subtracting the luminance signal from the blue signal, and one DCT block of 8 × 8 pixels which is obtained by subtracting the luminance signal from the red signal. Although two-dimensional DCT is performed, since the frequency components of the DCT block image data are generally high in the low frequency component and low in the high frequency component, the image data is DCT block image data.
And can be represented by the transform coefficients concentrated on the low frequency band.

Then, the DCT-processed image data (D
The (CT coefficient) is quantized by a quantizer. That is, the DCT coefficient is calculated and calculated by a predetermined quantization value by the quantizer. The quantization value is obtained by obtaining a two-dimensional frequency of 8 × 8 pixels as a quantization value weighted by visual characteristics. However, the quantized value used is a value scalar-multiplied by a predetermined quantization scale.

The quantized value is multiplied by an inverse quantized value obtained at the time of decoding the coded image data, so that the characteristic of the quantized value given at the time of encoding is canceled at the time of decoding. I have.

Next, the configuration of an MPEG encoder that performs encoding and decoding by such a method will be described. FIG.
5 shows the configuration of the MPEG encoder, and outlines the operation thereof.

The MPEG encoder 50 has an input terminal 5
1, adder 52, DCT unit 53, quantizer 54, VLC
Unit 55, buffer 56, code amount controller 57, inverse quantizer 61, inverse DCT unit 62, adder 63, image memory 64,
And a motion compensation predictor 65.

First, the moving picture signal supplied to the input terminal 51 is supplied to a motion compensation predictor 65 and an adder 52. In the adder 52, the signal supplied from the motion compensation predictor 65 is inverted and added. Are supplied to the DCT unit 53.

In the DCT unit 53, the supplied image signal is subjected to the above-described discrete cosine transform, and the obtained DTS
The CT transform coefficient is supplied to a quantizer 54, where the quantized data is quantized based on the predetermined quantized value, and the quantized data is quantized by an inverse quantizer 61 and VLC (variable length).
h coding) unit 55.

In the VLC unit 55, the supplied quantized data is variable-length coded, but a direct current (DC) component obtained by performing a DCT transform among the quantized values is a DP component.
CM (differential pulse code modulation) modulation is performed.

Further, an alternating current (AC) component is obtained while performing a zigzag scan in the order of data of a high frequency component from data of a low frequency component, and the obtained data has a zero run length and an effective run length. The Huffman coding is performed such that the coefficient value is regarded as one event, and codes having shorter code lengths are assigned in ascending order of occurrence probability.

The data subjected to the Huffman encoding, which is the variable length encoding, is temporarily stored in a buffer 56, and the temporarily stored data is supplied as an encoded data output at a predetermined transfer rate.

The generated code amount for each macro block of the supplied data is supplied to a code amount controller 57 and compared with a preset target code amount, and the generated code amount obtained by the comparison is obtained. Is supplied to the quantizer 54, and the quantizer 54 obtains encoded data at a predetermined transfer rate by changing the value of the quantization scale based on the code amount of the difference. Is performed to control the code amount.

On the other hand, the image data quantized by the quantizer 54 is supplied to an inverse quantizer 61 where the image data is inversely quantized.
The inversely quantized data is supplied to an inverse DCT unit 62, where the inverse DCT is performed, and the inverse DCT data is supplied to an adder 63.

In the adder 63, the signal is added to the reference image supplied from the motion compensation predictor 65, and the signal obtained by the addition is supplied to the image memory 64, where it is temporarily stored. The temporarily stored image data is used as a reference decoded image for calculating a difference image in the motion compensation predictor 65 so that the MPEG encoder 50 can obtain the motion-compensated encoded data. It has been done.

The coded data obtained in this way is M
It is supplied to a PEG decoder and decoded. FIG.
The configuration of the PEG decoder will be described, and an outline of the operation will be described.

An MPEG decoder 70 shown in FIG. 1 includes a data input terminal 71, a buffer 72, a VLD unit 73, an inverse quantizer 74, an inverse DCT unit 75, an adder 76, and an image memory 7.
7 and a motion compensation predictor 78.

First, the encoded data supplied to the input terminal 71 is temporarily stored in a buffer 72, and the temporarily stored encoded data is stored in a VLD (variable length de
coding) unit 73.

In the VLD unit 73, the data encoded by the VLC unit 55 is subjected to variable-length decoding, and the data relating to the direct current (DC) component and the alternating current (AC) component is obtained.

Of the obtained data, the AC component data is quantized arranged in an 8 × 8 matrix in the same zigzag scan from the low frequency band to the high frequency band component as performed by the MPEG encoder 50. Obtained as data,
The obtained quantized data is supplied to the inverse quantizer 74.

In the inverse quantizer 74, the inverse quantization is performed by the above-described quantization matrix, and the data obtained by the inverse quantization is supplied to the inverse DCT unit 75, where the inverse DCT operation is performed and the image is processed. Data is obtained as decoded data.

The obtained image data is temporarily stored in an image memory 77, the temporarily stored image data is supplied to a motion compensation predictor 78, and the supplied image data calculates a difference image in the motion compensation prediction. Is used as a reference decoded image for performing the decoding.

As described above, the image data constituting the moving picture is encoded and transmitted or recorded by the MPEG encoder 50, and the encoded data received or reproduced is decoded by the MPEG decoder 70. This method is obtained as moving image information, and such a method is used in both MPEG1 and MPEG2.

A profile and a level are defined as a method of using the moving picture signal encoding based on MPEG2. The profile includes a main profile used for recording on a DVD and digital satellite broadcasting. In addition, a simple profile in which a delay time is reduced by simplifying an encoding method, an SNR scalable profile described later, and a spatial scalable (Spatial) in which spatial resolution has a hierarchy.
There is a Scalable profile and a High profile having advanced functions.

In these profiles, the 4: 2: 0 image format is mainly used as the image format relating to the number of samples of the luminance signal and the color difference signal, and the 4: 2: 2 image format is a high profile,
Or 42 defined in parallel with the above profile
Use is allowed in only two profiles.

As described above, in recording and broadcasting of a normal video signal, the compression encoding of the image signal in the 4: 2: 0 image format is performed, and a high resolution for the color signal can be obtained. Recording and reproduction of signals in the two-image format and broadcasting are not performed.

However, in the recent information society, the generation of image signals using a computer has become widespread. Although it is not so necessary when encoding an image with a very low saturation as in the case of conventional camera shooting, it is not necessary to encode a high-saturation and high-resolution signal produced by a computer or the like. Deterioration is observed in the image quality obtained by performing encoding and decoding.
It has become desirable to encode a signal that does not reduce the resolution even for a color signal with high saturation such as a 2: 2 image format.

Japanese Patent Application Laid-Open No. 9-23 / 1999 relates to such encoding and decoding of image signals in which deterioration of color signal components is reduced.
No. 8366, "Image Encoding Apparatus, Image Decoding Apparatus, and Encoding / Decoding System" discloses that a block distortion component generated from a chrominance signal block generated at the time of image compression encoding and decoding spreads over the entire block. It discloses an encoding scheme for preventing this.

That is, according to the encoding method, as an encoder / decoder that dynamically switches the format conversion without lowering the compression efficiency, the digitized input image is format-converted, quantized, and encoded. In the image encoding device, at the time of format conversion, a plurality of format conversion units for converting to a plurality of spatial resolution image data by a predetermined luminance signal and color difference signal, the input image data or the quantized image data or other An image state determination unit that detects a change in the state of the image data and selects and outputs which of the plurality of spatial resolution image data is to be output. A method for supplying the signal to the quantization unit is disclosed.

The SNR scalability profile described above is one of the profiles defined in MPEG2 as a technique for realizing hierarchical coding in the frequency domain in which encoding and decoding are performed by selecting a spatial resolution.

The SNR scalability profile is a profile that can be used while maintaining the upward compatibility as an SNR profile that is higher than the main profile. First, encoding is performed by a base layer, with a layer having a small size as a higher layer (enhancement layer).

Next, in the higher layer to be encoded, the image of the base layer is decoded, the difference between the decoded image signal and the input image is calculated, and the obtained difference image is quantized to a smaller value than the base layer. It is stipulated that an encoded signal is obtained by performing quantization on a quantization scale.

By using the two encoded signals of the base layer and the higher layer as a function for obtaining the chrominance signal, a simulcast function of the chrominance signal is realized using the same technique as the SNR scalability. It is possible.

Using the simulcast function, for example, a luminance signal and a color difference signal of a 4: 2: 0 image are encoded in the base layer, and only a 4: 2: 2 color difference signal is encoded in the higher layer. By transmitting the two bit streams obtained in the simulcast system, the decoder side receives the 4: 2: 0 signal using only the lower layer and the 4: 2: 2 image format using the higher layer as well. Can be obtained.

The two bit streams coded by the base layer and the higher layer are multiplexed into one bit stream while being synchronized in a multiplexing system layer higher than the coding layer. Receiving the bit stream transmitted in this manner separates the multiplexed signal and decodes the separated two bit stream signals by two decoders. Then, decoding is performed using a decoder having a dedicated decoding function for obtaining a decoded signal in the 4: 2: 2 image format based on the two signals obtained by decoding.

[0049]

However, according to the method disclosed in Japanese Patent Laid-Open No. 9-238366,
An image state determination unit for determining which of a plurality of color signal formats is to be selected is required, and there are a plurality of bit stream formats according to the determination. In this case, decoding cannot be performed, and there is a problem in securing compatibility.

In the method of transmitting an image signal of two formats using the SNR scalability profile of MPEG2, a bit stream cannot be transmitted except for a multiplexed stream of two bit streams. 1 with a hierarchical structure
I cannot get a bit stream as one video stream.

In order to transmit two encoded signals in the 4: 2: 0 and 4: 2: 2 image formats using the SNR scalability profile,
Compatible with existing applications because it is not possible to have a hierarchical structure without changing the standard of the predetermined application that defines the bit stream generated by the base 4: 2: 0 image format and the application format. 4: 2: 2 while maintaining the quality
The only difference is that a difference signal from an image in an image format cannot be generated and multiplexed.

Accordingly, the present invention provides, for example, a predetermined application standard relating to a bit stream based on an existing digital broadcast and a DVD which is generated, recorded or transmitted according to a main profile of a standard system, and records or records. Generates an encoded bit stream for recording or transmitting image data of an upper hierarchical structure while maintaining the compatibility with existing applications by using the transmission format as it is and maintaining a hierarchical structure. Thereby, a method of encoding and decoding an image signal with enhanced function expandability of an application, a configuration of an encoding and decoding apparatus equipped with the method, and further recording such an encoded signal It is intended to provide a recording medium.

[0053]

The present invention comprises the following means 1) to 9) to solve the above-mentioned problems. That is,

1) A first step (1) of orthogonally transforming the supplied image signal to obtain a converted image signal, and performing variable length coding on the obtained converted image signal to obtain a compressed coded signal.
2) and a second step (13) of decoding the compression-encoded signal or the converted image signal to obtain a decoded image signal;
A third step (14) of calculating the decoded image signal and the supplied image signal to obtain a difference image signal, and a fourth step of compression-coding the difference image signal to obtain a difference image compression-coded signal (16), and a fifth step (17) of inserting the differential image compression-encoded signal into a user data recording area of the compression-encoded signal to obtain an extended image-encoded signal A method for encoding an extended image.

(2) The method according to (1), wherein a predetermined unique code is added as an identification signal and inserted into the user data recording area immediately before the differential image compression encoding signal in the fifth step. A method for encoding an extended image.

3) The converted image signal obtained in the first step is a signal in the 4: 2: 0 image format, and the difference image signal obtained in the third step is the 4: 2: 0 The extended image encoding method according to item 1), wherein the signal is a signal in an image format having a higher color signal resolution than the image format.

4) The supplied image signal is compression-encoded to obtain a first compression-encoded signal, and a decoded signal obtained by decoding the obtained first compression-encoded signal and the image signal Is obtained by compression-encoding the obtained differential image signal to obtain a second compression-encoded signal, and adding a unique code to the head of the obtained second compression-encoded signal. An extended image coded signal obtained by inserting the first compressed coded signal into a user data recording area is supplied,
The first compressed coded signal of the extended image coded signal is decoded to obtain a first decoded image signal, and the second compressed coded signal transmitted in the user data area of the extended image coded signal A first step (32, 33, 35) of decoding the signal to obtain a second decoded image signal, and a first decoded image signal and a second decoding obtained in the first step; A second step (36) of obtaining an extended decoded image by performing image signal addition processing.

5) The first compression coded signal is 4:
The image signal is a signal encoded by a 2: 0 image format, and the differential image signal is a signal encoded by an image format having a higher color signal resolution than the 4: 2: 0 image format. 4) A method for decoding an extended image according to the item.

6) First compression encoding means (12) for obtaining a converted image signal by orthogonally transforming the supplied image signal and performing variable length encoding on the obtained converted image signal to obtain a compressed encoded signal. And obtaining a compressed coded signal from the first compression coding means, or obtaining the converted image signal from the first compression coding means, and decoding at least one of the obtained signals. Decoding means (1) for obtaining a decoded image signal by
3), a decoded image signal obtained by the decoding means,
A calculating means (14) for obtaining a difference image signal by comparing and calculating the supplied image signal; and a second means for compression-coding the difference image signal obtained by the calculating means to obtain a difference image compression-coded signal. Compression encoding means (16);
User data insertion means (17) for inserting the differential image compression encoded signal obtained by the second compression encoding means into the user data area of the first compression encoded signal to obtain an extended image encoded signal An extended image encoding apparatus characterized by comprising:

7) The compression-encoded signal obtained by the first compression-encoding means is a signal in the 4: 2: 0 image format, and the differential image compression-encoding obtained by the second compression-encoding means The extended image encoding device according to item 6), wherein the signal is a signal in an image format having a higher color signal resolution than the 4: 2: 0 image format.

8) The supplied image signal is compression-encoded to obtain a first compression-encoded signal, and the decoded signal obtained by decoding the obtained first compression-encoded signal and the supplied decoded signal Obtain a difference image signal from the image signal, compression-encode the obtained difference image signal to obtain a second compression-encoded signal, and add a unique code to the head of the obtained second compression-encoded signal. Extended image coded signal obtained by inserting the extended image coded signal into the user data recording area of the first compressed coded signal, and decoding the supplied extended image coded signal Decoding means for decoding the first compressed encoded signal from the supplied extended image encoded signal to obtain a first decoded image signal (32)
And decoding a second compression-encoded signal transmitted in the user data recording area to obtain a second decoded image signal.
And an adder (36) for adding an extended decoded image signal by adding the second decoded image signal and the first decoded image signal. An extended image decoding apparatus, comprising:

9) In the user data recording area of the first encoded signal obtained by encoding the image signal in the 4: 2: 0 image format, the image signal encoded by the first encoded signal and the code An extended image recording medium for recording a difference image signal related to a difference from the image signal before the
The extended image recording medium, wherein the differential image signal is a second encoded signal encoded in an image format having a higher color signal resolution than the 4: 2: 0 image format.

[0063]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the extended image encoding method, extended image decoding method, extended image encoding device, extended image decoding device, and extended image recording medium of the present invention will be described below. This will be described below. FIG. 1 shows a first example of an extended image encoding device equipped with the extended image encoding method.
This embodiment will be described with reference to the drawings.

The extended image encoding device 10 shown in FIG. 1 has an image data input terminal 11, an MPEG (moving picture exp
erts group) encoder 12, MPEG decoder 13, adder 14, delay unit 15, DPCM (differential pulse)
code modulation) unit 16 and user data inserter 1
7.

Next, the operation of the extended image encoding apparatus 10 configured as described above will be described. First, input image data is supplied to an MPEG encoder 12 and encoded by a method according to an MPEG main profile, and an encoded signal obtained by encoding is supplied to a delay unit 15 and an MPEG decoder 13. You.

In the MPEG decoder 13, the encoded signal is decoded and supplied to one input terminal of an adder 14 as a negative signal, and the other input terminal receives input image data. Is supplied as a signal of positive polarity, a signal obtained by adding the signals supplied to the two input terminals is obtained from the adder 14, and the added signal is supplied to the DPCM encoder 16. .

In the DPCM encoder 16, the supplied signal is subjected to DPCM encoding to generate a DPCM encoded signal, and the generated DPCM encoded signal is supplied to one input terminal of the user data inserter 17. While being supplied, the other input terminal is supplied with an encoded signal delayed for synchronization processing by the delay unit 15, and the DPCM encoded signal is inserted into a user data area of the encoded signal. The encoded data obtained by insertion is supplied from the extended image encoding device 10 as an output signal of the device.

Thus, the signal encoded by the MPEG encoder 12 is decoded by the MPEG decoder 13, and the difference signal between the decoded signal and the input image data is converted by the DPCM encoder 16. The encoded data is obtained by performing DPCM encoding on the data and inserting it into the user data area. The MPEG encoder 12 and the MPEG decoder 13
The encoding and decoding performed by the DCT transform and the quantization of the input image are performed as local decoding on the signal subjected to the DCT transformation and the quantization, or VLC (variable length coding) is performed on the signal subjected to the quantization. ) To obtain the final bit stream data, and then VLD (variable length
Any of the cases where signal processing such as decoding) is additionally performed.

That is, in each case, an image signal based on encoding and decoding obtained by decoding an MPEG-encoded signal is compared with input image data, and a luminance signal and a color difference obtained by the comparison are compared. Each differential signal of the signal is transmitted by being DPCM-encoded.

Next, the operation of the extended image encoding apparatus 10 thus performed will be further described. FIG. 2 shows a DPC for DPCM encoding a difference signal related to a luminance signal.
3 shows an M code table.

In the table, DP for the range of the difference value
The figure shows the assignment of CM codes. That is, the difference image data obtained from the adder 14 is obtained from the upper left to the lower right with respect to the screen in the order in which the raster scan is performed. It is calculated as a value.

Then, the initial value of the first pixel is set to 0,
The difference from the difference value with respect to 0 is calculated and calculated in the horizontal direction, and a DPCM code is assigned according to the range of the difference value. The code is assigned to one line of pixels on the screen. Performed on data.

FIG. 3 shows the difference signal relating to the color difference signal
4 shows a DPCM code table for performing M encoding. In the table, since the color difference signal has a smaller horizontal frequency band than the luminance signal, a different DPCM code is assigned to the DPCM code for the color difference signal for a low-level change.

In the code tables for both the luminance signal and the chrominance signal, when the DPCM code is 1111, it is defined as ZeroRunESC, and in that case, the 0 data which occurs most frequently in the DPCM value of the difference image is determined. This code is used when transmitting a run length that continues with 0 without transmitting all. That is, a maximum of 8 bits (256
The number of bits of a pixel is transmitted, and the number information relating to a plurality of 0s is transmitted.

The DPCM code relating to the difference signal is −255 to
The 9-bit pixel data of +255 is converted into a 4-bit value based on these tables. However, in the case of a difference signal relating to an ordinary image, many zeros are generated. By performing zero-run length coding on the code, it is possible to obtain a shorter compressed code.

The data thus DPCM-encoded is supplied to the user data inserter 17, to which MPEG-encoded data delayed by a predetermined time by the delay unit 15 is supplied. However, DPCM encoded data of a differential image corresponding to the picture is inserted into a user data recording area in a picture data area having the same temporal relationship as the encoded picture.

FIG. 4 shows the syntax of a video layer in MPEG2. In the figure, the order of data for generating a bit stream related to an MPEG2 video signal is defined.

That is, first, a picture start code picture_start_code of 32 bits is described, and then a 10-bit I
(Intra-coded), P (Predictive-coded), and B (B
idirectionally predictive-coded) The information related to the transmission and display order of the picture is transmitted by Temporal_reference, and the information related to the image type such as I, P, and B pictures by 3 bits is transmitted by picture_coding_type. A stream is created.

Then, a 32-bit user data start code User_data_st at the end of this syntax
The insertion description of the DPCM encoded data is performed using 8-bit user_data described after the art_code.

That is, user_data () is user_start # co
The description is started from a byte-aligned start code for determining the position de, and it is defined so that the user data can be continuously described until the next 3-byte code of 0x000001 is received.

In this way, by describing DPCM encoded difference image data in the user data area of the video signal layer, image difference information is transmitted as data of the same hierarchy as the encoded image data, or the image difference information is transmitted. The signal can be recorded on a medium.

Since the user data area is a data area defined and permitted to be used by the user, user_data () is used by another user or another application, for example. Since there is a possibility, after user_start # code of user # data (), after describing about 4 bytes of identification code or unique code 0x22220204 indicating that it is differential image data of this method, For example, the codes of a DPCM-coded luminance signal, a color difference Cb signal (blue signal-luminance signal), and a color difference Cr signal (red signal-luminance signal) are sequentially described.

Thus, the signal encoded by the MPEG encoder 12 and the encoded signal are decoded,
A signal obtained by DPCM-encoding a signal having a difference from an input signal can be generated and transmitted as encoded data inserted in a user data area of the same image layer signal area.

Next, decoding of the encoded data generated and transmitted in this manner will be described. FIG. 5 shows a configuration of a normal image decoding device that decodes normal encoded data. The normal image decoding device 30 shown in FIG.
It is constituted by a decoder 32.

The decoding of the coded data by the MPEG decoder 32 is performed without detecting the user data described and transmitted, that is, in order to decode the coded data in such a manner that the user data is skipped. Since only the encoded data encoded by the encoder 12 is decoded, the ordinary image decoding device 30 obtains ordinary decoded data obtained by decoding the ordinary encoded data.

Next, a first embodiment of an extended image decoding apparatus for decoding extended user data, which is high-quality image data, by decoding user data inserted and transmitted in normal encoded data. State. FIG. 6 shows the configuration of the extended image decoding apparatus.

The extended image decoding device 30a shown in FIG.
Data input terminal 31, MPEG decoder 32a, user data separator 33, delay unit 34, DPCM decoder 3
5 and an adder 36.

Next, the operation of the extended picture decoding apparatus 30a thus configured will be described. First, the encoded data supplied to the data input terminal 31 is output to the MPEG decoder 3.
2a, where normal decoded data is obtained. The obtained decoded data is supplied to a delay unit 34, and the encoded data including user data is supplied to a user data separator 33. .

The user data separator 33 detects User_data_start_code in the syntax of the picture data in the supplied coded data, and detects the User_d
Unique code 0x22 transmitted after ata # start # code
220204 is detected.

If the unique code 0x22220204 is not detected, it is assumed that the user data transmitted thereafter is data that does not need to be decoded, and the user data is skipped.

When the unique code is detected,
It is determined that the differential image data for obtaining the extended image is described in the extended encoded data for improving the image quality, and the transmitted DPCM encoded data is supplied to the DPCM decoder 35.

The DPCM decoder 35 decodes the supplied DPCM coded data to obtain a luminance signal and a color difference Cb.
A signal and a chrominance Cr signal are obtained, and the obtained signals are used to decode a DPCM code based on the DPCM coding tables shown in FIGS. 2 and 3 to obtain respective difference image data. .

The obtained difference image data is supplied to one input terminal of an adder 36, and the other input terminal is provided with a normal MPE delayed by a predetermined time by a delay unit 34.
G-decoded image data is supplied, and the respective image data supplied to the two input terminals are added to obtain extended decoded image data. The obtained extended decoded image data is output from the extended image decoding device. It is supplied as an output signal from 30a.

As described above, the first embodiment for obtaining the extended decoded image data by decoding the encoded data generated by the extended image encoding device for transmitting the differential image data by DPCM encoding and transmitting the decoded image data by the extended image decoding device. As described above, a second embodiment in which differential image data is encoded by a different MPEG encoder and transmitted will now be described.

In the extended image encoding apparatus according to the second embodiment, the differential image data is DPCM-encoded and transmitted in the apparatus shown in the first embodiment. In this embodiment, an orthogonal transform technique is applied. This is different in that the encoding is performed according to the MPEG method, and will be described next with reference to the first embodiment. FIG. 7 shows a configuration of an extended image encoding apparatus equipped with an extended image encoding method according to a second embodiment, which will be described below with reference to the drawings.

The extended image encoding device 10a shown in FIG.
Image data input terminal 11, first MPEG encoder 12
a, MPEG decoder 13, adder 14, delay unit 15,
It comprises a fixed code amount indicator 18, a second MPEG encoder 19, and a user data inserter 17.

Next, the operation of the extended image encoding apparatus 10a thus configured will be described. First, the input image data is supplied to the first MPEG encoder 12a,
One of the encoded signals obtained by encoding according to the main profile of the EG is supplied to the delay unit 15, and the other is supplied to the MPEG decoder 13 to be decoded.

The decoded signal is supplied to one input terminal of the adder 14 for performing a polarity inversion operation, and the other input terminal is supplied with input image data. A signal representing the difference between the signals supplied to the two input terminals is generated as difference image data, and the generated signal is controlled by a fixed code amount indicator 18 to operate a second MPEG encoder 19 whose code amount is controlled. Supplied to

In the second MPEG encoder 19, the supplied differential image data is encoded for each macroblock in the same manner as in the conventional MPEG, and the encoded signal obtained is encoded as a user data. It is supplied to one input terminal of the inserter 17.

A signal encoded by the first MPEG encoder 12a delayed by the delay unit 15 is supplied to the other input terminal of the user data inserter 17,
The signal encoded by the second MPEG encoder 19 is inserted into the user data recording portion of the signal, and the encoded data obtained by the insertion is extended image encoding device 10a.
As an output signal.

As described above, the differential image data is
The data is supplied as encoded data that has been encoded according to the G method and inserted into the user data area. The method of inserting and describing the differential image data into the user data area is the same as that described in the first embodiment.
This is performed in the same manner as in the embodiment.

When the encoded data transmitted in this manner is decoded by using the ordinary decoder as shown in FIG. 5, the MPEG decoder 32 records the user data. Since the decoding is performed each time the encoded data recorded in the area is detected, the normal image decoding device 30
Can obtain image data obtained by decoding ordinary encoded data.

Next, to obtain extended decoded data for obtaining high-quality decoded image data by decoding the difference image data,
A second embodiment of the extended image decoding device will be described. FIG. 8 shows the configuration of the extended image decoding device 30b, and its operation will be described.

The extended image decoding device 30b shown in FIG.
It comprises a data input terminal 31, a first MPEG decoder 32b, a user data separator 33, a delay unit 34, a second MPEG decoder 37, and an adder 36.

The encoded data supplied to the data input terminal 31 of the extended picture decoding apparatus 30b thus configured is obtained as ordinary decoded data by the first MPEG decoder 32b, and the obtained decoded data is obtained. The data is supplied to the delay unit 34, and the encoded data including the user data is supplied to the user data separator 33.

The user data separator 33 detects User_data_start_code in the syntax of picture data in the supplied coded data, and detects the User_d
Unique code 0x22 transmitted after ata # start # code
If the unique code is not detected, the user data to be transmitted thereafter is skipped.

When the unique code is detected, the extended coded data transmitted subsequent to the unique code is supplied to the second MPEG decoder 37, and supplied to the second MPEG decoder 37. By decoding the encoded data, a luminance signal for obtaining an extended image and difference image data relating to two types of color difference signals are obtained.

The obtained difference image data is supplied to one input terminal of an adder 36, and the other input terminal is provided with a normal MPE delayed by a predetermined time by a delay unit 34.
G-decoded image data is supplied, and the respective image data supplied to the two input terminals are added to obtain extended decoded image data. The obtained extended decoded image data is output from the extended image decoding device. It is supplied as an output signal from 30b.

As described above, the second embodiment in which the coded data generated by the extended image coding apparatus for transmitting the differential image data by MPEG coding is decoded by the extended image decoding apparatus to obtain the expanded decoded image data After that, a case where the difference image data is encoded by a different image format and transmitted will be described.

The difference image in the case of the different image format is an image signal obtained by encoding and decoding in an image format in which the relationship between two chrominance signal pixel samples with respect to the luminance signal is expressed as 4: 2: 0. And a case where a signal of a difference between the image signal having the image format of 4: 2: 2 is transmitted as a differential data image in the user data area.

FIG. 9 shows the relationship between the number of pixels of a luminance signal and the number of pixels of a chrominance signal in a 4: 2: 0 image format. In the figure, an area of 720 × 480 pixels shown as Y indicates the number of pixels for the luminance signal, and 36 shown as U.
In the area of 0 pixels × 240 pixels, the number of pixels for the BY color difference signal obtained by subtracting the luminance signal Y from the blue signal B is
The area of 360 pixels × 240 pixels indicated by V indicates the number of pixels for the RY color difference signal obtained by subtracting the luminance signal Y from the red signal R.

FIG. 10 shows the relationship between the number of pixels of a luminance signal and the number of pixels of a chrominance signal in a 4: 2: 2 image format. In the figure, an area of 720 pixels × 480 pixels shown as Y indicates the number of pixels for the luminance signal, and 3 shown as U
The area of 60 pixels × 480 pixels ranges from the blue signal B to the luminance signal Y.
Is the number of pixels for the BY color difference signal obtained by subtracting, and the area of 360 pixels × 480 pixels indicated by V is the pixel for the RY color difference signal obtained by subtracting the luminance signal Y from the red signal R. Indicates a number.

As described above, in the image format of 4: 2: 0 and the image format of 4: 2: 2, the number of pixels for the luminance signal is 720 pixels × 480 pixels, which is the same, but BY , And R-Y, the number of pixels in the vertical direction is twice as large as 360 × 480 pixels, ie, 360 × 240 pixels.

As described above, the image quality of the 4: 2: 0 image format is lower than the image quality of the 4: 2: 2 image format in the vertical resolution of the color difference signal. For example, a thin line extending in the horizontal direction is used. If the line is a line that can be represented by a white or black luminance signal, it can be represented at the same resolution in both image formats, but a horizontal line with high saturation such as blue or red In this case, the resolution in the vertical direction is different, so that the image cannot be sufficiently expressed in the case of the 4: 2: 0 image format.

In the case of an image obtained by photographing a landscape or the like with a camera, such a high resolution in the vertical direction is not so required. However, an artificially produced image and a computer generated image, particularly a color character In the case of an image that includes, a resolution for a color difference signal with high saturation is required, and an encoding method using a 4: 2: 2 image format is more preferable.

Next, with respect to the encoded data encoded in the 4: 2: 0 image format, the difference image data from the image obtained by encoding in the 4: 2: 2 image format is stored in the user data area. An example of the case where transmission is performed is described.

The extended image encoding apparatus according to the third embodiment is configured to transmit the difference data by DPCM encoding.
In the first embodiment, image data obtained by subtracting an image in the 4: 2: 0 image format from an image in the 4: 2: 2 image format is transmitted as difference image data. This will be described with reference to the embodiments.

FIG. 11 shows the configuration of an extended image encoding apparatus equipped with an extended image encoding method according to the third embodiment, and its operation will be described. The extended image encoding device 10b shown in FIG. 1 includes a data input terminal 11, a 4: 2: 0 converter 21, an MPEG encoder 12, and an MPEG decoder 1.
It comprises a 3, 4: 2: 2 converter 22, a color difference data differentiator 23, a delay unit 15, a DPCM encoder 16 and a user data inserter 17.

Next, the operation of the extended image encoding device 10b thus configured will be described. First, input terminal 1
The input image data of 4: 2: 2 image format supplied to 1 is supplied to a color difference data differentiator 23 and a 4: 2: 0 converter 21.

In the 4: 2: 0 converter 21, the supplied image data in the 4: 2: 2 image format is 4:
Image format conversion is performed on data in the 2: 0 image format by vertical image filtering or the like.

The image data thus converted into the signal of the 4: 2: 0 image format is supplied to the MPEG encoder 1
The encoded data is supplied to the delay unit 15 and the MPEG decoder 13 by performing encoding in accordance with, for example, the specification of a main profile, which is usually performed, to obtain encoded data.

That is, one of the encoded signals obtained by encoding according to the MPEG main profile is
And a delay process for time alignment is performed, and the other one of the encoded signals is supplied to an MPEG decoder 13 to be decoded.

The MPEG decoder 13 decodes the image data to obtain the image data in the 4: 2: 0 image format, and supplies the image data to the 4: 2: 2 converter 22, where the 4: 2: 2 image data is output. In the 2 converter 22, the signal supplied in the 4: 2: 0 image format is converted into a signal in the 4: 2: 2 image format and is obtained as a signal in the 4: 2: 2 image format.

The 4: 2: 2 image format signal is a signal having the number of pixels corresponding to the 4: 2: 2 image format, but the information amount of the signal is 4: 2: 2. 0 has the same resolution as a signal of the image format, and has a low resolution for a color signal having high saturation in the vertical direction.

The signal thus converted into the 4: 2: 2 image format is supplied to the color difference data differentiator 23, where the difference image data from the supplied input image data is obtained by calculation, and the obtained difference image data is obtained. The obtained difference image data is supplied to the DPCM encoder 16, where the DPCM encoded data is generated in the same manner as described in the first embodiment.

The generated DPCM coded data is supplied to the user data inserter 17, and the supplied coded data is provided with a predetermined delay time by the delay unit 15 and has the same time relationship as the MPEG code. Is inserted into the user data area of the encoded data generated by the converter 13.

The DPCM encoded data inserted in this manner is difference image data obtained by subtracting image data in the 4: 2: 0 image format from image data in the 4: 2: 2 image format. Can be used to improve the resolution of 4: 2: 0 image format image data with reduced vertical resolution.

It should be noted that the data encoding and decoding of the 4: 2: 0 image format in the generation of the differential image data for improving the resolution is performed by the final encoding in the encoder 12 obtained from the MPEG encoder 12. Using a method of supplying the bit stream data in the stage to the MPEG decoder 13 for decoding, and data obtained in the middle stage where the signal processing up to the quantization is performed in the MPEG encoder 12,
There is a local decoding method that starts decoding from inverse quantization, and the encoding and decoding may be performed by any method.

In this manner, the difference image data between the image data in the 4: 2: 2 image format and the image data in the 4: 2: 0 image format is encoded data transmitted by the DPCM encoding and transmitted in the user data area. The method of inserting the difference image data into the user data area and the description method are performed in the same manner as in the first embodiment, and the coded data thus generated is used as the transmission signal. Made to be supplied.

In the extended image coding apparatus shown in this embodiment, the same operation can be performed by modifying the circuit configuration for generating the difference image. FIG.
Next, a configuration of an extended image encoding device obtained by modifying the third embodiment is shown.

In the extended image encoding device 10c shown in FIG. 11, the supplied image data of the 4: 2: 2 image format is supplied to the color difference data differentiator 23 and the 4: 2: 0 converter 21. The signal converted into the image format by the 4: 2: 0 converter 21 is output to the MPEG encoder 12 and the 4: 2: 0 converter.
It is supplied to a 2: 2 converter 22.

The signal supplied from the 4: 2: 2 converter is supplied to the color difference data differentiator 23 in the same manner as in the above-described extended image encoding device 10b, and thereafter the same processing operation is performed. However, the signal processing operation is different in that difference image data including a correction image signal for quantization noise generated at the time of encoding by the MPEG encoder 12 is not generated, but on the other hand, the difference image data is used as an information amount. The entropy of the data has a low value.

It is possible to reduce the number of codes required for coding the differential image data generated in this way, thereby reducing the number of arithmetic operations for coding in a coder arranged at a subsequent stage. In addition, since the amount of code transmitted by being inserted into the user data area can be reduced, the increase in the amount of code of the entire encoded data is small by inserting the difference image data into the user data area, and decoding of the difference image data can be performed. It is possible to reduce the load of signal processing on a user who performs normal decoding that is not performed.

As a method of reducing the burden on the user who normally performs decoding, the configuration of the 4: 2: 0 converter 21 is considered. That is, the image data in the 4: 2: 2 image format supplied to the 4: 2: 0 converter 21 needs to be converted into a signal of the image data in the 4: 2: 0 image format, and an image signal for that purpose is required. Is performed by, for example, adding two lines of color difference signals to match the characteristics of both image signals.

The change of the image format by the oversampling of the thus filtered signal in the 4: 2: 2 converter 22 in the vertical direction of the chrominance signal is performed, for example, by adding the chrominance signals of two scanning lines. 2
Is performed by interpolation calculation such as dividing by.

The reason for using such an interpolation calculation method is that the characteristics of the chrominance signal must be the same as those of the local decoder incorporated in the MPEG encoder 12, and the difference image signal When the difference data of the chrominance signal added as と し て does not have the same characteristics as the chrominance signal generated by the normal encoder, for example, DP
Obtaining an image signal having characteristics based on the 4: 2: 2 image format even when there is no deterioration due to encoding and decoding by a CM code, a decoder, or a second MPEG encoder and a decoder. Is no longer possible.

Therefore, the characteristics of the 4: 2: 2 converter 22 used in the extended image encoding device and the 4: 2: 2 converter 38 used in the extended image decoding device are different from each other in the vertical direction for the same color difference signal. Using the directional oversampling method, the image data portion encoded by the 4: 2: 0 image format has the same characteristics as the image data portion encoded by the conventional main profile,
The characteristic improvement of the image data having the same characteristic is obtained by decoding the signal transmitted in the user data area.

Next, an extended image decoding apparatus for decoding the encoded data generated by the extended image encoding apparatus shown in FIG. 11 or 12 will be described. FIG. 13 shows an example in which the coded data transmitted in such a manner is decoded by an ordinary MPEG decoder.

In the figure, when the supplied encoded data is generated by using the ordinary MPEG decoder 32 similar to that shown in FIG. 5, the encoding of the 4: 2: 0 image format is performed. Since the MPEG decoder 32 that decodes the signal performs the encoding without detecting the encoded data recorded in the user data recording area, the ordinary encoded data is decoded from the normal image decoding device 30. 4: 2:
Image data in the 0 image format is obtained.

A fourth embodiment of the extended image decoding apparatus for decoding difference image data according to the 4: 2: 2 image format to obtain high-quality decoded image data with excellent vertical resolution will be further described. FIG. 14 shows the configuration of the extended image decoding device 30c, and its operation will be described.

The extended image decoding device 30c shown in FIG.
A data input terminal 31, an MPEG decoder 32a, a user data separator 33, a delay unit 34, a DPCM decoder 35,
It comprises a 4: 2: 2 converter 38 and a color difference data adder 39.

The coded data supplied to the data input terminal 31 of the extended picture decoding apparatus 30c thus configured is decoded by the MPEG decoder 32a to obtain ordinary decoded data. The decoded data is supplied to the delay unit 34, and the encoded data including the user data is supplied to the user data separator 33.

The user data separator 33 detects User_data_start_code in the syntax of picture data in the supplied coded data, and
Unique code 0x transmitted after # data # start # code
If 22220204 is detected, but the unique code is not detected, the user data transmitted thereafter is skipped.

When the unique code is detected, the extended coded data transmitted following the unique code is supplied to the DPCM decoder 35,
The PCM decoder 35 decodes the supplied encoded data in the same manner as the extended image decoding device 30a shown in FIG. 6 described above, thereby increasing the vertical resolution of the 4: 2: 2 image format. Difference image data for correction is obtained for two types of color difference signals.

The obtained difference image data is supplied to one input terminal of the color difference data adder 39,
The other input terminal is supplied with the normal MPEG decoded image data delayed by a predetermined time by the delay unit 34, and the respective image data supplied to the two input terminals are added to increase the vertical resolution of the color signal. Improved extended decoded image data is obtained, and the obtained extended decoded image data is supplied as an output signal from the extended image decoding device 30c.

The difference image data between the image data transmitted in the 4: 2: 2 image format and the image data transmitted in the 4: 2: 0 image format is DP
The CM-encoded and inserted into the user data area of the bit stream and transmitted, and the transmitted 4: 2:
The image data transmitted in the 0 image format is decoded, and the differential image data for obtaining the image data transmitted in the 4: 2: 2 image format, which is transmitted in the DPCM encoding in the user data area, is decoded. The obtained difference image data is used to obtain image data with improved resolution.

The difference image signal for decoding and obtaining the image quality in the 4: 2: 2 image format is DP
In addition to being encoded and transmitted by the CM encoding, the encoded signal is generated by using the second MPEG encoder shown in the second embodiment, and further by another orthogonal transform method. The same extended image encoding device and extended image decoding device can be configured by using complementary techniques on the encoding side and the decoding side, such as the encoding method.

Further, the extended image encoding device and the extended image decoding device configured as described above can transmit and receive a high-quality image signal between the corresponding devices. Then, when the transmission of the differential image data is performed by a different method, by using different unique codes as identification codes, it is possible to receive and transmit signals generated and transmitted by an encoding device and a decoding device having the same characteristics. Decryption is enabled.

[0149] Then, by using these encoding and decoding devices, an extended image encoded signal can be recorded on a recording medium, and the recorded signal can be reproduced by the extended image decoding device. As the recording medium, there are a case where a disk-type recording medium such as a RAM-type DVD is used and a case where a tape-type recording medium such as a digital video recorder is used.

In particular, a recording signal using a recording apparatus using a digital video tape, for example, a D-VHS recording signal has a fixed transfer rate, and has high applicability especially when recording is performed at a fixed rate.

That is, although the transfer rate in the standard recording mode in the case of the D-VHS is about 14 Mbps, the transfer rate of about 7 to 9 Mbps is used when compression encoding of a normal NTSC moving image by the MPEG method is performed. Depending on the rate, video signals can be recorded with extremely high quality.

In this case, about 5 to 7 Mbps obtained by subtracting the transfer rate from 14 Mbps is a surplus area where no signal is recorded. By recording the above-mentioned user data in the surplus area, for example, recording is performed in the main profile. 4: 2: 0 for 4: 2: 0 image format
By recording a difference image from an image in the 2 to 4: 4: 4 image format, it is possible to reproduce image information obtained by expanding the pixel format of the color difference signal.

That is, an MPEG data recording device equipped with an extended image encoding device therefor comprises an MPEG encoder, a difference data generator for decoding the encoded data and generating difference data from the original image, and A user data recorder for recording the difference data in the user data in the stream to be encoded by the encoder.

Further, the MPEG data recording device comprises an MPEG encoder, a difference data generator for decoding the encoded data and generating difference data from the original image,
There is a method of including a user data recorder for recording difference data together with a predetermined unique code in user data of a video stream.

Further, in the MPEG data reproducing apparatus, the MPEG decoder and the difference data recorded together with the predetermined unique code in the user data of the video stream are obtained, and the decoded data is decoded by the MPEG decoding unit. And a difference data adder for adding the difference data to the data.

The MPEG data recording apparatus and the MPE equipped with the MPEG data reproducing apparatus thus configured
Using a G data recording / reproducing apparatus, a video tape recorded by the user with the apparatus can be reproduced with high quality by the apparatus, and the recorded tape can be reproduced by another reproducing apparatus which does not support differential image signal reproduction. Even in the case of reproduction by standard recording, standard reproduction is performed by a normal recording / reproducing function.

The standard recording / reproducing function is such that, in the case of encoding according to the MPEG main profile, transmission or recording is performed in the 4: 2: 0 image format, and the data transmitted or recorded in the user data area is 4: 4: 0. A signal obtained by encoding a difference image signal between a chroma signal of 2: 2 original image data and a chroma signal of 4: 2: 0, which is MPEG-decoded, is used for normal reproduction in a 4: 2: 0 image format, and in a differential reproduction mode. When an image signal can be decoded, an image signal having a high color signal resolution is reproduced in a 4: 2: 2 image format.

As described above, in the embodiment shown here, when a predetermined unique code and differential image information are recorded in the user data area of a video signal video stream encoded by the MPEG system, the bit stream itself is encoded. As the video encoding standard data to be converted, difference data information for improving image quality is recorded only in the user data area.

Although a digital VTR using fixed rate recording has been described as an example of the recording medium, the recording medium is a VT.
Not limited to R tape, for example, DVD-RW, DVD
-A recording medium using a RAM may be used.

Even in the case of such a disk-type recording medium, if the data capacity of the recorded program is smaller than the entire recording capacity, the difference image data is recorded in the user area in the bit stream. Alternatively, difference image data is recorded in a free area.

Since the recording in the user data area is performed together with the unique code, the signal recorded in the user area is reproduced while skipping the read even in the case of the reproduction by the DVD player whose reading speed is slow. Reproduction with a normal main profile is enabled.

As described above, examples of application to VTRs and DVDs have been described. This method can be applied to, for example, an image transmission system using a high-speed Internet line. That is, when a high transmission speed can be secured, differential image data for high image quality is transmitted.

As described in detail above, an extended image encoding device equipped with an extended image encoding method is a coded bit stream based on an international standard such as MPEG. In addition to the above, an image signal having a hierarchical structure without changing a predetermined application standard or a transmission format specified by the standard is provided in, for example, a user data recording area permitted to be inserted into the bit stream. It can generate a bit stream for transmitting image data, and can transmit image data with a higher hierarchical structure while maintaining full compatibility with existing image coding systems that are taking root in the market. A system that increases scalability can be configured.

Furthermore, since a single video stream can be given a hierarchical structure, a decoder can be designed with a simple configuration, so that it is possible to implement functional enhancement such as high image quality at a low price.

In particular, in the case of a recording medium at a fixed transfer rate, where the recording rate has more margin than the original recording data, transmission or recording of a high-quality video signal is performed by the method described in the above embodiment. , And a video signal with high chroma and high definition that extends the image format of the color signal from 4: 2: 0 to 4: 2: 2.

In the third embodiment described above, 4:
Obtaining a signal of a difference between an image signal obtained by decoding an image signal encoded in a 2: 0 image format and input image data supplied in a 4: 2: 2 image format;
The method of encoding the obtained difference signal and generating it as an encoded signal inserted into the user data area has been described, but the image format is limited to the 4: 2: 0 format and the 4: 2: 2 format. The image format used for the main coded data is, for example, the 4: 2: 0 system, for example, the 4: 1: 0 system, the 4: 1 :: 1 system, and the like, and is used as a differential image signal in the user data area. The image format to be used is, for example, 4: 2: 2 system and, for example, 4: 2
A signal format of an image format that gives higher resolution, such as a 4: 4 system, can be used.

In any case, an image format used as a difference image signal in the user data area uses a system having a higher resolution, such as a color difference signal, than the image format used for main encoded data.

[0168] The case where DPCM is used as the coding system of the coded signal transmitted in the user data area and the case where the coding system specified by MPEG is used have been described, but the coding system is not limited thereto. Never
Other encoding methods may be used as long as the encoding is performed with high efficiency on the difference image signal and the encoding and decoding can be performed relatively easily.

[0169]

According to the first aspect of the present invention, the supplied image signal is subjected to orthogonal transform and variable-length coding to obtain a compressed coded signal used as a standard, and the obtained compressed coded signal is obtained. And a supplied image signal to obtain a differential image signal. The obtained differential image signal is compression-encoded and inserted into a user data recording area of the compression-encoded signal to obtain an extended image encoded signal. Therefore, it is possible to obtain a decoded image signal with a standard image quality by standard decoding of the extended image coded signal and to decode the differential image signal transmitted in the user data area. When decoding is performed by a decoding device, there is an effect that a method of encoding an extended image signal that can obtain a higher-quality decoded signal can be provided.

According to the second aspect of the present invention, in particular, a predetermined unique code is added as an identification signal to the differential image signal transmitted in the user data area and inserted into the user data recording area. Claim 1
In addition to the effect described above, since the presence or absence of a differential image signal transmitted by a predetermined method in the user data area can be easily identified, when the target differential image data is inserted in the user data area, the signal is output. Has an effect of providing a method of encoding an extended image signal that can obtain a higher-quality decoded signal.

According to the third aspect of the present invention, in particular, a user of an encoded signal encoded by a 4: 2: 0 image format in which a differential image signal transmitted in a user data area is defined as a main profile. In the data recording area, a differential image signal corresponding to a signal encoded with a higher quality image such as a 4: 2: 2 image format is encoded, inserted, and transmitted. Using a decoding device that can decode the differential image signal transmitted in the user data area, there is an effect that an extended image signal encoding method that can obtain a high-quality image signal with a higher color signal resolution can be provided.

According to the fourth aspect of the present invention, an image signal is subjected to orthogonal transform and variable-length encoding to decode the compressed encoded signal into a normally used compressed encoded signal with high image quality. To obtain an extended coded signal that is compressed and coded and inserted and supplied to a user data recording area of the compressed coded signal, and decodes the obtained compressed coded signal of the extended coded signal. Obtaining a decoded image signal,
The differential image signal inserted into the user data area of the compressed and coded signal and transmitted is decoded to obtain a decoded differential image signal, and the obtained decoded image signal and the decoded differential image signal are added. Is performed, there is an effect that a method of decoding an extended image signal that can obtain a decoded image signal of higher image quality can be provided.

According to the fifth aspect of the present invention, the difference image signal transmitted in the user data area is defined as the main profile of the standard coding method.
In the user data recording area of the coded signal coded according to the 0 image format, a differential image signal corresponding to a signal coded with a higher quality image such as a 4: 2: 2 image format is coded, inserted, and transmitted. Since an encoded signal is obtained and decoded, in addition to the effect of claim 4, a method of decoding an extended image signal that can be obtained by decoding a high-quality image signal with a higher color signal resolution is provided. There are effects that can be provided.

According to the sixth aspect of the present invention, the supplied image signal is subjected to orthogonal transform and variable-length coding to obtain a compressed coded signal used as a standard, and the obtained compressed coded signal is obtained. And the supplied image signal are calculated to obtain a differential image signal, and the obtained differential image signal is compression-encoded and inserted into a user data recording area of the compression-encoded signal to obtain an extended image-encoded signal. As a result, it is possible to obtain a decoded image signal with standard image quality by standard decoding of the extended image coded signal, and to decode the differential image signal transmitted in the user data area. In the case where decoding is performed by a decoding device having the same, there is an effect that a configuration of an extended image signal encoding device capable of obtaining a decoded signal with higher image quality can be provided.

According to the seventh aspect of the present invention, in particular, the differential image signal transmitted in the user data area is encoded by a 4: 2: 0 image format defined as a standard main profile. 7. In addition to the effect of claim 6, a differential image signal corresponding to a signal encoded with a higher quality image such as a 4: 2: 2 image format is encoded, inserted and transmitted in a user data recording area of the signal. When the differential image signal transmitted in the user data area can be decoded, there is an effect of providing a configuration of an extended image signal encoding device capable of obtaining a high-quality image signal with a higher color signal resolution.

According to the present invention, an image signal is orthogonally transformed and variable-length coded to decode the compressed coded signal into a normally used compressed coded signal with high image quality. To obtain an extended coded signal that is compressed and coded and inserted and supplied to a user data recording area of the compressed coded signal, and decodes the obtained compressed coded signal of the extended coded signal. Obtaining a decoded image signal,
The differential image signal inserted into the user data area of the compression-encoded signal and transmitted is decoded to obtain a decoded differential image signal, and the obtained decoded image signal and decoded differential image signal are added. Is performed, there is an effect that a configuration of an extended image signal decoding device capable of obtaining a decoded image signal with higher image quality can be provided.

According to the ninth aspect of the present invention, the supplied image signal is subjected to orthogonal transform and variable-length encoding to obtain a standard encoded compression encoded signal, and the obtained compressed encoded signal is obtained. And the supplied image signal is obtained to obtain a differential image signal, and the obtained differential image signal is compression-encoded and inserted into a user data recording area of the compression-encoded signal to obtain an extended image encoded signal. And so on, the extended image decoded signal obtained as described above is recorded on a recording medium, and the recording medium decodes the extended image encoded signal with a standard image quality by a standard decoding device. Extended image recording in which a coded extended image signal for obtaining a higher quality decoded signal is recorded for a decoding device having a function of decoding a differential image signal transmitted in a user data area. Medium There is an effect that can offer.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an extended image encoding device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating D related to a luminance signal according to the first embodiment of the present invention.
It shows a PCM code table.

FIG. 3 is a diagram illustrating D according to a color difference signal according to the first embodiment of the present invention.
It shows a PCM code table.

FIG. 4 shows an encoding syntax of a video layer defined by MPEG2.

FIG. 5 is a diagram for describing decoding of encoded data generated by an extended image encoding device by a normal image decoding device.

FIG. 6 is a configuration diagram of an extended image decoding device according to the first embodiment of the present invention.

FIG. 7 is a configuration diagram of an extended image encoding device according to a second embodiment of the present invention.

FIG. 8 is a configuration diagram of an extended image decoding device according to a second embodiment of the present invention.

FIG. 9 is a diagram showing the relationship between the number of pixels of a luminance signal and the number of pixels of a color difference signal in a 4: 2: 0 image format.

FIG. 10 is a diagram illustrating a relationship between the number of pixels of a luminance signal and the number of pixels of a color difference signal in a 4: 2: 2 image format.

FIG. 11 is a configuration diagram of an extended image encoding device according to a third embodiment of the present invention.

FIG. 12 is a modified configuration diagram of the extended image encoding device according to the third embodiment of the present invention.

FIG. 13 is a diagram for describing decoding of encoded data generated by an extended image encoding device by a normal image decoding device.

FIG. 14 is a configuration diagram of an extended image decoding device according to a third embodiment of the present invention.

FIG. 15 is a diagram showing a configuration of a conventional MPEG encoder.

FIG. 16 is a diagram showing a configuration of a conventional MPEG decoder.

[Explanation of symbols]

 10, 10a, 10b, 10c Extended image encoding device 11 Data input terminal 12 MPEG encoder 12a First MPEG encoder 13 MPEG decoder 14 Adder 15 Delayer 16 DPCM device 17 User data inserter 18 Fixed code amount Indicator 19 Second MPEG encoder 21 4: 2: 0 converter 22 4: 2: 2 converter 23 Color difference data differencer 30 Normal image decoding device 30a, 30b, 30c Extended image decoding device 31 Data input terminal 32 , 32a MPEG decoder 32b First MPEG decoder 33 User data separator 34 Delayer 35 DPCM decoder 36 Adder 37 Second MPEG decoder 38 4: 2: 2 converter 39 Color difference data adder 50 MPEG code Quantizer 51 input terminal 52 adder 53 DCT unit 54 quantizer 55 VL Unit 56 buffer 57 transform code amount controller 61 inverse quantizer 62 inverse DCT unit 63 adder 64 image memory 65 motion compensation predictor 70 MPEG decoder 71 data input terminal 72 buffer 73 VLD unit 74 inverse quantizer 75 inverse DCT unit 76 adder 77 image memory 78 motion compensation predictor

 ──────────────────────────────────────────────────続 き Continuing on the front page F term (reference) 5C057 AA08 AA09 DA06 EA02 EA06 EM09 EM11 EM13 EM16 FB03 5C059 MA00 MA23 MA35 MC11 MC31 ME01 ME05 PP05 PP06 PP07 PP16 SS01 SS06 SS13 SS20 UA02 UA05 UA38 5A0BA BC01 BA04 BD02 BD03

Claims (9)

[Claims] A first step of orthogonally transforming a supplied image signal to obtain a converted image signal, and a variable length coding of the obtained converted image signal to obtain a compressed coded signal; A second step of decoding a signal or the converted image signal to obtain a decoded image signal; a third step of calculating the decoded image signal and the supplied image signal to obtain a difference image signal; A fourth step of compressing and encoding the differential image signal to obtain a differential image compression-encoded signal; and inserting the differential image compression-encoded signal into a user data recording area of the compression-encoded signal to extend the extended image encoded signal. 5. A method for encoding an extended image, comprising: at least: 2. The apparatus according to claim 1, wherein a predetermined unique code is added as an identification signal to the user data recording area immediately before the differential image compression encoding signal in the fifth step. A method for encoding an extended image. 3. The converted image signal obtained in the first step is a signal in the 4: 2: 0 image format, and the difference image signal obtained in the third step is the 4: 2: 0 image signal. 2. The method according to claim 1, wherein the signal is in an image format having a higher color signal resolution than the image format. 4. An image signal supplied is compression-encoded to obtain a first image signal.
, A differential image signal between the decoded signal obtained by decoding the obtained first compressed coded signal and the image signal, and the obtained differential image signal To obtain a second compression-encoded signal. A unique code is added to the head of the obtained second compression-encoded signal to obtain the first compression-encoded signal.
An extended image coded signal obtained by inserting it into the user data recording area of the compressed coded signal is supplied, and the first compressed coded signal of the extended image coded signal is decoded to form a first A first step of obtaining a decoded image signal and decoding a second compressed coded signal transmitted in a user data area of the extended image coded signal to obtain a second decoded image signal; A second step of performing an addition process of the first decoded image signal and the second decoded image signal obtained in step 1 to obtain an extended decoded image. A method for decoding an extended image. 5. The first compressed encoded signal is a signal encoded according to a 4: 2: 0 image format, and the differential image signal has a color signal resolution higher than that of the 4: 2: 0 image format. 5. The extended image decoding method according to claim 4, wherein the signal is a signal encoded in a high image format. 6. A first compression encoding means for orthogonally transforming a supplied image signal to obtain a converted image signal, and performing variable length encoding on the obtained converted image signal to obtain a compressed encoded signal. A compressed coded signal is obtained from the first compression coder, or the converted image signal is obtained from the first compression coder, and at least one of the obtained signals is decoded to obtain a decoded image. A decoding means for obtaining a signal; a calculation means for obtaining a differential image signal by performing a comparison calculation of the decoded image signal obtained by the decoding means with the supplied image signal; A second compression encoding means for compressing and encoding the differential image signal to obtain a differential image compression encoded signal; and a differential image compression encoded signal obtained by the second compression encoding means, User data of encoded signal An extended image encoding device, comprising: a user data inserting unit that obtains an extended image encoded signal by inserting into an area. 7. A compression-encoded signal obtained by said first compression-encoding means is a signal in a 4: 2: 0 image format, and is a differential image compression-encoding obtained by said second compression-encoding means. 7. The extended image encoding apparatus according to claim 6, wherein the signal is a signal in an image format having a higher color signal resolution than the 4: 2: 0 image format. 8. An image signal to be supplied is compression-encoded to obtain a first image signal.
And a difference image signal between the decoded signal obtained by decoding the obtained first compression-coded signal and the supplied image signal, and the obtained difference image signal To obtain a second compression-encoded signal, add a unique code to the head of the obtained second compression-encoded signal, and add a unique code to the user data recording area of the first compression-encoded signal. An extended image coded signal obtained by inserting the supplied extended image coded signal is supplied, and the supplied extended image coded signal is decoded. A first decoding unit that decodes the first compressed encoded signal to obtain a first decoded image signal; and a second decoding unit that decodes a second compressed encoded signal transmitted in the user data recording area. Second decoding means for obtaining a decoded image signal; Of the decoded image signal, said first decoded image signal and the extended image decoding apparatus characterized by configured by including an adding means for obtaining extended decoding image signals, the by adding. 9. An image signal coded by the first coded signal and the code, in a user data recording area of a first coded signal obtained by coding the image signal in a 4: 2: 0 image format. An extended image recording medium for recording a difference image signal relating to a difference from an image signal before conversion, wherein the difference image signal is encoded by an image format having a higher color signal resolution than the 4: 2: 0 image format. An extended image recording medium, which is a second encoded signal.