KR20100115238A - Device and method for codec design - Google Patents

Abstract

PURPOSE: A device and a method for designing a CODEC are provided to decode bit streams encoded in various types by the same information recognition method. CONSTITUTION: A CODEC design device finds the most similar RVC CODEC configuration(1310). The CODEC design device describes bit stream syntax information for a new CODEC(1315). The CODEC design device verifies the stream syntax information(1320). The CODEC design device updates a token pool(1325).

Description

Device and Method for codec design

The present invention relates to an image codec, and more particularly, to a method and apparatus for designing a new codec based on an input bitstream.

In general, a video is converted into a bitstream form by an encoder. In this case, the bitstream is stored according to an encoding type satisfying the constraint of the encoder.

MPEG requires syntax (hereinafter referred to as 'syntax') and semantics (hereinafter referred to as 'semantics') as constraints of the bitstream.

syntax indicates the structure, format, and length of the data, and in what order the data is represented. That is, syntax is to fit a grammar for encoding / decoding, and defines the order of each element included in the bitstream, the length of each element, and the data format.

Semantics means what each bit of data means. That is, semantics indicates what the meaning of each element in the bitstream is.

Therefore, various types of bitstreams may be generated according to encoding conditions of an encoder or an applied standard (or codec). In general, each standard (eg MPEG-1, MPEG-2, MPEG-4, MPEG-4 AVC, etc.) has a different bitstream syntax.

Accordingly, bitstreams encoded according to each standard or encoding condition may have different formats (ie, syntax and semantics), and a decoder corresponding to an encoder should be used to decode the bitstream.

As described above, the conventional bitstream decoder has a limitation of satisfying the constraints of the encoder, and this limitation causes a difficulty in implementing an integrated decoder corresponding to a plurality of standards.

Accordingly, the present invention is to solve the above-described problem, and is encoded in various formats (syntax, semantics) according to each standard (for example, MPEG-1, MPEG-2, MPEG-4, MPEG-4 AVC, etc.) It is an object of the present invention to provide a bitstream decoding method and apparatus capable of decoding a bitstream in the same information recognition scheme.

The present invention also provides a method and apparatus for designing a new codec based on an input bitstream.

According to one aspect of the present invention, a method and apparatus for designing a new codec based on input data are provided.

As described above, the integrated codec apparatus and method according to the present invention are encoded in various formats (syntax, semantics) according to each standard (for example, MPEG-1, MPEG-2, MPEG-4, MPEG-4 AVC, etc.). The decoded bitstream can be decoded with the same information recognition scheme.

In addition, the present invention can design a new codec based on the input bitstream.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term “and / or” includes any combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, preferred embodiments of the integrated codec method and apparatus according to the present invention will be described in detail with reference to the accompanying drawings, and in the following description with reference to the accompanying drawings, overlapping of the same or corresponding components regardless of reference numerals will be described. The description may be omitted.

FIG. 1 is a diagram schematically showing a configuration of a general decoder, and FIG. 2 is a diagram schematically showing a configuration of a general encoder.

As shown in FIG. 1, the MPEG-4 decoder 100 generally includes a variable length decoding unit 110, an inverse scan unit 115, and an inverse DC / AC prediction unit. / AC Prediction (120), Inverse Quantization (125), Inverse Discrete Cosine Transform (Inverse Discrete Cosine Transformation, 130), and Video Restoration (VOP Reconstruction, 135). It is apparent that the configuration of the decoder 100 may be different according to the applied standard, and some components may be replaced with other components.

When the transmitted bitstream 105 is parsed and parsed to extract header information and encoded video data, the variable length decoding unit 110 quantizes the Huffman table using a pre-stored Huffman Table. The inverse scan unit 115 performs the inverse scan to generate data in the same order as the moving image 140. That is, the inverse scan unit 115 outputs a value in reverse of the order of scanning in various ways during encoding. After quantization is performed during encoding, a scan direction may be defined according to a distribution of frequency band values. In general, a zig-zag scan method is used, but the scan method may vary by codec.

Syntax parsing may be performed integrally in the variable length decoding unit 110 or may be performed in any component that processes the bitstream 105 prior to the variable length decoding unit 110. In this case, since syntax parsing is the same as that of the standard applied to the encoder and the decoding period, the syntax parsing is performed only by a predetermined standard corresponding to the standard.

The inverse DC / AC predictor 120 determines the direction of the reference block for prediction by using the magnitude of the DCT coefficient in the frequency band.

The inverse quantizer 125 inverse quantizes the inversely scanned data. That is, the DC and AC coefficients are reduced by using a quantization parameter (QP) specified during encoding.

The inverse DCT unit 130 performs an Inverse Discrete Cosine Transform to obtain an actual video pixel value to generate a VOP (Video Object Plane).

The video reconstruction unit 135 reconstructs and outputs a video signal by using the VOP generated by the inverse DCT unit 130.

As shown in FIG. 2, the MPEG-4 encoder 200 generally includes a DCT unit 210, a quantizer 215, a DC / AC predictor 220, a scan unit 230, and a variable length encoder ( 235).

Each component included in the encoder 200 performs the inverse function of each component of the corresponding decoder 100, which is obvious to those skilled in the art. In brief, the encoder 200 performs encoding by converting a video signal (ie, a digital image pixel value) into a frequency value through a discrete cosine transform, quantization, and the like, and then encoding the information. Variable length encoding that differentiates the bit length according to the frequency is performed and output in the compressed bitstream state.

3 is a block diagram of an encoder according to an embodiment of the present invention.

The encoder according to the present invention further includes an extended bitstream generation and output unit 2410 as compared to the conventional encoder 200 described with reference to FIG. 2. The extended bitstream generation and output unit 2410 may control information (eg, a list and connection relations of the functional units used, input of the corresponding functional units) in the process of generating the conventional bitstream 316 generated by processing up to the front end. Data, syntax information, syntax connection information, etc.) to generate a decoder description. In addition, the extended bitstream 305 is generated using the generated decoder description 313 and the conventional bitstream 316 and transmitted to the decoder 300.

The encoder may include a tool box including a plurality of encoding functions and generate a bitstream according to one or more encoding standards through sequential or organic combinations of the functions included in the tool box.

In addition, the variable length encoder 235 herein refers to an arbitrary component (for example, an encoder) that performs encoding in order to generate the conventional bitstream 316 in the encoder. In addition, this does not limit the scope of the present invention.

3 is a diagram illustrating a case in which an extended bitstream generated by using decoder description information and a conventional bitstream is provided to a decoder.

However, the decoder description may be delivered to the decoder 300 in the form of separate data or bitstream. In this case, an encoded decoder description generation and an output unit (not shown) are positioned at the rear of the variable length encoding unit 235, and the encoded decoder description generated independently of the conventional encoding unit is provided to the decoder 300. It can be obvious.

Meanwhile, the decoder description includes functional unit identification information (FUID) of the corresponding functional units. The functional unit identification information (FUID) includes a tool box number (TBN) field indicating a tool box to which a corresponding function part belongs in the tool box unit, and a FU number field indicating unique identification information of the corresponding function part.

The functional unit identification information and the toolbox unit will be described later with reference to related drawings (FIGS. 10 to 12) and related tables (Table 5).

4 is a block diagram of an embodiment of a decoder according to the present invention, and FIG. 5 is a diagram illustrating an embodiment of a bitstream processing process in the decoder of FIG. 4.

The decoder description and the image bitstream illustrated in FIG. 4 may be, for example, information generated and provided by an encoder.

Referring to FIG. 4, the decoder 300 includes a decoder 305 and a separator 310. The decoder 305 includes a BSDL parser 320, a decoder forming unit 330, a toolbox 335, and a decoding solution 340.

The BSDL parser 320 interprets syntax information of an image bitstream input from the outside using a BSDL schema input from the separator 310. The video bitstream input to the BSDL parser 320 is data encoded by an arbitrary coding scheme (eg, MPEG-4, AVS, etc.). It will be readily understood by those skilled in the art that the BSDL parser 320 may interpret the BSDL Schema on its own or may be configured by an external algorithm through this specification.

The BSDL parser 320 includes a BSDL parser, which is an internal processor that reads the BSDL schema described in XML grammar and redefines the structure of the BSDL parser 320.

The rules to redefine using the BSDL schema can vary depending on the method applied by the producer. Therefore, the basic purpose is as follows. First, it is to be able to recognize the information about the length and meaning of the bitstream recorded on the BSDL schema. Second, to read the repetition structure and conditional execution structure defined in the BSDL schema, and to implement the programmatic routine that actually operates by the same repetition or conditional statement. Accordingly, the BSDL parser 320 before being redefined may be defined as a state in which only functions for achieving the above object are implemented, and the BSDL parser 320 that actually executes using the above-described parsing function may be implemented. It is a process of redefining the process.

The BSDL parser 320 is implemented as a program capable of constructing a flexible data flow under the control of the BSDL interpreter. For example, the BSDL parser 320 is implemented using a programming language such as CAL (Caltrop Actor Language), C, C ++, and Java. Can be.

BSDL internal processing unit 2525 and BSDL parser 320 may be implemented without limitation according to the design criteria of the decoder designer. Of course, existing BSDL management programs such as BSDL reference software may be applied. BSDL reference software is official software designed for smooth operation of BSDL standardized by MPEG standardization organization. It is obvious that BSDL parser 320 receiving BSDL schema can be more easily implemented using such software resources. .

As mentioned herein, the basic structure of BSDL parser 320 may be designed by various methods chosen by the decoder designer. That is, the decoder designer may autonomously select the application and design of the detailed algorithm to perform the designated function of the BSDL parser 320. However, the BSDL parser 320 may be redefined by the result of reading the BSDL schema, and the redefined result should be able to collaborate with (eg, communicate with) other components of the decoder 305.

The BSDL schema input by the BSDL parser 320 describes the details of the syntax information included in the bitstream. For example, the length of the syntax information, the meaning of the syntax information, the condition of the occurrence of the syntax information, and the repetition are described. The number of occurrences may be included. Herein, the length of information means a bit length occupied by specific information on a bitstream, and the meaning of syntax information indicates what information the information has. For example, if a functional unit is requesting information A, it may be necessary to distinguish which information A. In addition, in the case of the appearance condition or the number of recurrences, even when processing the video bitstream of the same standard using one BSDL schema, whether or not the appearance of some syntax information or the number of repetitions may vary depending on the attributes of the bitstream. Information that can be attached to the BSDL schema to define this. For example, an appearance condition may be necessary to avoid reading motion vector information when processing an intra frame, and the number of repetitions may be repeated if the macroblock has six blocks of the same structure. Can be used.

As illustrated in FIG. 5, the BSDL interpretation processor delivers the decoded result information regarding the details to the BSDL parser 320 so that the BSDL parser 320 reads the information included in the bitstream in the order specified in the BSDL schema. To help them.

The BSDL parser 320 converts the contents of the input bitstream into meaningful data by referring to the result information provided from the BSDL interpretation processing unit and provides the same to the decoder forming unit 330 and / or the decoding solution 340. In addition, the meaningful data provided by the BSDL parser 320 to the decoder forming unit and / or the decoding solution 340 include, for example, encoded image data of a predetermined macroblock size, AC for intra coded macroblocks. A prediction flag (ACpred_flag), MCBPC (MB type & coded block pattern for chrominance), CBPY (coded block pattern for luminance), and the like may be included. The data providing process may be performed regardless of whether the decoder forming unit 330 or the decoding solution 340 is driven.

This embodiment is intended to allow the decoder (decoder) to decode the bitstream using the decoder description, but to implement the decoder description using a BSDL language scheme and an XML-based format interoperable therewith. In this embodiment, the decoder description may have an XML format such as BSDL, CALML, BSDL schema, Syntax Parsing process, CALML can be distinguished its role to be used for the connection control between the functional part easily skilled in the art I can understand.

The BSDL language is described in the form of an XML document or XML schema that contains information about the structure and organization of the bitstream. The language is designed so that each can represent one or more video bitstream structures. By using the BSDL language, the decoder can obtain a high compatibility with other technologies even if the decoder applies the bitstream technology method that has been verified and used in the conventional MPEG standard. Since the language format and grammar related to BSDL are described in MPEG-B Part 5, the detailed description thereof will be omitted.

An example configuration of BSDL schema and connection control information using BSDL and XML is as follows. Of course, the configuration format of the BSDL schema and connection control information is not limited thereto.

BSDL Schema

<xsd: element name = "VideoObject">

<xsd: complexType>

    <xsd: sequence>

      <xsd: element name = "VOStartCode"

type = "m4v: StartCodeType" />

      <xsd: element name = "VOL">

           <xsd: complexType>

              <xsd: sequence>

              <xsd: element name = "header" type = "VOLHeaderType"

bs2: ifNext = "&volSC;" rvc: port = "0" />

              <xsd: element name = "VOP"

type = "VideoObjectPlaneType"

maxOccurs = "unbounded"

bs2: ifNext = "&vopSC;" rvc: port = "1" />

                 </ xsd: sequence>

             </ xsd: complexType>

        </ xsd: element>

    </ xsd: sequence>

  </ xsd: complexType>

</ xsd: element>

Connection control information

<Network name = "Decoder">

<Package>

<QID>

<ID id = "MPEG4 Simple Profile" />

</ QID>

</ Package>

<Port kind = "Input" name = "BITSTREAM" />

<Port kind = "Ouput" name = "YUV" />

<Instance id = "1">

<Class name = "Parser">

<QID>

<ID id = "c" />

</ QID>

</ Class>

<Note kind = "label" name = "Stream Parser" />

</ Instance>

<Instance id = "2">

<Class name = "VS">

<QID>

<ID id = "c" />

</ QID>

<Note kind = "label" name = "Video Session" />

</ Class>

</ Instance>

<Connection src = "" src-port = "BITSTREAM" dst = "1" dst-port = "BITSTREAM" />

<Connection src = "1" src-port = "CSCI" dst = "2" dst-port = "CSCI" />

<Connection src = "1" src-port = "DATA" dst = "2" dst-port = "DATA" />

<Connection src = "2" src-port = "YUV" dst = "" dst-port = "YUV" />

</ Network>

In another embodiment of the present invention, a decoder description is produced by a method of decoding a part or all of an image bitstream by a script program based on ECMAScript.

The BSDL schema is a method for describing video bitstream components according to the MPEG-B Part 5 standard. In addition to XML, script programs written in the ECMAScript language can also be used.

An example of calling a script program in the ECMAScript language within a BSDL schema is as follows:

  <xsd: simpleType name = "macroblock">

    <xsd: restriction base = "bs1x: userType">

      <xsd: annotation> <xsd: appinfo>

        <bs1x: script ref = "macroblock.js" />

      </ xsd: appinfo> </ xsd: annotation>

    </ xsd: restriction>

  </ xsd: simpleType>

An example of a script program in the ECMAScript language called in the above example is as follows.

function parserMain () {

CSCI.CBP = new Array (6);

var MV_X, MV_Y, MV_rX, MV_rY;

var read_length;

var return_value;

// var temp;

var i;

if (CSCI.VOP_coding_type! = 0) {

return_value = readBits (1);

if (return_value) {

// Not coded MB: Default CBP (all 0)

CSCI.MBtype = 0;

for (i = 0; i <6; i ++) CSCI.CBP [i] = 0;

return;

}

}

/ * MCBPC (MBtype & CBPC) * / {

if (CSCI.VOP_coding_type == 0) {

return_value = HuffmanVLD ("B-6");

} else {

return_value = HuffmanVLD ("B-7");

}

CSCI.MBtype = return_value [0];

CSCI.CBP [4] = return_value [1];

CSCI.CBP [5] = return_value [2];

}

if (CSCI.MBtype == 3 || CSCI.MBtype == 4) {

readBits (1); // AC pred. flag

}

/ * CBPY * / {

if (CSCI.MBtype == 3 || CSCI.MBtype == 4) {

return_value = HuffmanVLD ("B-8"); // CBPY Intra

CSCI.CBP [0] = return_value [0]? 1: 0;

CSCI.CBP [1] = return_value [1]? 1: 0;

CSCI.CBP [2] = return_value [2]? 1: 0;

CSCI.CBP [3] = return_value [3]? 1: 0;

} else {

return_value = HuffmanVLD ("B-8"); // CBPY Inter

CSCI.CBP [0] = return_value [0]? 0: 1;

CSCI.CBP [1] = return_value [1]? 0: 1;

CSCI.CBP [2] = return_value [2]? 0: 1;

CSCI.CBP [3] = return_value [3]? 0: 1;

}

}

if (CSCI.MBtype == 1 || CSCI.MBtype == 4) {

readBits (2); // DQuant

}

if (! (CSCI.MBtype == 3 || CSCI.MBtype == 4 || CSCI.VOP_coding_type == 0)) {

i = 0; // R121

do {

// motion vector

read_length = CSCI.VOP_Fcode_foward-1;

MV_X = HuffmanVLD ("B-12");

if (CSCI.VOP_Fcode_foward! = 1 && MV_X! = 0) {

MV_rX = readBits (read_length);

} else {

MV_rX = 0;

}

MV_Y = HuffmanVLD ("B-12");

if (CSCI.VOP_Fcode_foward! = 1 && MV_Y! = 0) {

MV_rY = readBits (read_length);

} else {

MV_rY = 0;

}

token_output ("MV_X", MV_X)

token_output ("MV_Y", MV_Y);

token_output ("MV_rX", MV_rX);

token_output ("MV_rY", MV_rY);

i ++;

} while (CSCI.MBtype == 2 && i <4); //R121.b

}

// BLOCK

for (i = 0; i <6; i ++) {

B_MP4_SP (CSCI, i)

}

}

Since the language describing the script program as in the above example follows the language system defined in the ECMA 262 standard, a detailed description thereof will be omitted.

The conventional BSDL standard presupposes that only one syntax element information is decoded by one script program call, and does not define a method in which the interpreted information is transferred to a functional unit through a data interface as shown in FIG. 5. Therefore, in the present embodiment, the following in-program functions are defined by the syntax parser to represent some or all of the syntax parsing process of the bitstream using ECMAScript.

parserMain (): The first entry point to run the syntax parsing algorithm written in ECMAScript, such as main () in C.

readBits (bit_length): Can be called in parserMain () function and reads bit_length bit from input bitstream and returns the value.

tokenOutput (token_name, token_data: A function that can be called within the parserMain () function and causes the syntax parser to output an output having an output value identifier called token_name and a data value of token_data. Sent to

In addition, the function of the decoder description is provided by additionally defining the following functions in the program.

-peekBits (bit_length): It can be called in parserMain () function. It checks bit_length bit from input bitstream and returns the value.

getToken (): A function that can be called from within the parserMain () function and returns an array of the full list of data types that the current syntax parser can output through the data interface.

It is obvious that the BSDL parser that receives the BSDL schema including syntax parsing process description written in ECMAScript should be able to run an ECMAScript-based script program. Each of the above functions corresponds to the role of each function in the script program driving process. It will be apparent to those skilled in the art that they should be called and handled in a manner.

In the above example, the case where ECMAScript is delivered as part or all of BSDL schema according to the manner defined in BSDL standard. However, ECMAScript is a general-purpose scripting language that is independently defined as a standard, so that it can be included in a transmission medium other than BSDL or transmitted independently. In addition, the syntax of ECMAScript dialects (Dialects), which have been tested to conform to the ECMAScript standard (ECMA 262), can also be used by making minor changes to the syntax program execution process of the syntax parser.

In the present embodiment, a description will be given on the assumption that information such as a decoder description 2560 and encoded video data 2580 is input from the outside, but one or more of the information is already present in an arbitrary component of the decoder 305. It is obvious that it may be implemented in a built-in manner.

Referring back to FIG. 4, the decoder forming unit 330 may include a portion (eg, a predetermined macroblock) of connection control information received from the separation unit 310 and / or bitstream data received from the BSDL parser 320. Using encoded image data of size, one or more of an AC prediction flag (ACpred_flag) for intra-coded macroblocks, an MB type & coded block pattern for chrominance (MCBPC), a coded block pattern for luminance (CBPY), and the like. Control the decoding solution 340 to be implemented.

That is, the decoder forming unit 330 controls such that some or all of the functional units included in the toolbox 335 are loaded and aligned in the decoding solution 340 using an organic connection relationship. Here, the connection control information may be written in CALML (CAL Markup Language). CALML is an XML format that can describe the decoder configuration of the CAL language (Caltrop Markup Language) method currently being discussed by the MPEG standardization organization. The CAL language consists of a connection between Actor, which is a program object, and each Actor. The structure of the CAL language is expressed in XML format. An example of this has already been presented as an example of representation of BSDL schema and connection control information.

In detail, the decoder forming unit 330 has a right to access the toolbox 335 composed of a set of various functional units, and sets input and output connections between the functional units provided in the toolbox 335 to decode the result. Configure solution 340. At this time, the input / output connection structure and execution order between the functional units are set with reference to the connection control information. In addition, some information for identifying the type of the input bitstream may be received from the BSDL parser 320 and may be referred to in the functional unit connection process. Once the connection structure between the functional units has been established, the connection structure can be regarded as an independent decoder capable of interpreting and decoding all kinds of video bitstreams intended by the decoder description maker, provided that continuous data input from the outside is assumed. Can be. In this case, the completed functional connection structure may be referred to as a decoding solution 340.

The toolbox 335 includes a plurality of functional units, each implemented to perform a predetermined process. Each of the functional units may each be implemented in a combination of program codes.

Each functional unit included in the toolbox 335 may be subdivided into a plurality of detailed toolboxes divided into sets for each application to which they are applied. For example, it may be subdivided into a first toolbox including MPEG functional units, a second toolbox including functional units other than the MPEG functional units, and the like. Or the first toolbox, which is a set of MPEG-2 functional units, the second toolbox, which is a set of MPEG-4 functional units, and the third toolbox, which is a set of AVS functional units, which is a digital TV compression standard of China. .

Of course, the toolbox 335 itself may be implemented in plural to have an independent connection relationship with the decoder forming unit 330 and the decoding solution 340. In this case, although not shown, the above-described first toolbox, second toolbox, etc. may be implemented as a toolbox of an independent type.

However, hereinafter, for convenience of description, a plurality of detailed toolboxes are included in one toolbox 335 or all functional units will be described based on the case in which they are interspersed without an aggregate configuration.

The toolbox 335 is an area including functional units (FUs) implemented to perform respective functions (that is, a predetermined process), and the respective functional units are decoded by the connection control of the decoder forming unit 330. The encoded image data included in the image bitstream 380 is output as decoded image data by loading the solution 340 to form a sequential connection operation relationship.

The toolbox 335 may include, for example, a de-blocking filter (DF) function, a VOP reconstructor (VR) function, a frame field reordering (FFR) function, an intra prediction and picture reconstruction (IPR) function, and an IT ( Functional units such as an Inverse Transform (IQ) function, an Inverse Quantization (IQ) function, an Inverse AC Prediction (IAP) function, an Inverse Scan (IS) function, and a DC Reconstruction (DCR) function may be included.

IT4x4 function unit, IQ4x4 function unit) and DCR4x4 function unit is characterized in that the block size to be processed is 4x4. This is because MPEG-4 AVC processes data with 4x4 block size, whereas MPEG-1 / 2/4 processes data with 8x8 block size during Transform, Quantization, and Prediction.

The tool box 335 may include all the functional units for performing data decoding regardless of the applicable standard, and may add necessary functions in the course of technology development, and may modify existing functions. It is obvious that it can be removed. For example, if an IS4x4 functional unit for processing data in a 4x4 block size is additionally required for the decoding process, the corresponding functional units may be added to the toolbox 335. In addition, a special prediction (SPR) function for performing intra prediction in MPEG-4 AVC may be further added.

Each functional unit provided in the toolbox 335 does not exist independently in each standard, and in the case of a functional unit capable of the same processing regardless of the standard, it is obvious that the functional unit may be integrated into one functional unit. The function of each functional unit is obvious to those skilled in the art and will be described briefly.

The DF function is a de-blocking filter of MPEG-4 AVC, and the VR function is a function that stores the reconstructed pixel values.

The FFR function is a function for interlaced mode, and the IPR function is a function for storing the reconstructed pixel value after intra prediction of MPEG-4 AVC. As described above, intra prediction of MPEG-4 AVC may be performed by the SPR function.

The IT function unit is a function unit that performs inverse transform of DC values and AC values, and the IQ function unit is a function unit that inverse quantizes AC values.

The IAP function is a function for inverse AC prediction of AC values, and the IS function is a function for inverse scan of AC values. The DCR function is a function that performs inverse prediction and inverse quantization of DC values.

The decoding solution 340 is a result generated by the decoder forming unit 330 and receives bitstream data (or encoded video data having a predetermined macroblock size) separated by syntax information units from the BSDL parser 320. .

As illustrated in FIG. 5, the input bitstream data may be input through a tangible or intangible data interface for inputting / outputting data. The data interface can be a specific memory buffer for software, a virtual port defining the flow of data, or a parameter on a program. The hardware may be a connection line on a circuit, and may be variously implemented.

Data can be input continuously through the interface and continuously (for example, parallel processing) without regard to the performance of a particular function. The decoding solution 340 processes the input data and outputs the decoded image data. As shown in FIG. 5, data may be delivered to each functional unit starting from a data interface, and the functional unit may process the data and deliver the data to subsequent functional units. All of this data flow is processed by the decoder forming unit 330 as predefined.

The decoding solution 340 may include a storage unit for storing data (eg, information extracted by syntax parsing of the bitstream) provided from the BSDL parser 320 and processing result data of each functional unit. Each functional unit loaded by the control of the decoder forming unit 330 may perform a designated process using one or more of data provided from the BSDL parser 320 and result data of a previously operated functional unit. In this case, the functional unit that will subsequently perform the process should recognize that the operation of the preceding functional unit is completed. To this end, the decoder forming unit 330 may continuously monitor whether the operation of each functional unit is completed and control whether to start the operation of a subsequent functional unit. In addition, if a separate area is provided for each functional unit in the storage unit, and the processing result data of the preceding functional unit is stored in the storage area for the subsequent functional unit under the control of the decoder forming unit 330, the subsequent functional unit You might be able to run a process as soon as the data you need to run it is stored in your storage area. In addition, it is obvious that various methods for controlling the start time of processing between the functional units may be additionally considered.

Of course, the storage unit may be provided in the decoder forming unit 330, and the decoder forming unit 330 is a function unit to perform a current process (eg, parsing syntax of a bitstream (eg, data received from the BSDL parser 320). Information extracted by the above), and the processing result data of each functional unit may be provided to the corresponding functional unit.

Hereinafter, an operation process of the decoder 305 will be briefly described with reference to FIG. 5.

When the input video bitstream and BSDL schema are input from the outside (assuming that information A and B exist at any point in the bitstream), the BSDL parser 320 reads the BSDL schema to the point corresponding to the information A. It is recognized that 5 bits of MB type data exists, and 2 bits of CBPY data exist at a point corresponding to information B.

The BSDL parser 320 then reads the specified number of bits at each point using the recognized information and delivers the read information to the decoding solution 340 according to the assigned meaning.

The decoding solution 340 receives and processes data named MB Type and CBPY from the BSDL parser 320. As described above, the decoding solution 340 is loaded and implemented by the functional units by the connection control of the decoder forming unit 330.

The data interface present in the decoding solution 340 receives data transmitted from the outside, and refers to the connection relationship of the functional units previously configured by the connection control information, and transmits the data to the functional units requesting the corresponding data.

Each functional unit also performs a decoding process according to a predetermined connection relationship (ie, a connection relationship for data processing). The connection relationship between all data flows and the functional units is based on the details previously configured by the decoder forming unit 330. The output image frame is output to the outside by sequential processing of the respective functional units.

As described above, a storage unit may be provided in the decoder forming unit 330 or the decoding solution 340. This is because, in receiving data from the BSDL parser 320, the delivery process is seamless and the data provision can be performed in parallel with the decoding process. In addition, each functional unit may read and use necessary data from the storage unit.

In addition, the BSDL parser 320 may provide corresponding data to the decoder forming unit 330 for decoding processing of the encoded image data so that the decoder forming unit 330 may provide the decoding solution 340, or the BSDL parser ( 320 may provide the data directly to the decoding solution 340.

Referring back to FIG. 4, the separator 310 separates the input decoder description 2560 into respective information and inputs the same to the decoder 305. The decoder description 2560 input to the separator 310 may include a BSDL schema 2565 for describing the structure of the bitstream and CALML data 2570 for describing the decoding process of the bitstream. The two types of data described above may be independently described by XML grammar, and two types of data may be integrated and transmitted for efficient decoder operation.

6 is a diagram illustrating a decoder description input process according to another embodiment of the present invention.

As illustrated in FIG. 6, the decoder 300 may further include a description decoder 510. The description decoder 510 may decode the input encoded decoder description 520, generate a decoder description 2560, and provide the decoder description 2560 to the separation unit 310.

By encoding and transmitting the decoder description 2560, the amount of data transmitted and received can be reduced.

7 is a block diagram of another embodiment of a decoder according to the present invention.

Referring to FIG. 4, the decoder description 2560 and the image bitstream are input to the decoder 305. The decoder description 520 and the image bitstream 380 encoded with reference to FIG. 305) has been described.

However, as illustrated in FIG. 7, it is apparent that configuration information of the decoder description 2560 may be input to the decoder 305 by being separated. In this case, it is apparent that the above-described separation unit 310 and decoder description 2560 may be omitted. 8 is a diagram illustrating a configuration of a decoding unit according to another embodiment of the present invention.

The decoder 305 in which the toolbox 335 and the decoder forming unit 330 are separated from each other has been described above with reference to FIGS. 4 to 7.

However, as illustrated in FIG. 8, it is obvious that the toolbox 335 may be implemented as one component of the decoder forming unit 330.

In this case, the decoder forming unit 330 may include not only a connection structure control function between the functional units but also a selection function of the functional units to be used, and the types of decoding solutions 340 that can be implemented through this may be various.

9 is a diagram showing the configuration of a BSDL parser according to another embodiment of the present invention.

The BSDL parser 320 including the BSDL analysis processor has been described above with reference to FIG. 4.

However, the BSDL parser 320 according to the present invention may be predefined and provided from outside the decoder 300 before starting decoding of the bitstream. Therefore, the BSDL analysis processor described above may be omitted. In this case, the BSDL parser maker 2610 may be configured by using an existing application program such as BSDL reference software.

So far, the description has been focused on the case where the BSDL parser processes the designated operation as an independent component. However, the BSDL parser may be implemented as one functional unit included in the toolbox or may be implemented to be included in advance as an independent component in the decoding solution. If the BSDL parser is provided in the toolbox, the decoder forming unit should load and control the BSDL parser to perform a process before the operation of the functional units operating for the bitstream decoding using the connection control information. Similarly, if the BSDL parser is previously included in the decoding solution, the decoder forming unit should control the BSDL parser to perform the process first before starting the process execution of each loaded functional unit. In each case, since the operation and function of the BSDL parser are the same as described above with reference to the related drawings, a detailed description thereof will be omitted. However, the subject initially receiving the BSDL schema or / and bitstream may need to be changed to a decoder forming unit and / or a decoding solution.

Although the decoding apparatus and the syntax parsing method for bitstream decoding according to the present invention have been described with reference to MPEG-4 AVC, video encoding / decoding of MPEG-1, MPEG-2, MPEG-4, AVS, and others are described. Naturally, the same can be applied without any limitation to the standard.

In addition, the information included in the connection control information is not described only with information about the connection relationship between the functional units for performing decoding by one standard, the processing process required for the corresponding functional unit, and the like. It is obvious that information may be described.

For example, suppose that an initial plurality of frames of an image bitstream are encoded in MPEG-2, subsequent frames are encoded in MPEG-4, and the remaining frames are encoded in MPEG-1. In this case, for the decoding of the encoded image data, the connection control information will be defined such that each frame having a different encoding method can be functionally combined and operated in accordance with each standard included in the toolbox 335. Do.

Hereinafter, another embodiment of a decoder according to the present invention will be described with reference to related drawings. However, in describing another embodiment, the description of the configuration that performs the same or extremely similar functions as the above-described embodiment will be omitted by the same description as the same reference numerals and reference numerals. For example, the toolbox 335, the decoder forming unit 330, and the decoding solution 330 illustrated in FIG. 11 are basically the same as the above-described configuration.

In order to form the decoder by loading and recombining the corresponding functional units in the decoding processor as described above, a mechanism capable of distinguishing and loading functional units is required. Hereinafter, a detailed configuration of the method for identifying and classifying the functional units FU according to the present invention will be described.

10 is an exemplary view showing a detailed configuration of a toolbox according to an embodiment of the present invention.

As shown in FIG. 10, the toolbox according to the present invention may be configured as a set of a plurality of toolboxes separately separated to store / manage a plurality of functional units according to types. Hereinafter, the set of the plurality of tool boxes will be referred to as a tool box unit. That is, the functional units are divided and stored / managed into a plurality of toolboxes in the toolbox unit according to their types, and each toolbox is divided into toolbox numbers (TBNs) and identified and managed. That is, the toolbox number is a kind of toolbox identification information.

That is, the toolbox unit according to the present invention includes an MPEG video toolbox for storing functional units related to MPEG video decoding; An MPEG audio toolbox that stores functional parts related to MPEG audio decoding; An MPEG graphics toolbox for storing functional parts related to MPEG graphic decoding; And a functional unit related to multimedia decryption such as a system toolbox storing functional units related to system decryption, and the toolbox unit may include a plurality of toolboxes.

The toolbox number of the toolbox may be defined as shown in Table 1 below.

TABLE 1

Tool Box Number (TBN)  Tool-library 0 MPEG video tool library One MPEG audio tool library 2 MPEG graphics tool library 3 System tool library 4 Reserved ... ... n Reserved

The toolbox unit and the toolbox may be logically divided into one storage means, or may be physically divided into a plurality of storage means.

11 is an exemplary diagram illustrating functional unit identification information (FUID) according to an embodiment of the present invention.

As illustrated in FIG. 11, the functional part identification information (FUID) according to the present invention includes a tool box number (TBN) field indicating a toolbox to which the corresponding function part belongs, and a FU number field indicating unique identification information of the corresponding function part. It is configured by.

The toolbox number field may be implemented with 4 bits, and the FU number field may be implemented with 28 bits. By implementing the FU number field with 28 bits, 268,435,456 functions can be stored, identified and used in one toolbox.

The FUID may be included in the decoder description through a method such as applied to the FUID field in the VNT mentioned above. In addition, it is obvious that even in XML-based connection control information containing information of the same meaning, it may be used to refer to each functional unit used in the decoder configuration.

12 is a conceptual diagram illustrating a functional unit classification / identification mechanism according to the present invention.

Referring to FIG. 12, the BSDL parser or decoder description analyzer of the decoder interprets the received decoder description to extract functional unit identification information (FUID) 1950, and the decoder forming unit of the functional unit identification information (FUID) 1950. Reading the TBN and FU numbers of the functions required to combine the decoders from them, and requesting the corresponding toolbox corresponding to the read TBN and FU numbers, the requested functions are loaded and connected to the decoding solution to form a recombinant decoder. To decode the input data.

For example, since the first FUID is TBN is 0 and the FU number is 69, among the functional units stored in the MPEG video toolbox 1910 in the toolbox, the functional unit having the FU number 69 is requested and loaded.

 13 is a flowchart illustrating a method of designing a new codec based on an input bitstream according to an embodiment of the present invention.

The following general design process can be applied to support new codecs or profiles using the RVC framework. Support of new codecs using RVC is expected to continue in the future, and it is expected that the following steps will help to manage tasks and tasks. After the target codec to be produced using RVC is determined, the following procedure can be performed.

In step 1310, the codec design apparatus finds the most similar RVC codec configuration. If possible, work is started by selecting the closest one to the codec to be made from existing RVC-based codec information. For example, MPEG-4 ASP can be derived from the MPEG-4 SP configuration.

In step 1315 the codec design apparatus describes bitstream syntax information (BSD) for the new codec. Bitstream syntax information of the target codec is described in RVC's Bitstream Syntax Description (BSD) format.

In step 1320, the codec design apparatus verifies BSD. That is, verify that the BSD created in step 2 meets the target codec standard. Make sure your BSD supports all the bitstream configuration methods available in the target codec.

In operation 1325, the codec design apparatus updates the token pool. Update the token pool for network configuration and information exchange between FUs. As a result of the BSD design, if a new token is added, the role of an existing token is changed, or some tokens are deleted, it is reflected in the token pool.

In operation 1330, the codec design apparatus checks the FU that needs to be added or modified. In order to implement the operation of the target codec in the RVC, some FUs in the existing VTL may need modification or a new FU may be needed. It summarizes which FUs should be modified or added, and which new features each FU specifically contains.

In operation 1335, the codec design apparatus embodies a new function of the FU. Based on the results of step 5, create a new textual description of each FU. In this step, the FU's input / output and internal processing algorithms are newly defined or changed. This process is then essential for the construction of a new FU network.

In more detail, the codec design apparatus determines the precision of the FU. When defining a new FU, you must determine the precision of that FU. Overly precise FUs may compromise the freedom of future implementations, and overly packed FUs can make it difficult to understand the codec's detailed features. For examples or guidelines, see the existing RVC standards (MPEG-B Part 4, MPEG-C Part 4).

The codec design device produces standard conformance verification information.

If at this stage you can define the “optimal response” of each FU (a successful outcome that must be given for a given input when a particular FU is operating normally), it can be useful in subsequent verification phases of the FU. There will be.

In operation 1340, the codec design apparatus describes FU network information (FND) for a new codec. The entire process of the target codec is described in FND. At this time, the new FUs defined in step 6 and the new tokens defined in step 4 can be used.

In operation 1345, the codec design apparatus verifies the FND. Verify the fabricated FND. Check that the connection of the FU is not possible due to deadlock and that the tokens of the input / output ports do not match each other.

In operation 1350, the codec design apparatus sets up the FU in a simulation environment. The FUs defined in step 6 and the FNDs defined in step 7 are set up for use in the existing simulation environment.

In more detail, the codec design apparatus manufactures a bitstream syntax parser FU. For the operation of the RVC decoder, the syntax parser FU must be able to operate on the simulation model. Convert the two stage BSDs to a parser FU, or use a generic syntax parser FU that can run BSD as input.

The codec design device then implements a new FU. Implement new or changed FUs according to the five-stage FU specification.

The codec design device verifies the FU's standard conformance. If the standard conformance verification information of step 6.2 has been produced, it is used to verify that the FU implemented in step 9.2 is working as intended.

In operation 1355, the codec design apparatus verifies the entire decoder configuration. Finally, a decoding solution is produced to verify that all the features of the implemented RVC codec work properly. Check if all FUs are working properly and the connection between FUs is sending and receiving data normally. Conformance bitstream and codec-level “optimal response” data can be used for verification.

Each step presented in the above method can be represented in the following table when the actual RVC-based codec is applied to the project itself. The following table applies to MPEG-4 ASP.

step O / X Note One Find the most similar RVC codec configuration O Development started based on MPEG-4 SP 2 Describes bitstream syntax information (BSD) for a new codec O Completed BSD production using ECMAScript and RVC-BSDL 3 Verify the BSD O Verify that the parsing process is working (using MPEG-4 visual conformance bitstream) 4 Token Pool Update O Checked the token pool and proposed a new token 5 Identify FUs that need to be added or modified r Necessary new features have been identified, but specifically, which FU should be changed 6 Embody new features of FU r 6.1 Determine the precision of FU X Coming 6.2 Create standard compliance verification information X Coming 7 Describes FU network information (FND) for new codecs X Coming 8 Verify the FND X Coming 9 Setup FU in the Simulation Environment X Coming 9.1 Create a bitstream syntax parser FU r Operable bitstream parser, but FU is not yet integrated with the network 9.2 Implement a new FU X Coming 9.3 Verify FU's standard conformance X Coming 10 Validate the entire decoder configuration X Coming

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

1 is a diagram schematically showing a configuration of a general decoder.

2 is a diagram schematically illustrating a configuration of a general encoder.

3 is a block diagram of an embodiment of an encoder according to the present invention;

4 is a block diagram of an embodiment of a decoder in accordance with the present invention.

5 is a diagram illustrating in detail a bitstream processing in a decoder according to an embodiment of the present invention.

6 is a diagram illustrating a decoder description input process according to another embodiment of the present invention.

7 is a block diagram of another embodiment of a decoder according to the present invention;

8 is a block diagram of another embodiment of the decoder of FIG. 4;

9 illustrates a configuration of a BSDL parser according to another embodiment of the present invention.

10 is an exemplary view showing a detailed configuration of a toolbox according to an embodiment of the present invention.

11 is an exemplary view showing functional unit identification information (FUID) according to an embodiment of the present invention.

12 is a conceptual diagram for explaining a functional unit classification / identification mechanism according to the present invention.

13 is a flowchart illustrating a method of designing a new codec based on an input bitstream according to an embodiment of the present invention.

Claims (1)

How to design a new codec based on input data.

Images