KR100738867B1 - Method for Coding and Inter-view Balanced Disparity Estimation in Multiview Animation Coding/Decoding System - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to an encoding method of a multiview video encoding / decoding system and a method of estimating inter-view correction variation.

2. The technical problem to be solved by the invention

The present invention proposes a GOP of a new structure, through which a simpler and more efficient multi-view video is encoded, and a method of encoding a multi-view video encoding / decoding system for reconstructing images of various viewpoints according to the situation of a decoder. The purpose is to provide

3. Summary of Solution to Invention

The encoding method of a multiview video encoding / decoding system for performing encoding on a multiview group (GOMV), which is a set of a plurality of multiview images within the same view, is a GGOP in which a GOP is extended to a view axis. Performing encoding on a frame of a base GOP (I am a GOP) including a base picture (OPP) in I; Performing encoding on a frame of a GOP including at least one or more GOP _B- P pictures using the reference GOP; And using the reference GOP and the GOP _B Performing encoding by performing a disparity estimation in both directions on a frame of a GOP _C -B picture including only a GOP _C picture existing between the reference GOP and at least one or more GOP _Bs .

4. Important uses of the invention

The present invention is used in an encoding / decoding system.

Encoding, Decoding, GOP, GGOP, GOMV, IBDE

Description

Method for Coding and Inter-view Balanced Disparity Estimation in Multiview Animation Coding / Decoding System}

1 is a structural diagram of a multiview profile encoding / decoding system implemented by applying time scalability of the MPEG-2 standard;

2 is a structural diagram of a stereo video encoding / decoding system using an MPEG-2 multiview profile;

FIG. 3 is an exemplary diagram for describing prediction encoding considering only parallax using two disparity predictions for bidirectional prediction; FIG.

4 is an exemplary diagram for describing prediction encoding using a disparity vector and a motion vector for bidirectional prediction.

5 is an exemplary diagram for explaining a picture form defined in MPEG-2;

6 is a structural diagram of one embodiment of a GOMV defined in accordance with the present invention;

7 is a structural diagram of one embodiment of a GGOP defined in accordance with the present invention;

8 is a structural diagram of an embodiment for explaining GOP _B in FIG. 7;

FIG. 9 is a diagram illustrating an embodiment of GOP _C in FIG. 7;

FIG. 10 is a structural diagram illustrating a bitstream storage location of a GGOP considering view scalability according to the present invention; FIG.

11 is a flowchart illustrating a method for estimating inter-view correction variation of a multiview video encoding / decoding system according to the present invention.

The present invention relates to an encoding method and an inter-view corrected variation estimation method of a multi-view video encoding / decoding system. In particular, the multi-view video is efficiently compressed, the correlation between the multi-view images is efficiently removed, and the imbalance between views is improved. The present invention relates to an encoding method and an inter-view correction variation estimation method of a multiview video encoding / decoding system.

Since the 1970s, information transmission technology centered on telephones and televisions has been developed as a medium for transmitting voice and video information. In particular, the development of computer technology has led to the development of data transmission technology such as text. In addition, the rapid development of communication, computer, and semiconductor technology gave birth to the era of multimedia communication that transmits data, voice, and video information that have been processed independently in one media.

Along with these technological developments, information and communication services required in the highly information society of the 21st century require a video medium that can express the information to be conveyed most realistically, naturally and intimately.

In the case of the most representative TV among the current image media, the research and development direction so far is high definition TV (HDTV), which gives the viewers a sense of presence and reality through optimization and enlargement of the existing TV screen, enhancement of resolution, and natural color of the image. However, they are limited to the realistic transfer of information because they are only possible to represent flat images and are different from the real world.

Therefore, today, with the development of the communication network, the interest in the image media technology, which is considered more natural and realistic user interface, for the three-dimensional representation, recording and reproduction of human visual sense and perception.

Three-dimensional stereoscopic TV broadcasting technology is expected to be applied to most video fields in the future according to the development of digital service and multimedia, and it will be one of the major industries in the 21st century in terms of demand and research and development prospects. Currently, the method of recognizing three-dimensional images so that humans can easily feel a three-dimensional effect is that the human visual system relies heavily on binocular disparity to recognize a sense of depth to an object. It is a stereo system that inputs the same image to each eye in a proper way.

This stereo system requires two planar images for the observer, so the existing video compression standards MPEG (Moving Picture Experts Group) or H.261, H.263 and H.264 can be used to process them. There are advantages to it.

In theory, the transmission of a stereo image requires twice the bandwidth of a conventional image, but since the two images are acquired at a distance from each other horizontally to each other, a large similarity (Correlation) is achieved. Have. Therefore, this similarity has the advantage of greatly reducing the amount of data.

The most basic of stereo image coding is disparity estimation, which finds similarity between two images and expresses the disparity as a vector. The method of disparity estimation is similar to the motion estimation used to eliminate the redundancy of time, so that the method used in the motion estimation is usually used as it is.

On the other hand, only a stereo image is sufficient to feel a three-dimensional effect, but in order to reproduce a more realistic image by providing various viewpoints, a multiview image obtained by acquiring images from various locations is required. When the multi-view image is used, the view image varies according to the viewer's position movement. As the number of viewpoints increases, more natural view movement becomes possible. However, in this case, since the amount of data increases by the number of viewpoints, an efficient compression method is required. Like a stereo image, since a multiview image has many overlapping points for each view, compression can be efficiently performed using a disparity estimation method.

In order to efficiently compress a multiview image, it is necessary to improve the structure of a group of pictures (GOP) which is a sequence of compressing a multiview image. Hereinafter, a multiview video encoding / decoding system and a GOP structure of MPEG-2 will be described in detail with reference to FIGS. 1 to 5.

1 is a structural diagram of a video encoding / decoding system of a multiview profile implemented by applying time scalability of the MPEG-2 standard.

The scalability provided by MPEG-2 is used to simultaneously decode video having different resolutions or formats by using one video device, and the time scalability among the scalability supported by MPEG-2 is frame rate (frame). It is a technique to improve visual quality by increasing rate. Multi-view video is applied to stereo video in consideration of this time scalability.

Substantially, the structure of an encoding / decoding system having a stereo video concept has a structure similar to the temporal scalability of FIG. 1, wherein left images of the stereo video are input to a base view encoder, and a right side of the stereo video Images are input to a temporally located auxiliary auxiliary view encoder. Such an encoder is for temporal scalability, and is an interlayer encoder that generates an image between images of a base layer in time.

Accordingly, a normal video can be obtained by separately encoding and decoding the left image, and a stereoscopic video can be realized by simultaneously encoding and decoding the left image and the right image. Here, a system multiplex and a system demultiplex capable of combining or separating sequences of two images are required for video transmission or storage.

2 is a structural diagram of a stereo video encoding / decoding system using an MPEG-2 multiview profile.

As shown in the figure, the base layer is coded using motion compensation and Discrete Cosine Transform (DCT) and decoded through inverse processes, and the temporally located auxiliary auxiliary view encoder is decoded. It plays the role of a temporal interlayer encoder predicted based on the image of the base layer.

That is, two variation predictions or one variation prediction and a motion compensation prediction may be used here, and temporal auxiliary view encoders that are temporally positioned like the base layer encoder and the decoder may be used for variation. And a motion compensation DCT encoder and decoder.

In addition, just as a motion predictor and a compensator are required in the motion prediction / compensation encoding process, the disparity compensation encoding process requires a disparity predictor and a compensator. DCT, quantization of DCT coefficients, and variable length coding of differential images. In contrast, the decoding process is a process of variable length decoding, inverse quantization, inverse DCT, and the like.

MPEG-2 encoding is a very efficient compression method due to bidirectional motion prediction for B-pictures, and since temporal scalability is also very efficient, B-pictures using only bidirectional prediction are used for encoding the right image. You can get it.

FIG. 3 is an exemplary diagram for describing prediction encoding considering only parallax using two disparity predictions for bidirectional prediction.

As shown in the figure, the left image is encoded using a non-scalable MPEG-2 encoder, and the right image is an MPEG-2 temporally positioned secondary view based on the decoded left image. Encoding is done using an encoder (temporal auxiliary view encoder). That is, the B-picture is encoded by using prediction obtained from two different left images. In this case, one of the two reference images is the left image when displayed in time, and the other is the left image that is next shown in time.

The two predictions make three prediction modes: forward, backward, and interpolated, similar to motion estimation / compensation. Here, the forward mode refers to the shift predicted from the left image at the same time, and the reverse mode refers to the shift predicted from the next left image. In the case of this method, since the prediction of the right image is performed through the disparity vectors of the two left images, this type of prediction method is called predictive encoding considering only the variation. In the end, the encoder encodes two variations for each frame of the right image. The vector is estimated, and the decoder decodes the right video from the left video using these two disparity vectors.

FIG. 4 is an exemplary diagram for explaining prediction coding using a disparity vector and a motion vector for bidirectional prediction. As shown in FIG. 3, B-picture using bidirectional prediction is used, but one variation is estimated in the direction of bidirectional prediction. And using one motion estimation. That is, one uses variation prediction from the left image of the same time zone and motion prediction from the right image of the previous time.

Like predictive coding considering only variation, bidirectional prediction produces three prediction modes called forward, reverse, and bidirectional modes. Here, the forward mode refers to motion prediction from the decoded right image, and the reverse mode refers to disparity prediction from the decoded left image.

Therefore, the specification of MPEG-2 Multi-View Profile (MVP) itself is designed to be suitable for real stereo video, so the structure of the encoder for multi-view video is not mentioned at all. There is a need for an encoder that can efficiently provide a multi-view video to provide a three-dimensional and realism at the same time.

On the other hand, MPEG-2 proposes a standard for encoding and decoding video.

5 is an exemplary diagram for describing a picture type defined in MPEG-2.

As shown in the figure, there are three types of pictures defined in MPEG-2: I picture, P picture, and B picture. An I (Intra-coded) picture simply does not use motion vector estimation / compensation. Only DCT-encoded, P (Predictive coded) picture performs motion estimation / compensation while referring to I picture or other P picture, and then DCT-encodes the remaining differential data, and B (Bidirectionally Predictive coded) picture is P picture Motion compensation is used as shown below, but motion estimation / compensation is performed from two frames on the time axis.

MPEG-2 pictures have the same structure as B, B, I, B, B, P, ..., and from the I picture to the next I picture are called GOP (Group of Picture), and the pictures in the GOP The number is N and the number of pictures between the I picture and the P picture or between the P picture and the P picture is defined as M.

In the conventional 2D video encoding / decoding system, the structure is simple because it only considers the prediction on the time axis. However, in order to compress a multiview image, both the time axis and the spatial axis need to be considered. It became.

In addition, the variation estimation is the biggest performance influence factor of the multi-view video encoding / decoding system, and the performance of the variation estimation may be greatly degraded due to inconsistency between cameras. For this reason, the performance of a multi-view video codec does not improve significantly compared to existing 2D codecs.

The present invention has been proposed to solve the above problems, and proposes a GOP of a new structure, thereby encoding a simpler and more efficient multi-view video, and reconstructing images of various views according to the situation of the decoder. An object of the present invention is to provide a method for generating a bit stream of a multiview video encoding / decoding system.

In addition, the present invention provides a method for estimating the inter-view correction variation of a multi-view video encoding / decoding system to efficiently remove the correlation between the multi-view images by performing the inter-view correction estimation. There is another purpose.

According to an aspect of the present invention, there is provided a coding method of a multiview video encoding / decoding system, characterized in that encoding is performed on a multiview group (GOMV), which is a set of a plurality of multiview images within the same viewpoint. A coding method of a multi-view video encoding / decoding system is provided.

In addition, the present invention extends a GOP to a view axis in an encoding method of a multiview video encoding / decoding system for encoding a multiview group (GOMV), which is a set of a plurality of multiview images within the same view. Performing encoding on a frame of a GOP including a base GOP-I picture in the GGOP; Performing encoding on a frame of a GOP including at least one or more GOP _B- P pictures using the reference GOP; And using the reference GOP and the GOP _B A multiview video encoding / operation is performed by performing disparity estimation in both directions on a frame of a GOP _C -B picture including only a GOP _C -B picture existing between the reference GOP and at least one or more GOP _Bs . A coding method of a decoding system is provided.

The present invention also provides a method for estimating the inter-view correction variation in a multi-view video encoding / decoding system, obtaining a global correction parameter, determining an initial representative value based on this, and performing the estimation of the variation using the values. Selecting an index having a good performance, calculating a probabilistic model for each index, and determining an optimal representative correction parameter using the index; A compression step of determining a correction parameter for the chrominance image using the disparity estimation vector obtained by using the representative correction parameter; And it provides a method for estimating the inter-view correction variation of a multi-view video encoding / decoding system comprising the step of applying each correction value to the image of a different time.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. First of all, in adding reference numerals to the components of each drawing, it should be noted that the same components have the same number as much as possible even if displayed on different drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In the present invention, a set of N multiview images within the same viewpoint will be defined as "Group of multiviews (hereinafter, simply referred to as 'GOMV'). In the conventional 2D video codec, I, P, and B pictures are allocated and encoded in a chronological order, but the coding order may be complicated when extended to a multiview image. Therefore, the present invention proposes a GOMV structure in order to maximize the compression efficiency while simplifying the structure of the encoder.

6 is an embodiment structural diagram of a GOMV defined in accordance with the present invention.

As shown in the figure, the position of encoding GOMV in the entire encoder is the same as the position of encoding one frame in the 2D video. A method of encoding one frame in the GOMV uses a function of encoding one frame in the existing 2D video. The encoding function of the existing two-dimensional video will be apparent to those of ordinary skill in the art, and thus a detailed description thereof will be omitted.

Meanwhile, in the present invention, as a set of GOPs of a multiview video, a structure in which the GOP is expanded on the view axis will be referred to as "Group of GOP" (hereinafter referred to as "GGOP").

7 is a structural diagram of an embodiment of a GGOP defined in accordance with the present invention.

As shown in the figure, the GGOP according to the present invention encodes other GOPs based on a base GOP (GOP including an I picture) using a variation estimation technique. The coding method on the time axis is the same as the conventional general GOP coding method.

8 and 9 are exemplary structural diagrams for describing GOP _B and GOP _C in FIG. 7, respectively.

The frame coding scheme in the GOMV encodes a frame of a base GOP (base GOP) as shown in FIG. 8 and encodes frames belonging to 'GOP _B ', which is a GOP including at least one P picture based on the frame. After encoding frames corresponding to GOP _B , encoding is performed on a frame corresponding to 'GOP _C ', which is a GOP composed of only B pictures. These frames use the pre-coded base GOP and GOP _B to perform disparity estimation in both directions as shown in FIG. 9.

In other words, GOP is _B with reference to a spatially pre-encoding the reference GOP GOP _B performs the compression, in time can be used for conventional two-dimensional coding method.

Since two or more reference images are provided in H.264, bidirectional disparity estimation can be performed even in space even if motion estimation is performed in both directions, thereby enabling more efficient compression. However, in the encoder before H.264, the estimation in two directions is maximum, so if the motion estimation is performed in one direction in a multiview image, the variation estimation also selects only one direction. However, in H.264, multiple references are used. Support for video maximizes efficiency in multiview video.

In particular, in the case of a frame corresponding to GOP _C , four adjacent reference images can be used in the up / down / left / right directions. In this case, a high quality image can be reconstructed even with a few bits. That is, GOP _C encodes the reference GOP and GOP _B spatially pre-coded as reference pictures, and temporally follows a conventional two-dimensional coding scheme.

FIG. 10 is a diagram for explaining a bitstream storage location of a GGOP in consideration of view scalability according to the present invention. FIG.

The encoding method of the present invention supports scalability in order to support the existing 2D display, stereo display, and multi-view display when the decoding unit outputs the image by decoding the image.

In the GGOP structure of FIG. 7, the base GOP becomes a reference image at another point in time, and since the GOP itself can be decoded, the decoder completely reconstructs the 2D image when only the information on the GOP is extracted and decoded. can do. Therefore, base GOP is transmitted through the base layer.

The time point closest to the base layer among the time points corresponding to the GOP _B is transmitted through the enhancement layer # 1. With the information of the base layer and the information of the enhancement layer # 1, the decoder can reconstruct stereo images. The remaining time points of GOP _B are transmitted through Enhabcement layer # 2. The decoders having received the base layer and the enhancement layer # 1 and # 2 may restore N / 2 views. Frames corresponding to GOP _C are transmitted through enhancement layer # 3. In order to reconstruct all the N multiview images in the GOMV, both the base layer and the enhancement layer # 1, # 2, and # 3 are required.

On the other hand, the variance estimation may be degraded due to inconsistency between views. One coordinate is mapped to a certain position in a multiview image in three-dimensional space, and normal shift estimation can be performed only if the mapped points have the same luminance and color difference values. However, due to the mismatch of parameters between cameras, they may not have the same value but may have different values. In the case of the pixel displaying the same position as described above, if the luminance and the color difference are different, the reliability of the disparity estimation decreases.

Much of the performance of multi-view video encoders comes from the elimination of inter-view correlation, which is done using disparity estimation. Therefore, the performance of low variance estimation causes a decrease in the performance of a multiview video encoder.

Usually, the problem of inconsistency between viewpoints has been solved by preprocessing. However, the preprocessing process may not completely solve the problem of inconsistency between multi-view points, and after the preprocessing process, the image may be damaged compared to the original image, and it is not preferable because the preprocessing process has a low sharpness. Therefore, the present invention will propose a variation estimation method in consideration of the inconsistency between the multi-view video.

11 is a flowchart illustrating an inter-view balanced disparity estimating method (IBDE) of a multiview video encoding / decoding system according to the present invention.

와

값을 구한다. 이후, 상기 전역 보정 파라미터를 기준으로 초기 대표값

와

를 정하고 최적의 대표값들(a[n]과 b[n])을 구하게 된다.As shown in the figure, the correction shift estimation method of the present invention is a global correction parameter.

Wow

Find the value. Thereafter, an initial representative value based on the global correction parameter

Wow

And the optimal representative values (a [n] and b [n]).

과

은 전역 보정 파라미터를 기준으로 영상의 특성을 고려하여 일정한 간격으로 배치한다. 이 값들을 이용하여 변이 추정을 수행한 후 가장 좋은 성능을 내는 인덱스(index) n과 m을 선택하고, 각 인덱스에 대해서 확률 모델을 계산한다. 이 확률 모델을 'Lloyd max 기법'을 이용하여 최 적의 대표 보정 파라미터

과

을 선택하게 되는 것이다. 여기서,

와

으로 대표 보정 파라미터의 수는 초기 보정 파라미터의 수보다 적다. Initial calibration parameters

and

Are arranged at regular intervals taking into account the characteristics of the image based on the global correction parameters. After performing the disparity estimation using these values, we select the indexes n and m that have the best performance, and compute the probability model for each index. This probabilistic model is then optimized using the 'Lloyd max technique'

and

Will be selected. here,

Wow

As a result, the number of representative correction parameters is smaller than the number of initial correction parameters.

The representative calibration parameters thus selected are applied to the cost function in the compression step. The correction parameter for the chrominance image is obtained by using the disparity vector obtained by using the representative correction parameter to obtain a matching area, and calculating the correction parameter for the area.

Hereinafter, as illustrated in FIG. 11, the method for estimating the inter-view correction shift according to the present invention will be described in detail by dividing the initial stage and the compression stage.

First, the initial stage will be described.

An imbalance correction algorithm for eliminating luminance imbalance between images of digital stereo pairs has been proposed in many existing papers. In the present invention, this theory is basically used for the calculation of the global correction parameter. The basic assumption of this approach is the mismatch of parameters between cameras.

A simple linear transformation is performed on the luminance component of the reference viewpoint to the average and variance of the luminance component of the current viewpoint, thereby generating a reference viewpoint converted into a preprocessing process. This is shown in Equation 1 below.

와

은 각각 원래 참조 영상의 휘도 성분값과 보정된 참조영상의 휘도 성분을 의미하고, a와 b는 보정 파라미터이다. At this time,

Wow

Denotes the luminance component value of the original reference image and the luminance component of the corrected reference image, respectively, and a and b are correction parameters.

The average value and the variance of the converted reference time point are shown in Equation 2 below.

는 각각 참조 영상과 보정된 참조 영상의 평균값,

와

는 각각 참조 영상과 보정된 참조 영상의 분산값을 의미한다. 파라미터

,

는 다음의 수학식 3에 의해 구할 수 있다.

Are the mean values of the reference image and the corrected reference image, respectively,

Wow

Denote variance values of the reference image and the corrected reference image, respectively. parameter

,

Can be obtained by the following equation (3).

As mentioned above, two conditions must be met: the mean, standard deviation of the current image, and the mean and standard deviation of the converted right image are equal. Therefore, the values of a and b can be obtained through Equation 4 below.

과

의 배열에서 n과 m의 수는 영상의 특성을 고려하여 적당한 값을 선택한다.

과

의 값은 수학식 5와 같이 일정한 간격으로 배치되며,

의 값은

에 의해서 결정된다.

의 값들의 중간값은

로 설정되도록

의 값을 결정한다.

의 값들도 와 마찬가지로 설정한다.The same applies to chrominance components. However, this method is used only in the global correction parameter of the luminance component of two viewpoints. After the global calibration parameter is obtained, the initial calibration parameter is determined based on this value. To avoid the complexity of the algorithm

and

The number of n and m in the array is selected appropriately considering the characteristics of the image.

and

The values of are arranged at regular intervals as shown in Equation 5,

The value of

Determined by

The median of the values of

To be set to

Determine the value of.

Set the values of as in.

As a cost function for estimating block-based variation, SAD (Sum absolute difference) of Equation 6 will be used.

과

에 대해서 확률 모델을 필요로 하는데, 이는 직접적으로 변이 추정을 수행해야만 구할 수 있다. 초기 보정 파라미터를 적용한 새로운 SAD 비용 함수는 다음의 수학식 7과 같다.To determine representative calibration parameters at an early stage

and

We need a probabilistic model for, which can be obtained only by performing the variance estimation directly. The new SAD cost function applying the initial correction parameter is shown in Equation 7 below.

는 변이 벡터를 의미하고,

,

은 초기 보정 파라미터의 인덱스이다.

와

는 블록의 가로 및 세로 크기를 의미한다. 이 비용 함수를 이용하여 각 매크로 블록마다

와

의 인덱스를 결정하게 되고, 확률 모델을 만든다. 또한, 이 인덱스를 저장하여 후에 색차 보정 파라미터를 구할 때 사용한다.

Means the variation vector,

,

Is the index of the initial calibration parameter.

Wow

Means the horizontal and vertical size of the block. Using this cost function, each macro block

Wow

Determine the index of, and create a probabilistic model. In addition, this index is stored and used later to obtain color difference correction parameters.

과

에 대해서 확률 모델을 만들고 난 후, 로이드 맥스 방법(Lloyd max method)을 이용해서 대표 보정 파라미터

과

를 결정하게 된다. 여기서

와

은

과

더욱 작은 값으로 선택한다. 시점간 보정 변이 추정을 수행한 후 대표 보정 파라미터의 인덱스를 보내야 복호화단에서 영상을 복원할 수 있기 때문에 너무 큰 수의

와

을 선택할 경우 부호화량이 증가하게 되어 성능이 악화될 수 있기 때문이다. So the initial calibration parameters

and

After creating a probabilistic model for, use the Lloyd max method to represent the representative calibration parameters.

and

Will be determined. here

Wow

silver

and

Choose a smaller value. After performing the point-to-point correction variation estimation, it is necessary to send the index of the representative correction parameter so that the decoder can reconstruct the image.

Wow

This is because the encoding amount may be increased if the performance is deteriorated.

Hereinafter, the 'compression step' of FIG. 11 will be described.

과

를 이용하여 직접적인 변이 추정을 수행한다. 시점간 보정 변이 추정의 SAD 비용함수는 상기 수학식 7과 같이 검색 범위를 나타내는

와

외에도 보정 파라미터에 서도 반복 문을 수행해야만 하기 때문에 계산량의 증가를 야기한다. In this step, the representative calibration parameters obtained in the previous step

and

We perform direct variation estimation using. The SAD cost function of the inter-view corrected variance estimation represents a search range as shown in Equation 7 above.

Wow

In addition, because the iterative statement must be executed in the calibration parameter, it increases the amount of computation.

와

값을 곱하고 더하는 과정이 들어가기 때문에 더욱 많이 계산량이 증가하게 된다. 이런 문제점을 해결하기 위해 대표 보정 파라미터가 결정되고 난 후 다음의 수학식 8과 같이 영상에 대한 새로운 히스토그램을 생성한다. 이때, 모든 경우의 수를 생각하여 새로운

을 생성한다.Also, to correct the reference image within the SAD function,

Wow

The process of multiplying and adding values increases the amount of computation. To solve this problem, after the representative correction parameter is determined, a new histogram for the image is generated as shown in Equation 8 below. At this time, considering the number of all cases

Create

0부터 255까지의 휘도 성분을 의미한다. 모든 휘도 성분에 대해서

과

에 매칭되는 새로운 값을 계산한 값을

에 저장하는 것이다. 이 배열의 계산에는 많은 연산량을 필요로 하지 않는다. 새로운 히스토그램을 이용하여 비용함수를 정의하면 다음의 수학식 9와 같다.

Means a luminance component from 0 to 255. For all luminance components

and

Computes a new value that matches

To save on. The computation of this array does not require much computation. The cost function is defined using the new histogram as shown in Equation 9 below.

Equation (9) shows the same performance as in Equation (8) while reducing the amount of calculation.

In order to find the chrominance correction parameter, representative correction index information was stored in the process of estimating luminance image variation. In addition, since the disparity estimation was performed, disparity vector information was obtained. The disparity vectors may be used to determine which part of the reference picture the current macroblock matches.

Equation 4 is performed in the same manner as in the luminance component to match the average variance value of the chrominance signal of the matched reference image after matching the chrominance image using a disparity compensation technique. Apply to create a correction value for the chrominance component.

The chrominance correction parameter can be obtained in the same way as the luminance correction parameter, but in this case, since the encoding amount must be transmitted to the decoder for the chrominance correction parameter, there is a disadvantage in that the coding amount is increased. Use the method.

In this case, if only the index information of the luminance correction parameter is known, the correction parameter can be known for both the luminance and the color difference. Therefore, it is confirmed that not only the information to be encoded has similar performance compared to when the index is independently obtained.

The method of the present invention as described above may be implemented as a program and stored in a computer-readable recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes are possible in the art without departing from the technical spirit of the present invention. It will be clear to those of ordinary knowledge.

As described above, the present invention has the effect of increasing the coding efficiency while performing multi-view video encoding considering scalability by using the GOMV and GGOP structures.

In addition, the present invention has an effect of improving the performance of the multi-view video encoder by effectively removing the correlation between the multi-view video by using the inter-view corrected variance estimation to solve the imbalance between the multi-view video.

Claims (16)

delete In the multi-view video encoding / decoding method for performing encoding on a multi-view group (GOMV), which is a set of a plurality of (N) multi-view pictures within the same view, Performing encoding on a frame of a GOP including a base GOP-I picture in a GGOP in which the GOP is extended with a view axis; Performing encoding on a frame of a GOP including at least one or more GOP _B- P pictures using the reference GOP; And Using the reference GOP and the GOP _B Performing encoding by performing a disparity estimation in both directions on a frame of a GOP _C -B picture including only a GOP _C picture existing between the reference GOP and at least one or more of the GOP _Bs . Multi-view video encoding / decoding method comprising a. The method of claim 2, The encoding for the GOP _B is Performing compression with reference to a reference GOP and a precoded GOP _B spatially Multi-view video encoding / decoding method characterized in that. The method of claim 2, The encoding for the GOP _C is Spatially referencing the reference GOP and GOP _B Multi-view video encoding / decoding method characterized in that. The method of claim 2, The reference GOP is, Transmitted to an encoder through a base layer, the encoder reconstructing a two-dimensional image Multi-view video encoding / decoding method characterized in that. The method of claim 2, The time point closest to the base layer among the time points corresponding to the GOP _B , Transmitted to an encoder through a first enhancement layer, wherein the encoder reconstructs a stereo image using the base layer and the first high layer. Multi-view video encoding / decoding method characterized in that. The method of claim 2, GOP _B other than the GOP _B transmitted through the first higher layer, Transmitted to an encoder through a second higher layer, wherein the encoder restores N / 2 viewpoints using the base layer and the first and second higher layers. Multi-view video encoding / decoding method characterized in that. The method of claim 2, The GOP _C is, Transmitted to an encoder through a second high layer, wherein the encoder reconstructs N multi-view images using the base layer and the first, second, and third high layers. Multi-view video encoding / decoding method characterized in that. In the multi-view video inter-view corrected shift estimation method, The global representative parameter is obtained, the initial representative value is determined based on these values, the variance estimation is performed using these values, the index that performs the best performance is selected, and the probability model is calculated for each index. An initial step of determining an optimal representative correction parameter; A compression step of determining a correction parameter for the chrominance image using the disparity estimation vector obtained by using the representative correction parameter; And Application step of applying each correction value to images of different time The multi-view video inter-view correction variation estimation method comprising a. The method of claim 9, The initial stage, Global Compensation Parameter

And

Determining; remind

And

The initial calibration parameter

and

Determining; Performing variation estimation using a cost function; And Representative Calibration Parameter

and

Steps to determine The multi-view video inter-view corrected variation estimation method comprising a. The method of claim 10, The global compensation parameter

And

Is, Determined by the following equation The method for estimating the correction variation between multi-view video points of view, characterized in that the.

The method of claim 10, The cost function is Having the following equation The method for estimating the correction variation between multi-view video points of view, characterized in that the.

The method of claim 10, The representative correction parameter

and

silver, What to determine by the Lloyd Max method The method for estimating the correction variation between multi-view video points of view, characterized in that the. The method of claim 9, The compression step, The representative correction parameter

and

Estimating the variation using; And Determining the color difference correction parameter by matching the luminance and color shift vectors of the color image. The multi-view video inter-view corrected variation estimation method comprising a. The method of claim 14, The cost function for variance estimation is Having the following equation The method for estimating the correction variation between multi-view video points of view, characterized in that the.

The method of claim 14, Determining the color difference correction parameter, Determining which part of the reference image the current macroblock matches using representative correction index information stored in the luminance image disparity estimation and the disparity vector information. The method for estimating the correction variation between multi-view video points of view, characterized in that.

Images