CN100415002C - Coding and compression method of multi-mode and multi-viewpoint video signal - Google Patents

Abstract

The invention discloses a multi-mode multi-viewpoint video encoding method, which analyzes the time correlation and inter-viewpoint correlation of multi-viewpoint video signals, and the system's compression efficiency, encoding complexity, random access performance, and encoding of multi-viewpoint video encoding. Delay and other comprehensive performance requirements, adaptively select the predictive coding mode that is suitable for the characteristics of the current coding multi-view video signal and the comprehensive performance requirements of multi-view video coding from the candidate predictive coding modes to encode the multi-view video signal, instead of A multi-reference frame multi-viewpoint video predictive coding method with single-mode and computationally complex joint temporal and spatial prediction reduces the computational complexity of multi-viewpoint video signal coding and compression, while ensuring coding compression efficiency and improving random access performance.

Description

Coding and compression method of multi-mode and multi-viewpoint video signal

technical field

The invention relates to a method for encoding and compressing multi-viewpoint video signals, in particular to a method for encoding and compressing multi-mode multi-viewpoint video signals based on the analysis of time correlation and inter-viewpoint correlation of multi-viewpoint video signals.

Background technique

3DAV (three-dimensional audio and video) is the development direction of the new generation of audio and video technology. As the core technology in 3DAV applications such as FTV (Free Viewpoint Television) and 3DTV (3D Television), multi-viewpoint video coding technology aims to solve problems such as compression, interaction, storage and transmission of 3D interactive video. The multi-viewpoint video signal is a group of video signals obtained by shooting the actual scene by the camera array. It can provide video image information from different angles of the shooting scene. Using one or more viewpoint information, the information of any viewpoint can be synthesized to achieve freedom. The purpose of switching viewpoints. Multi-viewpoint video is a new type of video with stereoscopic and interactive operation functions, and will have broad application prospects in interactive multimedia applications (such as digital entertainment, remote monitoring, and distance education, etc.) oriented to broadband and high-density storage media. Figure 1 is a schematic diagram of a commonly used multi-view video system at present. This system can perform imaging, encoding and compression, transmission, reception, decoding, and display of multi-view video signals, and the encoding and compression of multi-view video signals is the core of the entire system. core part.

Multi-viewpoint video signals have a huge amount of data, which is not conducive to network transmission and storage, as well as system resource consumption (high computational complexity, high storage capacity requirements, high power consumption, etc.), client random access (including fast forward, fast rewind, Viewpoint switching and viewing time freezing, viewing point sliding and other viewing access methods) and other issues. Therefore, how to improve the compression efficiency of multi-view video signal coding, reduce the resource consumption of the system, and enable the system to have flexible random access, partial decoding and rendering, etc., has become the current international research on multi-view video coding methods and standards. The pursuit of the goal has also become a research hotspot.

Utilizing the temporal correlation and inter-view correlation of multi-view video signals, motion compensation prediction and parallax compensation prediction are the basic ideas for encoding and compressing multi-view video signals. The time correlation and inter-viewpoint correlation of multi-viewpoint video signals change with the camera density of the imaging system, illumination changes, camera and object motion and other factors. When the cameras are dense and the imaging intensity of each viewpoint is consistent, the correlation between the viewpoints of the multi-viewpoint video signal is strong; when the cameras are sparse and the imaging intensity of each viewpoint is inconsistent, the temporal correlation of the multi-viewpoint video signal is relatively strong, while Correlation is weak. In addition, camera and object motion also have an impact on the correlation of multi-view video signals. Therefore, if a multi-view video coding framework with a single prediction structure mode is used to encode multi-view video signals with different correlation characteristics, it will either use a very complex multi-reference frame prediction mode to ensure high coding compression efficiency, but Causes the multiplied increase of the computational complexity and space complexity of the encoder, the decrease of random access performance, and the increase of encoding delay; Correlation, thus restricting the improvement of coding compression efficiency.

Due to the influence of factors such as different camera densities, illumination changes, camera and object motions, multi-view video signals show different statistical characteristics of content correlation in time and between viewpoints. The complex temporal and inter-view content correlation characteristics of multi-view video signals make the existing multi-view video coding schemes with a single structure unable to adapt well to the compression of multi-view video signals with complex and changeable content correlation characteristics. Obtaining an effective compression effect with comprehensive performance (encoding compression efficiency, random access, system resource consumption, partial decoding and rendering, encoding delay, etc.) is also an important problem common to existing multi-view video encoding methods.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for coding and compressing multi-view video signals, which can improve the comprehensive performance of multi-view video coding and compression while reducing the coding complexity.

The technical scheme adopted by the present invention to solve the above-mentioned technical problems is as follows: a multi-mode multi-viewpoint video signal encoding and compression method, the encoder is set as a multi-viewpoint video predictive encoding module, a correlation statistical analysis module, a prediction mode selection module and a mode update The trigger module has four functional modules. For the input multi-view video signal, at the beginning of encoding, the initial predictive encoding mode can be determined based on known information, such as camera array parameters, encoding complexity requirements, random access performance requirements, etc., by The described multi-viewpoint video predictive encoding module encodes, and then encodes according to the following steps: ① by the described prediction mode selection module according to the statistical analysis of the multi-viewpoint video signal correlation characteristics obtained by the statistical correlation analysis module and the According to the comprehensive performance requirements of multi-view video coding, such as compression efficiency, coding complexity, random access performance, and coding delay, dynamically select the predictive coding mode suitable for the characteristics of the multi-view video signal currently being coded from the candidate predictive coding modes; ② After encoding the input multi-view video signal with the selected predictive coding mode by the multi-view video predictive coding module, the encoded and compressed code stream signal is output; ③ when the mode update triggers the When the mode update trigger condition is not met, keep the current predictive coding mode, and when the mode update trigger condition in the mode update trigger module is satisfied, restart the correlation statistical analysis module to select and update the predictive coding mode.

The candidate predictive coding modes can be divided into three categories: the first type is a predictive coding mode applicable to multi-viewpoint video signals based on time correlation, and this type of predictive coding mode is mainly based on motion compensation prediction; the second type The first category is the predictive coding mode suitable for multi-viewpoint video signals based on inter-viewpoint correlation, and this kind of predictive coding mode is mainly based on parallax compensation prediction; The predictive coding mode of the viewpoint video signal, this type of predictive coding mode is a joint predictive coding mode that takes into account the temporal and spatial domains. Each of the above three categories of predictive coding modes can be composed of several predictive coding modes, which are respectively suitable for multi-view video signal coding with different correlation characteristics, and different requirements for the comprehensive performance of multi-view video coding (such as coding complexity, encoding compression efficiency, random access performance, encoding delay, etc.).

The statistical analysis of the described correlation statistical analysis module is to perform statistical analysis on the time correlation and inter-viewpoint correlation of the encoded or being encoded image group GOP (Group of picture), and define the correlation coefficient α for characterization to obtain The time correlation of the video signal is compared with the strength of the correlation between viewpoints.

In the correlation statistical analysis module, the number n _i ^D of intra-coded blocks in the image frame that is coded or coded by using only parallax compensation prediction and only using motion compensation prediction can be calculated. The number n _i ^P of intra-frame coding blocks in the coded image frame is counted, and the relationship between time and inter-viewpoint correlation of the current multi-view video signal is described by the proportional relationship between n _i ^D and n _i ^P.

In the correlation statistical analysis module, it is also possible to use only the prediction error of the image frame encoded by parallax compensation prediction and the prediction of the image frame encoded by only motion compensation prediction for the image group that has been encoded or is being encoded The proportional relationship of the error is used to analyze the time of the current multi-viewpoint video signal and the relationship between the strength and weakness of the correlation between viewpoints.

The prediction mode selection module is based on the multi-viewpoint video signal correlation characteristics obtained through statistical analysis by the correlation statistical analysis module, and the multi-viewpoint video coding compression efficiency, coding complexity, random access performance, coding delay, etc. The performance requirements for prediction mode selection are as follows:

(1) When the time correlation is obviously stronger than the inter-viewpoint correlation, further judge whether the distribution of the correlation in the time domain is relatively balanced, or whether the time correlation of the nearest moment is obviously stronger than that of the next nearest moment , select the prediction coding mode mainly based on motion compensation prediction;

(2) When the temporal correlation is obviously weaker than the inter-view correlation, select the predictive coding mode based on parallax compensation prediction;

(3) When the temporal correlation is roughly equivalent to the inter-view correlation, choose a joint predictive coding mode that takes both temporal and spatial domains into consideration.

The mode update triggering module may adopt a mode update scheme based on video content, and determine whether to re-enable the prediction mode selection module to update according to the variation of the correlation coefficient α obtained in the correlation statistical analysis module. Predictive coding mode.

The mode update triggering module may also adopt a timing update triggering method to regularly open the correlation statistical analysis module to perform statistical analysis on the time correlation and inter-viewpoint correlation of multi-viewpoint video signals, and enable the prediction mode selection module to determine the predictive coding mode.

In the multi-mode multi-viewpoint video encoder, the prediction structures of all candidate multi-viewpoint video prediction coding modes can have certain commonality, that is, the image frames located at the same moment as the intra-frame coding frames in the candidate prediction coding modes and The image frames located at the same viewpoint as the intra-coded frame are coded before other image frames in the image group, and the above-mentioned image frames have the same prediction method in all candidate prediction modes, and these image frames that are coded first can be coded At the same time as the image frame, obtain the correlation statistical analysis results of the multi-viewpoint video signal currently being encoded, and determine in time which predictive encoding is adopted for other frames in the image group currently being encoded after the encoding of these first encoded image frames is completed Structure, that is, from all candidate predictive coding modes, finally select a predictive coding mode that is suitable for the characteristics of the current multi-viewpoint video signal and the comprehensive performance requirements of multi-viewpoint video coding for coding.

Aiming at the phenomenon that the multi-viewpoint video signal time and content correlation between viewpoints change with different factors such as multi-viewpoint camera density, illumination, camera and object motion, the present invention proposes an analysis based on multi-viewpoint video signal time correlation and inter-viewpoint correlation And the multi-mode multi-view video coding framework required by the comprehensive performance of multi-view video coding. According to the changes of multi-view camera density, illumination, camera and object motion, different candidate predictive coding modes are designed. Through the multi-view video signal Simple statistical analysis of time correlation and inter-view correlation, as well as different requirements for the comprehensive performance of multi-view video coding (such as coding complexity, coding compression efficiency, random access performance, coding delay, etc.), from candidate predictive coding Among the modes, the predictive coding mode suitable for the characteristics of the current multi-viewpoint video signal is dynamically selected, thereby improving the comprehensive performance of the multi-viewpoint video signal coding.

Compared with the prior art, the advantage of the present invention is that by analyzing the temporal correlation of multi-view video signals and the inter-view correlation, dynamic selection is suitable for the characteristics of the currently encoded multi-view video signal and the comprehensive performance requirements of multi-view video coding. Predictive coding mode to replace the multi-reference frame predictive coding method of joint time and space prediction in the existing single mode, so as to effectively reduce the computational complexity of multi-viewpoint video signal coding and compression, and improve the random access of multi-viewpoint video system Performance, while ensuring the encoding compression performance.

Description of drawings

Fig. 1 is a schematic diagram of a multi-viewpoint video system;

Fig. 2 is a schematic diagram of the structure and encoding process of the multi-mode multi-viewpoint video encoder of the present invention;

Figure 3a is the first type of candidate predictive coding mode in the embodiment;

Figure 3b is a second type of candidate predictive coding mode in the embodiment;

Figure 3c is a third type of candidate predictive coding mode in the embodiment;

FIG. 4 is a sequential predictive coding mode PSVP using P frames;

FIG. 5 is a sequential predictive coding mode BSVP using B frames;

Fig. 6 is the multi-view video prediction coding mode of Mpicture;

Fig. 7 is a Joint multi-viewpoint video test sequence;

Fig. 8 is the average rate-distortion curve of the Xmas sequence part in the Joint multi-viewpoint video test sequence;

Fig. 9 is the average rate-distortion curve of the exit sequence part in the Joint multi-viewpoint video test sequence;

Figure 10 is the average rate-distortion curve of the ballroom sequence part in the Joint multi-viewpoint video test sequence;

Figure 11 is the average rate-distortion curve of the Joint multi-viewpoint video test sequence.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

Here, taking the representative 5×7 image group structure as an example (as shown in Fig. The four functional modules of the viewpoint video encoder and their cooperative working methods are described in detail.

1) Multi-view video predictive coding module

This module is responsible for encoding and compressing multi-viewpoint video signals, that is, using a candidate predictive coding mode dynamically selected by the prediction mode selection module to encode the current multi-viewpoint video signal.

According to the temporal correlation and inter-view correlation of multi-view video signals, the candidate multi-view video predictive coding modes are divided into three categories. The first category is the predictive coding mode suitable for multi-view video signals with temporal correlation ; The second category is the predictive coding mode suitable for multi-viewpoint video signals based on inter-viewpoint correlation; the third category is the predictive coding mode suitable for multi-viewpoint video signal with balanced temporal correlation and inter-viewpoint correlation. Each of the above three categories of predictive coding modes can be composed of several predictive coding modes to adapt to multi-view video signal coding with different correlation characteristics, and different requirements for the comprehensive performance of multi-view video coding (such as complex coding degree, encoding compression efficiency, random access performance, encoding delay, etc.).

Figure 3a, Figure 3b and Figure 3c respectively show the three different types of predictive coding modes adopted. In the figure, I represents the intra-frame coding frame, D represents the parallax compensation predictive coding frame, P represents the motion compensation predictive coding frame, and P' represents For time-space bidirectional predictive coding frames, D and P frames can be referred to, and B' is a time-space joint prediction frame, and D, P, and P' frames can be referred to. The predictive coding mode in Figure 3a is mainly based on motion compensation prediction, which is suitable for multi-view video signal coding based on temporal correlation, and belongs to the first type of predictive coding mode; the predictive coding mode in Figure 3b is mainly based on parallax compensation prediction, applicable It belongs to the second type of predictive coding mode for multi-view video signal coding based on inter-view correlation; the predictive coding mode in Fig. Video signal coding belongs to the third type of predictive coding mode. In this embodiment, there is only one candidate mode for each type of predictive coding mode in the three categories, and multiple different predictive coding modes can be designed according to requirements when the present invention is actually used.

2) Correlation statistical analysis module

The correlation coefficient α is defined to characterize the strength contrast between the temporal correlation and inter-viewpoint correlation of the video signal. This coefficient can be obtained by statistical analysis of the temporal correlation and inter-viewpoint correlation of the encoded or being encoded image group.

In multi-mode multi-viewpoint video coding, for images at the same moment as I frame but located at different viewpoints, such as several D frames located on the left and right sides of I frame in Figure 3a, Figure 3b and Figure 3c, only through parallax compensation Encoding, the number of I blocks (i.e. intra-frame coded blocks) in the D frame is expressed as n _i ^D ; for images with the same viewpoint but different moments with the I frame, as shown in Figure 3a, Figure 3b and Figure 3c, they are located above and below the I frame Several P frames on side 2 (actually shown as frames before and after the I frame in terms of time) are coded only by motion compensation prediction, and the number of I blocks in the P frame is expressed as n _i ^P . The correlation coefficient α can be defined as

$\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; no</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; /</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; P</span>}}$

Wherein, n and m represent the frame numbers of the D frame and the P frame used to calculate the correlation coefficient respectively. The correlation coefficient α can be used to characterize the time correlation of the video signal and the strength and weakness of the correlation between viewpoints. Moreover, the number of I-blocks required to calculate α can be statistically obtained at the same time of encoding, and the additional calculation overhead is extremely low. Therefore, α can be used to effectively implement statistical analysis of video signal correlation in multi-mode and multi-view video coding. In this embodiment, the correlation coefficient α is calculated by using the proportional relationship between the number of I blocks in the D frame and the P frame, and the threshold method is used in the prediction mode selection module from the three candidate predictions shown in Fig. 3a, Fig. 3b and Fig. 3c Among the encoding modes, one predictive encoding mode is finally selected and submitted to the multi-view video predictive encoding module for encoding.

In addition to the above schemes, it is also possible to use the prediction errors (such as SAD values) of those image frames (D frames) that are coded or are coded only using disparity compensation prediction for coding, and those that only use motion compensation prediction for coding The proportional relationship of the prediction error of the image frame (P frame), statistical analysis of the time of the current multi-viewpoint video signal and the strength of the correlation between the viewpoints.

3) Prediction mode selection module

According to the statistical analysis results of multi-viewpoint video signal correlation of the correlation statistical analysis module, and the comprehensive performance requirements of multi-mode multi-viewpoint video coding such as compression efficiency, coding complexity, random access performance, and coding delay, the candidate predictive coding Select a predictive coding mode that is suitable for the characteristics of the current multi-viewpoint video signal and the comprehensive performance requirements of coding. The predictive coding mode is selected as follows:

(1) When the time correlation is obviously stronger than the inter-viewpoint correlation, it can be further judged whether the distribution of the correlation in the time domain is relatively balanced, or whether the time correlation at the nearest moment is obviously stronger than that at the next nearest moment to select and determine an appropriate Type 1 predictive coding mode.

(2) When the temporal correlation is obviously weaker than the inter-view correlation, select a type 2 predictive coding mode mainly based on disparity compensation prediction, so as to use this predictive coding mode in the multi-view video predictive coding module for coding.

(3) When the temporal correlation is approximately equivalent to the inter-view correlation, select a third type of joint predictive coding mode that takes into account temporal and spatial domains.

In this embodiment, due to the three candidate predictive coding modes adopted, the image frame located on the central cross in Fig. 3a, Fig. 3b and Fig. These image frames located on the central cross in the mode have the same prediction method, so while encoding these image frames, the n _i ^D and n _i ^P required by the correlation statistical analysis module can be obtained, so that the currently encoding The statistical analysis results of the correlation of multi-viewpoint video signals, in order to determine in time which prediction mode to adopt for other frames in the image group after the encoding of these image frames located on the central cross is completed, that is, from the Among the three candidate predictive coding modes, a predictive coding mode suitable for the characteristics of the current multi-view video signal is finally selected, and submitted to the multi-view video predictive coding module for coding.

4) Mode update trigger module

A mode update scheme based on video content can be used, that is, according to the change of the correlation coefficient α obtained in the correlation statistical analysis module, the threshold method is used to determine whether to re-enable the prediction mode selection module to update the corresponding prediction coding mode; or In the way of timing updating and triggering, the correlation statistical analysis module is regularly enabled to statistically analyze the time correlation and inter-viewpoint correlation of multi-viewpoint video signals, and the prediction mode selection module is enabled to determine the predictive coding mode to be adopted. This embodiment adopts a mode update scheme based on video content.

The following describes the performance of multi-viewpoint video coding in this embodiment:

1) Random access performance of multi-mode multi-view video coding scheme

For multi-viewpoint video, its random access includes fast forward, rewind, viewpoint switching, viewing moment freezing, viewpoint sliding and other access methods. Assuming v views for encoding, the total number of multi-view video frames s=v×t for each view t frames is limited. Let x _i represent the number of frames that need to be decoded in advance before decoding the i-th frame, p _i is the probability of the user randomly accessing the i-th frame, then the mathematical expectation of the random access cost ${\mathrm{E.}}_{\mathrm{no}}=\underset{t=1}{\overset{v\&times;t}{\mathrm{\&Sigma;}}}{x}_i{p}_i$ It is an important index to evaluate the support degree of a predictive coding mode n to random access. The higher the cost, the lower the decoder's ability to support random access, and the more resources consumed to support random access. Let k _n be the probability of encoding a multi-view video signal using the nth predictive coding mode, and the number of candidate predictive coding modes is N, then the random access cost of multi-mode multi-view video coding can be expressed as $\mathrm{E.}\left(x\right)=\underset{\mathrm{no}=1}{\overset{N}{\mathrm{\&Sigma;}}}(k_{\mathrm{no}}\&times;{\mathrm{E.}}_{\mathrm{no}}).$

The coding probability k _n of each mode in multi-mode multi-view video coding is directly related to the characteristics of the actual multi-view video signal. In this embodiment, N=3, and assuming that the encoding probability of each mode is the same, that is, k _n =1/3 (n=1, 2, 3), the random access costs of different schemes are shown in Table 1. In the table, PSVP and BSVP respectively represent the sequential prediction method using P frame and B frame, and their predictive coding modes are shown in Fig. 4 and Fig. 5 respectively. Mpiture is the Mpicture multi-viewpoint video coding method developed by Japan Fujii et al., and its predictive coding mode is shown in FIG. 6 . PSVP, BSVP and Mpiture are single-mode multi-reference frame predictive coding methods. MMVC is a multi-mode multi-viewpoint video coding method of the present invention using three candidate predictive coding modes as shown in FIG. 3 (this embodiment is a representative of the solution of the present invention). It can be seen from Table 1 that in terms of random access performance, PSVP is the worst, while BSVP and Mpicture are relatively better. However, the multi-mode multi-viewpoint video coding method MMVC of the present invention has the lowest random access cost, compared with PSVP, BSVP and Mpicture methods, its random access cost is reduced by 49% to 72%, and the random access performance is obviously improved.

2) Computational complexity of multi-mode multi-view video coding scheme

The high-precision disparity compensation prediction and motion compensation prediction based on the H.264/AVC coding framework account for more than 75% of the computational complexity of the entire multi-view video encoder. Therefore, the disparity compensation prediction and The number of motion-compensated predictions characterizes the computational complexity of the entire encoder. Computational complexity comparison of various schemes is shown in Table 1. Due to the use of multiple reference frame methods, the computational complexity of PSVP, BSVP and Mpicture schemes is very large, especially the BSVP and Mpicture methods. Compared with the PSVP, BSVP and Mpicture schemes, the computational complexity of the scheme of the present invention is relatively reduced by 29% to 57%.

Table 1 Comparison of Random Access Cost and Computational Complexity of the MMVC of the present invention

encoding scheme E(X) Random access cost multiplier Computational complexity Computational complexity multiple PSVP 11.0 364% 58 141% BSVP 7.5 248% 83 202% Mpicture 6.0 199% 97 237% MMVC 3.02 100% 41 100%

3) Rate-distortion performance of multi-mode multi-view video coding schemes

In order to evaluate the coding efficiency of the MMVC scheme of the present invention, based on the H.264/AVC (JM8.5mainprofile) video coding framework, a multi-viewpoint video coding experiment was carried out (quantization parameters QP are 24, 30, 36, 40 respectively). The multi-viewpoint video test sequence uses the multi-viewpoint test sequence set of Xmas (camera distance 9mm, high correlation between viewpoints), exit (slow motion, large parallax, camera distance 19.5cm) and ballroom (vigorous movement) from Tanimoto Lab and MERL , all three sequences were captured by a parallel camera system with a resolution of 640×480. Select 5 viewpoints, 5 scenes, and 5 image groups in each scene, and stitch them into a Joint sequence as shown in Figure 7, that is, each viewpoint video has 5×5×7=175 frames. In the experiment, the scene switching of the actual video is simulated by sequence splicing. In this embodiment, MMVC can adaptively select an appropriate predictive coding mode from the candidate predictive coding modes shown in FIG. 3a, FIG. 3b and FIG. 3c according to the video content Encode the Joint sequence.

Figures 8, 9, 10, and 11 are comparisons of the rate-distortion performance of joint test sequence encoding using MMVC of this embodiment and methods such as sequential prediction methods PSVP, BSVP, and Mpicture. Among them, Figures 8, 9, and 10 are the average rate-distortion curves of the Xmas, exit, and ballroom sequences in the Joint sequence, respectively. The overall average rate-distortion curve of the Joint sequence shown in Figure 11 shows that the rate-distortion performance of MMVC is basically equivalent to that of BSVP, PSVP and Mpicture.

In summary, compared with the prior art, the present invention has the advantage of dynamically selecting the characteristics of the currently encoded multi-view video signal and the multi-view video signal by analyzing the temporal correlation and inter-view correlation of the multi-view video signal. The predictive coding mode required by the comprehensive coding performance can replace the multi-reference frame predictive coding method of the existing single mode, which is computationally complex and joint temporal and spatial prediction, so as to effectively reduce the computational complexity of multi-viewpoint video signal coding and compression, and improve the performance of multi-viewpoint video. The random access performance of the system, while ensuring the encoding compression performance.

It is obvious that the multi-view video predictive coding mode is not limited to the form of this embodiment, so the invention is not limited to the specific details and examples shown here without departing from the spirit and scope of the general concept defined by the claims and equivalents. Example with description.

Claims (9)

1. A multi-mode multi-viewpoint video signal coding compression method is characterized in that encoder is set to four functional modules of multi-viewpoint video predictive coding module, correlation statistical analysis module, prediction mode selection module and mode update trigger module, for The input multi-viewpoint video signal first determines the initial predictive coding mode according to the known information, encodes by the multi-viewpoint video predictive coding module, and then performs coding according to the following steps: ① by the prediction mode selection module according to the described The statistical correlation analysis module of the multi-view video signal obtained by statistical analysis and the requirements for the comprehensive performance of multi-view video coding, dynamically select from the candidate predictive coding modes to determine the prediction suitable for the characteristics of the multi-view video signal currently being coded Coding mode; 2. After the multi-view video predictive coding module encodes the input multi-view video signal with the selected predictive coding mode, the encoded and compressed code stream signal is output; 3. When the mode update triggers When the mode update triggering condition in the module is met, the correlation statistical analysis module is restarted to select and update the predictive coding mode. 2. The method for encoding and compressing multi-mode and multi-viewpoint video signals as claimed in claim 1, wherein the candidate predictive coding modes are divided into three categories: the first category is applicable to multi-viewpoints based on temporal correlation The predictive coding mode of the video signal, this type of predictive coding mode is based on motion compensation prediction; the second type is the predictive coding mode suitable for multi-viewpoint video signals based on inter-viewpoint correlation, this type of predictive coding mode is based on parallax compensation Prediction-based; the third category is the predictive coding mode of multi-view video signals suitable for the balance of temporal correlation and inter-view correlation. 3. The method for encoding and compressing multi-mode and multi-viewpoint video signals as claimed in claim 1, wherein the statistical analysis of the statistical analysis module of correlation is to the temporal correlation between the encoded or the image group being encoded and between the viewpoints. The correlation is statistically analyzed, and the correlation coefficient α is defined to characterize the strength comparison between the time correlation of the obtained video signal and the correlation between viewpoints. 4. The method for encoding and compressing multi-mode and multi-viewpoint video signals as claimed in claim 3, characterized in that in the described correlation statistics analysis module, only parallax compensation prediction is used for encoding in the image group that has been encoded or is being encoded The number n _l ^D of the intra-coded blocks in the image frame and the number n _l ^P of the intra-coded blocks in the image frame encoded only by motion compensation prediction are counted, and the ratio between n _l ^D and n _l ^P is calculated Describe the relationship between time and inter-view correlation strength of the current multi-view video signal. 5. The method for encoding and compressing multi-mode and multi-viewpoint video signals as claimed in claim 3, characterized in that in the statistical correlation analysis module, the group of images that have been encoded or are being encoded are only predicted using parallax compensation The proportional relationship between the prediction error of the coded image frame and the prediction error of the coded image frame using only motion compensation prediction is used to analyze the temporal and inter-view correlation strength of the current multi-viewpoint video signal. 6. The multi-mode multi-viewpoint video signal coding and compression method as claimed in claim 2, wherein said prediction mode selection module obtains multi-viewpoint video signal correlation characteristics and The method of selecting the prediction mode for the comprehensive performance requirements of multi-view video coding is as follows: (1) When the time correlation is obviously stronger than the inter-viewpoint correlation, further judge whether the distribution of the correlation in the time domain is relatively balanced, or whether the time correlation of the nearest moment is obviously stronger than that of the next nearest moment , select the prediction coding mode mainly based on motion compensation prediction; (2) When the temporal correlation is obviously weaker than the inter-view correlation, select the predictive coding mode based on parallax compensation prediction; (3) When the temporal correlation is roughly equivalent to the inter-view correlation, choose a joint predictive coding mode that takes both temporal and spatial domains into account. 7. The method for encoding and compressing multi-mode and multi-viewpoint video signals as claimed in claim 1, wherein said mode update trigger module adopts a mode update scheme based on video content, and obtains according to said correlation statistical analysis module The variation of the correlation coefficient α determines whether to re-enable the prediction mode selection module to update the prediction coding mode. 8. The method for encoding and compressing multi-mode and multi-viewpoint video signals as claimed in claim 1, wherein said mode update trigger module adopts the mode of timing update trigger, and regularly opens said correlation statistics analysis module for multi-viewpoint video The temporal correlation and inter-view correlation of the signal are statistically analyzed and the predictive mode selection module is enabled to determine the predictive coding mode. 9. The multi-mode multi-viewpoint video signal coding and compression method as claimed in claim 1, wherein in all candidate modes of said candidate predictive coding mode, the image frame located at the same moment as the intra-frame encoded frame and the image frame located at the same time as the intra-frame encoded frame The image frames of the same viewpoint of the coded frame are encoded before other image frames in the image group, and these image frames encoded first have the same prediction mode in all candidate modes.

Images