CN100415002C - Multi-mode multi-viewpoint video signal code compression method - Google Patents
Multi-mode multi-viewpoint video signal code compression method Download PDFInfo
- Publication number
- CN100415002C CN100415002C CNB2006100528959A CN200610052895A CN100415002C CN 100415002 C CN100415002 C CN 100415002C CN B2006100528959 A CNB2006100528959 A CN B2006100528959A CN 200610052895 A CN200610052895 A CN 200610052895A CN 100415002 C CN100415002 C CN 100415002C
- Authority
- CN
- China
- Prior art keywords
- correlation
- viewpoint
- video signal
- mode
- predictive coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000007906 compression Methods 0.000 title claims description 37
- 230000006835 compression Effects 0.000 title claims description 37
- 238000007619 statistical method Methods 0.000 claims description 37
- 230000002123 temporal effect Effects 0.000 claims description 33
- 238000004458 analytical method Methods 0.000 abstract description 5
- 238000004364 calculation method Methods 0.000 abstract description 4
- 238000012360 testing method Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 238000010219 correlation analysis Methods 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a video encoding method with multi-mode and multi-view. According to the analysis of the time correlation for the multi-view video signals and view correlation , as well as the comprehensive characters such as the compressing efficiency, complexity of encoding, random access performance and coding delay demanded by the system, the method adaptively dynamic-chooses the predictive coding mode which meets the features and the comprehensive encoding requirements of the current signals to encode them, in order to replace the method with single mode and complex calculation that forecasts by the union of time and space, and decrease the calculation complexity of compressing the signal coding.
Description
Technical field
The present invention relates to the code compression method of multi-view point video signal, especially relate to multi-mode multi-viewpoint video signal code compression method based on correlation analysis between the temporal correlation of multi-view point video signal and viewpoint.
Background technology
3DAV (three-dimensional audio frequency and video) is the developing direction of audio frequency and video technology of new generation.Core technology in using as FTV (free view-point TV), 3DTV 3DAV such as (three-dimensional televisions), multiple view video coding technology are intended to solve problems such as the compression of 3D interactive video, mutual, storage and transmission.Multi-view point video signal is actual scene to be taken the one group of vision signal that obtains by camera array, it can provide the video image information of photographed scene different angles, utilize one or more view information can synthesize the information of any viewpoint, reach the purpose of freely switching viewpoint.Multi-view point video is a kind of novel video with third dimension and interactive operation function, will have wide practical use in the interactive multimedia application (as digital amusement, remote monitoring, long-distance education etc.) towards broadband and high-density storage media.Fig. 1 is the schematic diagram of the present multi-view video system of using always, this system can carry out imaging, encoding compression, transmission, reception, decoding, demonstration of multi-view point video signal etc., and wherein the encoding compression of multi-view point video signal is the core of whole system.
Multi-view point video signal exist data volume huge, be unfavorable for Network Transmission and storage, and system resources consumption (high computation complexity, high storage capacity requirement, high power consumption etc.), user side random access (comprise F.F., rewind down, viewpoint switch and watch constantly freeze, access mode is watched in viewpoint slip etc.) etc. problem.Therefore, how to improve the compression efficiency of multi-viewpoint video signal code, the resource consumption of reduction system, make system have performances such as random access, partial decoding of h and drafting flexibly, become the target of being pursued in present multi-view point video encoding method in the world and the standard formulation research, also become the research focus.
Utilize the temporal correlation of multi-view point video signal, the correlation between viewpoint, adopting motion compensated prediction, parallax compensation prediction is the basic ideas of carrying out the multi-viewpoint video signal code compression.Correlation changes with the factors vary such as camera density, illumination variation, camera and object motion of imaging system between the temporal correlation of multi-view point video signal, viewpoint.When camera is intensive, when each viewpoint imaging intensity is consistent, correlation is strong between the viewpoint of multi-view point video signal; When camera is more sparse, when each viewpoint imaging intensity is inconsistent, the temporal correlation of multi-view point video signal then relatively strong and between viewpoint correlation a little less than.In addition, camera and object motion also exert an influence to the correlation of multi-view point video signal.Therefore, if adopting the multiple view video coding framework with single predict pattern encodes to the multi-view point video signal with different correlation characteristics, to cause it or adopt very complicated multi-reference frame predictive mode to guarantee high encoding compression efficient, the rising at double of encoder computation complexity and space complexity, random access performance descend, encoding time delay increases but cause; Adopt simple relatively predict, but encoder is difficult to make full use of correlation between the temporal correlation of multi-view point video signal and viewpoint, thus the raising of restriction encoding compression efficient.
Because the influence of factors such as different cameral density, illumination variation, camera and object motion, cause multi-view point video signal on its time, show different relevance statistical properties between viewpoint.The time of this complexity of multi-view point video signal is gone up the relevance characteristic that reaches between viewpoint, make the multiple view video coding scheme of existing single structure can not finely be adapted to the compression of relevance characteristic multi-view point video signal complicated and changeable, be difficult to obtain effectively compression effectiveness of combination property (encoding compression efficient, random access, system resources consumption, partial decoding of h and drafting, encoding time delay etc.), this also is the existing ubiquitous major issue of multi-view point video encoding method.
Summary of the invention
Technical problem to be solved by this invention provides a kind of multi-viewpoint video signal code compression method, when reducing encoder complexity, improves the combination property of multiple view video coding compression.
It is as follows that the present invention solves the problems of the technologies described above the technical scheme that is adopted: a kind of multi-mode multi-viewpoint video signal code compression method, encoder is arranged to multi-view point video predictive coding module, the correlation statistical analysis module, predictive mode is selected module and four functional modules of schema update trigger module, multi-view point video signal to input, when coding is initial, can be earlier according to Given information, as the camera array parameter, the encoder complexity requirement, random access performance requirement etc., determine the initial predicted coding mode, encode by described multi-view point video predictive coding module, encode according to the following steps then: 1. select the module multi-view point video signal correlative character that statistical analysis obtains according to described correlation statistical analysis module and to the compression efficiency of multiple view video coding by described predictive mode, encoder complexity, the random access performance, the requirement of several combination properties of encoding time delay, Dynamic Selection determines to be fit to the predictive coding pattern of current multi-view point video signal characteristics of encoding from candidate's predictive coding pattern; 2. after the multi-view point video signal of input being encoded with this selected predictive coding pattern by described multi-view point video predictive coding module, output encoder compressed code flow signal; 3. when the schema update trigger condition in the described schema update trigger module does not satisfy, keep current predictive coding pattern, when the schema update trigger condition in the described schema update trigger module satisfies, again open described correlation statistical analysis module, to select to upgrade the predictive coding pattern.
Described candidate's predictive coding pattern can be divided into three major types: the 1st class is the predictive coding pattern that is applicable to based on the multi-view point video signal of temporal correlation, and such predictive coding pattern is based on motion compensated prediction; The 2nd class is the predictive coding pattern that is applicable to based on the multi-view point video signal of correlation between viewpoint, and such predictive coding pattern is predicted as the master with parallax compensation; The 3rd class is the predictive coding pattern that is applicable to the multi-view point video signal of correlation equilibrium between temporal correlation and viewpoint, and such predictive coding pattern is when taking into account, the associated prediction coding mode in spatial domain.Each class in the above-mentioned three major types predictive coding pattern can be made up of several predictive coding patterns again, be applicable to multi-viewpoint video signal code respectively, and the difference of multiple view video coding combination property is required (as encoder complexity, encoding compression efficient, random access performance, encoding time delay etc.) with different correlation characteristics.
The statistical analysis of described correlation statistical analysis module be to encode or just between the temporal correlation of image encoded group GOP (Group of picture) and viewpoint correlation carry out statistical analysis, and definition relative coefficient α is used to characterize the power contrast of correlation between the temporal correlation of the vision signal that obtains and viewpoint.
In described correlation statistical analysis module, can be to encoding or just in the image encoded group, only adopting the parallax compensation prediction to carry out the quantity n of the Intra-coded blocks in the image encoded frame
i DOnly adopt motion compensated prediction to carry out the quantity n of Intra-coded blocks in the image encoded frame
i PAdd up, with n
i DAnd n
i PProportionate relationship the strong or weak relation of correlation between time of current multi-view point video signal and viewpoint is described.
In described correlation statistical analysis module, also can be to encoding or just in the image encoded group, analyze correlation strong or weak relation between time of current multi-view point video signal and viewpoint with the proportionate relationship that only adopts the parallax compensation prediction to carry out the predicated error of image encoded frame and only adopt motion compensated prediction to carry out the predicated error of image encoded frame.
Described predictive mode is selected the module multi-view point video signal correlative character that statistical analysis obtains according to described correlation statistical analysis module and the mode that predictive mode selects is carried out in the requirement of combination properties such as the compression efficiency of multiple view video coding, encoder complexity, random access performance, encoding time delay as follows:
(1) when the time correlation obviously is better than between viewpoint correlation, further judge that correlation is in the distribution situation of time domain inside relative equilibrium whether, or the temporal correlation in the contiguous moment obviously is better than time contiguous temporal correlation constantly, selects the predictive coding pattern based on motion compensated prediction;
(2) when the time correlation obviously is weaker than between viewpoint correlation, select to be predicted as main predictive coding pattern with parallax compensation;
(3) when correlation between time correlation and viewpoint roughly quite the time, when selecting to take into account, the associated prediction coding mode in spatial domain.
Described schema update trigger module can adopt the schema update scheme based on video content, according to the situation of change of the relative coefficient α that obtains in the described correlation statistical analysis module, determine whether to reactivate described predictive mode and select module to upgrade the predictive coding pattern.
Described schema update trigger module also can adopt the mode that triggers of regularly upgrading, regularly open described correlation statistical analysis module correlation between the temporal correlation of multi-view point video signal and viewpoint is carried out statistical analysis, and enable predictive mode and select module to determine the predictive coding pattern.
In the multi-mode multi-viewpoint video encoder, can make the predict of all candidates' multi-view point video predictive coding pattern have certain general character, be to be arranged in described candidate's predictive coding pattern with the picture frame of intracoded frame synchronization and the picture frame that is positioned at the same viewpoint of intracoded frame all to be encoded prior to other picture frame of image sets, and above-mentioned these picture frames all have identical prediction mode in all candidate's predictive modes, can be in these picture frames that are encoded at first of coding, obtain the correlation statistic analysis result of current multi-view point video signal of encoding, and these picture frames that are encoded at first coding finish the back in time determine current just in the image encoded group other frame take which kind of forecast coding structure, promptly a final selected predictive coding pattern that is fit to current multi-view point video signal characteristics and the requirement of multiple view video coding combination property is encoded from all candidate's predictive coding patterns.
The present invention is directed to content relevance between multi-view point video signal time and viewpoint with many view camera density, illumination, different and the phenomenon that changes of factor such as camera and object motion, proposition is based on the multi-mode multi-viewpoint video coding framework of correlation analysis between multi-view point video signal temporal correlation and viewpoint and the requirement of multiple view video coding combination property, density according to many view camera, illumination, the variation of camera and object motion etc., design corresponding different candidate's predictive coding pattern, by correlation between the temporal correlation of multi-view point video signal and viewpoint is carried out simple statistical characteristic analysis, and the difference of multiple view video coding combination property required (as encoder complexity, encoding compression efficient, the random access performance, encoding time delay etc.), Dynamic Selection is adapted to the predictive coding pattern of current multi-view point video signal characteristics from candidate's predictive coding pattern, thereby improves the combination property of multi-viewpoint video signal code.
Compared with prior art, the invention has the advantages that by to correlation analysis between multi-view point video signal temporal correlation and viewpoint, Dynamic Selection is suitable for the predictive coding pattern of current multi-view point video signal characteristics that are encoded and the requirement of multiple view video coding combination property, multi-reference frame predictive coding method with the associating time and space prediction of the calculation of complex that replaces existing single-mode, thereby effectively reduce the computation complexity of multi-viewpoint video signal code compression, improve the random access performance of multi-view video system, guaranteed the encoding compression performance simultaneously.
Description of drawings
Fig. 1 is the multi-view video system schematic diagram;
Fig. 2 is multi-mode multi-viewpoint video encoder structure of the present invention and cataloged procedure schematic diagram;
Fig. 3 a is the 1st a class candidate predictive coding pattern among the embodiment;
Fig. 3 b is the 2nd a class candidate predictive coding pattern among the embodiment;
Fig. 3 c is the 3rd a class candidate predictive coding pattern among the embodiment;
Fig. 4 is for adopting the sequential prediction coding mode PSVP of P frame;
Fig. 5 is for adopting the sequential prediction coding mode BSVP of B frame;
Fig. 6 is the multi-view point video predictive coding pattern of Mpicture;
Fig. 7 is a Joint multi-view point video cycle tests;
Fig. 8 is the average rate distortion curve of Xmas sequence part in the Joint multi-view point video cycle tests;
Fig. 9 is the average rate distortion curve of exit sequence part in the Joint multi-view point video cycle tests;
Figure 10 is the average rate distortion curve of ballroom sequence part in the Joint multi-view point video cycle tests;
Figure 11 is the average rate distortion curve of Joint multi-view point video cycle tests.
Embodiment
Embodiment describes in further detail the present invention below in conjunction with accompanying drawing.
Here, with 5 * 7 representative image group structures is that example is (shown in Fig. 3 a, Fig. 3 b and Fig. 3 c, each image sets has 5 viewpoints, 7 moment, totally 35 frames), be elaborated with regard to 4 functional modules and the collaborative work mode thereof of multi-mode multi-viewpoint video encoder.
1) multi-view point video predictive coding module
This module is responsible for the encoding compression of multi-view point video signal, promptly adopts by predictive mode and selects certain candidate's predictive coding pattern of module Dynamic Selection that current multi-view point video signal is encoded.
According to correlation situation between multi-view point video signal temporal correlation and viewpoint, candidate's multi-view point video predictive coding pattern is divided into three major types, and the 1st class is the predictive coding pattern that is applicable to based on the multi-view point video signal of temporal correlation; The 2nd class is the predictive coding pattern that is applicable to based on the multi-view point video signal of correlation between viewpoint; The 3rd class is the predictive coding pattern that is applicable to the multi-view point video signal of correlation equilibrium between temporal correlation and viewpoint.Each class in the above-mentioned three major types predictive coding pattern can be made up of several predictive coding patterns again, adapting to multi-viewpoint video signal code, and the difference of multiple view video coding combination property required (as encoder complexity, encoding compression efficient, random access performance, encoding time delay etc.) with different correlation characteristics.
Fig. 3 a, Fig. 3 b and Fig. 3 c represent 3 kinds of different classes of predictive coding patterns being adopted respectively, I represents intracoded frame among the figure, D represents the parallax compensation encoded predicted frame, P represents the motion compensated predictive coding frame, during P ' expression, empty bi-directional predictive coding frame, can be with reference to D, P frame, when B ' is, empty associated prediction frame, can be with reference to D, P and P ' frame.The predictive coding pattern of Fig. 3 a is applicable to the multi-viewpoint video signal code based on temporal correlation based on motion compensated prediction, belongs to the 1st class predictive coding pattern; The predictive coding pattern of Fig. 3 b is predicted as the master with parallax compensation, is applicable to the multi-viewpoint video signal code based on correlation between viewpoint, belongs to the 2nd class predictive coding pattern; The predictive coding pattern of Fig. 3 c then when taking into account, the associated prediction in spatial domain, be applicable to the multi-viewpoint video signal code of correlation equilibrium between the time viewpoint to belong to the 3rd class predictive coding pattern.In the present embodiment, each the class predictive coding pattern in the three major types only has a candidate pattern, and actual use can design a plurality of different predictive coding patterns when of the present invention as required.
2) correlation statistical analysis module
Definition relative coefficient α is used to characterize the power contrast of correlation between the temporal correlation of vision signal and viewpoint, this coefficient can by to encode or just between the temporal correlation of image encoded group and viewpoint correlation carry out statistical analysis and obtain.
In the multi-mode multi-viewpoint video coding, for with I frame synchronization but be positioned at the image of different points of view, as being positioned at some D frames of 2 sides about the I frame among Fig. 3 a, Fig. 3 b and Fig. 3 c, only it to be encoded by the parallax compensation prediction, the quantitaes of I piece in the D frame (being Intra-coded blocks) is n
i DFor being same viewpoint but the image in the different moment with the I frame, as being positioned at I frame some P frames of 2 sides (the actual front and back frame that shows as the I frame from the time) up and down among Fig. 3 a, Fig. 3 b and Fig. 3 c, only by motion compensated prediction it is encoded, the quantitaes of I piece is n in the P frame
i PRelative coefficient α may be defined as
Wherein, n, m represent to be used to calculate the D frame of relative coefficient and the frame number of P frame respectively.This relative coefficient α can be used for characterizing the power contrast of correlation between the temporal correlation of vision signal and viewpoint.And calculate the required I number of blocks of α can coding simultaneously statistics obtain, additional computational overhead is extremely low, thereby can effectively realize the vision signal correlation statistical analysis of multi-mode multi-viewpoint video coding by α.Present embodiment promptly adopts the proportionate relationship of I number of blocks in D frame and the P frame to calculate relative coefficient α, and selects to adopt in the module threshold method 1 predictive coding pattern of final selection from 3 candidate's predictive coding patterns shown in Fig. 3 a, Fig. 3 b and Fig. 3 c to submit to multi-view point video predictive coding module at predictive mode and encode.
Except that such scheme, also can be by encoding or just those only adopt parallax compensation prediction to carry out the predicated error (for example sad value) of image encoded frame (D frame) in the image encoded group, and those only adopt motion compensated prediction to carry out the proportionate relationship of the predicated error of image encoded frame (P frame), correlation strong or weak relation between the time of the current multi-view point video signal of statistical analysis and viewpoint.
3) predictive mode is selected module
Multi-view point video signal correlation statistic analysis result according to the correlation statistical analysis module, and, from candidate's predictive coding pattern, select to be fit to certain predictive coding pattern of current multi-view point video signal characteristics and coding comprehensive performance requirement to the requirement of combination properties such as the compression efficiency of multi-mode multi-viewpoint video coding, encoder complexity, random access performance, encoding time delay.The selection mode of predictive coding pattern is as follows:
(1) when the time correlation obviously is better than between viewpoint correlation, can judge further that correlation is in the distribution situation of time domain inside relative equilibrium whether, or the temporal correlation in the contiguous moment obviously is better than time contiguous temporal correlation constantly, to select to determine certain the 1st suitable class predictive coding pattern.
(2) when the time correlation obviously is weaker than between viewpoint correlation, select certain to be predicted as the 2nd main class predictive coding pattern, so that in multi-view point video predictive coding module, adopt this predictive coding pattern to encode with parallax compensation.
(3) when correlation between time correlation and viewpoint roughly quite the time, when then selecting certain the 3rd class to take into account, the associated prediction coding mode in spatial domain.
In the present embodiment, because the picture frame that 3 candidate's predictive coding patterns that adopted are arranged in Fig. 3 a, Fig. 3 b and Fig. 3 c on the central cross is encoded prior to other picture frame of image sets, and these picture frames that are positioned on the central cross of these 3 predictive coding patterns have identical prediction mode, therefore can in these picture frames of coding, obtain the required n of correlation statistical analysis module
i DAnd n
i PThereby can obtain the correlation statistic analysis result of current multi-view point video signal of encoding, so that being arranged in picture frame coding on the central cross at these finishes in time definite other frame of image sets in back and takes which kind of predictive mode, promptly a final selected predictive coding pattern that is fit to current multi-view point video signal characteristics from 3 candidate's predictive coding patterns shown in Fig. 3 a, Fig. 3 b and Fig. 3 c is submitted to multi-view point video predictive coding module and is encoded.
4) schema update trigger module
Can adopt schema update scheme, promptly, determine whether to reactivate predictive mode by threshold method and select module to upgrade corresponding predictive coding pattern according to the situation of change of the relative coefficient α that obtains in the correlation statistical analysis module based on video content; Perhaps also can adopt the mode that triggers of regularly upgrading, regularly enable the correlation statistical analysis module correlation between the temporal correlation of multi-view point video signal and viewpoint is carried out statistical analysis, and enable predictive mode and select the predictive coding pattern of module to determine to adopt.Present embodiment adopts the schema update scheme based on video content.
Below carry out multiple view video coding with regard to present embodiment performance describe:
1) the random access performance of multi-mode multi-viewpoint Video Coding Scheme
For multi-view point video, its random access comprise F.F., rewind down, viewpoint switch and watch constantly freeze, access mode such as viewpoint slip.Suppose that v the viewpoint that is used to encode, the multi-view point video frame sum s=v * t of each viewpoint t frame are limited.Make x
iBeing illustrated in needs the frame number of decoding in advance, p before the i frame decoded
iBe the probability of user's random access i frame, the then mathematic expectaion of random access cost
Be to estimate the important indicator of a predictive coding pattern n to the random access degree of support.This cost is high more, illustrates that decoding end is low more to the tenability of random access, and the resource that consumes for the support random access is just many more.If k
nFor adopting the probability of n predictive coding pattern-coding multi-view point video signal, candidate's predictive coding number of modes is N, and then the random access cost of multi-mode multi-viewpoint video coding can be expressed as
The probability k of each pattern-coding in the multi-mode multi-viewpoint video coding
nDirectly relevant with the characteristics of actual multi-view point video signal.N=3 in the present embodiment, and suppose that each pattern-coding probability is identical, i.e. k
n=1/3 (n=1,2,3), then the random access cost of different schemes is as shown in table 1.PSVP and BSVP represent the sequential prediction method that adopts P frame, B frame respectively in the table, and its predictive coding pattern respectively as shown in Figure 4 and Figure 5.Mpiture is people's such as Japanese Fujii a Mpicture multi-view point video encoding method, and its predictive coding pattern as shown in Figure 6.PSVP, BSVP and Mpiture are the multi-reference frame predictive coding method of single-mode.MMVC is the multi-mode multi-viewpoint method for video coding (is the present invention program's representative with the present embodiment) of employing of the present invention 3 kinds of candidate's predictive coding patterns as shown in Figure 3.By table 1 as seen, with regard to the random access performance, PSVP is the poorest, and BSVP is relative with Mpicture quite a lot of.And the random access cost of multi-mode multi-viewpoint method for video coding MMVC of the present invention is minimum, relative PSVP, BSVP and Mpicture method, and its random access cost has reduced by 49%～72%, and the random access performance is significantly improved.
2) computation complexity of multi-mode multi-viewpoint Video Coding Scheme
Prediction of high accuracy parallax compensation and motion compensated prediction based on coding framework H.264/AVC account for the computation complexity of whole multiple view video coding device 75% or more, therefore can predict and the number of times of motion compensated prediction characterize the computation complexity of whole encoder by one 5 * 7 required parallax compensation of image sets of on average encoding.Each scheme computation complexity is more as shown in table 1, owing to adopted the multi-reference frame method, the computation complexity of PSVP, BSVP and Mpicture scheme is all very big, especially BSVP and Mpicuture method.And compare with the Mpicture scheme with PSVP, BSVP, the present invention program's computation complexity has then reduced by 29%～57% relatively.
The random access cost of table 1 the present invention program MMVC and computation complexity are relatively
| Encoding scheme | E(X) | Random access cost multiple | Computation complexity | The computation complexity multiple |
| PSVP | 11.0 | 364％ | 58 | 141％ |
| BSVP | 7.5 | 248％ | 83 | 202％ |
| Mpicture | 6.0 | 199％ | 97 | 237％ |
| MMVC | 3.02 | 100％ | 41 | 100％ |
3) distortion performance of multi-mode multi-viewpoint Video Coding Scheme
In order to estimate the code efficiency of MMVC scheme of the present invention,, carried out multiple view video coding experiment (quantization parameter QP is respectively 24,30,36,40) based on (JM8.5mainprofile) video coding framework H.264/AVC.The multi-view point video cycle tests selects for use Xmas (correlation is big between camera spacing 9mm, viewpoint), the exit of Tanimoto laboratory and MERL (to move slowly, big parallax, camera spacing 19.5cm) and many viewpoints cycle tests collection of ballroom (motion violent), it is captured that 3 sequences are parallel camera system, and resolution is 640 * 480.Choose 5 viewpoints, 5 scenes, 5 image sets of each scene, and it is spliced into as shown in Figure 7 Joint sequence, i.e. each 5 * 5 * 7=175 frame of each viewpoint video.In the experiment, the scene of simulating actual video by the mode of sequence assembly is switched, in the present embodiment, MMVC can self adaptation select suitable predictive coding pattern that the Joint sequence is encoded from the candidate's predictive coding pattern shown in Fig. 3 a, Fig. 3 b and Fig. 3 c according to video content.
Fig. 8,9,10,11 distortion performance of the Joint cycle tests being encoded for the MMVC and the methods such as sequential prediction method PSVP, BSVP and Mpicture of employing present embodiment compare.Wherein Fig. 8,9,10 is respectively Xmas, exit and three sequences of the ballroom average rate distortion curve separately in the Joint sequence.The population mean rate distortion curve of Joint sequence shown in Figure 11 shows that the distortion performance of MMVC and BSVP, PSVP and Mpicture is suitable substantially.
In sum, compared with prior art, the invention has the advantages that by to correlation analysis between multi-view point video signal temporal correlation and viewpoint, Dynamic Selection is suitable for the predictive coding pattern of current multi-view point video signal characteristics that are encoded and the requirement of multiple view video coding combination property, multi-reference frame predictive coding method with the associating time and space prediction of the calculation of complex that replaces existing single-mode, thereby effectively reduce the computation complexity of multi-viewpoint video signal code compression, improve the random access performance of multi-view video system, guarantee the encoding compression performance simultaneously.
Obviously, multi-view point video predictive coding pattern is not limited only to the form of present embodiment, therefore under the situation of the spirit and scope of the universal that does not deviate from claim and equal scope and limited, the example that the present invention is not limited to specific details and illustrates here and describe.
Claims (9)
1. multi-mode multi-viewpoint video signal code compression method, it is characterized in that encoder is arranged to multi-view point video predictive coding module, the correlation statistical analysis module, predictive mode is selected module and four functional modules of schema update trigger module, multi-view point video signal to input, earlier determine the initial predicted coding mode according to Given information, encode by described multi-view point video predictive coding module, encode according to the following steps then: 1. select the module multi-view point video signal correlative character that statistical analysis obtains according to described correlation statistical analysis module and to the requirement of the combination property of multiple view video coding, Dynamic Selection determines to be fit to the predictive coding pattern of current multi-view point video signal characteristics of encoding from candidate's predictive coding pattern by described predictive mode; 2. after the multi-view point video signal of input being encoded with this selected predictive coding pattern by described multi-view point video predictive coding module, output encoder compressed code flow signal; 3. when the schema update trigger condition in the described schema update trigger module satisfies, open described correlation statistical analysis module again, to select to upgrade the predictive coding pattern.
2. multi-mode multi-viewpoint video signal code compression method as claimed in claim 1, it is characterized in that described candidate's predictive coding pattern is divided into three major types: the 1st class is the predictive coding pattern that is applicable to based on the multi-view point video signal of temporal correlation, and such predictive coding pattern is based on motion compensated prediction; The 2nd class is the predictive coding pattern that is applicable to based on the multi-view point video signal of correlation between viewpoint, and such predictive coding pattern is predicted as the master with parallax compensation; The 3rd class is the predictive coding pattern that is applicable to the multi-view point video signal of correlation equilibrium between temporal correlation and viewpoint, and such predictive coding pattern is when taking into account, the associated prediction coding mode in spatial domain.
3. multi-mode multi-viewpoint video signal code compression method as claimed in claim 1, the statistical analysis that it is characterized in that described correlation statistical analysis module be to encode or just between the temporal correlation of image encoded group and viewpoint correlation carry out statistical analysis, and definition relative coefficient α is used to characterize the power contrast of correlation between the temporal correlation of the vision signal that obtains and viewpoint.
4. multi-mode multi-viewpoint video signal code compression method as claimed in claim 3, it is characterized in that in described correlation statistical analysis module, to encoding or just in the image encoded group, only adopting the parallax compensation prediction to carry out the quantity n of the Intra-coded blocks in the image encoded frame
l DOnly adopt motion compensated prediction to carry out the quantity n of Intra-coded blocks in the image encoded frame
l PAdd up, with n
l DAnd n
l PProportionate relationship the strong or weak relation of correlation between time of current multi-view point video signal and viewpoint is described.
5. multi-mode multi-viewpoint video signal code compression method as claimed in claim 3, it is characterized in that in described correlation statistical analysis module, to encoding or, analyze correlation strong or weak relation between time of current multi-view point video signal and viewpoint with the proportionate relationship that only adopts the parallax compensation prediction to carry out the predicated error of image encoded frame and only adopt motion compensated prediction to carry out the predicated error of image encoded frame just in the image encoded group.
6. multi-mode multi-viewpoint video signal code compression method as claimed in claim 2 is characterized in that described predictive mode selects the module multi-view point video signal correlative character that statistical analysis obtains according to described correlation statistical analysis module and the mode that predictive mode selects is carried out in the requirement of the combination property of multiple view video coding as follows:
(1) when the time correlation obviously is better than between viewpoint correlation, further judge that correlation is in the distribution situation of time domain inside relative equilibrium whether, or the temporal correlation in the contiguous moment obviously is better than time contiguous temporal correlation constantly, selects the predictive coding pattern based on motion compensated prediction;
(2) when the time correlation obviously is weaker than between viewpoint correlation, select to be predicted as main predictive coding pattern with parallax compensation;
(3) when correlation between time correlation and viewpoint roughly quite the time, when selecting to take into account, the associated prediction coding mode in spatial domain.
7. multi-mode multi-viewpoint video signal code compression method as claimed in claim 1, it is characterized in that described schema update trigger module adopts the schema update scheme based on video content, according to the situation of change of the relative coefficient α that obtains in the described correlation statistical analysis module, determine whether to reactivate described predictive mode and select module to upgrade the predictive coding pattern.
8. multi-mode multi-viewpoint video signal code compression method as claimed in claim 1, it is characterized in that described schema update trigger module adopts the mode that triggers of regularly upgrading, regularly open described correlation statistical analysis module correlation between the temporal correlation of multi-view point video signal and viewpoint is carried out statistical analysis, and enable predictive mode and select module to determine the predictive coding pattern.
9. multi-mode multi-viewpoint video signal code compression method as claimed in claim 1, it is characterized in that being arranged in all candidate pattern of described candidate's predictive coding pattern with the picture frame of intracoded frame synchronization and the picture frame that is positioned at the same viewpoint of intracoded frame and all be encoded, and these picture frames that take the lead in being encoded all have identical prediction mode in all candidate pattern prior to other picture frame of image sets.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100528959A CN100415002C (en) | 2006-08-11 | 2006-08-11 | Multi-mode multi-viewpoint video signal code compression method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100528959A CN100415002C (en) | 2006-08-11 | 2006-08-11 | Multi-mode multi-viewpoint video signal code compression method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1913640A CN1913640A (en) | 2007-02-14 |
| CN100415002C true CN100415002C (en) | 2008-08-27 |
Family
ID=37722378
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006100528959A Expired - Fee Related CN100415002C (en) | 2006-08-11 | 2006-08-11 | Multi-mode multi-viewpoint video signal code compression method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100415002C (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101291434A (en) * | 2007-04-17 | 2008-10-22 | 华为技术有限公司 | Encoding/decoding method and device for multi-video |
| JP5263901B2 (en) * | 2007-04-17 | 2013-08-14 | トムソン ライセンシング | Virtual reference decoder for multiview video coding |
| CN101087400B (en) * | 2007-06-26 | 2011-09-21 | 中兴通讯股份有限公司 | Video frame delay detection method and system |
| CN101389034B (en) * | 2007-09-14 | 2010-06-09 | 华为技术有限公司 | Image encoding/decoding method, apparatus and an image processing method, system |
| CN101170702B (en) * | 2007-11-23 | 2010-08-11 | 四川虹微技术有限公司 | Multi-view video coding method |
| CN101547010B (en) * | 2008-03-24 | 2011-07-06 | 华为技术有限公司 | System, method and device for coding and decoding |
| WO2009131703A2 (en) * | 2008-04-25 | 2009-10-29 | Thomson Licensing | Coding of depth signal |
| WO2010021666A1 (en) | 2008-08-20 | 2010-02-25 | Thomson Licensing | Refined depth map |
| KR20110126103A (en) | 2009-01-07 | 2011-11-22 | 톰슨 라이센싱 | Joint depth estimation |
| CN102148952B (en) * | 2010-02-05 | 2013-12-11 | 鸿富锦精密工业（深圳）有限公司 | Video image compression method and playing method thereof |
| CN101937578B (en) * | 2010-09-08 | 2012-07-04 | 宁波大学 | Method for drawing virtual view color image |
| CN103596012B (en) * | 2013-11-14 | 2017-05-10 | 山东电子职业技术学院 | Interframe macro block type selecting method used in real-time AVS-based video frame rate transcoding |
| CN103974070B (en) * | 2014-04-25 | 2017-08-15 | 广州市香港科大霍英东研究院 | Wireless video transmission method and system based on multi-user input and output |
| CN111669601B (en) * | 2020-05-21 | 2022-02-08 | 天津大学 | Intelligent multi-domain joint prediction coding method and device for 3D video |
| CN116962715A (en) * | 2022-03-31 | 2023-10-27 | 华为技术有限公司 | Encoding method, apparatus, storage medium, and computer program product |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030202592A1 (en) * | 2002-04-20 | 2003-10-30 | Sohn Kwang Hoon | Apparatus for encoding a multi-view moving picture |
| CN1613263A (en) * | 2001-11-21 | 2005-05-04 | 韩国电子通信研究院 | 3d stereoscopic/multiview video processing system and its method |
| WO2006062377A1 (en) * | 2004-12-10 | 2006-06-15 | Electronics And Telecommunications Research Institute | Apparatus for universal coding for multi-view video |
-
2006
- 2006-08-11 CN CNB2006100528959A patent/CN100415002C/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1613263A (en) * | 2001-11-21 | 2005-05-04 | 韩国电子通信研究院 | 3d stereoscopic/multiview video processing system and its method |
| US20030202592A1 (en) * | 2002-04-20 | 2003-10-30 | Sohn Kwang Hoon | Apparatus for encoding a multi-view moving picture |
| WO2006062377A1 (en) * | 2004-12-10 | 2006-06-15 | Electronics And Telecommunications Research Institute | Apparatus for universal coding for multi-view video |
Non-Patent Citations (4)
| Title |
|---|
| Encoding and reconstruction of multiview video objects. Olm, J-R.Signal Processing Magazine, IEEE,Vol.16 No.3. 1999 |
| Encoding and reconstruction of multiview video objects. Olm, J-R.Signal Processing Magazine, IEEE,Vol.16 No.3. 1999 * |
| 多视点视频编码中的视频目标提取与视差匹配. 朱仲杰，蒋刚毅，郁梅，吴训威.电子学报，第5期. 2004 |
| 多视点视频编码中的视频目标提取与视差匹配. 朱仲杰，蒋刚毅，郁梅，吴训威.电子学报，第5期. 2004 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1913640A (en) | 2007-02-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100415002C (en) | Multi-mode multi-viewpoint video signal code compression method | |
| CN103220528B (en) | Method and apparatus by using large-scale converter unit coding and decoding image | |
| CN106063271B (en) | For executing cross-layer video coding/decoding method and its equipment and the cross-layer video coding method and its equipment for executing the prediction based on sub-block of the prediction based on sub-block | |
| CN101461248B (en) | Method and apparatus for adaptively determining a bit budget for encoding video pictures | |
| CN103618907B (en) | Based on compressed sensing multiple views distributed video coding and frame collating unit and method | |
| CN105308966A (en) | Video encoding method and apparatus thereof, and a video decoding method and apparatus thereof | |
| CN101867813B (en) | Multi-view video coding method oriented for interactive application | |
| CN102948145A (en) | Video-encoding method and video-encoding apparatus based on encoding units determined in accordance with a tree structure, and video-decoding method and video-decoding apparatus based on encoding units determined in accordance with a tree structure | |
| CN105684409A (en) | Representing blocks with hash values in video and image coding and decoding | |
| CN106131546B (en) | A method of determining that HEVC merges and skip coding mode in advance | |
| CN103067704B (en) | A kind of method for video coding of skipping in advance based on coding unit level and system | |
| CN103858431A (en) | Multiview video data encoding method and device, and decoding method and device | |
| KR102312335B1 (en) | Method and apparatus for video encoding for illumination compensation, method and apparatus for video decoding for residual prediction | |
| CN104853210A (en) | Method for decoding video | |
| KR20110083367A (en) | Method and apparatus for video encoding using pattern information of hierarchical data unit, and method and apparatus for video decoding using pattern information of hierarchical data unit | |
| CN103238321A (en) | Video encoding method for encoding hierarchical-structure symbols and a device therefor, and video decoding method for decoding hierarchical-structure symbols and a device therefor | |
| TW201415904A (en) | Motion prediction method, motion compensation method, motion prediction apparatus, motion compensation apparatus, and computer readable recording medium for inter-prediction | |
| CN103636222A (en) | Method and apparatus for unified scalable video encoding for multi-view video and method and apparatus for unified scalable video decoding for multi-view video | |
| CN102265611A (en) | Localized in-loop filtering with multiple filters in hybrid video coding | |
| CN106416256B (en) | For carrying out coding or decoded method and apparatus to depth image | |
| CN106961603A (en) | Intracoded frame code rate allocation method and device | |
| CN106507106A (en) | Video interprediction encoding method based on reference plate | |
| CN104396251A (en) | Method for encoding multiview video using reference list for multiview video prediction and device therefor, and method for decoding multiview video using refernece list for multiview video prediction and device therefor | |
| CN104837019A (en) | AVS-to-HEVC optimal video transcoding method based on support vector machine | |
| CN109688407A (en) | Reference block selection method, device, electronic equipment and the storage medium of coding unit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: SHANGHAI SILICON INTELLECTUAL PROPERTY EXCHANGE CE Free format text: FORMER OWNER: NINGBO UNIVERSITY Effective date: 20120105 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 315211 NINGBO, ZHEJIANG PROVINCE TO: 200030 XUHUI, SHANGHAI |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20120105 Address after: 200030 Shanghai City No. 333 Yishan Road Huixin International Building 1 building 1704 Patentee after: Shanghai Silicon Intellectual Property Exchange Co.,Ltd. Address before: 315211 Zhejiang Province, Ningbo Jiangbei District Fenghua Road No. 818 Patentee before: Ningbo University |
|
| ASS | Succession or assignment of patent right |
Owner name: SHANGHAI SIPAI KESI TECHNOLOGY CO., LTD. Free format text: FORMER OWNER: SHANGHAI SILICON INTELLECTUAL PROPERTY EXCHANGE CENTER CO., LTD. Effective date: 20120217 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 200030 XUHUI, SHANGHAI TO: 201203 PUDONG NEW AREA, SHANGHAI |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20120217 Address after: 201203 Shanghai Chunxiao Road No. 350 South Building Room 207 Patentee after: Shanghai spparks Technology Co.,Ltd. Address before: 200030 Shanghai City No. 333 Yishan Road Huixin International Building 1 building 1704 Patentee before: Shanghai Silicon Intellectual Property Exchange Co.,Ltd. |
|
| ASS | Succession or assignment of patent right |
Owner name: SHANGHAI GUIZHI INTELLECTUAL PROPERTY SERVICE CO., Free format text: FORMER OWNER: SHANGHAI SIPAI KESI TECHNOLOGY CO., LTD. Effective date: 20120606 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| C56 | Change in the name or address of the patentee | ||
| CP02 | Change in the address of a patent holder |
Address after: 200030 Shanghai City No. 333 Yishan Road Huixin International Building 1 building 1706 Patentee after: Shanghai spparks Technology Co.,Ltd. Address before: 201203 Shanghai Chunxiao Road No. 350 South Building Room 207 Patentee before: Shanghai spparks Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20120606 Address after: 200030 Shanghai City No. 333 Yishan Road Huixin International Building 1 building 1704 Patentee after: Shanghai Guizhi Intellectual Property Service Co.,Ltd. Address before: 200030 Shanghai City No. 333 Yishan Road Huixin International Building 1 building 1706 Patentee before: Shanghai spparks Technology Co.,Ltd. |
|
| DD01 | Delivery of document by public notice |
Addressee: Shi Lingling Document name: Notification of Passing Examination on Formalities |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200121 Address after: 201203 block 22301-1450, building 14, No. 498, GuoShouJing Road, Pudong New Area (Shanghai) pilot Free Trade Zone, Shanghai Patentee after: Shanghai spparks Technology Co.,Ltd. Address before: 200030 Shanghai City No. 333 Yishan Road Huixin International Building 1 building 1704 Patentee before: Shanghai Guizhi Intellectual Property Service Co.,Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20080827 Termination date: 20200811 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |