CN112929667B - Encoding and decoding method, device, equipment and readable storage medium - Google Patents
Encoding and decoding method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN112929667B CN112929667B CN202110325525.2A CN202110325525A CN112929667B CN 112929667 B CN112929667 B CN 112929667B CN 202110325525 A CN202110325525 A CN 202110325525A CN 112929667 B CN112929667 B CN 112929667B
- Authority
- CN
- China
- Prior art keywords
- image
- frame
- encoding
- determining
- frames
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000004044 response Effects 0.000 claims abstract description 29
- 230000015556 catabolic process Effects 0.000 abstract description 7
- 238000006731 degradation reaction Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 description 14
- 230000005540 biological transmission Effects 0.000 description 8
- 230000000903 blocking effect Effects 0.000 description 8
- 230000009467 reduction Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a coding and decoding method, a device, equipment and a readable storage medium, wherein the coding method comprises the following steps: determining at least two consecutive image frames from a current group of pictures GOP in response to a received instantaneous decoding refresh IDR request; determining a first partial image region from the image regions of each of the at least two consecutive image frames, respectively, as intra-coded image regions of each of the at least two consecutive image frames; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame; in encoding the at least two consecutive image frames, performing: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode. The scheme well solves the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality degradation.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a coding and decoding method, device, apparatus, and readable storage medium.
Background
In the prior art, when a packet loss error code occurs in a network (i.e., a video is in a screen-display phenomenon), an IDR (Instantaneous Decoding Refresh, instant decoding refresh) refresh request is usually sent to a transmitting end through a receiving end, after the transmitting end receives the IDR request sent by the receiving end, the transmitting end encodes a video sequence using an IDR video frame as an initial frame immediately and sends the video sequence to the receiving end, so that the screen-display phenomenon can disappear after the IDR video frame is completely received and decoded by the receiving end.
However, in the prior art, the code rate is controlled and the video sequence is transmitted in the manner of GOP (Group of Pictures ), and the bandwidth occupied by the IDR frame is generally 10 times that occupied by the P frame (forward predictive coding frame), so if a new IDR frame is suddenly inserted, the code rate may be suddenly increased, resulting in network jitter, packet loss, blocking, and jamming, which may not solve the problem of screen-churning, but also introduce the problems of network packet loss, sudden increase of image quality, and the like, resulting in more jamming of the subsequent video and more decoding failures.
As described above, the conventional codec scheme for the IDR request has a problem that the image quality is easily degraded.
Disclosure of Invention
The invention aims to provide a coding and decoding method, a device, equipment and a readable storage medium, which are used for solving the problem that a coding and decoding scheme aiming at an IDR request in the prior art is easy to cause image quality degradation.
In order to solve the above technical problems, an embodiment of the present invention provides an encoding method, including:
determining at least two consecutive image frames from a current group of pictures GOP in response to a received instantaneous decoding refresh IDR request;
determining a first partial image region from the image regions of each of the at least two consecutive image frames, respectively, as an intra-coded image region of each of the at least two consecutive image frames; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame;
in encoding the at least two consecutive image frames, performing: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode.
Optionally, the method further comprises:
determining second local image areas from the image areas of the designated image frames respectively, wherein the second local image areas are used as inter-frame coding image areas of the designated image frames respectively; the specified image frame is other image frames than the first image frame among the at least two consecutive image frames; the inter-coded image area of each designated image frame satisfies: the intra-coded image area of the image frame preceding the designated image frame is located at the same position in the corresponding image area;
In encoding the specified image frame, performing: and encoding the inter-frame encoded image region by adopting a forward prediction encoding mode.
Optionally, determining at least two consecutive image frames from the current group of pictures GOP in response to the received immediate decoding refresh IDR request, includes:
determining an index value characterizing network quality in response to the IDR request;
and determining the at least two continuous image frames from the current GOP according to the index value.
Optionally, in response to the IDR request, determining an index value characterizing network quality includes:
responding to the IDR request, judging whether the current image frame accords with the limited image frame judging condition;
if yes, determining the index value.
Optionally, determining whether the current image frame meets the limited image frame determination condition includes:
and determining whether the current image frame to be encoded meets the limited image frame judging condition according to the arrangement position of the current image frame in the current GOP.
Optionally, determining whether the current image frame to be encoded meets the limited image frame determination condition according to the arrangement position of the current image frame in the current GOP includes:
Judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward;
if yes, determining that the current image frame meets the limited image frame judging condition;
if not, determining that the current image frame does not meet the limited image judgment condition.
Optionally, the index value includes a value of round trip delay rtt;
determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining a number of continuous image frames matching the size of the target time window from the GOP as the at least two continuous image frames according to the size of the target time window.
Optionally, determining the first local image region from the image regions of each of the at least two consecutive image frames, respectively, as the intra-coded image region of each of the at least two consecutive image frames, includes:
and respectively determining a first local image area from the image areas of each of the at least two continuous image frames according to the size of the target time window and the height value of the image frame, and taking the first local image area as an intra-frame coding image area of each of the at least two continuous image frames.
The embodiment of the invention also provides a decoding method, which comprises the following steps:
sending an immediate decoding refresh IDR request;
receiving an image frame obtained by an encoding end by encoding at least two consecutive image frames in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas, and can collectively cover the entire image area of a single image frame.
The embodiment of the invention also provides a coding device, which comprises:
a first determining module, configured to determine at least two consecutive image frames from a current group of pictures GOP in response to a received instantaneous decoding refresh IDR request;
a second determining module, configured to determine, from the image areas of each of the at least two consecutive image frames, a first local image area as an intra-coded image area of each of the at least two consecutive image frames, respectively; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame;
A first encoding module for performing, when encoding the at least two consecutive image frames: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode.
Optionally, the apparatus further includes:
a third determining module, configured to determine, from respective image areas of a designated image frame, respective second local image areas as inter-frame coded image areas of the designated image frame; the specified image frame is other image frames than the first image frame among the at least two consecutive image frames; the inter-coded image area of each designated image frame satisfies: the intra-coded image area of the image frame preceding the designated image frame is located at the same position in the corresponding image area;
a second encoding module, configured to, when encoding the specified image frame, perform: and encoding the inter-frame encoded image region by adopting a forward prediction encoding mode.
Optionally, determining at least two consecutive image frames from the current group of pictures GOP in response to the received immediate decoding refresh IDR request, includes:
determining an index value characterizing network quality in response to the IDR request;
And determining the at least two continuous image frames from the current GOP according to the index value.
Optionally, in response to the IDR request, determining an index value characterizing network quality includes:
responding to the IDR request, judging whether the current image frame accords with the limited image frame judging condition;
if yes, determining the index value.
Optionally, determining whether the current image frame meets the limited image frame determination condition includes:
and determining whether the current image frame to be encoded meets the limited image frame judging condition according to the arrangement position of the current image frame in the current GOP.
Optionally, determining whether the current image frame to be encoded meets the limited image frame determination condition according to the arrangement position of the current image frame in the current GOP includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward;
if yes, determining that the current image frame meets the limited image frame judging condition;
if not, determining that the current image frame does not meet the limited image judgment condition.
Optionally, the index value includes a value of round trip delay rtt;
Determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining a number of continuous image frames matching the size of the target time window from the GOP as the at least two continuous image frames according to the size of the target time window.
Optionally, determining the first local image region from the image regions of each of the at least two consecutive image frames, respectively, as the intra-coded image region of each of the at least two consecutive image frames, includes:
and respectively determining a first local image area from the image areas of each of the at least two continuous image frames according to the size of the target time window and the height value of the image frame, and taking the first local image area as an intra-frame coding image area of each of the at least two continuous image frames.
The embodiment of the invention also provides a decoding device, which comprises:
the first sending module is used for sending an immediate decoding refresh IDR request;
a first receiving module, configured to receive an image frame obtained by encoding at least two consecutive image frames by an encoding end in response to the IDR request;
The encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas, and can collectively cover the entire image area of a single image frame.
The embodiment of the invention also provides coding equipment, which comprises a memory, a processor and a program which is stored in the memory and can run on the processor; the above-described encoding method is implemented when the processor executes the program.
The embodiment of the invention also provides decoding equipment, which comprises a memory, a processor and a program which is stored in the memory and can run on the processor; the processor implements the decoding method described above when executing the program.
The embodiment of the invention also provides a readable storage medium, on which a program is stored, which when executed by a processor, implements the steps of the above-described encoding method; alternatively, the program, when executed by a processor, implements the steps in the decoding method described above.
The technical scheme of the invention has the following beneficial effects:
in the above scheme, the encoding method determines at least two consecutive image frames from the current group of pictures GOP by responding to the received instantaneous decoding refresh IDR request; determining a first partial image region from the image regions of each of the at least two consecutive image frames, respectively, as an intra-coded image region of each of the at least two consecutive image frames; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame; in encoding the at least two consecutive image frames, performing: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode. According to the scheme of the embodiment of the application, after the IDR request is received, the local image areas of the image frames are subjected to intra-frame coding to achieve the effect of the complete IDR frame, instead of directly coding at least one image frame into the complete IDR frame respectively for transmission, so that the IDR frame with larger occupied bandwidth is divided into the local intra-frame coding frames with smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, the problems of more packet loss, network jitter, image quality reduction and the like caused are avoided, and the problem that the coding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality reduction is well solved.
Drawings
FIG. 1 is a schematic diagram of a coding method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating the coverage of an entire image area according to an embodiment of the present invention;
FIG. 3 is a flow chart of a decoding method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an implementation framework of an encoding method according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a coding apparatus according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a decoding apparatus according to an embodiment of the invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The present invention provides an encoding method, as shown in fig. 1, for solving the problem that the encoding and decoding scheme for IDR request in the prior art is liable to cause degradation of image quality, including:
step 11: in response to receiving the immediate decoding refresh IDR request, at least two consecutive image frames are determined from the current group of pictures GOP.
The current GOP refers to a GOP to which a first image frame to be encoded belongs when the IDR request is received. The image frames in the determined "at least two consecutive image frames" may include a frame before the first image frame, and/or a frame after the first image frame, which are not limited herein.
Step 12: determining a first partial image region from the image regions of each of the at least two consecutive image frames, respectively, as an intra-coded image region of each of the at least two consecutive image frames; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame.
Wherein "the entire image area together covering a single said image frame" is to be understood as: an entire image area corresponding to one image frame; the entire image area of the different image frames may be the same; the common overlay may be one where there is or is not overlap of image areas. Taking the case of overlapping and common coverage as an example, it is assumed that two consecutive frames of a frame and B frame are selected, and as shown in fig. 2, an intra-coded image area a of the a frame and an intra-coded image area B of the B frame can commonly cover the entire image area c of one image frame.
Step 13: in encoding the at least two consecutive image frames, performing: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode.
In this step, the intra-frame coding part of the IDR frame fed back after the IDR request is split into a plurality of intra-frame coding parts of continuous frames to be respectively coded, so that one IDR frame with larger occupied bandwidth is split into a plurality of local intra-frame coding frames with smaller occupied bandwidth, thereby reducing the requirement on bandwidth and avoiding the situations of packet loss, blocking and the like.
From the above, the encoding method provided by the embodiment of the present invention determines at least two consecutive image frames from the current group of pictures GOP by responding to the received instantaneous decoding refresh IDR request; determining a first partial image region from the image regions of each of the at least two consecutive image frames, respectively, as an intra-coded image region of each of the at least two consecutive image frames; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame; in encoding the at least two consecutive image frames, performing: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode. According to the scheme of the embodiment of the application, after the IDR request is received, the effect of carrying out intra-frame coding on the local image areas of a plurality of image frames to achieve the complete IDR frame can be achieved, instead of directly respectively coding at least one image frame into the complete IDR frame for transmission, so that an IDR frame with larger occupied bandwidth is divided into a plurality of local intra-frame coding frames with smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, and the problems of more packet loss, network jitter, image quality reduction and the like are avoided; the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality degradation is well solved.
Further, the method further comprises: determining second local image areas from the image areas of the designated image frames respectively, wherein the second local image areas are used as inter-frame coding image areas of the designated image frames respectively; the specified image frame is other image frames than the first image frame among the at least two consecutive image frames; the inter-coded image area of each designated image frame satisfies: the intra-coded image area of the image frame preceding the designated image frame is located at the same position in the corresponding image area; in encoding the specified image frame, performing: and encoding the inter-frame encoded image region by adopting a forward prediction encoding mode.
Thus, since there is an intra-frame coding frame that can be forward-referenced starting from the first frame (i.e., the first frame of limited coding) that determines at least two consecutive image frames, it is possible to perform coding using the partial forward predictive coding mode starting from the second frame. In the above process, the inter-coded image region generally corresponds to the image region of the previous frame to which it refers, but is not limited thereto.
In the implementation of the present invention, further, the inter-coded image region may refer to the first partial image region and the second partial image region in the previous image frame.
Specifically, in the embodiment of the present invention, the determining the first partial image area and the second partial image area may include:
determining a restricted area (including a first partial area image area and a second partial area image area) of each of the at least two consecutive image frames; wherein the n+1th frame is one more restricted area than the N-th frame, N being an integer greater than or equal to 0;
and encoding the last limited area (namely the first local image area) in the limited areas corresponding to each image frame in an intra-frame encoding mode, and encoding the residual limited area in an inter-frame encoding mode when the residual limited area (namely the second local image area) exists, wherein the inter-frame encoding refers to all the limited areas in the previous image frame.
The number of the limited areas of the first image frame (i.e. the current image frame) in the at least two continuous image frames may be 1, the number of the limited areas of the next image frame in the first image frame may be 2, and thus, the number of the limited areas of the next image frame may be sequentially increased; the "preceding image frame" refers to a frame preceding the present image frame (the image frame corresponding to the remaining limited area).
The sample scheme can realize block refreshing (intra-frame coding), and the coding mode can ensure that the coded data can realize normal display; in addition, the intra-frame coding mode can ensure that data refreshing is realized and subsequent normal display is ensured; and inter-frame coding can ensure that refreshed data is referenced, thereby ensuring subsequent normal display.
Whether to use the method shown in fig. 1 in the embodiment of the present invention may be determined according to the network quality. This is because the quality of the network determines to some extent whether or not the conventional IDR coding has the problems as described in the background art.
Based on the above considerations, in one embodiment, determining at least two consecutive image frames from the current group of pictures GOP in response to the received instantaneous decoding refresh IDR request may include:
determining an index value characterizing network quality in response to the IDR request;
and determining the at least two continuous image frames from the current GOP according to the index value.
Specifically, before the index value is determined, it may be determined whether or not the current image frame meets a limited image frame determination condition (to be described later) in response to the IDR request. If yes, determining the index value. The index value may include a value of rtt (round-trip time).
In the embodiment of the invention, under the condition of relatively good network quality, the value of rtt is relatively small, and a relatively small number of image frames needing limited processing can be determined according to the value of rtt; under the condition of relatively poor network quality, the value of rtt is relatively large, and a relatively large number of image frames needing limited processing can be determined according to the value of rtt; thereby ensuring that the frame refreshing is realized as better as possible regardless of the network quality; therefore, the image frames needing limited processing can be determined according to the network quality, so that the frame refreshing can be better realized, and the problems of clamping and the like are avoided.
The term "determining the index value" is also understood to mean acquiring the index value, i.e., the first image frame referred to above.
In the embodiment of the invention, judging whether the current image frame meets the judging condition of the limited image frame comprises the following steps: and determining whether the current image frame to be encoded meets the limited image frame judging condition according to the arrangement position of the current image frame in the current GOP.
In the embodiment of the present invention, determining whether the current image frame to be encoded meets the limited image frame determination condition according to the arrangement position of the current image frame in the current GOP includes: judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward; if yes, determining that the current image frame meets the limited image frame judging condition; if not, determining that the current image frame does not meet the limited image judgment condition.
With respect to whether relatively forward or relatively backward is meant whether the current image frame is relatively forward or backward in its position throughout the current GOP. For example, the current image frame is located most forward in the entire current GOP and may be considered relatively forward; alternatively, the current image frame may be considered relatively backward if it is last in position throughout the current GOP.
The arrangement position may be represented by a position ratio, that is, it is determined whether the above-described determination condition is satisfied according to the position of the current image frame in the current GOP. The position ratio refers to: the ratio between the position ordering value (i.e., number M) and (the duration T and frame rate fps corresponding to the GOP) of the current image frame in the current GOP.
Specifically, it may be determined whether the position ratio corresponding to the arrangement position is within a preset value range, where the preset value range is 0 to 0.2, or 0.8 to 1. That is, it is determined whether the arrangement position is at the first 20% or the last 20% of the entire GOP. The special processing is performed on the frame before or after, because the frame before (or after) is relatively close to the IDR frame of the previous (or next) picture group, the occupied bandwidth is large, and the blocking is caused, so that the special processing can reduce the occupied bandwidth to a certain extent and avoid the blocking. It can also be understood that: in the embodiment of the present invention, it is considered that 20% of the frames before and after the GOP are subjected to limited encoding, because if the frame too far before is used as IDR, the problem occurs: just after sending one IDR and another IDR, the bandwidth is too much. Similarly, if too late a frame is used as an IDR, the problem arises: the latter IDR may be affected. Therefore, these two partial frames need to be selected for limited encoding.
In the embodiment of the present invention, the absolute value of the difference between the values at the two ends of the preset value range is multiplied by the total frame number corresponding to the GOP, and is greater than or equal to 3 frames, but not limited to this.
In the embodiment of the invention, the index value comprises a value of rtt (round-trip time); determining the at least two consecutive image frames from the current GOP according to the index value, including: determining the size of a target time window according to the rtt value and the rtt threshold value; and determining a number of continuous image frames matching the size of the target time window from the GOP as the at least two continuous image frames according to the size of the target time window.
This can control the effect of achieving I-frame refresh with a minimum of frames. The rtt threshold is greater than or equal to 100ms and less than or equal to 300ms, which may be 200ms in particular.
Specifically, when the rtt value is less than or equal to the rtt threshold, determining a size (specifically, a corresponding frame number) of the target time window by adopting a first formula; determining the size of the target time window by adopting a second formula under the condition that the rtt value is larger than the rtt threshold value; wherein, the first formula is: w=min (3, max (500/rtt, 5)); the second formula is: w=max (3,500/rtt); w represents the size of the target time window, and rtt represents the value of rtt. The first formula represents taking the maximum value of 500/rtt and 5, and then taking the minimum value of the maximum value and 3; the second formula represents taking the maximum of 3 and 500/rtt.
Thus if rtt is less than or equal to the rtt threshold, W is equal to 3; if rtt > rtt threshold, then W is at least equal to 3, and if 500/rtt is greater than 3, W is decreasing as rtt increases; further, the smaller W is, the larger the limited area is, and the larger W is, the smaller the limited area is; therefore, to a certain extent, rtt is inversely proportional to W, and rtt is directly proportional to the limited area, so that the effect of refreshing the I frame by using the least frames as possible when the network transmission delay is large can be realized.
In an embodiment of the present invention, determining, from respective image areas of the at least two consecutive image frames, a first local image area as an intra-coded image area of each of the at least two consecutive image frames includes: and respectively determining a first local image area from the image areas of each of the at least two continuous image frames according to the size of the target time window and the height value of the image frame, and taking the first local image area as an intra-frame coding image area of each of the at least two continuous image frames.
This allows an accurate determination of the intra-coded image area of each image frame and finally the complete coverage of the entire image area of a single said image frame.
Specifically, the Height of the first local image area may be a Height/W line, where the Height represents a Height value of the image frame, and W represents a size of the target time window; the above-mentioned n+1 is less than or equal to W. The Height of the second partial image region may also be a Height/W line. The width of the first and second partial image areas may be the width of the image frame.
Therefore, no overlap between the local image areas can be ensured, so that the intra-frame coding area is reduced to the greatest extent, and the occupied width is reduced.
The embodiment of the invention also provides a decoding method, as shown in fig. 3, comprising the following steps:
step 31: sending an immediate decoding refresh IDR request;
step 32: receiving an image frame obtained by an encoding end by encoding at least two consecutive image frames in response to the IDR request; the encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode; the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas, and can collectively cover the entire image area of a single image frame.
The coding end is the coding equipment.
The decoding method provided by the embodiment of the invention comprises the steps of sending an instant decoding refresh IDR request; receiving an image frame obtained by an encoding end by encoding at least two consecutive image frames in response to the IDR request; the encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode; the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas, and can collectively cover the entire image area of a single image frame. According to the scheme of the embodiment of the application, after the IDR request is received by the encoding end, the partial image areas of the image frames are subjected to intra-frame encoding to achieve the effect of a complete IDR frame, instead of directly encoding at least one image frame into the complete IDR frame for transmission, so that the IDR frame with larger occupied bandwidth is divided into a plurality of partial intra-frame encoding frames with smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, and the problems of more packet loss, network jitter, image quality reduction and the like are avoided; the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality degradation is well solved.
The following further describes the encoding method and decoding method provided by the embodiments of the present invention, where the IDR request is sent by the decoding end, the encoding end is sent by the sending end, and the decoding end is received by the receiving end.
In view of the above technical problems, the embodiments of the present invention provide an encoding method and a decoding method, which may be specifically implemented as a method for fast refreshing real-time audio and video. The transmitting end selects one of 2 modes according to preset conditions: one is to encode a complete IDR frame; the other is to send limited P frame or B frame according to the network condition; therefore, the problems of more packet loss, network jitter, image quality degradation and the like caused by the occupation of bandwidth by huge IDR frames are avoided.
First, the following is a specific example of the content related to the embodiment of the present invention:
(1) The sending terminal A initiates an audio-video call to the receiving terminal B.
(2) The receiving end B receives the invitation of the sending end A and establishes a communication link.
(3) The camera of the transmitting end A collects YUV data (color coded data), encodes into an ES stream (Elementary Stream, basic code stream) according to a video coding standard, encapsulates the ES stream into an RTP (Real-time Transport Protocol ) packet, transmits the video data to the receiving end B through RTP-RTCP (Real-time Transport Control Protocol ), and processes the audio data in the same manner.
(4) And after receiving the audio and video stream of the sending end A, the receiving end B decodes and renders the audio and video stream. Meanwhile, the sending end of the equipment of the receiving end B also collects YUV data, microphone data and the like, performs coding compression and encapsulation, and transmits the stream to the receiving end of the equipment of the sending end A.
(5) Under normal conditions of the network, the above (3) and (4) are always performed. However, it is assumed that the receiving end B network suddenly deteriorates in a mobile environment. The receiving end B finds out a large number of packet loss, error codes, disordered frames and the like, so that the decoding failure of the receiving end B occurs, and the phenomena of screen display, blocking and the like are also understood as the condition of poor network quality. The receiving end B sends an IDR request to the sending end A.
(6) The sending end A processes the IDR request of the receiving end B, can immediately encode the current frame into an IDR frame, and sends the IDR frame to the receiving end B; referring to the processing framework shown in fig. 4 (the a21 code reference a11, the a31 code reference a32 code reference a21 and the a22 code reference a22 shown in fig. 4, where N is an integer greater than 0), in the case of weak network (i.e. low network quality), the receiving end needs to quickly refresh the transmitting end to adapt to the weak network condition when the receiving end has a screen, a clip, a packet loss, a decoding failure, etc. The scheme provides a stable and effective refreshing mechanism, namely the screen and the blocking can be rapidly solved, network jitter and more packet loss can not be caused, and user experience can be greatly improved. The method comprises the following steps:
(1) The sender a processes the IDR request of the receiver B.
In this step, the value of the current network rtt packet (i.e. the value of rtt described above) may be obtained by the server. Alternatively, the value of rtt may be determined after the operation (3) of determining that the current image frame satisfies the limited image frame determination condition.
(2) And starting a network bandwidth estimation module and a limited reference frame module.
(3) The network bandwidth estimation module determines whether the current frame (the frame to be currently encoded and transmitted, i.e., the current image frame) enters the limited reference frame module according to the position (i.e., the arrangement position) of the current frame in the whole GOP and a preset condition. Wherein, the preset conditions are: the position ratio of the arrangement position is equal to or less than 0.2 (M/(T×fps)), or the position ratio is equal to or more than 0.8 (M/(T×fps); wherein T represents the duration of the GOP (i.e., the duration corresponding to the GOP), such as T seconds; m represents the number of the current image frame in the GOP (i.e., the number of the current image frame in the GOP described above); fps represents the frame rate; if the result obtained by calculation according to the above formula is smaller than or equal to 0.2 or larger than or equal to 0.8, entering a limited reference frame module. Otherwise, the current frame may be immediately encoded as an IDR frame and transmitted to the receiving end B.
Thus, special treatment can be carried out on the front frame or the rear frame; because the front (or back) frame is relatively close to the larger IDR frame of the previous (or next) frame group, the occupied bandwidth is large, and the blocking is caused, so that the occupied bandwidth can be reduced to a certain extent by special processing, and the blocking is avoided.
The embodiment of the invention can be as follows: 0.2 XT x fps.gtoreq.3. The GOP involved is encoder customized, containing 30 frames or 100 frames, etc.
(4) Limited reference frame module: the (limited) reference frame sliding window size W (i.e. the size of the target time window) is determined from the value of rtt (i.e. the value of the rtt packet). If rtt.ltoreq.Tm, the sliding window size is W=min (3, max (500/rtt, 5)). Wherein Tm is a preset threshold (i.e. the rtt threshold may be specifically greater than or equal to 100ms and less than or equal to 300 ms), which represents a delay time, generally 200ms. If rtt > Tm, the sliding window size is w=max (3,500/rtt). The effect of refreshing the I frame by using the least frames as much as possible when the network transmission delay is large can be realized:
thus, if rtt+.Tm, then W equals 3; if rtt > Tm, then W is at least equal to 3, and in the case where 500/rtt is greater than 3, W is decreasing as rtt increases; further, the smaller W is, the larger the limited area is, and the larger W is, the smaller the limited area is. Therefore, to a certain extent, the value of rtt is inversely proportional to W, and the value of rtt is directly proportional to the limited area, so that the effect of refreshing the I frame by using the least frames as possible when the network transmission delay is large can be realized.
(5) A reference frame (P-frame) restricted area (a specific implementation of the restricted area of the image frame described above) is calculated for each frame within the sliding window, the image Height of the restricted area being (Height/W) lines. It can also be understood that the size of the reference frame limited area is determined according to the value of rtt in this scheme.
(6) Continuing to encode as P or B frames (bi-predictive interpolation encoded frames) starting from the current 0 th frame (i.e., the current image frame described above) to the W-1 th frame, the following involves special encoding of the restricted area:
1) Frame 0 has only one limited area, specifically an area with the width of the image frame from 0 to (Height/W) row; the current frame is intra-coded in this region coding mode. Other regions are normally encoded (normally encoded according to the encoding requirements). For example, if the 0 th frame is originally a B frame or a P frame, the regions outside the limited region are encoded into the B frame or the P frame according to the original principle; wherein a P frame belongs to forward predicted inter-frame coding, referring only to the I or P frame nearest to it before; the B frame is a bi-directional predictive coded frame, inter-coded with the preceding I or P frame and the following P frame as reference frames.
2) Frame 1 has 2 restricted areas, specifically two areas ranging in Height from 0 to (Height/W) and from (Height/W) to 2× (Height/W) rows, the width of which is the area of the image frame width. The first limited area of the current frame is inter-coded with reference to the first limited area of the 0 th frame (inter-coding is a limited area that requires reference to the previous frame, where reference to the previous frame is defined). The second limited area of the current frame is intra-coded. Other regions are normally encoded.
3) The 2 nd frame has 3 limited areas, specifically three areas with heights from 0 to (Height/W), from (Height/W) to 2× (Height/W), and from 2× (Height/W) to 3× (Height/W), the three areas being areas of the image frame width. The first and second limited regions of the current frame are inter-coded with reference to the first and second limited regions of frame 1. The third limited area of the current frame is intra-coded. Other regions are normally encoded.
And so on, the limited area from the 3 rd frame to the W-2 nd frame and the coding can be similar processed with reference to the above;
4) W-1 frames have W restricted areas, in particular from 0 to (Height/W) row, from (Height/W) to 2X (Height/W) row from 2× (Height/W) row to 3× (Height/W) row, ····from (W-1) × (Height/W) row to W× (Height/W) row, the width of these W areas is an area of the image frame width. The first and second to W-1 th restricted areas of the current frame are inter-coded, referring to the first and second to W-1 th restricted areas of the W-2 th frame. The W-th limited area of the current frame is intra-coded. Other regions are normally encoded.
Thus, the block refreshing (intra-frame coding) can be realized, and the coding mode can ensure that the coded data can be normally displayed; the intra-frame coding mode can ensure that data refreshing is realized and subsequent normal display is ensured; and inter-frame coding can ensure that refreshed data is referenced, thereby ensuring subsequent normal display.
5) Frames that exceed the sliding window are encoded as normal.
From the above, the scheme provided by the embodiment of the invention can be implemented: the quick refreshing can be realized under the weak network without causing network jitter; the limited reference frame only carries out intra-frame coding on a specific limited area, so that the image quality in the weak network can be ensured to be better.
The embodiment of the invention also provides a coding device, as shown in fig. 5, comprising:
a first determining module 51, configured to determine at least two consecutive image frames from the current group of pictures GOP in response to the received immediate decoding refresh IDR request;
a second determining module 52, configured to determine, from the image areas of each of the at least two consecutive image frames, a first local image area as an intra-coded image area of each of the at least two consecutive image frames, respectively; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame;
a first encoding module 53, configured to, when encoding the at least two consecutive image frames, perform: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode.
The encoding device provided by the embodiment of the invention determines at least two continuous image frames from the current group of pictures GOP by responding to the received instant decoding refresh IDR request; determining a first partial image region from the image regions of each of the at least two consecutive image frames, respectively, as an intra-coded image region of each of the at least two consecutive image frames; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame; in encoding the at least two consecutive image frames, performing: and encoding the intra-frame encoded image area by adopting an intra-frame encoding mode. According to the scheme of the embodiment of the application, after the IDR request is received, the local image areas of the image frames are subjected to intra-frame coding to achieve the effect of the complete IDR frame, instead of directly coding at least one image frame into the complete IDR frame respectively for transmission, so that the IDR frame with larger occupied bandwidth is divided into the local intra-frame coding frames with smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, the problems of more packet loss, network jitter, image quality reduction and the like caused are avoided, and the problem that the coding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality reduction is well solved.
Further, the device further comprises: a third determining module, configured to determine, from respective image areas of a designated image frame, respective second local image areas as inter-frame coded image areas of the designated image frame; the specified image frame is other image frames than the first image frame among the at least two consecutive image frames; the inter-coded image area of each designated image frame satisfies: the intra-coded image area of the image frame preceding the designated image frame is located at the same position in the corresponding image area; a second encoding module, configured to, when encoding the specified image frame, perform: and encoding the inter-frame encoded image region by adopting a forward prediction encoding mode.
In an embodiment of the present invention, determining at least two consecutive image frames from a current group of pictures GOP in response to a received instantaneous decoding refresh IDR request, includes: determining an index value characterizing network quality in response to the IDR request; and determining the at least two continuous image frames from the current GOP according to the index value.
Wherein determining an index value characterizing network quality in response to the IDR request comprises: responding to the IDR request, judging whether the current image frame accords with the limited image frame judging condition; if yes, determining the index value.
In the embodiment of the invention, judging whether the current image frame meets the judging condition of the limited image frame comprises the following steps: and determining whether the current image frame to be encoded meets the limited image frame judging condition according to the arrangement position of the current image frame in the current GOP.
Wherein determining whether the current image frame to be encoded meets the limited image frame determination condition according to the arrangement position of the current image frame in the current GOP includes: judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward; if yes, determining that the current image frame meets the limited image frame judging condition; if not, determining that the current image frame does not meet the limited image judgment condition.
In the embodiment of the present invention, the index value includes a value of round trip delay rtt; determining the at least two consecutive image frames from the current GOP according to the index value, including: determining the size of a target time window according to the rtt value and the rtt threshold value; and determining a number of continuous image frames matching the size of the target time window from the GOP as the at least two continuous image frames according to the size of the target time window.
In an embodiment of the present invention, determining, from respective image areas of the at least two consecutive image frames, a first local image area as an intra-coded image area of each of the at least two consecutive image frames includes: and respectively determining a first local image area from the image areas of each of the at least two continuous image frames according to the size of the target time window and the height value of the image frame, and taking the first local image area as an intra-frame coding image area of each of the at least two continuous image frames.
The implementation embodiments of the encoding method are applicable to the embodiment of the encoding device, and the same technical effects can be achieved.
The embodiment of the invention also provides a decoding device, as shown in fig. 6, which comprises:
a first sending module 61, configured to send an immediate decoding refresh IDR request;
a first receiving module 62, configured to receive an image frame obtained by encoding at least two consecutive image frames by an encoding end in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode;
The intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas, and can collectively cover the entire image area of a single image frame.
The decoding device provided by the embodiment of the invention sends an immediate decoding refresh IDR request; receiving an image frame obtained by an encoding end by encoding at least two consecutive image frames in response to the IDR request; the encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode; the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas, and can collectively cover the entire image area of a single image frame. According to the scheme of the embodiment of the application, after the IDR request is received by the encoding end, the partial image areas of the image frames are subjected to intra-frame encoding to achieve the effect of a complete IDR frame, instead of directly encoding at least one image frame into the complete IDR frame for transmission, so that the IDR frame with larger occupied bandwidth is divided into a plurality of partial intra-frame encoding frames with smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, and the problems of more packet loss, network jitter, image quality reduction and the like are avoided; the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality degradation is well solved.
The embodiments of the decoding method are applicable to the embodiments of the decoding device, and the same technical effects can be achieved.
The embodiment of the invention also provides coding equipment, which comprises a memory, a processor and a program which is stored in the memory and can run on the processor; the above-described encoding method is implemented when the processor executes the program.
The implementation embodiments of the encoding method are applicable to the embodiment of the encoding device, and the same technical effects can be achieved.
The embodiment of the invention also provides decoding equipment, which comprises a memory, a processor and a program which is stored in the memory and can run on the processor; the processor implements the decoding method described above when executing the program.
The implementation embodiments of the decoding method are applicable to the embodiment of the decoding device, and the same technical effects can be achieved.
The embodiment of the invention also provides a readable storage medium, on which a program is stored, which when executed by a processor, implements the steps of the above-described encoding method; or,
the program when executed by a processor implements the steps of the decoding method described above.
The implementation embodiments of the encoding method or the decoding method are applicable to the embodiment of the readable storage medium, and can achieve the corresponding same technical effects.
It should be noted that many of the functional components described in this specification have been referred to as modules, in order to more particularly emphasize their implementation independence.
In an embodiment of the invention, the modules may be implemented in software for execution by various types of processors. An identified module of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different bits which, when joined logically together, comprise the module and achieve the stated purpose for the module.
Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Likewise, operational data may be identified within modules and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices.
Where a module may be implemented in software, taking into account the level of existing hardware technology, a module may be implemented in software, and one skilled in the art may, without regard to cost, build corresponding hardware circuitry, including conventional Very Large Scale Integration (VLSI) circuits or gate arrays, and existing semiconductors such as logic chips, transistors, or other discrete components, to achieve the corresponding functions. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that various modifications and changes can be made without departing from the principles of the present invention, and such modifications and changes should also be considered as being within the scope of the present invention.
Claims (13)
1. A method of encoding, comprising:
determining at least two consecutive image frames from a current group of pictures GOP in response to a received instantaneous decoding refresh IDR request;
determining a first partial image region from the image regions of each of the at least two consecutive image frames, respectively, as an intra-coded image region of each of the at least two consecutive image frames; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame;
In encoding the at least two consecutive image frames, performing: encoding the intra-frame encoded image area by adopting an intra-frame encoding mode;
wherein determining at least two consecutive image frames from the current group of pictures GOP in response to the received immediate decoding refresh IDR request comprises:
responding to the IDR request, judging whether the current image frame accords with the limited image frame judging condition; if yes, determining an index value representing the network quality;
determining the at least two consecutive image frames from the current GOP according to the index value;
wherein determining whether the current image frame meets the restricted image frame determination condition includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward;
if yes, determining that the current image frame meets the limited image frame judging condition;
if not, determining that the current image frame does not meet the limited image frame judging condition.
2. The method according to claim 1, wherein the method further comprises:
determining second local image areas from the image areas of the designated image frames respectively, wherein the second local image areas are used as inter-frame coding image areas of the designated image frames respectively; the specified image frame is other image frames than the first image frame among the at least two consecutive image frames; the inter-coded image area of each designated image frame satisfies: the intra-coded image area of the image frame preceding the designated image frame is located at the same position in the corresponding image area;
In encoding the specified image frame, performing: and encoding the inter-frame encoded image region by adopting a forward prediction encoding mode.
3. The method of claim 1, wherein the index value comprises a value of round trip delay, rtt;
determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining a number of continuous image frames matching the size of the target time window from the GOP as the at least two continuous image frames according to the size of the target time window.
4. A method according to claim 3, wherein determining the first partial image region from the respective image regions of the at least two consecutive image frames as the respective intra-coded image region of the at least two consecutive image frames, respectively, comprises:
and respectively determining a first local image area from the image areas of each of the at least two continuous image frames according to the size of the target time window and the height value of the image frame, and taking the first local image area as an intra-frame coding image area of each of the at least two continuous image frames.
5. A decoding method, comprising:
sending an immediate decoding refresh IDR request;
receiving an image frame obtained by an encoding end by encoding at least two consecutive image frames in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame;
the at least two consecutive image frames are determined by the encoding end by:
responding to the IDR request, judging whether the current image frame accords with the limited image frame judging condition; if yes, determining an index value representing the network quality;
determining the at least two continuous image frames from the current GOP according to the index value;
wherein determining whether the current image frame meets the restricted image frame determination condition includes:
Judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward;
if yes, determining that the current image frame meets the limited image frame judging condition;
if not, determining that the current image frame does not meet the limited image frame judging condition.
6. An encoding device, comprising:
a first determining module, configured to determine at least two consecutive image frames from a current group of pictures GOP in response to a received instantaneous decoding refresh IDR request;
a second determining module, configured to determine, from the image areas of each of the at least two consecutive image frames, a first local image area as an intra-coded image area of each of the at least two consecutive image frames, respectively; wherein, the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame;
a first encoding module for performing, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image area by adopting an intra-frame encoding mode;
wherein determining at least two consecutive image frames from the current group of pictures GOP in response to the received immediate decoding refresh IDR request comprises:
Responding to the IDR request, judging whether the current image frame accords with the limited image frame judging condition; if yes, determining an index value representing the network quality;
determining the at least two consecutive image frames from the current GOP according to the index value;
wherein determining whether the current image frame meets the restricted image frame determination condition includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward;
if yes, determining that the current image frame meets the limited image frame judging condition;
if not, determining that the current image frame does not meet the limited image frame judging condition.
7. The apparatus of claim 6, wherein the apparatus further comprises:
a third determining module, configured to determine, from respective image areas of a designated image frame, respective second local image areas as inter-frame coded image areas of the designated image frame; the specified image frame is other image frames than the first image frame among the at least two consecutive image frames; the inter-coded image area of each designated image frame satisfies: the intra-coded image area of the image frame preceding the designated image frame is located at the same position in the corresponding image area;
A second encoding module, configured to, when encoding the specified image frame, perform: and encoding the inter-frame encoded image region by adopting a forward prediction encoding mode.
8. The apparatus of claim 6, wherein the index value comprises a value of round trip delay, rtt;
determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining a number of continuous image frames matching the size of the target time window from the GOP as the at least two continuous image frames according to the size of the target time window.
9. The apparatus of claim 8, wherein determining a first partial image region from each of the at least two consecutive image frames as an intra-coded image region for each of the at least two consecutive image frames, respectively, comprises:
and respectively determining a first local image area from the image areas of each of the at least two continuous image frames according to the size of the target time window and the height value of the image frame, and taking the first local image area as an intra-frame coding image area of each of the at least two continuous image frames.
10. A decoding apparatus, comprising:
the first sending module is used for sending an immediate decoding refresh IDR request;
a first receiving module, configured to receive an image frame obtained by encoding at least two consecutive image frames by an encoding end in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: encoding the intra-frame encoded image areas of each of the at least two consecutive image frames by adopting an intra-frame encoding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the positions of the respective intra-frame coding image areas in the corresponding image areas can jointly cover the whole image area of a single image frame;
the at least two consecutive image frames are determined by the encoding end by:
responding to the IDR request, judging whether the current image frame accords with the limited image frame judging condition; if yes, determining an index value representing the network quality;
determining the at least two continuous image frames from the current GOP according to the index value;
Wherein determining whether the current image frame meets the restricted image frame determination condition includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively forward or relatively backward;
if yes, determining that the current image frame meets the limited image frame judging condition;
if not, determining that the current image frame does not meet the limited image frame judging condition.
11. An encoding apparatus comprising a memory, a processor, and a program stored on the memory and executable on the processor; the encoding method according to any one of claims 1 to 4, characterized in that the processor executes the program.
12. A decoding device comprising a memory, a processor, and a program stored on the memory and executable on the processor; the decoding method according to claim 5 is implemented when the processor executes the program.
13. A readable storage medium having stored thereon a program, which when executed by a processor, implements the steps of the encoding method according to any one of claims 1 to 4; alternatively, the program when executed by a processor implements the steps of the decoding method according to claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110325525.2A CN112929667B (en) | 2021-03-26 | 2021-03-26 | Encoding and decoding method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110325525.2A CN112929667B (en) | 2021-03-26 | 2021-03-26 | Encoding and decoding method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112929667A CN112929667A (en) | 2021-06-08 |
| CN112929667B true CN112929667B (en) | 2023-04-28 |
Family
ID=76176183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110325525.2A Active CN112929667B (en) | 2021-03-26 | 2021-03-26 | Encoding and decoding method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112929667B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117412053A (en) * | 2022-07-08 | 2024-01-16 | 中兴通讯股份有限公司 | Video encoding method, video encoding device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105163117A (en) * | 2015-08-26 | 2015-12-16 | 华为技术有限公司 | Image coding and processing methods and devices |
| CN106998328A (en) * | 2017-03-30 | 2017-08-01 | 北京奇艺世纪科技有限公司 | A kind of video transmission method and device |
| CN110784740A (en) * | 2019-11-25 | 2020-02-11 | 北京三体云时代科技有限公司 | Video processing method, device, server and readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11265599B2 (en) * | 2019-06-11 | 2022-03-01 | Nvidia Corporation | Re-encoding predicted picture frames in live video stream applications |
-
2021
- 2021-03-26 CN CN202110325525.2A patent/CN112929667B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105163117A (en) * | 2015-08-26 | 2015-12-16 | 华为技术有限公司 | Image coding and processing methods and devices |
| CN106998328A (en) * | 2017-03-30 | 2017-08-01 | 北京奇艺世纪科技有限公司 | A kind of video transmission method and device |
| CN110784740A (en) * | 2019-11-25 | 2020-02-11 | 北京三体云时代科技有限公司 | Video processing method, device, server and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112929667A (en) | 2021-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10321138B2 (en) | Adaptive video processing of an interactive environment | |
| US6357028B1 (en) | Error correction and concealment during data transmission | |
| US6611561B1 (en) | Video coding | |
| US8606025B2 (en) | Encoding and decoding images using refreshed image and recovery point | |
| EP0933948A2 (en) | Video encoder and video encoding method | |
| RU2283543C2 (en) | Method for processing error code of compressed image during transmission | |
| US20170094294A1 (en) | Video encoding and decoding with back channel message management | |
| US20210385504A1 (en) | Quality aware error concealment technique for streaming media | |
| US10819976B2 (en) | Long-term reference for error recovery without back channel | |
| CN112929667B (en) | Encoding and decoding method, device, equipment and readable storage medium | |
| WO2021052500A1 (en) | Video image transmission method, sending device, and video call method and device | |
| CN101312536B (en) | Video coding method and apparatus for video transmission error control | |
| JP5275810B2 (en) | Method and apparatus for video error concealment using reference frame selection rules | |
| CN112040234B (en) | Video encoding method, video decoding method, video encoding device, video decoding device, electronic equipment and storage medium | |
| CN112995214B (en) | Real-time video transmission system, method and computer readable storage medium | |
| CN112437319A (en) | Method for switching multiple video streams based on webrtc | |
| US11265583B2 (en) | Long-term reference for error recovery in video conferencing system | |
| CN116866604A (en) | Image processing method and device | |
| CN116962613A (en) | Data transmission method and device, computer equipment and storage medium | |
| US10200694B2 (en) | Method and apparatus for response of feedback information during video call | |
| JP2002010265A (en) | Transmitting device and its method and receiving device and it method | |
| JP5122448B2 (en) | Video encoding method and apparatus | |
| KR101942371B1 (en) | Apparatus and method for video frame scene change detection and encoding in mobile wireless environment | |
| JPH07193821A (en) | Animation picture and its method | |
| US20140289369A1 (en) | Cloud-based system for flash content streaming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |