KR100923961B1 - System and method for low-delay video telecommunication - Google Patents

Abstract

Even when a scene change frame exists, a transmission delay can be prevented due to overflow of a buffer, and a video communication system capable of ensuring a certain level or higher of a playback image, a device constituting the same, and a device for the same Discuss the algorithm. According to an embodiment of the present invention, the encoding unit of the transmitting apparatus sequentially inputs input image data of a current frame that is input in macroblock line units and is inter-mode in block units of a predetermined size, The scene change determination unit determines whether the scene change frame corresponds to the scene change frame based on the image data from the coding unit to the block group in which encoding has been performed on the current frame that is divided into N (N is a natural number of two or more) blocks. As described above, in the encoding unit that determines whether the scene change frame is in block group units, the frame determined to be not the scene change frame performs encoding on the entire frame, but encoding is performed on the subsequent block group with respect to the scene change frame. Do not perform. The receiving side apparatus of the video communication system repeatedly plays back the image of the previous frame at the time corresponding to the scene change frame.

Description

System and method for low-delay video telecommunications

The present invention relates to a video communication system, and more particularly, to a video communication system capable of supporting low-delay video communication with a certain level or more of quality and a procedure therefor.

A system for providing real-time video communication such as video telephony or video conferencing is called a 'video communication system'. A video communication system is a device for allowing voice and video communication in real time between two or more users who are separated from each other, and includes a sending device and a receiving device. The transmitting device encodes the video and audio, and then transmits the encoded data through wired and wireless channels, and the receiving device receives the encoded data and decodes the received encoded data. Restore voice Such a transmitting device and a receiving device are generally provided together in one device such as a terminal or a user equipment. Of course, the video communication is performed between the transmitting device and the receiving device of different user equipment.

Various video and audio codec technologies have been proposed to implement a video communication system. For example, H.261, the international standard for video telephony and video conferencing over Integrated Services Digital Network (ISDN), and international videophone services over public switched telephone network (PSTN). H.263, the standard. In addition, digital storage media (DSM) or MPEG (MPEG) 1 or MPEG 2, which are video compression standards for broadcast communication, may also be used as an image codec technology of a video communication system. The codec technology of video and audio has a significant influence on image quality and sound quality in video communication.

On the other hand, in a video communication system, it is very important to ensure real-time communication with image quality or sound quality. Codec technology indirectly affects the real-time of communication by affecting the amount of data generated, but the transmitter of the transmitting device and the receiver of the receiving device (these transmitters and receivers can be implemented as transceivers of the user device). Data processing speed and transmission-related technologies directly affect the real-time of video communication. For example, in order to support real-time video communication, end-to-end delay should be prevented or minimized by minimizing buffer delays at the transmitter and the receiver.

One reason for the end-to-end delay caused by the buffer delay is when a scene change frame exists in the input image, because the amount of data generated as a result of encoding in the scene change frame is larger than other frames. 'Scene change frame' is a frame when the scene before and after the scene is changed, and even if it is not the scene change, the correlation between the images before and after is low so that the amount of generated bits per frame at the time of encoding is relatively scene changed even when the average bit generation amount is relatively high. May be included in the frame. In general, when it is not a scene change frame, the encoder of the transmitting apparatus compresses and encodes the input video using inter mode, that is, inter-screen prediction. In this case, not only the load on the encoder is small but also the amount of encoded data Because of the relatively small number, the problem of transmission delay in the transmitter buffer is less likely. On the other hand, in the case of the scene change frame, since the encoding is not possible in the inter mode but in the intra mode, the generated bit amount is relatively large, and the load on the encoder increases. In addition, when the amount of bits generated is large, a problem of transmission delay in the buffer of the transmitter may be caused.

Video coding techniques that do not take into account real-time communication, such as MPEG-4 or H.264 / AVC, do not consider whether the frame is a scene change frame, and the entire video is the same group of pictures (GOP). Coding is performed using the following structure. When the existing video encoding technology is applied to a real-time video communication system as it is, the amount of bits generated per frame is relatively increased as compared to when the video is not a scene change frame. In this way, encoding so that the amount of generated bits varies greatly in accordance with the characteristics of the frame (eg, whether or not it is a scene change frame) is called variable bit rate (VBR) encoding.

As described above, when the image is encoded by the VBR method, a reproduced image having a constant image quality may be obtained regardless of whether the frame is a scene change frame. However, the VBR method causes a large fluctuation in the amount of bits generated by the encoder, which causes a significant transmission delay in the transmitter buffer or all the encoded data generated by the transmission channel having a limited bandwidth. It is difficult to transmit.

For example, suppose that the amount of bits generated by the encoder due to the scene change frame instantaneously increases. In this case, if the encoding is performed using the VBR method, not only data accumulation occurs in the buffer of the transmitter, but also a buffer overflow phenomenon may occur in the buffer of the transmitter in severe cases. When the buffer overflow occurs, data loss that cannot be restored in the video communication system is a problem that must be avoided in the video communication system.

Even if the buffer overflow does not occur, an increase in the amount of bits generated by the encoder may cause another problem of 'transmission delay'. This is because the bitstream entered into the transmitter buffer should be transmitted to the receiving device through the transmission channel, since the data rate in the transmission channel is generally limited. That is, when a large amount of data is generated temporarily, a frame in which a large amount of data is generated through a transmission channel cannot be transmitted at a time, and as a result, a delay occurs in the transmission of the bitstream. This problem of transmission delay also hinders real-time video communication.

In addition, if the amount of bits generated by the encoder increases, the amount of data to be decoded by the decoder increases by that amount, so that it is difficult to proceed with decoding normally when there is a scene change frame. If the decoding of the data received from the decoder is not performed normally, it greatly affects the image quality of the reproduced video, and as a result, it is difficult to perform a smooth video call or a video conference. Therefore, a real-time video communication system supporting video call, video conferencing, etc. needs to prevent such problems from occurring.

One way to solve the problems of the above-described VBR scheme is to keep the amount of bits generated per frame in the encoder almost constant regardless of the characteristics of the image. In this case, the bit amount generated by the encoder is almost constant regardless of whether it is a scene change frame, and this type of encoding is called constant bit rate (CBR) encoding. However, when the fixed bit rate encoding method is used, the image quality of the display image is not constant, and in particular, in the case of a scene change frame, the image quality is considerably worsened.

However, even when an image is encoded by the CBR method, there may be a variation in the amount of bits generated and output by the encoder. In this case, in general, the transmission rate of data transmitted through the transmission channel is kept constant by adjusting or controlling the buffer in the transmitter. However, controlling the data rate using the transmitter buffer is a major factor in increasing the end-to-end delay caused by the buffer delay.

As described above, when a frame having a low correlation with a previous frame, that is, a scene change frame, enters an input image, and according to a conventional video communication system or procedure, there is a problem of overflow due to a buffer of a transmitter or a delay due to buffering, Various problems may occur such as a transmission delay problem in the transmission channel and / or deterioration of the image quality of the reproduced video.

The problem to be solved by the present invention is the problem of delay due to overflow or buffering in the buffer of the transmitter and transmission in the transmission channel even if there is a scene change in the input video sequence or even if the scene change is not correlated with the previous frame The present invention provides an apparatus and method for low latency video communication, which can prevent a delay problem from occurring and also ensure a certain level or higher of a playback image.

One embodiment of the present invention for solving the above problems relates to a transmitting side device of a video communication system including a video encoding device, the video encoding device is input in units of macroblock lines, and inter- A coding unit for sequentially encoding input image data of a current frame of a current frame) in macroblock units, and encoding is performed in the coding unit with respect to the current frame that is divided into N (N is a natural number of two or more) block groups. And a scene change determiner configured to determine whether the current frame corresponds to a scene change frame based on the image data up to the k (1 ≦ k ≦ N−1) th group of blocks. If it is determined that the scene is not a scene change frame, encoding is not performed on the entire current frame. Do not perform the encoding for the image by the image data belonging to the present case that the frames are determined to be a scene change frame, the (k + 1) th block from the second group of N group in the block to the k-th group of blocks.

According to an aspect of the embodiment, the scene change determination unit (SAD) of the difference between the pixel value of the image from the first block group to the k-th block group of the current frame and the image of the corresponding portion of the previous frame or A mean (MAD) or an average value of the sum of the differences of each block group may be used to determine whether the current frame is a scene change frame. In this case, the corresponding portion of the previous frame is a corresponding portion of the immediately preceding frame of the current frame, or if the immediately preceding frame of the current frame corresponds to a scene change frame and there is no corresponding portion, the immediately preceding frame of the previous frame. May be the corresponding portion of.

According to another aspect of the embodiment, N may vary according to a frame position (eg, frame number) of an image to be encoded. That is, it is not necessary to set the same number of block groups for all frames, and the number of block groups may also vary according to the characteristics of the frame. In this case, there is a difference in time at which the scene change determination unit determines whether to change scenes. In addition, the sizes of each of the plurality of block groups constituting one frame are not necessarily the same.

According to another aspect of the present invention, there is provided a receiving apparatus of a video communication system, including: a receiving unit for receiving image data transmitted through a transmission channel, and reconstructed image data by decoding the received image data. And a decoding unit for generating and outputting a signal, and when the output frame is a scene change frame, the decoding unit repeatedly outputs an image of a frame that is not a previously output scene change frame.

According to an aspect of the embodiment, the image data received by the receiving unit includes flag information for indicating whether the corresponding frame is a scene change frame, and the decoding unit may include an image of the current frame or The image of the previous frame can be output. The decoding unit may determine whether the decoded image data is data for a last macroblock of a current frame, and whether the current frame is a scene change frame based on encoding mode information for the last macroblock. May reproduce the image of the current frame or the image of the previous frame based on the determination result in the decoding unit.

The video encoding method for a video communication system according to an embodiment of the present invention for achieving the above-mentioned problems is to encode the input image data of the current frame which is input in units of macroblock lines and is also inter-mode (Inter-Mode) Image data up to the k (1≤k≤N-1) th block group in which the encoding step is performed in the encoding step for the current frame that is divided into a coding step and N (N is a natural number of two or more) block groups. And a scene change determination step for determining whether the current frame corresponds to a scene change frame, and in the encoding step, if it is determined that the current frame is not a scene change frame, encoding is performed on the entire current frame. However, the current frame is determined by the image data up to the k-th block group. If it is determined that the side switch frame (k + 1) does not carry out the image coding for belonging to the second group of blocks from the N-th group of blocks.

According to another aspect of the present invention, there is provided a video decoding method for a video communication system, the receiving step of receiving image data transmitted through a transmission channel and the reconstructed image data by decoding the received image data. And a decoding step for generating and outputting the data. In the decoding step, when the frame to be played is a scene change frame, the image of a frame other than the previously played scene change frame is repeatedly output.

According to an embodiment of the present invention, it is possible to prevent the occurrence of transmission delay problems due to overflow or buffering in the buffer of the transmitting device of the video communication system and transmission delay problems in the transmission channel, as well as the quality of the playback image. Can be guaranteed above a certain level.

1 is a block diagram showing the configuration of a real-time video communication system according to an embodiment of the present invention.

2 is a block diagram illustrating a block group based encoding method according to an embodiment of the present invention.

3 is a block diagram illustrating a block group based encoding method according to another embodiment of the present invention.

4 is a block diagram illustrating a display method according to an embodiment of the present invention when encoded according to the method shown in FIG. 2.

FIG. 5 is a block diagram illustrating a display method according to an embodiment of the present invention when encoded according to the method shown in FIG. 3.

FIG. 6 is a graph illustrating a comparison of bit generation rates when a proposed algorithm and an existing algorithm according to TMN8 are applied according to an embodiment of the present invention.

7 is a graph showing a comparison of the PSNR in the case of applying the proposed algorithm according to the embodiment of the present invention and the conventional algorithm according to TMN8.

Hereinafter, with reference to the accompanying drawings will be described in detail an embodiment of the present invention. The embodiments described below are disclosed for the purpose of illustrating the technical idea of the present invention, and should not be construed as limiting the technical idea of the present invention. And the names of each component in the description of the following embodiments may be called other names in the art. Similarly, some modifications to the configuration on the drawings may be regarded as substantially the same as the configuration on the drawings disclosed in the present embodiment, provided that the functional similarity or the same. Reference numerals added in the drawings and descriptions of the embodiments are only for convenience of description, and like reference numerals denote like elements throughout the specification.

1 is a block diagram showing the configuration of a real-time video communication system according to an embodiment of the present invention. Referring to FIG. 1, a real-time video communication system includes a transmitting device 100 and a receiving device 200, and transmission of video and audio data from the transmitting device 100 to the receiving device 200 is maintained. This is accomplished through a wireless transmission channel. The following describes only encoding and transmission of video data excluding audio data, since the present invention is related to a scene change frame. Therefore, in the embodiments described below, voice data can be encoded and transmitted in the same manner as in the prior art, and there is no limitation in the method.

The transmitting device 100 includes an encoding unit 110, a bit-rate controlling unit 120, and a transmitting unit 130. An image signal (input image) input to the encoding unit 110 of the transmitting device 100 may be obtained using a camera (not shown), but is not limited thereto. For example, the input image may be an image previously stored in the storage medium or an image received through another transmission channel. When acquiring an input image using a camera, the camera may be provided in combination with the transmitting device 100 or may be provided as a separate component. The information about the image complexity of the input image acquired by the camera or the like is transmitted to the bit rate control unit 120.

According to an aspect of the present embodiment, input image data obtained by a camera or the like may be input to the encoding unit 110 in units of macroblock lines. Information on image complexity of the input image may also be input to the bit rate control unit 120 in units of macroblock lines. As such, when image data is input in macroblock units using the encoding unit 110 or the like, bit rate control may be performed in macroblock line units.

The encoding unit 110 is a means for compressing and encoding an image input in units of macroblock lines in units of macroblocks or smaller blocks having a predetermined size, and there is no particular limitation on a codec method applied to an input image. For example, in the encoding unit 110, all image compressions known to date such as H.261, H.263, H.264 / AVC, MPEG-1, MPEG-4, MPEG-4, or VC-1, etc. Any one of the encoding schemes defined in the international standard related to the encoding may be used, or a different new scheme may be used.

The encoding unit 110 according to an embodiment of the present invention encodes the first frame of the input image in the intra mode, but encodes the remaining frames except the first frame in the inter mode. Can be. Alternatively, according to an exemplary embodiment, encoding of one frame may be performed in an intra mode at a predetermined interval with respect to the input image, and all other frames may be encoded in an inter mode. In the case of encoding in the inter mode, encoding is performed through motion estimation and compensation using an arbitrary frame as a reference frame among various frames immediately before, similar to that defined in the existing codec procedure, or the current frame and the immediately preceding frame. The encoding may be performed by using a difference frame composed of a difference value with the frame as a skin encoding frame. In the embodiment of the present invention, the latter case is more preferable.

The encoding unit 110 may include a means for determining whether the skin encoded frame is a scene change frame, for example, the scene change determiner 112. The scene change determination unit 112 is means for determining whether a frame input to the encoding unit 110 corresponds to a scene change frame by using the image of the current frame and the image of the previous frame. The scene change determination unit 112 may not only detect the actual scene change but may also determine a frame having a low correlation between the current frame and the previous frame even if not the scene change as the scene change frame.

According to an embodiment of the present invention, the scene change determination unit 112 may determine whether to change scenes in units of a group of blocks (GOB). Determining whether or not to switch scenes in units of GOB means determining whether a corresponding frame corresponds to a scene change frame by dividing into two or more block groups of a frame and then using image data encoded up to a predetermined block group. . That is, it is determined whether the current frame is a scene change frame based on the encoding result up to the first block group, and if it is determined that the current frame is not a scene change frame by the encoding result up to the first block group, It is determined whether or not it is a scene change frame based on the encoding result up to the block group. On the other hand, when it is determined that the scene is a transition frame based on the encoding result up to the first block group, the encoding is not performed on the subsequent block group.

In this way, for determination in units of block groups, one frame is divided into a plurality of block groups, for example, M blocks. Here, M is two or more. In this case, each block group in the same frame may have the same size but different sizes. For example, since an object is generally located at the center of the image rather than the edge of the image, the frame may be divided into a plurality of block groups so that the size of the block group located at the center is smaller than the block group at both ends. In addition, the number of block groups divided according to frames does not necessarily need to be the same.

There is no particular limitation on how many blocks are divided into one frame group, but it may be divided into two to ten block groups, preferably three to five block groups. This is because, in the case of dividing into two block groups, it is difficult to fully achieve the effects of the present invention. In the case of dividing into more than five block groups, the number of times the previous frame is repeatedly played by the receiving device increases. This is because there is a disadvantage.

According to an aspect of the present embodiment, the scene change determination unit 112 is a difference frame consisting of the difference value of the image of the current frame and the image of the previous frame or the difference consisting of the difference value of the reconstructed image of the image of the current frame and the previous frame The frame may be used to determine whether the current frame corresponds to the scene change frame. Alternatively, the scene change determination unit 112 may determine whether to change scenes by using statistical characteristics of image information between the current image and the previous image. In an embodiment of the present invention, any method among the known methods may be used. There is no special restriction as to whether to use it. Existing known methods include, for example, Difference of Gray-level Sums, Sum of Gray-level Differences, Difference of Gray-level Histograms, Colored Template Matching, Difference of Color Histograms and Comparison of Color Histograms or Brightness Difference of Histograms and Pixels between Successive Frames or Variance of DC coefficients and Motion Vectors to Analyze Scene Changes or Count the Number of Valid Motion Vectors in Predictive-coded Pictures or P-pictures and Bidirectionally Predictive-coded Pictures or B-pictures DC Image Sequences to Analyze Scene Changes, and so on.

Hereinafter, an example of dividing a frame into three block groups having the same size will be described in detail with reference to FIG. 2 or 3 along with FIG. 1. FIG. 2 and 3 are block diagrams each illustrating an encoding scheme including block group based scene change frame determination according to an embodiment of the present invention. Here, the frame numbers 1 to 5 are all frames encoded in the inter mode.

1 and 2, when image data of frame 1 is sequentially input in macroblock line units, the encoding unit 110 may first input a macro of a previous frame corresponding to the macroblock with respect to the input image data. The SAD (Sum of Absolute Difference) with the block is calculated and stored. This process proceeds sequentially to all macroblock lines in the first block group of frame 1. The scene change determination unit 112 of the encoding unit 110 obtains an average value of the SADs up to the calculated first block group, and determines whether the current frame is a scene change frame. For example, by comparing whether the average value of the SAD is smaller or larger than a predetermined threshold, it may be determined that the former is not a scene change frame, and the latter may be determined to be a scene change frame. Here, the predetermined threshold is a value used as a reference for determining whether or not the scene change frame is determined in consideration of the capacity of the buffer, the capacity of the transmission channel, or the like, or is experimentally determined.

If it is determined that the current frame is not a scene change frame using the first block group, the SAD of the second block group of the current frame and the previous frame corresponding to the second block group are calculated and stored. do. The scene change determination unit 112 of the encoding unit 110 calculates an average value of the SADs up to the second block group (first and second block groups), and compares it with a predetermined threshold to determine whether the current frame is a scene change frame. do. As a result of the determination, when it is determined that the image is not a scene change frame, the calculation of the SAD, determination of the scene change frame, etc. are performed for the remaining block groups in the same manner as in the previous block group. In the present embodiment, frame 1 is finally determined not to be a scene change frame, and as in the conventional method, the entire frame is encoded in the inter mode.

Subsequently, referring to FIG. 1 and FIG. 2, it is determined whether the frame corresponds to a scene change frame after the encoding is completed in the block group unit in the same manner with respect to the second frame. More specifically, when the image data of frame 2 is sequentially input in units of macroblock lines, the encoding unit 110 first calculates an SAD with the macroblock of the previous frame corresponding to the macroblock with respect to the input image data. Save it. This process proceeds sequentially up to all macroblocks or macroblock lines in the first block group of frame two. The scene change determination unit 112 of the encoding unit 110 determines whether the current frame is a scene change frame after obtaining the average value of the SADs up to the first block group. For example, when the average value of the SADs up to the first block group is equal to or greater than a predetermined threshold value, the scene change determination unit 112 determines that the current frame is a scene change frame. In this case, the encoding unit 110 skips the remaining block groups of the current frame (that is, the second and third block groups) without performing encoding anymore (that is, the remaining block groups of the current frame). COD (Coded or Not-coded) of the macroblock included in the value is set to '1'.

Subsequently, after encoding is completed in units of block groups in the same manner with respect to frame 3, it is determined whether the frame corresponds to a scene change frame. More specifically, when the image data of the third frame is sequentially input in units of macroblock lines, the encoding unit 110 first calculates an SAD with the macroblock of the previous frame corresponding to the macroblock with respect to the input image data. Save it. This process proceeds sequentially up to all macroblock lines in the first block group of frame 3. The scene change determination unit 112 of the encoding unit 110 obtains an average value of the SADs up to the first block group, and determines whether frame 3 is a scene change frame. However, in the present embodiment, since the frame 2 is determined as the scene change frame in the first block group of the frame 2, the first block group of the frame 3 is based on the block group based on the frame 2 as the reference frame. Encoding is performed, and the second block group of frame 3 performs block group based encoding on frame 1 as a reference frame. This is because frame 2 is only encoded with the first block group, and the remaining block groups are not encoded. Referring to FIG. 2, frame 3 is determined to be a scene change frame as a result of using the average value of the SADs up to the second block group (first and second block groups), and thus no encoding is performed on the last third block group. (I.e., all macroblocks included in the third block group have a COD of '1').

Subsequently, encoding is performed in units of block groups in the same manner with respect to frame 4, and an average value of SADs to the corresponding block group is obtained to determine whether frame 4 is a scene change frame. However, in the present exemplary embodiment, the frame 2 is determined as the scene change frame in the first block group of frame 2, and the frame 3 is determined as the scene change frame in the second block group of frame 3, The first and second block groups of frame 4 perform block group based encoding using frame 3 as a reference frame, and the third block group of frame 4 performs block group based encoding using frame 1 as a reference frame. To perform. This is because, according to the embodiment of the present invention, the third block group of frame 2 and frame 3 is not encoded. Referring to FIG. 2, since frame 4 is not determined as a scene change frame (when the SAD values of the first and second block groups are small, the SAD average value up to the third block group may be smaller than the threshold value. It may be determined that this is not a scene change frame.) Encoding is performed on all block groups. In the case of a general frame that is not a transition frame such as frame 4, depending on whether or not the motion vector of the macroblock is (0,0), the COD of the macroblock is treated as 1 or 0, and the motion vector is Macroblocks with a COD of 1 after (0,0) are processed in the corresponding macroblock mode as a skip mode.

 Subsequently, encoding is performed in units of block groups in the same manner with respect to frame 5, and an average value of SADs to the corresponding block group is obtained to determine whether frame 5 is a scene change frame. However, in this embodiment, since it is determined that frame 4 is not a scene change frame, all block groups of frame 5 perform block group-based encoding on frame 4 as a reference frame. Referring to FIG. 2, since frame 5 is not determined to be a scene change frame, encoding is performed on all block groups.

1 and 3, the encoding unit 110 encodes image data input by one macroblock line in units of macroblocks or smaller block sizes. After encoding the first block group of frame 1, the encoding unit 110 calculates and stores the SAD of the previous frame corresponding to the first block group. The scene change determination unit 112 of the encoding unit 110 obtains an average value of the SADs up to the first block group, and determines whether the current frame is a scene change frame. For example, by comparing whether the average value of the SAD is smaller or larger than a predetermined threshold, it may be determined that the former is not a scene change frame, and the latter may be determined to be a scene change frame.

If it is determined that the current frame is not a scene change frame using the first block group, the SAD of the second block group of the current frame and the previous frame corresponding to the second block group are calculated and stored. do. The scene change determination unit 112 of the encoding unit 110 obtains an average value of the SADs up to the second block group, and compares it with a predetermined threshold to determine whether the current frame is a scene change frame. As a result of the determination, when it is determined that the image is not a scene change frame, the calculation of the SAD, determination of the scene change frame, etc. are performed for the remaining block groups in the same manner as in the previous block group. In the present embodiment, frame 1 is finally determined not to be a scene change frame, and as in the conventional method, the entire frame is encoded in the inter mode.

1 and 3, the encoding unit 110 sequentially encodes frame 2 in macroblock units, and the scene change determination unit 112 of the encoding unit 110 performs frame 2. The average value of the SAD up to the first block group of is used to determine whether the current frame is a scene change frame. For example, when the average value of the SADs up to the first block group is smaller than a predetermined threshold, the encoding unit 110 calculates the SAD with the previous frame corresponding to the second block group, After obtaining the SAD average value of the first block group, it is determined whether the current frame is a scene change frame. As a result of the determination, when the SAD average value up to the second block group is equal to or greater than a predetermined threshold value, the scene change determination unit 112 determines that the current frame is a scene change frame. In this case, the encoding unit 110 skips the remaining block groups (ie, the third block group) of the current frame without performing encoding anymore (ie, a macro included in the third block group of the current frame). Set the block's COD (Coded or Not-coded) to '1'.

Subsequently, it is determined whether frame 3 is a scene change frame in units of block groups in the same manner with respect to frame 3, and an average value of SADs to the corresponding block group is obtained to determine whether frame 3 is a scene change frame. However, in the present embodiment, since the frame 2 is determined as the scene change frame in the second block group of the frame 2, the first block group and the second block group of the frame 3 refer to the frame 2 as a reference frame. Encoding is performed to determine whether it is a scene change frame, and the third block group of frame 3 performs block group based encoding on frame 1 as a reference frame. This is because in frame 2, only the first and second block groups are encoded, and the third block group is not encoded.

Subsequently, frames 4 and 5 are sequentially encoded in macroblock units or blocks of a predetermined size in the same manner, and frame 4 or 5 using the average value of SADs to the corresponding block group. It is determined whether the first frame is a scene change frame. However, in the present embodiment, since it is determined that frame 3 is not a scene change frame, all block groups of frame 4 encode frame 3 as a reference frame and determine whether to change scenes, and frame 5 All of the block groups in the A-frame encode frame 4 as a reference frame and determine whether to switch scenes. Referring to FIG. 3, since frames 4 and 5 are not determined to be scene change frames, encoding is performed on all block groups.

When it is determined that the current frame is a scene change frame according to the above-described embodiment of the present invention, the encoding unit 110 transmits the information indicating the fact (for example, a scene change instruction flag) to a bitstream to be transmitted. Can be included. This is to inform the receiving apparatus 200 of the scene change frame so that decoding can be adaptively performed corresponding to an encoding method in the case where there is a scene change frame, as will be described later. In some embodiments, the information indicating that the current frame is a scene change frame may be transmitted to the bit rate control unit 120. In this case, the bit rate control unit 120 may also receive the received scene change frame indication information. Can be used to control

For example, the following two methods may be used to inform the decoding unit 220 of the receiving side apparatus 200 of the scene change occurrence.

The first method is to implicitly indicate whether or not a transition frame is made. There are various methods. For example, there may be a method of defining a block pattern and informing the decoder side of a frame matched with the block pattern as a scene-transformed frame. As an example of this, the encoding unit 110 forcibly encodes a different encoding mode between a frame determined as a frame to which the scene has been changed and the last macroblock of the frame other than the scene change. For example, in the case of a scene change frame, the last macroblock is unconditionally encoded in the intra mode, and in the non-scene change frame, the last macroblock is unconditionally encoded in the inter mode, so that the current frame is It can tell if it is a transition frame.

 Another method for notifying the decoding unit 220 of the receiving apparatus 200 of the frame in which the scene change has occurred is to use an explicit method, for example, adding an element for notifying the scene change to the data syntax. There may be a way. More specifically, define a new flag (flag) to inform the Picture Layer whether a transition frame, and if the current frame is a transition frame, set the flag to '1', the transition If it is not a frame, it can be set to '0'.

1, the bit rate control unit 120 uses the received image complexity information, the bit rate information received from the encoding unit 110, and the like to determine the encoded data generated as a result of the encoding in the encoding unit 110. Control the bit rate of the bitstream. For example, the bit rate control unit 120 may control the bit rate by adjusting the size of a quantization coefficient (QP) used in the encoding unit 110 at the time of encoding. In addition, according to an aspect of an embodiment of the present invention, the bit rate control unit 120 may also use information indicating whether a scene change frame is used as an element for determining the size of the quantization coefficient. This bit rate control process may be performed in units of macroblock lines.

The transmitting unit 130 is an apparatus for transmitting the data received from the encoding unit 110. For example, the bitstream generated and received by the encoding unit 110 may be temporarily stored through the encoder buffer of the transmission unit 130 and then transmitted to the receiving side apparatus 200 at a predetermined transmission rate through the transmission channel. The bitstream may be transmitted in a packet unit, for example, and the transmission rate of the transmission channel may be constant or may vary according to various network conditions.

Subsequently, the receiving side apparatus 200 will be described with reference to FIG. 1. The receiving device includes a receiving unit 210, a decoding unit 220, and a display unit 230. When the bitstream is received through the antenna of the receiving device 200, the received encoded data is sent to the decoding unit 220 through the buffer of the receiving unit 210. The decoding unit 220 generates a reconstructed image frame by decoding the encoded data as before, and sends the reconstructed image frame (output image) generated to be reproduced by the display unit 230. According to the present embodiment, the decoding unit 220 performs decoding using information on the encoding method as well as information on the received scene change frame.

According to the exemplary embodiment of the present invention, the reconstructed image generated by the decoding unit 220 and reproduced by the display unit 230 does not include images of all frames. For example, when the encoding unit 210 performs encoding by the above-described block group-based encoding method, the image signal for the skipped block group (blocks in which the COD of the macroblock is indicated by '1') is received from the receiving side apparatus. Since it is not transmitted to 200, the decoding unit 220 may not generate a reconstructed image of the entire frame for these frames. Therefore, the display unit 230 of the receiving-side apparatus 200 according to the present embodiment, for example, skips only the scene change frame or skips the scene change frame and one or more subsequent frames, instead of the previous frame. The video is reproduced by repeatedly playing the video by skipping frames.

FIG. 4 illustrates an image reproduced according to an embodiment of the present invention when encoding is performed by the encoding unit 110 to determine whether each frame is a scene change frame on a block group basis as in the example shown in FIG. 2. A block diagram showing. Referring to FIG. 4, when frames 2 and 3, which are scene transition frames, are reproduced on the display unit 230, images of frame 1, which is the previous frame, are repeatedly played, and frames 1, 4, or 5 are repeated. If it is not a scene change frame such as the burn frame, the reconstructed video of the corresponding number is played as it is.

FIG. 5 is a block diagram illustrating an image reproduced according to an exemplary embodiment of the present invention when encoding is performed in units of block groups as in the example illustrated in FIG. 3. Referring to FIG. 5, when the second frame, which is a scene change frame, is reproduced on the display unit 230, the image of the first frame, which is the previous frame, is repeatedly played, and the first frame, the third frame, the fourth frame, or the fifth frame. If it is not a scene change frame such as a frame, the reconstructed video of the corresponding number is reproduced as it is.

As described above, according to the embodiment of the present invention, encoding is performed in units of macroblocks or macroblock lines, and it is determined whether the current frame corresponds to a scene change frame by using difference data up to a predetermined block group of the frame. do. As a result of the determination, when the current frame is determined to be a scene change frame by difference data up to some block groups instead of the entire block group of the current frame, the transmitting apparatus does not perform encoding on the images of the remaining block groups. Information indicating that the frame is a scene change frame is also transmitted to the transmitting device. When the scene is not a scene change frame, the receiving device reproduces the original reconstructed image as it is, but the reconstruction image of the previous frame is repeatedly reproduced for the scene change frame.

Experiment result

In order to test the performance of the algorithm for the low latency video communication system according to the embodiment of the present invention described above, the algorithm was implemented in the international standard H.263 Baseline Profile. Table 1 shows the experimental images and bit rates applied to these tests. In Table 1, the delta bit rate is a value added or subtracted based on the target bit rate (1 Mbps), and represents an upper limit and a lower limit of the amount of bit generation allowed in the coding unit of the transmitting apparatus.

division Experimental video Image size / bit rate 720 x 480 (1 Mbps) Delta bitrate 0.5 Mbps Experiment Screen 300 screen Cutaway occurrence screen 64, 163, 255, 260

FIG. 6 is a graph illustrating a comparison of bit generation rates when the proposed algorithm and the conventional algorithm according to the TMN8 are applied according to the above-described embodiment of the present invention. Referring to FIG. 6, it can be seen that the proposed algorithm according to the embodiment of the present invention is located between the upper and lower limits of the bit rates disclosed in Table 1.

FIG. 7 is a graph illustrating a comparison of a power signal-to-noise ratio (PSNR) when a proposed algorithm according to an embodiment of the present invention and an existing algorithm according to the TMN8 are applied. Referring to FIG. 7, the PSNR of the proposed algorithm according to the embodiment of the present invention shows an improved trend over the algorithm according to TMN8 in most frames. However, according to the embodiment of the present invention, since the proposed algorithm encodes only a part of the screen in the scene change frame and the following two frames, the PSNR is decreased, but since these frames are not displayed at the receiving device, this is not a problem at all. Do not.

As described above, according to the embodiment of the present invention, since the bit generation amount can be controlled within a predetermined fluctuation range so that the bit generation amount does not vary greatly in the bit generation amount of the video communication system, It is possible to prevent the occurrence of an overflow phenomenon and a transmission delay. In addition, according to the embodiment of the present invention, the PSNR also shows an improvement of an average of 2.0 dB over the conventional algorithm. Further, according to the embodiment of the present invention, since the image of the previous frame is repeatedly played back in the scene change frame, the image of the scene change frame is delayed, but the image quality of the image played therebetween is prevented from being deteriorated. You can do it.

Claims (11)

A transmission side apparatus of a video communication system including a video encoding apparatus, wherein the video encoding apparatus A coding unit for sequentially encoding input image data of a current frame that is an inter-mode; For the current frame configured by dividing N (N is a natural number of two or more) block groups, the current frame is configured based on image data up to a k (1≤k≤N-1) th block group in which encoding is performed in the coding unit. A scene transition determination unit for determining whether the current frame corresponds to a scene transition frame, If it is determined that the current frame is not a scene change frame, the encoding unit encodes the entire current frame, but determines that the current frame is a scene change frame based on the image data up to the k-th block group. If not, encoding is not performed for the video belonging to the (k + 1) th block group to the Nth block group. The apparatus of claim 1, wherein the scene change determination unit uses an average value of a sum of absolute difference (SAD) between an image of a first block group to a kth block group of the current frame and an image of a corresponding portion of a previous frame. And determining whether the current frame is a scene change frame. 3. The frame of claim 2, wherein the corresponding portion of the previous frame is a corresponding portion of the immediately preceding frame of the current frame, or when the immediately preceding frame of the current frame corresponds to a scene change frame and there is no corresponding portion. A transmission side device of a video communication system, characterized in that it is a corresponding portion of a previous frame. The apparatus of claim 1, wherein the N may vary according to a frame position of an image to be encoded. The apparatus of claim 1, wherein the size of each of the N divided block groups may not be the same. A receiving unit for receiving image data transmitted through a transmission channel; A decoding unit for generating and outputting reconstructed image data by decoding the received image data; And A display unit for reproducing the reconstructed image data; And if the frame to be decoded is a scene change frame, the decoding unit repeatedly outputs an image of the previous frame rather than the decoded scene change frame. The image data according to claim 6, wherein the image data received by the receiving unit includes flag information for indicating whether the corresponding frame is a scene change frame, and the decoding unit includes an image of a current frame or a previous frame based on the flag information. A receiving device of a video communication system, characterized in that for outputting an image of a video. The apparatus of claim 6, wherein the decoding unit determines whether the decoded image data is data for a last macroblock of a current frame, and the current frame is a scene change frame based on encoding mode information for the last macroblock. And the display unit reproduces the image of the current frame or the image of the previous frame based on the determination result in the decoding unit. A video communication system comprising a transmitting device and a receiving device, The video encoding apparatus of the transmitting apparatus includes an encoding unit for sequentially encoding input image data of a current frame having an inter-mode; The current frame is divided based on image data up to a k (1≤k≤N-1) th block group in which encoding is performed in the coding unit with respect to the current frame divided into N (N is a natural number of two or more) block groups. A scene transition determination unit for determining whether a frame corresponds to a scene transition frame, and the encoding unit performs encoding on the entire current frame when it is determined that the current frame is not a scene transition frame, but the k When it is determined that the current frame is a scene change frame based on the image data up to the first block group, encoding is not performed on the images belonging to the (k + 1) th block group to the Nth block group. The receiving device is A receiving unit for receiving image data transmitted through a transmission channel; And And a decoding unit for generating and outputting reconstructed image data by decoding the received image data. And if the frame to be decoded is a scene change frame, the decoding unit repeatedly outputs an image of the previous frame rather than the decoded scene change frame. As a video encoding method for a video communication system, An encoding step of sequentially encoding input image data of a current frame that is an inter-mode; The current frame, which is divided into N (N is a natural number of 2 or more) block groups, is based on image data up to a k (1≤k≤N-1) th block group in which encoding is performed in the encoding step. A scene transition determination step for determining whether the current frame corresponds to a scene transition frame, In the encoding step, when it is determined that the current frame is not a scene change frame, encoding is performed on the entire current frame, but it is determined that the current frame is a scene change frame based on the image data up to the k-th block group. If not, the video encoding method for a video communication system, characterized in that the encoding is not performed for the video belonging to the (k + 1) -th block group to the N-th block group. A receiving step of receiving image data transmitted through a transmission channel; And And a decoding step of generating and outputting reconstructed image data by decoding the received image data. And in the decoding step, if the frame to be decoded is a scene change frame, the image of the previous frame is repeatedly output instead of the decoded scene change frame.