JP4721028B2 - Moving image distribution system, moving image distribution method and program thereof - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving image distribution system that distributes a compression-coded moving image from a transmitting station to a plurality of receiving stations, and a technology related thereto.
[0002]
[Prior art]
In general, a moving image is transmitted to a remote place. For example, as shown in FIG. 10, the imaging camera 1 is installed toward the monitoring target, and the monitoring video (moving image) captured by the imaging camera 1 is transmitted from the server 2 to the remote clients 4 a and 4 b through the network 3. .., And monitor display is performed by the clients 4 a, 4 b... To monitor the monitoring video, and the monitoring video is recorded as necessary.
[0003]
By the way, when transmitting a moving image through the network 3, it is desirable to reduce the data size by compressing and encoding the moving image in consideration of the communication speed and the data capacity at the time of recording by the clients 4a, 4b. That is, as shown in FIGS. 10 and 11, for example, a moving image captured by the imaging camera 1 is compressed and encoded by the encoder 6 and transmitted from the server 2 to each client 4 a, 4 b. On the client 4a, 4b... Side, the compressed and encoded moving image given from the server 2 is received through the client device 4, and then the moving image is decoded by the decoder 7 and displayed on the monitor device 5.
[0004]
As a method of compressing and encoding such a moving image such as a surveillance video, a method (first prior art) that combines DCT (Discrete Cosine Transform) and motion compensated predictive encoding is common. This method is also adopted in the MPEG (Moving Picture Experts Group) system and the like. In general, DCT is applied to intra-frame (intra) coding in which coding is performed using only information in a frame (still image) so as to reduce redundancy in the spatial direction. In addition, motion compensation predictive coding (inter-frame (inter) coding) predicts a frame to be encoded from a frame at another time in order to reduce redundancy in the time direction, Are subjected to DCT, quantization, or the like. In this case, in order to keep the difference small, the encoding target frame is often predicted from temporally adjacent frames. Such intra-frame coding and motion compensation predictive coding (inter-frame coding) processing is performed with a block obtained by dividing a frame into a plurality of basic processing units.
[0005]
The outline of the conventional differential encoding method will be described with reference to FIG. A plurality of frames f1, f2,... Sequentially output from the imaging camera 1 are sequentially input to the encoder 6 (FIGS. 10 and 11). As described above, when each frame fn (n = 1, 2,...) Is input to the encoder 6 (ST100), whether or not the input frame fn is a key frame is determined in step ST101. If the frame fn is a key frame, intra-frame encoding processing is executed in step ST102. That is, the frame fn is divided into blocks, DCT is performed for each block, and the conversion coefficient is calculated. Next, a quantized coefficient obtained by quantizing the transform coefficient is output. Next, in step ST103, encoded data obtained by variable-length encoding (entropy encoding) the quantized coefficient is generated, output as a bit stream. Further, the quantization coefficient calculated in step ST102 described above is stored in the key frame memory 100 after being decoded (inverse quantization and inverse DCT) in step ST104.
[0006]
Next, when the next frame fm (m = n + 1) is input in step ST100, it is determined whether or not the frame fm is a key frame in step ST101. If the frame fm is not a key frame, the process proceeds to step ST105, and a difference value of pixel values is calculated in block units between the key frame fn and the frame fm stored in the key frame memory 100. Next, in step ST106, it is determined whether or not the difference value is within a predetermined range. If the difference value is within the predetermined range, inter-frame (inter) encoding, that is, DCT and quantization are performed on the difference signal between the key frame and the input frame fm in step ST107. On the other hand, if the difference value exceeds the predetermined range, intra-frame coding is executed in step ST108. As described above, the quantization coefficients calculated in steps ST107 and ST108 are variable length encoded in step ST103, converted into a bit stream, and then output to the network 3 through the server 2.
[0007]
An example of such a bitstream decoding process will be described below with reference to FIG. When the above bit stream is input to the clients 4a, 4b,... Via the network 3 (ST110), the decoder 7 extracts a compressed encoded signal from the bit stream, performs variable length decoding, and performs the quantization coefficient. Is obtained. In the subsequent step ST111, if the quantized coefficient has been intra-frame encoded in step ST102 of the compression encoding process, it is decoded (intra-frame decoding) and stored in a predetermined key frame memory 101. In step ST108, when the quantized coefficient is intra-frame encoded, the decoding is performed. On the other hand, if the quantized coefficient has been inter-frame encoded at step ST107, the decoding (inter-frame decoding) is performed with reference to the key frame stored in the key frame memory 101. Then, the decoded image that has been decoded within the frame or between the frames is displayed on the monitor device 5 (ST112).
[0008]
In such a moving image compression coding system represented by the MPEG system, high compression efficiency can be obtained by compressing difference information between frames.
[0009]
In addition, a still image compression method (intraframe coding method) such as the Motion JPEG method is adopted, and such a still image is displayed in a time-series manner so that it is reproduced like a moving image. There is also a method (second prior art).
[0010]
[Problems to be solved by the invention]
By the way, in the differential encoding method according to the first prior art described above, for example, many frames (each frame other than the key frame) include time-series differential information in a compressed and encoded moving image (video data). Since it is composed of frames, it is a series of data by continuation of each frame.
[0011]
Here, video data including such a time-series difference (inter-frame coding) is called a P picture, while a difference (intra-frame coding) is taken between adjacent images in a frame without including the time-series difference. The obtained video data is called an I picture.
[0012]
In the case of the first prior art, the number of inserted P pictures is generally fixed. In the case of such a compression encoding method, for example, as shown in FIG. 14, when transmission of video data is started to one client 4a at the timing T1, first, the intra-frame encoded I picture 8a is stored in the server 2 From the I picture 8a to the client 4a, the P picture 9a obtained by taking the time series difference (interframe coding) one after another is transmitted, and after transmitting the P picture 9a for a certain frame, Periodic I picture 8b and P picture 9b are transmitted.
[0013]
By the way, for example, it is assumed that there is a request for new distribution from another client 4b at timing T2. At the timing T2, the server 2 is in the process of transmitting the P picture 9a to the network 3. By the way, the P picture 9a is a frame obtained by taking time series differences (interframe coding) one after another from the I picture 8a as a starting point and chaining one after another. Unless the picture 8a is obtained, it is impossible to decode a moving picture only from the P picture 9a. However, since the first I picture 8a has already been discharged to the network 3, the first I picture 8a cannot be obtained on the client 4b side. Therefore, the client 4b cannot decode the video data until the next I picture 8b is given from the server 2.
[0014]
However, for example, in the video distribution such as MPEG2 having a frame rate of about 30 fps, for example, when an I picture appears every second, for example, the P picture is transmitted to one client 4a. Even if there is a request for new distribution from another client 4b, it is possible to transmit the I picture to the client 4b within at least one second, so that the other client 4b can decode the video data without delay. can do.
[0015]
However, for example, when recording a monitoring video for security purposes in the server 2, for example, it may be recorded by dropping frames at a rate of less than 30 fps, for example, an I picture at a timing of once every 30 seconds as real time. May be recorded. When such a moving picture having a long appearance period of an I picture is distributed to a plurality of clients 4a and 4b, for example, when a P picture is transmitted to one client 4a, a new distribution request is issued from another client 4b. When there is, there may be a case where an I picture cannot be transmitted for up to 30 seconds. For this reason, it takes a long time to wait for the decoding of the moving image in the other client 4b. That is, when a moving image compression encoding method with a long I picture appearance cycle is applied, such as a moving image with dropped frames, the waiting time cannot be ignored for the client 4b that has requested new distribution.
[0016]
On the other hand, in the second prior art, since all the frames are intra-frame encoded, all the clients 4a and 4b that have requested distribution at an arbitrary timing are intra-frame encoded at the respective timings at that time. Distributed video data can be distributed. However, this second prior art has a drawback that the frame rate is lowered because of the large amount of data.
[0017]
In view of such a problem, the present invention intends to solve the problem that a moving image with a low waiting time when a new distribution request is requested during the distribution of video data while efficiently distributing a moving image at a high compression rate. An object is to provide an image distribution system and a technology related thereto.
[0018]
[Means for Solving the Problems]
  In order to solve the above-mentioned problem, the invention according to claim 1 receives a transmitting station that compresses and encodes a moving image and distributes the moving image data that is compressed and encoded by the transmitting station and distributed over a network. A plurality of receiving stations, wherein the transmitting station has an internal frame of the moving image.Love-Frame coding using informationByGenerated intraframe encoded data and difference information between different framesGenerated by the interframe coding usedEncoding means for compressing and encoding the moving image while inter-frame encoded data is mixed along the time axis in units of frames, the intra-frame encoded data and the frame compressed and encoded by the encoding means Data transmitting means for transmitting inter-coded data to the network along a time axis, and the encoding means of the transmitting stationIsWhenever there is a new distribution request from any of the receiving stations,GenerationAnd thisTheThe intra-frame encoded data is transmitted through the data sending means and the network.ReceivedDelivered to the bureauAnd the encoding means of the transmitting station designates a key frame for each predetermined period from the plurality of frames of the moving image, and the encoding means of the transmitting station includes a plurality of other frames other than the key frame. In each frame, the frame is divided into a plurality of block areas, a specific area is designated from among the plurality of block areas, and the positions of the specific areas of the plurality of other frames included in the designated period of the key frame Are different from each other, and the encoding means of the transmitting station includes a reference frame for one frame generated by combining specific areas of each of the plurality of other frames included in the specified period, and a key frame. If the second difference between them is larger than a predetermined value, intra-frame coding is performed on the key frame, while if the second difference is smaller than the predetermined value, the key frame It is intended to apply the reference inter-frame coding has the reference frame to.
[0019]
  Invention of Claim 2 is a moving image delivery system of Claim 1, Comprising:The encoding means of the transmitting station applies intra-frame encoding to the block area when the first difference between the block area excluding the specific area and the key frame is larger than a predetermined value, When the difference is smaller than a predetermined value, inter-frame encoding is performed on the block area with reference to the key frame, and intra-frame encoding is performed on the specific area.
[0020]
Invention of Claim 3 is a moving image delivery system of Claim 1 or Claim 2, Comprising: The said transmission station has an imaging camera which images a predetermined imaging target, The said receiving station is, A moving image captured by the imaging camera is displayed or recorded.
[0021]
  The invention according to claim 4A moving image distribution method for compressing and encoding a moving image at a transmitting station and distributing the compression-encoded moving image to a plurality of receiving stations through a network, comprising: a) one frame of the moving image at the transmitting station Time frame of intra-frame encoded data generated by intra-frame encoding using internal information and inter-frame encoded data generated by inter-frame encoding using difference information between different frames in units of frames B) compressing and encoding the moving image while being mixed along the line; b) transmitting the intra-frame encoded data and the inter-frame encoded data to the network along the time axis at the transmitting station; The step a) includes: a-1) generating the intra-frame encoded data every time there is a request for new distribution from any one of the receiving stations. And a-2) designating a key frame at regular intervals from a plurality of frames of the moving image, and in each of a plurality of other frames other than the key frame, the frames are divided into a plurality of block areas. Dividing and designating a specific area from among the plurality of block areas, and the positions of the specific areas of the plurality of other frames included in the designated period of the key frame are different from each other, In the step a), the second difference between the reference frame for one frame generated by combining the specific areas of each of the plurality of other frames included in the specified cycle and the key frame is greater than a predetermined value. If it is larger, intra-frame coding is applied to the key frame, and if the second difference is smaller than a predetermined value, the reference frame for the key frame is used. In which reference inter-frame coding and the is applied.
[0023]
  According to a fifth aspect of the present invention, in order to implement the moving image distribution method according to the fourth aspect on a computer, the computer executes the steps.thingIt is.
The invention according to claim 6 includes a transmitting station that compresses and encodes a moving image and distributes it, and a receiving station that receives data of the moving image that is compressed and encoded by the transmitting station and distributed over a network, The transmitting stationIsIntra-frame coding data generated by intra-frame coding using intra-frame coding using information in one frame of the moving image and inter-frame coding using difference information between different frames Encoding means for compressing and encoding the moving image while mixing data along the time axis in units of frames, the intra-frame encoded data and the inter-frame encoded data compressed and encoded by the encoding means Data transmitting means for transmitting to the network along a time axis, and the encoding means of the transmitting station designates a key frame at a certain period from a plurality of frames of the moving image, and The encoding means of the transmitting station divides the frame into a plurality of block areas in each of a plurality of other frames other than the key frame, and A specific area is designated from the designated areas, and the positions of the specific areas of the plurality of other frames included in the designated period of the key frame are different from each other, and the encoding means of the transmitting station uses the designated period. When the second difference between the reference frame for one frame generated by combining the specific areas of each of the plurality of other frames included in the key frame and the key frame is larger than a predetermined value, the key frame On the other hand, when the second difference is smaller than a predetermined value, intra-frame coding with reference to the reference frame is performed on the key frame.
[0024]
DETAILED DESCRIPTION OF THE INVENTION
<Basic concept of video distribution system>
First, the basic concept of a moving image distribution system used in an embodiment of the present invention will be described. In this moving image distribution system, as in the conventional example shown in FIGS. 10 and 11, for example, a moving image such as a monitoring video imaged by the imaging camera 1 is compression encoded by the encoder 6, and this moving image (compressed frame) is compressed. Data) is stream-distributed from the server 2 as a transmitting station to a plurality of clients 4a and 4b (client machines 4) as receiving stations at remote locations through the network 3. On the clients 4 a and 4 b side, the compression-coded moving image (compressed frame data) given from the server 2 is decoded by the decoder 7 and displayed on the monitor device 5.
[0025]
Here, the encoder 6 is an existing method such as the existing MPEG system (see FIGS. 12 and 13), the MPEG2 system, or the like such as Japanese Patent Application No. 2000-368510 proposed by the present applicant. A moving image is compression-encoded using a moving image compression encoding method (hereinafter referred to as “SRVC method”: see later).
[0026]
Then, as shown in FIG. 1, the encoder 6 uses the I picture 11a as intra-frame encoded data generated only by intra-frame encoding as the first frame, and a plurality of frames following the I picture 11a are A moving image is compression-encoded so as to become a plurality of P pictures 12a as inter-frame encoded data including information of time-series differences (inter-frame encoding) sequentially obtained starting from the I picture 11a. . For example, in the case of the MPEG system or the MPEG2 system, the number of frames (for example, 29 frames) of a P picture following one I picture 11a is fixed in principle, and the I picture 11a of one frame and the P picture 12a of 29 frames are constant. Cycle.
[0027]
In particular, when there is a moving image distribution request from each of the clients 4a and 4b, the encoder 6 has received the distribution request even when the P picture should be transmitted as a basic distribution timing. Immediately after the time points Th1, Th2, and Th3, the I pictures 11b, 11c, and 11d that are independently intra-coded are forced to appear, and the I pictures 11b, 11c, and 11d are used as the first frame in time series. Compression encoding is performed so that the P pictures 12b, 12c, and 12d are followed by the difference (interframe encoding).
[0028]
The server 2 and the client machine 4 use computer equipment including a predetermined data recording device such as a RAM, a ROM, and a hard disk drive, and a central processing unit (CPU). The stored compressed image data is managed, and the processing order of the compressed image data for performing reproduction according to the distribution of moving images is determined. The operation procedure of the central processing unit is regulated by a software program stored in advance in the data recording device, and is executed according to this software program.
[0029]
<Compression encoding method>
Hereinafter, an example in which this embodiment is applied to an SRVC compression encoding method will be described. FIG. 2 is a schematic diagram for explaining the SRVC compression encoding method, and FIG. 3 is a flowchart for realizing this compression encoding method.
[0030]
This compression encoding method of the SRVC method is different from the MPEG method in which a difference is made with reference to the immediately preceding frame in the encoder (encoding means) 6 (see FIG. 13). As shown in FIG. It is fixed and compressed, so that thinning during reproduction and reproduction in the reverse time direction are relatively easy.
[0031]
In this compression encoding method, as shown in FIG. 2, for example, a plurality of still images (frames) sequentially taken along the time axis by an imaging camera 1 (see FIG. 13) such as a CCD (charge coupled device) sensor or a CMOS sensor. are input to the encoder 6 (see FIG. 13) (step ST1 in FIG. 3).
[0032]
The key block memory 21A in FIG. 3 stores a reference frame f0 (not shown) configured by connecting predetermined specific areas (key GOB) as they are before the frame f1 is input. The reference frame f0 is compressed and transmitted to the key block memory 21B (see FIG. 8) of the decoder (decryption means) 7, and then decrypted and stored. However, for the first frame at the time of starting compression encoding, there is no key GOB preceding the first frame, and therefore no reference frame is present.
[0033]
In the next step ST1a, the encoder 6 determines whether there is a request for new distribution from any of the clients 4a and 4b (see FIG. 12).
[0034]
If there is a request for new distribution, the process proceeds to step ST1b, and it is necessary to transmit the I pictures 11b, 11c, and 11d. In ST19, the data is output to the server 2 as bit stream data. Specifically, the intra-frame coding processing at this time performs orthogonal transform such as DCT on the pixel value of the frame f1, and the quantized coefficient obtained by quantizing the transform coefficient is calculated. With this processing procedure, the encoder 6 generates I pictures 11b, 11c, and 11d and outputs them to the clients 4a and 4b through the server 2 every time a new distribution request is received from each client 4a and 4b. . Here, the I pictures 11b, 11c and 11d distributed to the clients 4a and 4b are processed as key frames in the encoder 6 and the decoder 7 on the clients 4a and 4b side. That is, the process proceeds to step ST8 in FIG. 3, where local decoding (inverse orthogonal transformation such as inverse quantization and inverse DCT) is performed and stored in the key frame memory 20A shown in FIG. Accordingly, the key frame memory 20A stores a changed key frame including a quantization error through intra-frame coding (ST1b) and local decoding (ST8). As a result, the image of the key frame becomes the same as the image of the key frame that is referred to when being decoded (interframe decoding) by the decoder 7 described later, and the image quality of the moving image to be decoded is deteriorated. Disappears. Thus, the compression encoding process for the I pictures 11b, 11c, and 11d (key frames) when a new distribution request is received from each of the clients 4a and 4b is completed.
[0035]
On the other hand, if there is no request for new distribution from each client 4a, 4b in step ST1a, the encoder 6 determines that the input frame f1, unless there is a request for new distribution from each client 4a, 4b. A key frame is designated periodically from f2, f3, f4,. In FIG. 2, the input frame f1 (corresponding to the I picture 11a in FIG. 1) is a key frame.
[0036]
Then, in step ST2 in FIG. 3, it is determined whether or not the input frame f1 is a key frame. In FIG. 2, since the frame f1 is a key frame as described above, the process proceeds to step ST3 as it is, and intra-frame (intra) / inter-frame (inter) encoding processing is executed.
[0037]
The processing procedure of step ST3 is shown in FIG. As shown in FIG. 4, the difference value of the pixel value between the reference frame f0 (not shown) and the input frame f1 (key frame) stored in the key block memory 21A in step ST4 and the absolute value sum of the difference values (Difference absolute value sum) S is calculated. However, if the input frame f1 is the first frame at the time when compression encoding is started, there is no reference frame as described above, so that data in which all pixels are zero is used as the reference frame. As the data, an absolute value sum (difference absolute value sum) S of difference values is calculated.
[0038]
Next, in step ST5 in FIG. 4, it is determined whether or not the sum of absolute differences is equal to or less than a predetermined threshold value. For example, when the difference value is expressed by ΔPi (i: number corresponding to each pixel), the difference absolute value sum S is expressed as S = | ΔP1 | + | ΔP2 | + ... + | ΔPn | (n: number of pixels) It is expressed by When the difference absolute value sum S is equal to or smaller than the threshold value, the input frame f1 is subjected to inter-frame encoding using the frame f0, assuming that the temporal change between the frames f0 and f1 is small (step ST6). Specifically, an orthogonal transform such as DCT (Discrete Cosine Transform) is performed on the difference signal between the input frame f1 and the frame f0, and a quantized coefficient obtained by quantizing the transform coefficient is calculated. Such interframe coding processing is executed in units of small blocks (macroblocks) having a size of 8 × 8 pixels or 6 × 6 pixels. The same applies to the subsequent processing. In this embodiment, orthogonal transformation such as DCT is adopted as a transformation method, but DWT (discrete wavelet transformation) may be adopted instead of DCT. In this case, the inter-frame encoding process may be executed in units of frames by dividing the frame into a plurality of areas called tiles in consideration of the execution memory capacity or the like instead of being performed in units of the small blocks. .
[0039]
On the other hand, in step ST5 in FIG. 4, when the difference absolute value sum S calculated in step ST4 exceeds a predetermined threshold, the process proceeds to step ST7. For example, when the input frame f1 is the first frame at the time when compression encoding is started (corresponding to the I picture 11a in FIG. 1), the data in which all pixels are zero as described above is used. Since the absolute value sum (difference absolute value sum) S of the difference values is calculated as the reference frame data, the difference absolute value sum S calculated in step ST4 exceeds a predetermined threshold value. In step ST7, intraframe coding is performed on the input frame f1 (key frame) using only the information in the frame. Specifically, orthogonal transformation such as DCT is performed on the pixel value of the frame f1, and a quantized coefficient obtained by quantizing the transform coefficient is calculated.
[0040]
Actually, from the viewpoint of increasing the compression rate of the frame, color space conversion is performed on the input frame before the intra-frame coding (steps ST1b and ST7) or the inter-frame coding (step ST6). Applied. For example, when the original signal is composed of RGB spaces of “R (red component)”, “G (green component)”, and “B (blue component)”, this is adopted in the NTSC (National Television System Committee) system or the like. A YUV coordinate system, a YIQ coordinate system, a YCbCr coordinate system, or the like may be used. For example, when the YCbCr coordinate system is used, the RGB component is converted into a YCbCr component coordinate system composed of a luminance signal Y and two color difference signals Cb and Cr. Since the YCbCr component has a smaller correlation between components than the RGB component, the image size can be compressed.
[0041]
After execution of the intra-frame / inter-frame encoding process (step ST3, that is, the process of FIG. 4), the process proceeds to step ST19 in FIG. At this time, the quantized coefficients calculated in steps ST6 and ST7 (FIG. 4) are subjected to entropy coding including Huffman coding, and then image information such as the image size of the frame and the number of quantization bits, Are multiplexed together with compression information such as a coding table and a coding method (intraframe coding and interframe coding) for each small block area, and output to the server 2 as a bit stream. Further, the quantization coefficients calculated in steps ST6 and ST7 (FIG. 4) are subjected to local decoding (inverse orthogonal transformation such as inverse quantization and inverse DCT) in step ST8 in FIG. 3, and are shown in FIG. Stored in the key frame memory 20A. Accordingly, the key frame memory 20A stores a changed key frame including a quantization error through encoding (ST6, ST7) and local decoding (ST8). As a result, the image of the key frame becomes the same as the image of the key frame that is referred to when being decoded (interframe decoding) by the decoder 7 described later, and the image quality of the moving image to be decoded is deteriorated. Disappears. The compression encoding process for the input frame f1 (key frame) is thus completed.
[0042]
Next, as shown in FIG. 2, the frame f2 is input to the encoder 6 following the frame f1. Then, whether or not the frame f2 is a key frame is determined in step ST2 in FIG. In FIG. 2, the frame f2 is not a key frame. Therefore, in FIG. 3, the process proceeds to step ST9.
[0043]
In step ST9, the frame f2 is divided into a plurality of block areas (GOB). Here, the GOB is composed of, for example, an area of 6 lines or 32 lines in an image of one frame. For example, when an area of 6 lines is one GOB, an image of 640 × 480 pixels is 1 A frame is divided into 30 (= 480/6) GOBs.
[0044]
Next, in step ST10, one or more specific areas (key GOB) are designated from these GOBs.
[0045]
Here, FIG. 6A schematically shows a frame f2 divided into four GOBs. The frame f2 is divided into four GOBs in units of ten to several tens of pixels in the vertical direction, and the first stage GOB is designated as the key GOB (“key GOB1”). As shown in FIGS. 2B to 2D, frames f3 to f5 sequentially input to the encoder 6 following the frame f2 are also divided into a plurality of GOBs. However, as the respective key GOB, the second stage GOB (“key GOB2”) of the frame f3, the third stage GOB (“key GOB3”) of the frame f4, and the fourth stage GOB (“key” of the frame f5). GOB4 ") is specified. These frames f2 to f5 including the key frame f1 and the keys GOB1 to GOB4 are arranged along the time axis as shown in FIG.
[0046]
Returning to the description of the frame f2 again. At this point, the process proceeds to step ST11 in FIG. 3, and thereafter, the frame f2 is sequentially processed for each small block obtained by further dividing the GOB into basic processing units of about 8 × 8 pixels or 6 × 6 pixels. become. However, in this step ST11, it is determined whether or not the block to be processed belongs to the key GOB. If the block belongs to the key GOB, the process proceeds to step ST12. Then, after the block is subjected to the intra-frame encoding, it is entropy encoded in step ST19, multiplexed with the image information and the compression information, and output to the server 2 as a bit stream. In addition, the quantized coefficients output after intra-frame coding of the block in step ST12 are stored in the key block memory 21A after being subjected to local decoding (inverse orthogonal transformation such as inverse quantization and inverse DCT) in step ST3. The
[0047]
When the key GOB is thus encoded in the frame, a flag indicating that the key GOB is used is added to the head of the data of each key GOB.
[0048]
On the other hand, if the block does not belong to the key GOB in step ST11, the process shifts to the intraframe / interframe encoding process in step ST14. Here, FIG. 5 is a flowchart showing a processing procedure of a subroutine of intra-frame / inter-frame encoding processing. As shown in FIG. 5, first, in step ST5, the difference value between the block of the input frame and the key frame stored in the key frame memory 20A and the difference absolute value sum S are calculated. Next, in step ST6, a condition determination is made as to whether or not the difference absolute value sum S is less than or equal to a threshold value. The inter-frame coding is performed with reference to the key frame stored in the above. On the other hand, when the difference absolute value sum S exceeds the threshold value, the process proceeds to step ST18, and the block is subjected to the intra-frame coding.
[0049]
The quantized coefficients encoded in steps ST7 and ST18 are subjected to variable length encoding (entropy encoding) and the multiplexing process in step ST19 shown in FIG. Is output. Thus, the compression encoding process for the input frame f2 is completed.
[0050]
Next, in FIG. 2, the frames f3, f4,... Input to the encoder 6 subsequent to the frame f2 are processed in the same manner as in the case of the frame f2 until a key frame is input. However, as described above, the positions of the key GOB (key GOB1 to key GOB4) in the frames f2, f3, f4,. In this way, the key GOB1 to key GOB4 locally decoded in step ST3 is stored in the key block memory 21A for one frame, and as schematically shown in FIG. 2, the keys GOB1 to GOB4 are: In the key block memory 21A, it is synthesized with the reference frame A. This reference frame A is used when a key frame to be input later is inter-frame encoded in the subroutine of step ST3.
[0051]
The moving image (compressed image data) compressed and encoded by the encoder 6 is streamed and distributed from the server 2 to the clients 4a and 4b through the network 3 using a predetermined protocol such as TCP / IP.
[0052]
In this way, intra-frame coding and inter-frame coding are selectively executed according to the difference between the difference from the reference frame stored in the key block memory 21A in step ST3 and input in steps ST9 and ST10. A frame is divided into a plurality of GOBs, a key GOB is designated, a key GOB for one frame is distributed to a plurality of frames along the time axis, and each of these keys GOB is encoded in the frame. For this reason, the intraframe encoding processing amount is dispersed in time, the rapid increase in the compression encoding processing amount is suppressed, the encoding processing amount is flattened in time, and the playback speed of the moving image at the transmission destination High quality moving images can be compressed and transmitted without changing. This is particularly effective for transmission lines with limited bandwidth, such as the Internet.
[0053]
In the key block memory 21A, the key GOB distributed in a plurality of frames is stored, and a reference frame A including these keys GOB is configured. This reference frame A is an accumulation of key GOBs at different times. In this embodiment, intra-frame coding and inter-frame coding are selectively executed depending on the difference between the reference frame A and the key frame. For this reason, the difference in image quality between GOBs resulting from the use of the reference frame A made up of the key GOBs at different times is alleviated, and it becomes possible to compress and transmit high-quality moving images.
[0054]
<Decryption method>
Next, the decoding method will be described in detail. FIG. 7 is a schematic diagram for explaining this decoding method, and FIG. 8 is a flowchart for realizing this decoding method.
[0055]
First, similarly to the conventional example shown in FIGS. 10 and 11, the compressed image data given from the server 2 through the network 3 is given to the client machine 4 as bit stream data and inputted to the decoder 7 ( Step ST20 in FIG. 8). The compressed image data is separated from the bit stream and then decoded in step ST21. That is, the compressed data of the frames f1, f2,... Shown in FIG. At this time, in step ST21 in FIG. 8, the compressed data of the key frame f1 is decoded by intra-frame coding or inter-frame coding in the same manner as the processing procedure shown in steps ST4 to ST7 in FIG. Processing is performed in units of small blocks of about 8 × 8 pixels or 6 × 6 pixels. Here, when decoding the compressed data of the key frame f1 in FIG. 7, a reference frame f0 (not shown) stored in advance in the key block memory 21B (FIG. 8) is used. The key frame f1 decrypted here is stored in the key frame memory 20B (FIG. 8).
[0056]
Further, in FIG. 7, the intraframe coding in steps ST12, ST14 to ST18 in FIG. 3 or the compressed data of frames f2, f3,... Input to the decoder 7 following the compressed data of the key frame f1 or The same processing as the decoding processing of interframe coding is performed in units of the small blocks. Note that the key frame f1 stored in the key frame memory 20B is used when performing decoding processing of inter-frame coding. When the frames f2, f3,... Are decoded, if a small block that is a basic processing unit belongs to the key GOB, the small block is stored in the key block memory 21B. When the key GOB for one frame is accumulated, the reference frame A (see FIG. 2) composed of these keys GOB is synthesized. The combined reference frame A is used when the compressed data of the key frame input to the decoder 7 is decoded. For example, when the compressed data of the frames f2 to f5 shown in FIGS. 6 (a) to 6 (d) are input to the decoder 7, the compressed data of the blocks constituting each key GOB is subjected to intraframe decoding. Later, the frame is sequentially stored in the key block memory 21B to reconstruct the reference frame A.
[0057]
By the way, when the frame groups f1, f2,... Decoded in step ST21 are displayed as moving images in this way, between the GOB that has been intra-frame encoded by the encoder 6 and the GOB that has been inter-frame encoded. Therefore, a difference in the image quality of the moving image is likely to appear, and in particular, the key GOB encoded in the frame may be clearly seen in the moving image. In consideration of this, in this embodiment, in step ST22 shown in FIG. 8, only the key GOB decoded in step ST21 is encoded again, and then the key GOB requantization process is executed.
[0058]
FIG. 9 is a flowchart showing a processing procedure of a subroutine of key GOB requantization processing in step ST22. As shown in FIG. 9, first, a small block of about 8 × 8 pixels or 6 × 6 pixels is input (step ST30). Next, in step ST31, it is determined whether or not the block belongs to the key GOB. Specifically, in step ST31, whether or not the data is the key GOB is determined based on whether or not there is a flag indicating the key GOB at the head of the data.
[0059]
Here, if the block does not belong to the key GOB, the block is not requantized, the key GOB requantization process ends, and the process moves to step ST23 shown in FIG.
[0060]
On the other hand, if it is determined in step ST31 that the block belongs to the key GOB, the process proceeds to step ST32, and the difference value between the key frame stored in the key frame memory 20B and the pixel value of the block, The difference absolute value sum S of the difference values is calculated.
[0061]
Next, in step ST33, a condition determination is made as to whether or not the difference absolute value sum S is equal to or smaller than a threshold value. If the difference absolute value sum S exceeds the threshold value, the block is not requantized, and key GOB requantization processing is performed. Is completed, and the process proceeds to step ST23 shown in FIG.
[0062]
On the other hand, if it is determined in step ST33 that the sum of absolute differences S is equal to or less than the threshold value, the process proceeds after step ST34. First, in step ST34, the difference signal between the block and the key frame is transform-coded, and then in step ST35, the transform coefficient is quantized. The processing of these steps ST33 to ST35 includes determination processing (ST6) of an encoding method (interframe encoding, intraframe encoding) based on the sum of absolute differences S performed by the encoder 6, and orthogonal transformation such as DCT. And it is the same as the quantization process (ST7).
[0063]
After that, in step ST36, the quantized coefficient is inversely quantized, and then in step ST37, transform coding decoding (inverse orthogonal transform such as inverse DCT) in step ST34 is performed. As a result, in accordance with the processing in steps ST34 to ST37, quantization is performed in the same manner as when the block 6 other than the key GOB is interframe-encoded by the encoder 6 and then the encoded signal is decoded by the decoder 7. An irreversible difference signal including an error is obtained.
[0064]
Next, the block is reconstructed from the difference signal using the key frame stored in the key frame memory 20B in step ST38 and output.
[0065]
The block subjected to the key GOB requantization processing is output after being combined with a frame (decoded image) in step ST23 shown in FIG.
[0066]
The above key GOB requantization process will be described by taking the frames f2 to f5 shown in FIG. 6 as an example. As schematically shown in FIG. 7, the keys of the frames f2 to f5 decrypted in step ST21 described above are used. The difference between the GOB and the key frame stored in the key frame memory 20B is taken.
[0067]
Next, in step ST32, it is determined whether or not the difference absolute value sum S of the difference values is equal to or less than a threshold value, that is, whether or not to perform interframe encoding. Is subjected to inter-frame coding (transform coding and quantization) and then subjected to decoding of the inter-frame coding (inverse quantization and inverse transform decoding), so that the frames f2 to f5 Decoded images F1 to F5 corresponding to are generated.
[0068]
In this manner, if the difference between the key GOB and the key frame is small in the same procedure as the compression encoding process (ST5 to ST7), the compression encoding is performed on the difference signal between the key GOB and the key frame. After that, the decoding is performed and the key GOB is reconstructed, so that the encoder 6 performs inter-frame coding on the block area other than the key GOB and then the decoder 7 decodes the encoded signal. In addition, errors due to interframe coding and decoding are mixed in the key GOB. Therefore, when displaying the decoded moving image, the key GOB does not stand out in the moving image, and there is an effect that the person watching the moving image does not feel uncomfortable.
[0069]
In addition to distributing the I picture 11a as the first frame, every time there is a request for new distribution from each of the clients 4a and 4b, the independent I pictures 11b, 11c, and 11d are forcibly changed immediately thereafter. Since the P pictures 12b, 12c, and 12d including GOBs that are inter-frame encoded based on the I pictures 11b, 11c, and 11d are subsequently distributed, a new distribution request is made later. The I pictures 11b, 11c, and 11d can be promptly transmitted to the clients 4a and 4b, and the clients 4a and 4b display the I pictures 11b, 11c, and 11d, and then the subsequent P pictures 12b, 12c, Since 12d can be quickly decoded, the waiting time for moving image reproduction of each of the clients 4a and 4b is reduced. In particular, when the moving image to be compression-encoded by the encoder 6 is information that has already been thinned out in time series, such as a frame dropping video of a monitoring video, I When the appearance cycle of pictures is as long as 30 seconds as the real time at the time of imaging, it is possible to avoid a serious situation such as oversight of monitoring by reducing the waiting time on the client 4a, 4b side. There are advantages.
[0070]
In addition, when a frame other than a key frame is distributed, only a part of the blocks (key GOB) is intra-frame encoded. Therefore, according to the first prior art that distributes only inter-frame encoded data. Although the data size is slightly increased as compared with the differential encoding method, it is only necessary to transmit time-series differential data at the time of transmitting a key frame. Compared with the differential encoding method according to the technology, the transmission data amount of the key frame can be greatly reduced. Therefore, in the distribution of streaming data, it is possible to suppress a rapid increase in the coding processing amount of the key frame, which is the most problematic in the first prior art, and the coding processing amount is temporally flattened and the moving image is animated. The image quality can be improved. Further, there is an advantage that the amount of data can be greatly reduced as compared with the second prior art in which all the frames are intra-coded.
[0071]
In the above-described embodiment, an example in which the moving image that has been distributed by streaming is displayed on the monitor device 5 on the client 4a, 4b side has been described, but instead of or in addition to this, the client 4a, 4b side In this case, the moving image may be recorded on a predetermined recording device in an accumulative manner.
[0072]
【The invention's effect】
According to the first to sixth aspects of the present invention, in the transmitting station, the moving image is an inter-frame including intra-frame encoded data (I picture) consisting only of intra-frame encoded information and inter-frame encoded information. When the encoded data (P picture) is compressed and encoded along the time axis based on a certain rule and streamed to a plurality of receiving stations, during streaming delivery to one receiving station, When there is a request for new distribution from another receiving station, immediately after that, the transmitting station generates intra-frame encoded data (I picture) and performs streaming distribution to each transmitting station. The receiving station can obtain the independent intra-frame encoded data (I picture) immediately after issuing the request and immediately decode it. Therefore, the waiting time at each receiving station is reduced.
[0073]
  Also,Claim 1 toAccording to the invention of claim 6,In compression coding for key frames, intra-frame coding and inter-frame coding are selectively executed depending on the difference between the reference frame and the key frame, so that all key frames are coded within the frame. Compared toThe amount of key frame transmission data can be greatly reduced. Accordingly, in the distribution of streaming data, it is possible to suppress a rapid increase in the encoding processing amount of key frames other than the intra-frame encoded data, flatten the encoding processing amount in time, and improve the image quality of moving images. It becomes possible to improve.
[Brief description of the drawings]
FIG. 1 is a diagram showing the appearance timing of an I picture according to an embodiment of the present invention.
FIG. 2 is a schematic diagram for explaining a compression coding method according to an embodiment of the present invention.
FIG. 3 is a flowchart for realizing a compression encoding method according to an embodiment of the present invention.
FIG. 4 is a flowchart for realizing a compression encoding method according to an embodiment of the present invention.
FIG. 5 is a flowchart for realizing a compression encoding method according to an embodiment of the present invention.
FIG. 6 is an explanatory diagram showing each frame divided into four block areas.
FIG. 7 is a schematic diagram for explaining a decoding method according to an embodiment of the present invention.
FIG. 8 is a flowchart for realizing a decoding method according to an embodiment of the present invention.
FIG. 9 is a flowchart for explaining requantization processing;
FIG. 10 is a conceptual diagram showing an outline of a conventional moving image distribution system.
FIG. 11 is a block diagram showing a conventional moving image distribution system.
FIG. 12 is a flowchart for explaining a conventional encoding method.
FIG. 13 is a flowchart for explaining a conventional encoding method.
FIG. 14 is a diagram illustrating the appearance timing of an I picture in a conventional encoding method.
[Explanation of symbols]
1 Imaging camera
2 servers
3 network
4a, 4b client
5 Monitor device
6 Encoder
7 Decoder
11a-11d I picture
12a-12d P picture

Claims (6)

A transmitting station that compresses and encodes moving images, and
A plurality of receiving stations that receive the moving image data compressed and encoded by the transmitting station and distributed over the network;
The transmitting station is
Interframe coding which is generated by the inter-frame coding using the intra-frame coded data generated by intra-frame coding using internal information of one frame, the difference information between different frames of the moving image Encoding means for compressing and encoding the moving image while mixing data along the time axis in units of frames;
Data transmitting means for transmitting the intra-frame encoded data and the inter-frame encoded data compressed and encoded by the encoding means to the network along a time axis;
Wherein said encoding means of the transmitting station generates one of said intraframe-coded data each time there is a request for new delivery from the receiving station, the data transmission means and said intra-coded data equivalent 該Fu frame and delivered to the receiving station through the network,
The encoding means of the transmitting station designates a key frame at a certain period from a plurality of frames of the moving image,
The encoding means of the transmitting station divides a frame into a plurality of block areas in each of a plurality of other frames other than the key frame, and specifies a specific area from the plurality of block areas,
The positions of the specific areas of each of the other frames included in the specified period of the key frame are different from each other,
The encoding means of the transmitting station includes a second frame between a reference frame for one frame generated by combining specific regions of each of the plurality of other frames included in the specified period, and a key frame. When the difference is larger than a predetermined value, intra-frame encoding is performed on the key frame. On the other hand, when the second difference is smaller than a predetermined value, inter-frame encoding with reference to the reference frame is performed on the key frame. A video distribution system characterized by this.   The moving image distribution system according to claim 1,
  The encoding means of the transmitting station applies intra-frame encoding to the block area when the first difference between the block area excluding the specific area and the key frame is larger than a predetermined value, When the difference between the two is smaller than a predetermined value, the moving picture distribution system is characterized by performing inter-frame coding with reference to the key frame for the block area and intra-frame coding for the specific area. . The moving image distribution system according to claim 1 or 2, wherein
The transmitting station has an imaging camera for imaging a predetermined imaging target;
The moving image distribution system, wherein the receiving station displays or records a moving image captured by the imaging camera.   A moving image distribution method for compressing and encoding a moving image at a transmitting station and distributing the compression-encoded moving image to a plurality of receiving stations through a network,
  a) In the transmitting station, generated by intra-frame coding using intra-frame coding data generated by intra-frame coding using information in one frame of the moving image and difference information between different frames. A step of compressing and encoding the moving image while mixing the encoded data between frames along the time axis in units of frames;
  b) In the transmitting station, sending the intra-frame encoded data and the inter-frame encoded data to the network along a time axis;
With
  The step a)
    a-1) generating the intra-frame encoded data every time there is a request for new distribution from any of the receiving stations;
    a-2) A key frame is designated at regular intervals from a plurality of frames of the moving image, and the frame is divided into a plurality of block areas in each of a plurality of other frames other than the key frame, Designating a specific area from the plurality of block areas;
Have
  The positions of the specific regions of each of the other frames included in the specified period of the key frame are different from each other.
  In the step a), a second difference between a reference frame for one frame generated by combining specific areas of each of the plurality of other frames included in the specified period and a key frame is predetermined. If it is larger, intra-frame coding is performed on the key frame, and if the second difference is smaller than a predetermined value, inter-frame coding with reference to the reference frame is performed on the key frame. Video distribution method.   The program for making a computer perform the said each process in order to implement | achieve the moving image delivery method of Claim 4.   A transmitting station for compressing and distributing moving images, and
  A receiving station that receives the video data compressed and encoded by the transmitting station and distributed over the network;
With
  The transmitting station is
  Intra-frame encoded data generated by intra-frame encoding using information in one frame of the moving image and inter-frame encoded data generated by inter-frame encoding using difference information between different frames Encoding means for compressing and encoding the moving image while mixing along the time axis in units of frames,
  Data transmission means for transmitting the intra-frame encoded data and the inter-frame encoded data compressed and encoded by the encoding means to the network along a time axis;
With
  The encoding means of the transmitting station designates a key frame at a certain period from a plurality of frames of the moving image,
  The encoding means of the transmitting station divides the frame into a plurality of block areas in each of a plurality of other frames other than the key frame, and designates a specific area from the plurality of block areas,
  The positions of the specific regions of each of the other frames included in the specified period of the key frame are different from each other.
  The encoding means of the transmitting station includes a second frame between a reference frame for one frame generated by combining specific regions of each of the plurality of other frames included in the specified period, and a key frame. When the difference is larger than a predetermined value, intra-frame encoding is performed on the key frame. On the other hand, when the second difference is smaller than a predetermined value, inter-frame encoding with reference to the reference frame is performed on the key frame. A video distribution system characterized by this.

Images