KR101008753B1 - Multimedia data streaming system - Google Patents

Abstract

PURPOSE: A multimedia data streaming system is provided to immediately watch a broadcast of HD(High Definition) quality without buffering time, thereby contributing to a domestic IPTV broadcast service. CONSTITUTION: An encoder configures an RTP head of an RTP(Real-time Transport Protocol) packet by a standard fixed head and a standard extension head. An instant on head is placed in front of the standard extension head. The instant on head includes an instant identifier, an instant size, and meta data. The instant size is the size of the meta data. The meta data is compression data of main digital data.

Description

Multimedia Data Streaming System {Multimedia Data Streaming System}

The present invention relates to a file compression, transmission and playback method for transmitting a large amount of multimedia content to the Internet Protocol (IP). In the current technical field, MPEG-4 (H.264) compression technology can digitally encode video, voice, and text to transmit HD-quality multimedia to IP, and transmit data to IP through TCP and UDP. There is an improved Real-time Transport Protocol (RTP), and a Session Description Protocol (SDP) for accessing and managing sessions, and an RTSP for real-time streaming and VCR functions that maintain the data reliability of RTP at the TCP level. Real Time Streaming Protocol (RTC), Real Time Control Protocol (RTCP) for control and monitoring of data jams, was developed and standardized by the Internet Engineering Task Force (IETF). QoE (Quality of Experience) is achieved by streaming HD quality multimedia through IP in real time, and Q through VCR function, monitoring function, and real-time control function. In order to achieve the goal of quality of service (OS), it means that RTP, SDP, RTSP, and RTCP technical specifications are the latest technologies.

Among the current IPTV international standards, the relevant items of the present invention are H.264 and MPEG-4 Series, which are adopted as multimedia compression codecs, and the detailed specifications thereof are ISO / IEC 14496-1 MPEG-4 Part1-Systems / 14496-. 2 MPEG-4 Part-2-Visual / 14496-3 MPEG-4 Part3-Audio / 14496-10 MPEG-4 Part10-AVC (H.264) / 14496-12 MPEG-4 Part12-ISO Media File Format / 14496- 14 MPEG-4 Part14-MPEG-4 File Format.

 There are RTP, SDP, RTSP, and RTCP adopted as Internet Protocol, and in the detailed specification, RTP is IETF RFC Document No. 3550/3551/5506/2862/3016/3095/3243/3409/3759/4362/4815/5225 / 3558/4788/5188/3640/3711/5506/3984/4867/4103/4351/4170/4348/4424/4352/4383/5219

SDP is IETF RFC Document No. 3388/3407/3556/3605/4566/4568/4570/4572/4574/4583/4796/5159/5432/5547

RTSP is IETF RFC Document No. 2326/4567.

RTCP is IETF RFC Document No. 3550/3611/3711/4571/4585/4586/4961/5093/5124/5506.

However, even with this technology, in providing IPTV broadcasting in public networks (ADSL, VDSL), the existing TV broadcasting is required because it requires a buffering latency of several seconds to several tens of seconds before the user selects and watches digital multimedia contents or channels. Unlike watching TV, users cannot watch IPTV broadcasting at the same time as moving and selecting a real-time channel. Therefore, users are inconvenient when watching IPTV broadcasting.

With the advent of the IPTV era globally, ITU-T, an international standardization organization, has the IPTV International Technology Standardization Organization under its umbrella to secure interoperability among countries and provide better services. We received technical proposals for various fields and adopted high-efficiency technologies as IPTV International Technical Standards. In 2009, video (H.264) / Audio (MPEG-4 AAC) on the compression of digital multimedia IPTV International Technical Standards for Codec and IP Transport Protocol (RTP / SDP / RTSP / RTCP) are released. This proves that the IPTV industry will continue to grow for a very long time, and the market size is estimated at 65 trillion won per year.

 In Western Europe and North America, IPTV broadcasting services are provided based on ADSL / VDSL, a public Internet network (also called public network) that we commonly use by introducing products that meet IPTV international technical standards. In France, 2009 As of November, about 15 million subscribers are receiving services.

 In Korea, KT's Cook TV and LGT's MyLG TV were launched in 2006, starting with Hana TV (now: SKT's Broad & TV), a pre-IPTV service. Although the era of full-fledged IPTV has been opened, the IPTV broadcasting platforms of the above-mentioned three IPTV companies are composed of products that do not meet the IPTV international technical standards, and thus are faced with a situation in which real-time broadcasting is not transmitted. It is trying to provide a full-fledged IPTV service by newly installing a premium network with cores with functions.As of December 2009, a premium internet network was established in parts of Seoul to provide IPTV broadcasting services including real-time broadcasting services. However, the province is still receiving Pre IPTV broadcasting service, which does not provide such services. All.

 Therefore, in the IPTV industry, which will continue to grow and explode for the next 10 years, Korea, which is an advanced IT country, is falling behind and emerged as a serious concern. The necessity of 'Open IPTV Broadcasting Service' is being raised, and the three terrestrial broadcasting companies are already considering providing such services. Is developing an open IPTV broadcasting transmission center to support the industry. This is a very important issue that will contribute to the balanced development of the country as an alternative to grow together 100 domestic cable broadcasters, which are in danger of being destroyed due to IPTV broadcasting service, in the age of convergence.

 In this domestic and foreign market environment, only three companies worldwide including the company have products that meet the IPTV international standard in the field of digital multimedia compression and IP transmission, and products that meet the IPTV international technical standard. Even if the contents are selected and the channel is switched, the waiting time for the buffering and the playback time is needed for 5 seconds or longer, and thus there is an inconvenience in not being able to watch the broadcast at the same time as the channel switching as in the conventional analog TV. .

The current IPTV broadcasting service does not provide service satisfaction to viewers due to the above-mentioned problems. Representatively, in general TV, the channel can be viewed immediately at the same time as the channel is selected, but the current IPTV broadcasting service can not be immediately viewed at the same time as the user selects the channel or content.

 This phenomenon is caused by the network kernel response → network library response → network application response → library kernel response to run the streaming engine. In response to streaming library response → streaming application response, response delay of more than 1 second (1,000mm) occurs.

In the next step, the existing ISO / IEC file format is a method of sending IP packets after payloading 13MB to 30MB of data, which causes transmission delay time of 3 to 5 seconds.

The next step is to transfer 14,900bytes (15KB) per second when transferring large amounts of data, and 43,690bytes per second (43KB) per second for the RTP method, so 192KB (SD) to 1,152KB (1080P HD) data within 1 second. Since it cannot transmit, the waiting time for buffering is 13 seconds to 77 seconds for the HTTP method and 5 seconds to 27 seconds for the RTP method.

Even in the playback of multimedia files, the conventional method receives and decrypts 13MB to 30MB of multimedia data, loads and plays necessary resources, and thus additional delay time for receiving data and delay time for decryption occur. A buffering wait time and a refresh wait time were required.

Therefore, in order to immediately watch a user without a waiting time for buffering in an IPTV broadcasting service, a corresponding transmission system must respond immediately, and a time for paying the multimedia data by the transmission system must be shortened. It is an object of the present invention to shorten the time for decrypting multimedia data and to provide a protocol and method for transmitting a large amount of data.

On the other hand, in IPTV on-demand broadcasting service, users must search sequentially through double speed playback until they find the scene they want, so they can't do real-time search (Jog-shuttle) like the home DVD player on the market. Inconvenience to wait as much time as it takes, and network bandwidth also needs to be relatively higher than the ratio of double speed playback, so the Internet used by individuals is inconvenient that cannot escape the speed limit. Another method other than double speed playback is to predict the timeline of the scene to find and move the position bar to search. However, it is difficult to accurately predict the timeline. Even if the prediction is accurate, the time that the server responds by requesting the video from the client to the server At least 5 seconds of waiting time, such as the wait time for the buffering and the waiting time for the playback is suffering from the inconvenience.

Currently, the IPTV on-demand broadcasting service cannot provide a real-time search (Jog-shuttle) service of the front and rear video even in the public network-based IPTV as well as the public network-based IPTV due to the above-mentioned problems. Representatively, the home DVD player that has already been distributed can search the front and back video in real time through the jog-shuttle function, but in the current public and premium network based IPTV broadcasting service, the user can search for the desired scene. After the video can not be searched (Jog-shuttle) in real time.

This phenomenon means that the existing double speed playback technology transmits all the data of the video on the hard disk of the streaming transmission system through the network, or finds the data requested by the client in the sector of the hard disk and pays it again. This is because there is a time required for transmission by loading, and a buffering time until the client receives the corresponding data and a waiting time for playback. Another object of the present invention is to solve the above problems and to enable real-time search (Jog-shuttle) of the front and rear video even in the IPTV on-demand broadcasting service.

The present invention is 45mm Sec. In the Linux operating system based on the monolithic kernel, to make it possible to respond within a short time, the application program's transmission program is optimized to the kernel layer so that the transmission system can respond immediately, and the time when the transmission system pays multimedia files. When compressing a multimedia file to reduce the speed of the file, format it as a file in the form of an Instant-on track. By payloading only 3,072 KB (3,145,728 bytes) of data within a short time, the response delay time and transmission delay time of the transmission system can be shortened, and a large capacity of 1,152 kb / 805 mm sec. Is transmitted, and a multimedia player (device for IPTV broadcasting) Receives and decrypts SDP standard information file, which is 1KB size, generated when payloading data in the transmission system, which is not the conventional method of receiving 13MB ~ 30MB multimedia data and preparing for decryption. It can be played back within 1,000mm sec.

To this end, the present invention is an encoder for converting multimedia data into main digital data and generating an RTP packet using the main digital data, receiving the RTP packet from the encoder, and interpreting the head information of the RTP packet. In the multimedia data streaming system comprising a streaming server having a module, a client for receiving and restoring and playing the RTP packet from the streaming server, wherein the encoder comprises the RTP head of the RTP packet as a standard fixed head and a standard expansion head And an instant-on head including an identifier (ID), a length, and metadata in front of the standard expansion head, wherein the length is a size of the metadata, and the metadata compresses the main digital data. Multimedia data characterized in that the data Provide a reaming system.

Preferably, the meta data is the main digital data of the field immediately following the name and length in the standard extension head.

Preferably, the identifier ID, length, and metadata of the instant-on head are composed of one field.

The encoding system may generate an identifier (ID) corresponding to the name of the standard extension head with an 8-bit size.

The first 4 bits of the identifier (ID) are preferably track IDs which are determined according to video, audio or text data types.

The streaming server preferably generates an SDP file to which the track ID is assigned and transmits it to a separate port according to the track ID.

The streaming system preferably embeds the streaming engine in the network kernel of the Linux operating system in order to immediately respond to the streaming system at the request of a client.

Preferably, the client receives the SDP file, acquires information of the streaming media transmitted from the streaming system, and prepares for the playback of the streaming media.

The streaming server may store an I-frame address of the multimedia data stored in a hard disk in an SDP file and transmit the same to a client.

The instant on head is repeatedly provided between the standard expansion head,

When there is a movement of the position bar on the client, the streaming server pays the I-frame of the instant-on head corresponding to the position of the position bar on the hard disk using the address of the I-frame stored in the SDP file. It is desirable to send to the client.

 According to the present invention, in watching an IPTV broadcast in the current public Internet network, HD quality broadcasts can be viewed immediately after the user selects content and channels without waiting for buffering, thereby contributing to domestic IPTV broadcasting service. Of course, it is possible to provide a comfortable IPTV viewing environment even if Premium network cannot be established.

On the other hand, IPTV viewers have experienced analog TV and DVD player and demand Jog-shuttle function to search the front and rear video in real time like the existing DVD player in IPTV broadcasting. It is not supported, causing inconvenience to users, and it is limited in providing various services because it cannot implement all the features of IPTV service only due to technical limitations. In this situation, the present invention can not only meet the needs of users, but also contribute to the activation of domestic IPTV broadcasting by overcoming the above technical limitations, and has the effect of enabling exclusive export to the IPTV broadcasting platform field abroad.

1 is a block diagram illustrating a video encoder;
2 is a block diagram illustrating an H.264 encoder;
3 is a block diagram illustrating an H.264 decoder;
4 is an explanatory diagram showing an H.264 Baseline, Main, and Extended profile;
5 is a conceptual diagram illustrating a 4: 2: 0 field configuration;
6 is a flowchart of NAL units;
7 is an explanatory diagram showing an RTSP operation process;
8 is a block diagram showing a TCP / UDP / RTP header format;
9 is an explanatory diagram showing an SDP session description method;
10 is a block diagram showing a digital multimedia real-time compression, streaming, and playback system;
11 is an explanatory diagram showing a method of implementing a real-time streaming system;
12 is a structural diagram showing a real time streaming server;
13 is a flowchart showing Start-Up / ShutDown of the RTSP server;
14 is a flowchart showing a processing procedure of an RTSP server;
15 is a flowchart showing an RTSP processor processing procedure;
16 illustrates an RTSP filter roll.
17 shows an example of an RTSP Route role;
18 is an exemplary diagram illustrating an RTSP Preprocessor role;
19 shows an example of an RTSP Request role;
20 is an exemplary diagram illustrating an RTSP Postprocessor role;
21 is an exemplary diagram illustrating an RTSP Send Packet role;
22 is an exemplary diagram illustrating an RTSP Processor role;
23 is an exemplary diagram illustrating an SDP Preocessor role;
24 is an explanatory diagram showing an RTP extension header;
25 is an explanatory diagram showing RTP standard format data;
Fig. 26 is an explanatory diagram showing a name field value;
27 is an explanatory diagram showing an instant-on extension header;
28 is an explanatory diagram showing an instant-on extension header and a standard extension header;
29 illustrates an instant on Hinted role;
30 is a block diagram showing a configuration of a multimedia data streaming system according to the present invention;
31 is an explanatory diagram showing an embodiment using an instant-on extension header according to the present invention;
32 shows an SDP file according to the present invention.

The terms used in the present invention have been selected as widely used general terms as possible in consideration of functions in the present invention, but may vary according to the intention or custom of the person skilled in the art or the emergence of new technology. In addition, in certain cases, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the corresponding description of the invention. Therefore, it is intended that the terms used in the present invention should be arranged based on the meanings of the terms and the general contents of the present invention, rather than the names of the simple terms.

Among the terms used in the present invention, IP (Internet Protocol) refers to a protocol used when transferring data to a network. In general, when a file is downloaded, FTP (File Transfer Protocol) is used, and when a web site is surfed Use Hyper Text Transfer Protocol (HTTP).

Among the terms used in the present invention, Digital Multimedia Contents is usually stored in a film or the like, unlike analog multimedia transmitted over a cable or over the air due to a difference in frequency, it is usually stored in a hard disk (CD, DVD, USB Memory) and transmitted as data. It includes films, animations, graphics, images, etc., in which audio, video, text, and graphics are synthesized.

Among the terms used in the present invention, Encoder is a device that receives an analog signal and compresses and stores it in digital multimedia, and Encoding is an operation of compressing analog multimedia into digital multimedia. Decoding refers to the task of playing digital multimedia. Players, set-top boxes, etc. are used. Trans-Coding refers to the act of compressing or converting digital multimedia into a separate codec or into a separate format.

Among the terms used in the present invention, Streaming is a method of continuously receiving water as it flows so that only the first screen of an image is downloaded and immediately viewed in real time, and a broadcaster receives an analog signal and compresses it in digital multimedia in real time. This equipment transmits IP packets. The streaming server is a device that transmits the streaming data received from the broadcaster to the accessor in real time and streams the digital multimedia file received from the encoder in real time according to the user's request.

Among the terms used in the present invention, IPTV is a broadcasting format that transmits digital multimedia through an IP network and enables the user to watch digital multimedia using a set-top box and a TV receiver.

In calculating the data transmission amount for instant viewing in the present invention, SD (Standard Definition) class has the same image quality as that of analog TV and has a size of 740 × 480 or 640 × 480 when the unit of pixels is 24fps (per second). Number of frames) or 30 fps. 1fps is 6,554bytes / 33.33mm at 1.5Mbps, 192KB / sec, and 30fps to ensure the quality and quality of SD video when digital multimedia video is compressed using H.264 Video Codec and MPEG-4 AAC Audio Codec. has a data rate of sec.

D1 level is the picture quality of DVD and has the size of 720 × 480 and is composed of 24fps or 30fps. 1fps is 8,738bytes / 33.33mm sec at 2Mbps, 256KB / sec, and 30fps to ensure the quality and quality of D1 level video when digital multimedia video is compressed using H.264 Video Codec and MPEG-4 AAC Audio Codec. Has a data rate of.

480P is the quality of ED-TV, which is 848 × 480 and consists of 24fps or 30fps. When digital multimedia video is compressed using H.264 Video Codec and MPEG-4 AAC Audio Codec, 1fps is 13,107byte / 33.33mm sec at 3Mbps, 384KB / sec, and 30fps to ensure the quality and quality of 480P video. Has a data rate of.

720P is HD (High Definition) quality and has a size of 1280 × 720 and is composed of 24fps, 30fps, and 60fps. To ensure the quality and quality of 720P video when digital multimedia video is compressed using H.264 Video Codec and MPEG-4 AAC Audio Codec, 1fps is 26,214byte / 33.33mm at 6Mbps, 1,152KB / sec, and 30fps. has a data rate of sec.

1080P is HD quality and has 1920 × 1080 size and consists of 24fps or 30fps. When digital multimedia video is compressed using H.264 Video Codec and MPEG-4 AAC Audio Codec, to ensure the quality and quality of 1080P video, 1fps is 39,322byte / 33.33mm at 9Mbps, 1,152KB / sec, and 30fps. has a data rate of sec.

In the transport capacity of the present invention, RTP (Real-time Transport Protocol) data can be expanded to a maximum of 1,466 bytes, the transport capacity of the RTP is 1,466byte / 1mm sec, 48,867 bytes / 33.33mm sec, so 30 frames and 60 frames One scene of 1080P multimedia can be transmitted in real time.

Among general matters for a codec of multimedia compression implemented in the present invention, compression refers to a process of making or compressing data into fewer bits. 'Raw' or uncompressed digital video typically requires a large bit rate (about 216 Mbits required for one second of uncompressed TV-quality video). Compression is required.

Compression requires a compressor (encoder) and a decompressor (decoder), which are mutually opposing systems. The encoder converts the original data into compressed form (using a reduced number of bits) before storing and transmitting, and the decoder converts the compressed form data into the original video data. This pair of encoders / decoders is called a codec.

 Network bitrates continue to increase, and high-speed network connections at home have become commonplace, and the storage capacity of hard disks, flash memory, and optical media is very large. Prices continue to drop per bit transmitted or stored, and video compression allows the use of digital video in transmission and storage environments that do not support uncompressed 'raw' video. Do it. For example, current Internet throughput is not sufficient to deal with uncompressed video in real time.

 Digital Versatile Disk (DVD) can only store a few seconds of original video at TV quality resolution and frame rate, making DVD and video storage impractical without compressing video and audio. Compression makes efficient use of transmission media and storage media and plays an important role as a key element of multimedia even if storage capacity and transmission capacity are improved.

 The purpose of the compression algorithm is to compress by minimizing distortion caused by the compression process to obtain efficient compression and to remove redundancy from the signals of the transmitted signals in the temporal, spatial and frequency domains. For example, even if a low-pass filter is applied to the background area on the same frame to remove information of high frequency components, the human eye and the brain (Human Visual System) are more sensitive to low frequencies. There is no problem to find out.

 The video codec codes the original image or video image in compressed form and restores it again to produce a copy of the original image or an image close to the original. If the reconstructed video image is identical to the original, a lossless coding method is used. If the reconstructed video is different from the original, a lossy coding method is used.

 The codec uses a model to represent the original video image, which means that the model can be used to reconstruct the video data by effectively encoding and representing the original video. Ideally, the model should represent the image as faithfully as possible to the original with as few bits as possible. The lower the compressed bit rate, the lower the image quality of the image reconstructed by the decoder, so these two goals (compression efficiency and image quality) generally conflict. As shown in FIG. 1, a video encoder is composed of three main parts: a temporal model, a spatial model, and an entropy encoder.

The input of the temporal model is an uncompressed video image. The temporal model generally uses a method of predicting the current video frame to remove temporal overlap using similarity between adjacent video frames.

MPEG-4 Visual and H.264 improve prediction by performing predictions from one or more previous or future frames and compensating for differences between frames. The output of the temporal model is an error frame and a set of model parameters such as a motion vector that tells how the motion was compensated.

 The error frame allows input of a spatial model, which uses the similarity between adjacent samples in the error frame to eliminate spatial redundancy. MPEG-4 Visual H.264 eliminates spatial redundancy by performing transforms on error samples and quantizing the results.

 During the transformation, the sample is transformed into another domain and represented as transform cofficient. Coefficients are quantized so that non-significant values are removed and only a few significant coefficient values remain. In this way, the error frame can be represented more simply. The output of the spatial model is a series of quantized transform coefficient values.

 The parameters of the temporal model (generally motion vectors) and the parameters of the spatial model (coefficients) are compressed by an entropy encoder. The entropy encoder produces a statistical or compressed file present in the data. Compressed data consists of coded motion vector parameters, coded error coefficients, and header information. The video decoder recovers video frames from the compressed bit stream.

The coefficients and motion vectors are reconstructed by the entropy decoder and the spatial model is decoded to reconstruct the error frame. The decoder uses the motion vector parameter and one or more reconstructed previous frames to produce the predictive frame of the current frame. The current frame is reconstructed by adding an error frame to the prediction frame thus made.

  When defining the syntax of the H.264 CODEC bitstream used in the present invention and a method of decoding such a bitstream, the encoder and decoder generally include the functional elements shown in Figs.

Except for the deblocking filter, most of the functional elements (prediction, transform, quantization, entropy coding) are covered by previous standards (MPEG-1, MPEG-2, MPEG-4, H). .261, H.263), but significant changes in H.264 occur in the details of each functional block. The encoder of FIG. 2 includes two data flow paths, a 'forward' path (left to right) and a 'restore' path (right to left). The input frame or field Fn is processed in macro block units. Each macro block is encoded in intra mode or inter mode, where the predictive block PRED ('P' in FIG. 2) for each block in the macro block is formed by the reconstructed picture sample and in intra mode the PRED is present in the current slice. Generated from previously encoded, decoded and reconstructed samples.

In the intermode PRED is generated by predicting motion compensation from one or two reference frames selected from List 0 and / or List 1 reference frames. In the figure, the reference picture is shown as the previously encoded picture F´n- 예측, but the predictive reference frame for each macroblock partition (in inter mode) has already been encoded and reconstructed and filtered (displayed order) past or future. It can be selected from the picture.

The predictive block PRED comes out of the current block to produce an error block Dn, the error (difference) block is transformed (transformed using a block transform), quantized and represented by X in the picture. Entropy-encoded coefficients form a compressed bitstream along with additional information (prediction mode, quantization, parameters, motion vector information, etc.) needed to decode each block in the macroblock and transmitted via the Network Abstraction Layer (NAL). Or stored.

¹)되고 역변환(T

¹)되어 오차 블록 D´n을 생성한다. The data flow path of the decoder of FIG. 3 shows the similarity between the encoder and the decoder. The encoder (restore path) not only encodes and transmits each block in the macroblock but also decodes (restores) this information again to generate reference data for later prediction. Coefficient X is inverse quantization (Q

¹) and inverse transform (T

¹) to generate the error block D'n.

The predictive block PRED is added to D'n to generate a reconstructed block uF'n (the version from which the original block was decoded: u means no filter). A filter is applied to reduce block distortion, and a reconstructed reference frame is generated from block F'n.

The decoder of FIG. 3 receives the compressed bitstream from the NAL and performs entropy decoding on the data elements to produce quantized coefficients X. The generated coefficients are inversely quantized and inversely transformed to produce D'n (same as D'n indicated on the encoder). The decoder uses the header information decoded from the bitstream to generate the same prediction block PRED as the original prediction block PRED generated at the encoder. PRED is added to D'n to generate uF'n, and uF'n passes through a filter to produce each decoded block F'n.

In the structure of H.264 used in the present invention, three profiles that support a specific function are defined, and the requirements of the encoder and decoder are defined to be compatible with the profile. Baseline profiles are intra coding and inter coding using I-slices containing only I macro blocks and P (predicted) slices containing P macro blocks and / or I macro blocks, and context-adaptive variable length codes. It supports entropy coding using (context-adaptive variable-length codes, CAVLC).

The Main profile consists of intercoding with Bi-predictive (B) slices, including interlaced video, B macroblocks and / or Imacroblocks, intercoding using weighted predictors, and kentext based arithmetic coding. Supports entropy coding using context-based arithmetic coding (CABAC).

The Extended Profile does not support interlaced video or CABAC, but coded bitstreams, or SP-slices (Switching P with P and / or I macro blocks to facilitate switching between coded bitstreams) and SI-slices. A mode is added to enable efficient switching and improved error recovery (Data Partitioning) between (Switching I, a special form of intra-coded macroblocks that facilitate switching between coded bitstreams).

For each application, the Baseline profile is capable of video telephony, video conferencing, and wireless communication. The main profile is for TV broadcast and video storage. The extended profile is useful for streaming media applications.

4 shows the relationship between the three profiles and the coding tools. The Baseline profile belongs to the Extended profile but does not belong to the Main profile. The codec's performance limits are defined by levels, with each level having parameter limits such as sample processing rate, picture size, compression bit rate, and memory requirements.

The H.264 video format supports the encoding and decoding of 4: 2: 0 progressive or interlaced video. The default sampling format for 4: 2: 0 progressive scan frames is shown in Figure 8, and the color difference (Cb and Cr) samples are located horizontally every second luminance sample and vertically located between two luminance samples. The interlaced frame consists of two fields (upper field and lower field) separated in time and the basic sampling format is shown in FIG.

The coded data format of H.264 makes a difference between the Video Coding Layer (VCL) and the Network Abstraction Layer (NAL). The output of the encoding process is VCL data (contiguous bits representing coded video data) and is mapped to NAL units before transmission or storage.

As illustrated in FIG. 6, each NAL unit includes RAW Byte Sequence Payload (RBSP), which is data corresponding to compressed video data or header information. The compressed video image is expressed in consecutive NAL units, which may be transmitted through a packet-based network or a bitstream transmission link or stored in a file.

The purpose of defining the VCL and NAL separately is to distinguish between the compression characteristics of the VCL and the transmission characteristics of the NAL. The encoder of H.264 may use one or two of several previously encoded pictures as reference frames for motion compensation prediction of an inter coded macroblock or macroblock partition. This allows the encoder to search the closest portion to the current macro block partition, not just from the previous encoded picture, but from more pictures.

The encoder and decoder each have one or two previously encoded and decoded reference pictures (pictures that appear before and / or after the current picture in the order in which they are displayed).

A video picture consists of one or more slices, each slice containing an integer number of macroblocks with values between 1 (one macroblock per slice) to the number of all macroblocks in the picture (one slice per picture). do.

The number of macroblocks per slice need not be constant within one picture. There is minimal inter-dependency between coded slices, which helps limit the propagation of errors.

There are five kinds of coded slices mentioned above, and a coded picture may be made of different kinds of slices. For example, a picture coded with a baseline profile may include a mixture of I slices and P slices, and a main profile or extended profile picture may contain a mixture of I slices, P slices, and B slices.

Syntax elements

Explanation
mb_type

Determines whether a macro block is coded in intra mode or in inter (P or B) mode. Determine the size of macroblock partitions.

mb_pred

Determine intra prediction mode
sub_mb_pred

Determine the submacroblock partition size for each submacroblock.

coded_block_pattern

Indicates which 8x8 block (luminance and chrominance) contains the coded transform coefficients.

mb_qp_delta

Change the quantization parameter.
residual

Coded transform coefficients corresponding to the error image samples after prediction.

The macro block includes coded data corresponding to 16 × 16 sample areas (16 × 16 luminance samples, 8 × 8 Cb, and 8 × 8 Cr samples) of a video frame and syntax elements as shown in Table 1 below. Macroblocks are numbered in sequential scan order within a frame.

In the present invention, an H.264 video image encoded for real time streaming transmission of H.264 is composed of a series of NAL units, and each NAL unit includes RBSP Table 2. All other elements are just NAL units, while coded slices (including data partition slices and Instantaneous Decoder Refresh slices) and the End of Sequence RBSP are defined as VLC NAL units. Each unit of a typical RBSP is transmitted in an independent NAL unit.

RBSP TYPE

Explanation
Parameter Set

'Global' parameters for the picture, such as picture resolution, video format, and macroblock placement map.
Supplemental
Enhancement
information


Additional messages that are not essential to the proper decoding of video footage.
Picture Delimiter

Boundaries between video pictures. If Picture Delimiter is not used, decode delimits based on the frame number contained within each slice header.

Coded slice

Header and data for the slice.
(This RBSP unit contains the actual coded video data.)


Data Partition
A, B or C

Three units containing data partitioned slice layer data (useful for error recovery)
Partition A: Header data for all MBs in the slice.
Partition B: Intra coded data.
Partition C: Intercoded data.

End of sequence

Indicates that the next picture (in decoding order) is an IDR picture.
(Not necessary information for decoding)

End of stream

Indicates that there are no more pictures in the bitstream.

Filler data

(Which can be used to increase the number of bytes in the image)
'Dummy' data.

The header of the NAL unit includes information on the type of RBSP, and the rest of the NAL unit is composed of RBSP data. There are several differences between the method of transmitting the NAL unit between packet-based packet networks and the transmission of a continuous data stream.

 In a packet-based network, each NAL unit can be carried in an independent packet and must be configured in the correct order before decoding. In a continuous data stream transmission environment, a start code prefix (uniquely identified delimiter) is placed before each NAL unit and creates a byte stream before transmission. This allows the decoder to search the stream to find the start code prefix that distinguishes the start of the NAL unit.

In general, the applied coding video should be transmitted or stored along with the associated audio track and side information. To do this, you can use the following real-time transport protocols, such as Real Time Protocol (RTP) and User Datagram Protocol (UDP). H.264 provides a video coding method that is optimized for compression efficiency and aims to meet the needs of practical multimedia communication applications. The range of compression tools available is limited to that of MPEG-4 Visual, but compression parameters and various strategies are available. Therefore, H.264's digital video compression technology is the core technology of multimedia applications. It has been adopted as video compression algorithm of mobile phone video compression and VOD service, satellite and terrestrial DMB service and IPTV service. Adopted as the multimedia compression standard of the IPTV international technical standard, the present invention applied H.264 to real-time high-quality video transmission.

In the present invention, it is derived from a technology called streaming multimedia developed by Real Network Co., Ltd. to perform transmission using an IP network of digital multimedia. The principle of the technology is that a large amount of multimedia data can be individually executed. As a stream flows into smaller pieces, the receiver continuously sends and receives data, without having to wait for all the data to be received. That is, the continuous media data of the application layer is cut into short pieces, and the packet is transmitted. On the receiving side, the data is reproduced by continuous decoding while maintaining a certain amount of real-time characteristics whenever a certain unit of data is received. Decoded playback is not.

The term streaming now means broadcasting on the Internet, and streaming is a kind of broadcasting. On the other hand, streaming is divided into On Demand and Live, and technically, it is divided into Unicast and Multicast. The standard protocol used by streaming technology is RTSP (Real Time Streaming Protocol).

In order to view the media image by streaming the first on-demand streaming, a media file prepared in advance, it is necessary to click on the file and wait for the driving signal of the file.

The second way is live streaming. This way, when you click on the relevant place, a player supporting streaming will appear and send video or sound. This method has a big impact on sound quality and image quality up to now. When more data is needed than it is needed, it is temporarily stored in the buffer. When less data is needed than it is needed, the sound quality or image quality is reduced. However, in addition to the transmission speed suggested by the present invention, a real time streaming is performed through a study on a compression scheme or a transmission protocol.

Real Time Streaming Protocol (RTSP) for real-time streaming in the present invention, RTSP (Real Time Streaming Protocol) refers to the protocol of the application layer for real-time media transmission in the On Demand format. RTSP is a protocol that handles various requests related to service request, response or service connection setting, and stream playback in streaming service on the Internet. It is jointly developed by Real Network, Netscape, Corporation, Columbia University, etc. Has been standardized. The protocol for streaming uses RTSP and RTP, and packetizes it to perform streaming service.

The RTSP protocol is an application layer protocol that aims to provide a powerful protocol for streaming multimedia in multi-pointers using both unicast and multicast. It creates and controls temporally synchronized streaming such as audio and video. That is, the continuous medium itself does not transmit and performs a network remote control role for the streaming server.

In the MMS / HTTP protocol, TCP-based media data is packetized, and reliable transmission is performed regardless of the order. On the receiving side, in order to restore and reproduce the media data, the buffering occurs by destroying the time relationship of media packets. When requesting retransmission when a frame is lost, jitter or skew of continuous media data occurs.

According to the basic operation principle of the RTSP protocol, the client requests video or audio information having real-time characteristics from the server, and the server operates by transmitting the information. Here, streaming refers to a server that cuts and transmits a compressed series of messages into packets. On the receiving side, instead of receiving and decoding / reproducing the entire message, the receiving side decodes each time a certain unit of message is received, while maintaining some real-time characteristics. It is a technology that enables continuous playback.

The features of the RTSP protocol enable simultaneous control of multiple media information streams in a Unicast or Multicast environment, and can operate over a variety of transport layer protocols, including TCP and UDP, in combination with the RTP and RTCP protocols.

The RTSP establishes an RTP / RTCP channel using reliable TCP for transmission of control messages and then forwards the RTP / RTCP packets. That is, session setup and release are controlled by RTSP, actual A / V data is transmitted through RTP, and has similar syntax and operation as HTTP, but both server and client can send request and receive response. , Has a playback state. And protocol specification for commands used to set up and release sessions is done through SDP.

As shown in FIG. 7, during the operation of the RTSP, the player of the client makes a connection request through the RTSP using the Session Description Protocol (SDP), and the streaming server's response to the SDP also includes session and information set by the session. Use the information contained in it as it is appropriate for RTSP allocation.

The player's control commands are sent via RTSP, and the response is also via RTSP. You can use this information to send setup requests to the streaming server. If the Play request is accepted, the actual data is sent to the client using the RTP / RTCP protocol. Use the Pause method to temporarily stop playback and send a Teardown message to close the session completely.

In the present invention, during implementation of Real-time Transport Protocol (RTP) compression and transmission for real-time streaming, Real-time Transport Protocol (RTP) was approved as Internet proposal standard by Internet Engineering Streering Group (IESG) in November 1995 and RFCI1889 (Request For Comments 1889) and RFC1890 (RTCP: RTP Profile for Audio and Video Conferences with Minimal Control 1890).

RTP provides end-to-end transport capabilities suitable for applications that transmit real-time data such as voice, video, or mock data over multicast or unicast. However, RTP does not cover resource reservations and does not provide features such as timely data delivery, QoS guarantees, and reversal of transmission. Therefore, the meaning of transport can be said to be a standard established by focusing on the characteristics of real-time data. RTP packets are delivered using UDP. Multiplexing in RTP is provided by the destination transport address, and in conferences using multiple media, each media article is transmitted within RTP sessions with different destination addresses. Therefore, multiple media are transmitted together in one RTP session and then not demultiplexed based on payload type or SSRC field value.

There are four characteristics of the RTP protocol. First, it uses a simple frame that can be used directly by an application with the application layer framing (ALF) structure. Real-time applications do not need to use complex protocols such as TCP because they do not need the services provided by TCP. When the loss of data occurs, the TCP method of retransmitting stably transmits information but generates time delay. In real-time programs, time lag is more serious than loss of data.

ALF is the way most network software is organized, and the advantage is that it performs better than a multi-layered structure. If you use different layers, inefficient methods, such as interprocess communication, in order to allow different protocols to interact with different layers. However, ALF can use efficient methods such as direct function calls. RTP defines the format, operation, and basic roles as frames. An application program sends data in RTP frames along with the information needed for the data, such as encoding headers.

The second feature is that time recording of real time data is possible. TCP / IP networks do not maintain constant time relationships between messages, allowing real-time data to arrive at irregular intervals. You may arrive out of order, or even not.

The receiver's application program should reconstruct the received data at the same interval as the original data creation interval, and reconstruct it using the time stamp of the received data. The timestamp for reconstruction is part of the RTP frame because it is a common part of all real-time applications.

The third feature is multicast and unicast support. Applications such as video conferencing typically have a large number of participants, so RTP is designed to support multicast. Participate in meetings and speak and all participants receive information by multicasting.

In addition to multicasting only RTP packets, feedback also multicasts. Multicasting feedback seems to be a waste of network bandwidth, but it allows you to know the number of participants, so you can see the bandwidth used. Not only is it easy, but it has the advantage of knowing whether the transmission failure is local or global.

Fourth feature is Translators and Mixers function. In real time transmission, the sender and receiver play an important role. RTP adds two important roles to them, allowing systems using RTP to act as translators and mixers. Of course, Translators and Mixers are not necessary in every case.

However, sometimes they are the only way the network can support real time by: Translators and Mixers are in the middle of the sender and receiver, and Translators simply switch to another format. For example, if users of different bandwidths participate in a video conference together, they should be tailored to the side with the lowest bandwidth because of the different bandwidths. However, if the format is changed to the format suitable for each bandwidth by the role of Translator, each user can receive the data of the bandwidth suitable for himself.

Mixer does not change the format, but combines multiple streams into a single stream while maintaining the original format. This method is very efficient for synthesizeable audio data.

8 is a comparison of the TCP / UDP / RTP head format. Among them, in the form of RTP packet, the header has a fixed size and specific information and data are attached to the header according to the multimedia information. V is the version field and the latest version is 2.0. P is used to construct a packet in units of 32 bits. If P is set, it means that padding octets are included at the end of the packet, not the payload part. If the X bit is used as 1, it indicates that exactly one extension header follows the fixed header. CC indicates the number of CSRC identifiers in the fixed header. CRSC is the source of the RTP packet stream that contributed to the RTP mixer's combined stream. In other words, while RTP packets are transmitted through the network, the intermediate system receives RTP packets from various sources, combines them properly, creates a new type of RTP packet, and delivers it to the next system. An intermediate system that performs this function is called an RTP mixer. do.

M is used to indicate a frame area, mainly frame boundary, for multimedia information. That is, it is used to distinguish between voice and image information in the packet. The PT field refers to the RTP Payload form of the profile defined in RFC 1890 and is interpreted by the application. The profile is a payload type code that is specified in the payload format and has a fixed correspondence. That is, when PT is 0, it indicates that the audio information is encoding scheme and has 800Hz clock rate and one audio channel. Currently 33 payload types are defined.

The Sequence Number is incremented by 1 each time an RTP packet is sent. The receiver uses this field to detect packet loss and restore the packet order. The TimeStamp field represents the point in time at which the first octet of the RTP packet was sampled. The sampling point is generated from a constantly increasing clock. This is used to synchronize real-time data and calculate jitter. The initial value is chosen randomly.

The SSRC field indicates an identifier of a data source such as a camera or a microphone. In other words, it allows you to identify the synchronization source. This identifier should be chosen as a random number that has a different value from the sources in the RTP session, but there may be a conflict, so RTP has a way to detect and resolve it. The CSRC field indicates identifiers that can distinguish the sources when the RTP packets are mixed in the intermediate system. It consists of 0-15 CSRCs, each with 32 bits, which identifies the contributing sources for the payload in the packet, and the mixer populates the SSRCs of those sources with the CSRC. The number of identifiers is shown in the CC above.

Audio does not require special parsing when constructing an RTP packet, but simply cuts it to a constant length. Audio only needs to be transmitted unilaterally to the user, so unlike the response to a connection request and a control command, there is no need for a binding process. However, because video streams do not have a constant number of bits per frame, they cannot produce RTP packets of constant length, such as audio. Therefore, carry out the parsing process.

During the implementation of the Session Description Protocol (SDP) for real-time streaming in the present invention, the SDP protocol is a MMUSIC (IETF) of IETF for the purpose of advertising the information required by the user to participate in the multimedia session on the Internet and defining the multimedia session in real time. Multiparty Multi-media Session Control) A protocol standardized to RFC2327 by the Working Group.

The SDP contains information on session creation, session invitation, and session description to define a multimedia session. This information includes the address and port for the media control server, the media type, and the media server address. To describe a session, use the SDP protocol. The SDP protocol is used to describe multimedia sessions and to initiate various types of sessions.

Built on the Internet using IP multicast, MBone is widely used for multi-media multimedia applications. However, if you want to join a session that is currently open on MBone, you need to know the contents of the session or the multicast address and port number each session uses.

The SDP protocol is a protocol used to describe multimedia sessions. It is mainly used to describe the payload part when announcing a session by periodically sending packets to a predetermined multicast address and port. The purpose of the use of SDP is to convey information about the media stream of a multimedia session to beneficiaries who wish to participate in the session. SDP defines this multimedia session as a set of media streams that exist for a period of time. It has the ability to advertise the existence of a session and convey enough information to participate in the session.

In the Internet environment, SDP provides two important functions. One is to announce the existence of a session, and the other is to deliver enough information to join the session. To this end, the SDP contains the following information: Session name and purpose, the time the session is established, the media making up the session, media reception information (address, port, format), media type (video, audio), transport protocol (RTP / UDP / IP), media format (MPEG -4, H.264), multicast address for media, transport port for media, and bandwidth information. It also provides enough bandwidth information for the client to join the session and the contact information of the person responsible for the session. The method of describing an SDP session is described in text. Each SDP item is composed of several lines of text as shown below.

<type> = <value>

<type> is written as one character. <value> is a structured text string that depends on <type>. There should be no spaces on either side of the '=' sign. In general, <value> consists of several parts, separated by spaces. SDP sessions consist of a session level and several media levels. The session level part starts with the 'v =' line and the media level part starts with the 'm ='. Some items must be filled in and some are optional. However, the order between the lines must be followed.

Referring to FIG. 9, the session description method may be described in detail. The * marked above is optional, and the parser will ignore <type> characters that it does not understand. The following is an example of describing a session in SDP.

v = 0

o = Administrator 743235897 866589328 IN IP4 203.1.118.230

s = ICAST Demo

i = Demo

u = http: //www.icast.com

e=sales@icast.com

p = + 01 408 8740701

t = 3075578128 3076222528

r = 565200 39600 0

a = recvonly

m = audio 4640 RTP / AVP PCM

c = IN IP4 218.3.3.10/127

a = orient: portait

a = framerate: 30

a = datarate: 407

a = quality: 6

a = grayed: 0

We are using a new line to end the record. Now, the specific information on each item is as follows.

Protocol Version (v = 0): Indicates the version of SDP.

Origin

  o = <user name> <session ID> <version> <network type> <address>

The 'o' part shows the session creator the session ID and session version number. <User name> is the user login on the producing host. Session ID provides a globally unique identifier for the session. <Version> is the version number for this advertisement. This version value increments with every change. <Network type> is a text string describing the network type. The string 'IN' in front means 'internet'. <Address> represents the unique address of the host that created the session.

Session Name

It looks like s = <session name>, which shows the session name.

Information about sessions and media

  I = <session description> Used to enter the purpose and various detailed information about the session.

URI and email address and telephone number

   u = <URI> e = <email address> p = <phone number>

   It is used to indicate additional information about the meeting.

Connection data

c = <network type> <address type> <connection address> / ttl

The 'c' part contains connection data information. The first detail is the network type. 'IN' stands for the Internet. In 'Address Type', it is defined as IPv4. The connection address mainly uses a class-D IP multicast group address. Time To Live (TTL) defines the range to which multicast packets are sent. MBone uses several standards for TTL as follows.

Bandwidth

 b = <modifier>: <bandwidth value>

Modifiers are 'CT (Conference Total) and' AS (Application Specific Maximum). CT refers to the total bandwidth for all media and AS refers to the bandwidth for one media.

Time, repeat count

t = <start time> <end time>

The 't' part describes the start and end times for the conference session. Time uses NTP values. If the end time is zero, the session continues indefinitely. The start time is zero, indicating that the session is persistent.

r = <repeat interval> <live time> <option from start time>

 The 'r' part describes the number of iterations for a session. For example, if a session is live for one hour over three months, starting from 10:00 on Monday to 11:00 on Tuesday, <start time> in the 't' section displays the first Monday's 10 o'clock in NTP format and repeats < Interval> displays the time of the last session of the week, in NTP format.

Examples of the above are as follows.

   t = 3034423619 3042462419

   r = 604800 3600 0 90000

· Media Announcement

m = <media> <port> <transport protocol> <list>

The Media section describes one of audio, video, whiteboard, text, and data. The second part shows the transport port to which the media stream will be sent. The port value depends on the transport protocol. When the protocol is RTP, an even port (3456) is used for data transmission and one port (3457) larger than RTP is used for control information. The third part shows the transport protocol. For example, a media described as 'm = video 3456/2 RTP / AVP 31' is used when the RTP media stream runs under the RTP audio / media profile. For example, if the protocol part is 'RTP / XYZ', it indicates RTP whose profile name is 'XYZ'. The next section is about media formats. For audio and video, this is the media payload type defined in the RTP audio / video profile. As an example of static payload type, u-law PCM sampling at 8KHz has payload type defined as 0 and can be expressed as follows.

m = video 3456 RTP / AVP 0

An example of a dynamic payload type is 16-bit stereo audio sampling at 16 KHz, using payload type 98. If you want to use this payload type, you need the following additional information:

m = video 3456 RTP / AVP 98

a = rtpmap: 98 L16 / 16000/2

The general form of rtpmap is:

a = rtpmap <payload><encoding> / <clock rate> / [<encoding parameter>]

The implementation of the system configuration for real time streaming in the present invention is shown in Figure 10, which is a real-time digital multimedia transmission system configuration Caster is responsible for packetized transmission and file storage by compressing the digital multimedia, codec method MPEG-4 AVC (H.264) Video / AAC Audio CODEC is applied and the transmission method supports both Unicast and Multicast, the operating system supports WINDOWS XP and Linux, and the performance is CPU Pentium 4 2.8GHz Hyper-Threading function and Memory It uses 256MB, Soundcard and Captureboard with Ospray or DRC board, 1CCD for indoor camera and 3CCD for outdoor camera.

Streaming Server (Real Time Streaming Transmission System) is a function to send the streaming packet and file received from the Caster to the player in real time.It performs RTSP authentication, RTP reception, RTSP authentication, SDP connection, RTP transmission and Unicast, Multicast transmission function. The operating system uses Linux, the CPU is Pentium 3 1GHz, and the memory is 512MB. It is equipped with RTP / UDP Unicaster and Multicast Module.

Finally, the real-time player's H / W performance is CPU Pentium 3 1GHz with 128MB of memory, 8MB of video memory and soundcard, RTSP / RTP, SDP / HTTP Tunneling and RTSP authentication.

As shown in FIG. 11, three methods were implemented by dividing a method of configuring a real-time high-quality video transmission system using an RTP protocol according to whether media data is accumulated and a transmission method of the RTSP server. The streaming server (real-time streaming transmission system) can be composed of on-demand broadcasting and live broadcasting according to the file format and packetization format of media data, and divided into multicast and unicast according to the transmission scheme. In order to receive real-time streaming multimedia data from a streaming server, when a client requests media data from the server, the server pays the media data immediately and sends it to the client in real time through RTP / UDP. Apply the transmission method.

Real-time broadcast without transmitting media data and transmitting multi-source (audio and video) video to H.264 directly from Caster equipped with capture card to client without configuring streaming server. This configuration is suitable for the multicast transmission method used. In the unicast transmission method, the streaming server is configured to efficiently use the limited bandwidth, and the encoded multimedia packet is transmitted to the streaming server to be relayed and transmitted back to the client.

In the present invention, the configuration for the implementation of the real-time streaming server (real-time streaming transmission system) can be shown in Figure 12, which is a real-time streaming server structure between the GUI program management unit for managing the server control, management, access tool program between the server and the client It consists of a streaming control unit responsible for session connection, disconnection, and transmission of real-time media data, and a player unit (interpretation) that pays for and recognizes multimedia files.

The transmission port number of the real-time video transmission protocol is that the RTSP is used for port 554 and 7070, and the RTP port is used for TCP port 80 and 8080, whereas 50 ports are used for 6,950 to 6,999 for media data transmission. Since a parallel port can be configured, RTP / UDP / IP based packet transmission rather than TCP / IP based packet transmission can be quite efficient for continuous media data transmission. In the above contents, the digital multimedia compressed into digital multimedia and packetized or filed in the RTP format is responded by the server according to the RTSP request of the client, and the player unit (interpretation) of FIG. 12 pays the multimedia file requested by the client. According to the following procedure, the SDP file is generated by the SDP of the streaming control unit of FIG. 12 and the session is created and transmitted to the client player of the SDP file by RTSP, and then the main data of the digital multimedia is streamed to the client by RTP in real time. .

13 to 15 are flowcharts of a start-up, a shut-down, an authentication sequence of an RTSP protocol, and an RTSP preprocessor as a processing procedure of a real-time streaming server.

In the present invention for the implementation of a real-time string server for real-time streaming Of the Protocol Roles applied to RTSP, RTP, and SDP, the RTSP Filter role is the server calling the RTSP Filter Module for the RTSP request from the client as shown in FIG. 16, and the Filter responds to the processed video data packet by packet, but changes to RTSP. It responds proactively, so it responds immediately to any point in the video.

The RTSP Route role is shown in FIG. 17. After the Server calls all packets in the RTSP Filter Module, the Server calls the RTSP Route role, which serves to call the root directory for each RTSP request.

The RTSP Preprpcessor role is shown in Figure 18. After the server calls all modules registered for the RTSP path role, the server calls the RTSP preprocessor role and sends the appropriate RTSP response to the client.

The RTSP Request role is shown in FIG. 19. If the RTSP preprocessor does not respond, the server calls RTSP again. Only one RTSP is called and RTSP is registered first when Server starts.

The RTSP Postprocessor role is shown in FIG. 20. When the RTSP Request is registered, the Server calls the RTSP Postprocessor in response to the request, which is used to record statistical information.

RTP Send packet role is shown in Figure 21. When a packet calls the player of the client, the server sends the packet through the RTP, its role is transferred to the RTP to send data to the client, and RTP continuously calls the data of the server. Send to client.

The RTSP Processor role is shown in FIG. 22. The Server receives the RTCP signal from the client and waits for a request to send a unit packet (when requesting any point of the video). This allows you to respond to any query from the client, based on frames per second of audio, including data loss.

As for the SDP Processor role, the transmission server address is 218.233.155.XX as shown in FIG. 23, the broadcast control protocol uses the RTSP protocol, the property of the media uses the RTP protocol used for video and audio formats, and the compression technology CODEC is H. The .264 method is applied.

Instant-on for instant viewing in the present invention is based on the real-time streaming that can send all the data of one frame at a time using RTSP / RTP as described above, and the streaming transmission system (Server) to respond immediately This is a technique for payloading and transmitting only small data (3% of original media data) of a header without paying the multimedia main data file. When a client (digital player) requests digital multimedia from the server, the streaming server The streaming control unit of FIG. 12 is added to each component of the Streaming Server by adding the information (media property) of the multimedia data to the head of the multimedia file by transmitting the head data information to the client without paying the multimedia files. Hinted module to interpret the information of the compressed head It was added to only the head page loading data by creating a SDP file multimedia attribute information of a character by the SDP Parsing roll is applied to the technique for directly transmitting the file to the Client to transmit and receive stage from the compression stage of the multimedia.

Implementation of Instant-on RTP Fomat for Instant-on of Instant View in the present invention compresses the head of the RTP Payload Standard configuration (FIGS. 24 and 25) to create the Instant-on RTP payload meta-information (FIG. 27) head. Instant-on Mixed RTP payload meta-information (FIG. 28) data is created. As such, RTP meta information is provided by writing information about the sequential number and the amount of data per time stamp in the header of the video data, so that RTP can payload only the head without streaming all the video data so that the video data can be streamed. . RTP Data needs the following meta information embedded in RTP to provide RTP clients with the following information.

Transmission Time sends the transmission time as an integer of 4 octets representing the transmission time of the RTP packet in milliseconds. The transmission time is always canceled at the beginning of the medium. For example, if the RTSP or SDP response includes a range of 0-729.45 by the request of the URL and the client requests to be played in the range of 100-729.45, the server responds immediately to find the time of the nearest frame. .

During frame type configuration, video is composed of key frame, b-frame, p-frame. All video sources have key frame (including background screen), which is defined as 16bit integer and sent to client.

Packet Number is a simple 64-bit integer sent by a streaming server. For example, if the response from the URL contains a range of 0-729.45, and the client wants to play in the same range, the first packet is zero and increments by one for each successive packet, the first packet being 1000 in 60 seconds. If there is a play request of 60-729.45 between packets of 0, the number of first packets is 1001, and is increased by 1 for each successive packet.

The Packet Position configuration sends the position of the packet as a simple 64-bit integer from the streaming server. If the client plays the range 100-729.45 when the entire video is in the range 0-729.45, the RTP packet of the first video is between 0 and 100. Bytes are the sum of the RTP packets of the video. In order to calculate the location of each packet, the location is calculated by subtracting the sum of packets of the requested location from the total number of bytes of the RTP transmission packet, so that the desired image is immediately viewed.

Media data is the main data of the multimedia, and the server delivers the multimedia by the RTP protocol. The sequence number transmits 2 octets of the RTP sequence number from the streaming server (real time streaming transmission system), and the sequence number is used to express the RTP meta information on the amount of video data loaded.

RTP Standard Format Configuration FIG. 24 and FIG. 25 show that a data field is loaded by RTP meta information in a real-time streaming transmission system, and is composed of information of each packet of a head and data. When this data is sent to the server in a standard format, the first bit of the head part is zero (ie uncompressed), and the first bit is determined by the 15-bit name field. This 15-bit name field must contain two ASCII characters for the list of packets contained in the RTP data. Therefore, the head of the RTP Standard Format occupies 12% of the total multimedia data, and the streaming head system takes one to three seconds of delay time to payload and parse SD to HD multimedia files to create SDP files. By compressing by applying Instant-on roll (Fig. 29), Hinted in the form of Instant-on RTP payload meta-information as shown in Fig. 27, the size of Instant-on RTP Format Head is less than 3% (3,072KB) of the total multimedia data. 45mm Sec. The real-time streaming transmission system responds within a short time, and adds a Hinted module to the streaming server controller of FIG. The SDP file is generated and sent to the client, and the player of the client receives the SDP file and receives 50mm Sec. By decoding the multimedia data within 1,000mm Sec. Up to 1152KB / 805mm Sec. Should be made to play within.

The RTP port constituting the present invention can use 50 ports from 6950 to 6999, and simultaneously transmits video, audio, and text data to the divided ports to enable real-time streaming transmission, thereby eliminating waiting time for playback. Implement viewing immediately.

Meanwhile, in the present invention, the movement of the player position bar executed in the client is transmitted and received to the streaming transmission system by using the Real Time Control Protocol (RTCP) using the port 554, and the streaming transmission system pays the data of the scene. In order to access the I-frame only by directly accessing the address stored in the SDP file without searching the hard disk, only the I-frame data is transmitted to the unused port among the 50 ports from 6,950 to 6,999. This enables real-time search (Jog-Shuttle) without increasing network bandwidth.

<Examples>

A basic configuration of a multimedia data streaming system according to the present invention will be described with reference to FIG. 30 as follows.

The multimedia data streaming system according to the present embodiment includes an analog multimedia data input means 200, an encoder 300, a streaming server 100, and a client 400.

The analog multimedia data input means 200 is a video and audio input means such as a camera, a tape, and the like, and is a device into which video and audio data due to physical phenomena are input.

The video and audio data input by the analog multimedia data input means 200 is transmitted to the encoder 300. The encoder 300 converts analog data into main digital data and executes an RTP encoding process in which an RTP header is attached to the main digital data to generate an RTP packet.

The header of the RTP packet includes a standard fixed head 310, a standard expansion head 330 and the instant-on head 320.

Since the standard fixing head 310 and the standard expansion head 330 have been described above, description thereof will be omitted.

Referring to FIG. 31, the instant on head 320 is provided in front of the standard expansion head 330, and an instant identifier 322, an instant size 324, and metadata 326 are configured in one field.

The instant identifier 322 compresses the Name field of the standard extension head 330. That is, the name field of the standard extension head 330 is composed of 16 bits, whereas the instant identifier 322 is composed of 8 bits.

The encoder 300 generates the instant identifier 322 in random numbers and letters. The instant identifier 322 is generated so that the instant identifier 322 does not overlap with another identifier through a search process.

In this case, the first 4 bits of the instant identifier 322 are determined differently according to the data type, and are identically assigned to the same data type. For example, the first 4 bits for video data, 4326 for audio data, and 4328 for text data may be determined, and the first 4 bits may be determined identically for the same data type.

The instant size 324 represents the size of the metadata 326, and the metadata 326 compresses data of the field immediately following the Name and Length fields in the standard extension head 330. It is. Preferably, the compression is lossless compression that removes and compresses empty memory space.

When the channel is changed or a new menu is selected in the IP by the instant-on header 320, the instant-on header 320 can immediately watch without buffering time. Specifically, when a transmission request is received by the client, the streaming server generates an SDP file based on the contents of the instant identifier 322 and the instant size 324 and transmits the SDP file to the client. At this time, since the metadata 326, which is mainly data about the first scene, is in the same field as the instant identifier and the instant size, the SDP file is transmitted and the metadata 326 is simultaneously transmitted. Thereafter, the original main data 328 which is not compressed into the metadata 326 is transmitted.

That is, in the related art, the buffering takes a long time since all head information up to 30MB is transmitted and then reproduced. However, the instant-on header 320 enables instant viewing without the buffering time. The size of the instant-on header 320 is less than 3% of the total multimedia data, SDP parsing 50mm Sec. Run it within to generate the SDP file and send it to the client. In addition, the client receives the SDP file and 50mm Sec. Decrypt within.

In this embodiment, the RTP port can use 50 ports from 6950 to 6999. In particular, the video, audio, and text data can be classified by type and transmitted to the divided ports at the same time so that there is no waiting time for playback. Implement

Specifically, it was difficult to commercialize the transmission port to a port divided by the increase in congestion between ports in the past, but in the present invention, when the SDP file is generated as shown in FIG. 32, the starting IP 520 and the track ID 1 ( 530 and track ID 2 540 are designated. The track ID consists of the first four bits of the instant identifier 322 in the instant-on header. Therefore, video, audio, and text data are automatically classified by the track ID. In this example, track ID 1 530 means video and track ID 2 540 means audio.

Since the track ID is transmitted to and recognized by the client through the SDP file, congestion does not occur even when video data, audio data and text data are transmitted to the divided ports. In addition, it is preferable that the data use port numbers separated from each other without using adjacent port numbers.

Since the transmission protocol used in the present invention is 1,466byte / 1mm sec. As RTP (Real-time Transport Protocol), it becomes 48,867byte / 33.33mm sec. Transmission is possible.

When the streaming server pays off the multimedia file, the streaming server's En-De Paser API makes the following SDP roll based on the IETE standard.

v = 0

o = mhandley 2890844526 2890842807 IN IP4 126.16.64.4

Seminar to s = S

i = A Seminer on the session description protocol

c = IN IP4 224.2.17.12/127

t = 2873397496 2873404696

m = audio 49170 RTP / AVP 0

b = AS: 64

b = RS: 800

b = RR: 2400

m = video 51372 RTP / AVP 31

b = AS: 256

b = RS: 800

b = RR: 2400

IETE's SDP specification uses option 'a' to allow a user to apply a separate developed technology. By using this, the multimedia file is linked with option 't' having time information of the standard SDP file standard. I-frame and the sectors of the hard disk in which the audio associated with it are stored are encrypted and added as shown in FIG. This addition of the SDP roll is programmed to be done in the server's En-De Paser.

The SDP file stores the I-frame address of the multimedia data stored in the hard disk. The SDP file is transmitted to the client when the multimedia data is first transmitted. Therefore, if there is a movement of the player position bar executed in the client later, only I-frame can be faded by directly accessing the address stored in the SDP file without searching the hard disk to payload the data of the scene. . In this case, the I-frame is payloaded in association with the instant identifier of the instant-on head.

Referring to FIG. 31, the instant on head according to the present embodiment is repeatedly provided between the standard expansion heads. This configuration can be modified to include an instant-on head between two or three standard expansion heads. The address stored in the SDP file is matched with the instant identifier of the instant-on head, so that the user can directly access the corresponding instant-on head as the user moves the position bar.

When there is a movement of the position bar from the client, a signal is transmitted by Real Time Control Protocol (RTCP) using port 554, and the streaming server does not search the hard disk using the address stored in the SDP file. It is accessible and also directly accesses the instant-on head corresponding to the position of the position bar among the repeated instant-on heads by using the address, and pays only the I-frames in the metadata of the instant-on head. .

In this way, since the I-frame is transmitted by the instant-on head by skipping the multimedia data, real-time search (Jog-Shuttle) is possible without increasing the bandwidth.

The present invention adds instant-on technology to products in the field of compression and transmission on the basis of technology implementing the IPTV international technical standard in the field of compression and transmission. As an invention that solves the inconvenience of viewers by supporting viewing as soon as there is no need, the industrial applicability is very large.

100: streaming server 200: analog multimedia data input means
300: encoder 310: standard fixing head
320: Instant-on head 330: Standard expansion head
400: client

Claims (10)

A streaming server which converts multimedia data into main digital data and generates an RTP packet using the main digital data, receives the RTP packet from the encoder, and has a hinted module for interpreting head information of the RTP packet; In the multimedia data streaming system including a client that receives the RTP packet from the streaming server to restore and play,
The encoder consists of a standard fixed head and a standard expansion head of the RTP head of the RTP packet,
In front of the standard expansion head is provided an instant-on head consisting of an instant identifier, instant size and metadata,
The instant size is the size of the metadata, the metadata is multimedia data streaming system, characterized in that the compressed data of the main digital data. The method of claim 1,
The metadata is a multimedia data streaming system, characterized in that the main digital data of the field immediately following the name (Length) in the standard extension head. The method of claim 1,
And the instant identifier, instant size, and metadata of the instant-on head are composed of one field. The method of claim 1,
The encoder is a multimedia data streaming system, characterized in that for generating an 8-bit size instant identifier corresponding to the name (Name) of the standard extension head. The method of claim 1,
And the first 4 bits of the instant identifier are track IDs determined according to video, audio or text data types. The method of claim 5,
The streaming server generates the SDP file designated by the track ID and transmits to a separate port according to the track ID. The method of claim 6,
The streaming system is a multimedia data streaming system embedded in the network kernel of the Linux operating system (Embedded) for the immediate response of the streaming system by the client's request. The method of claim 6,
The client receives the SDP file, the multimedia data streaming system to obtain the information of the streaming media transmitted from the streaming system, and ready to play for it and play immediately. The method of claim 1,
The streaming server is a multimedia data streaming system, characterized in that for storing the address of the I-frame of the multimedia data stored in the hard disk in the SDP file to the client. 10. The method of claim 9,
The instant on head is repeatedly provided between the standard expansion head,
When the movement signal of the position bar is transmitted from the client, the streaming server uses the address of the I-frame stored in the SDP file to determine the I-frame of the instant-on head corresponding to the position of the position bar on the hard disk. Multimedia data streaming system, characterized in that the payload to transmit to the client.

Images