WO2000072517A1

WO2000072517A1 - System and method for streaming media over an internet protocol system

Info

Publication number: WO2000072517A1
Application number: PCT/CA2000/000133
Authority: WO
Inventors: Joe Toth; James Schellenberg; David Graves
Original assignee: Edge Networks Corporation
Priority date: 1999-05-21
Filing date: 2000-02-15
Publication date: 2000-11-30
Also published as: AU2653000A; CA2280662A1; AU2652800A; WO2000072602A1; AU2652900A; WO2000072599A1

Abstract

A method of communication between a client and a continuous media server in a data communication backbone network, the method comprising the steps of the server composing data to be transmitted into a backbone common format; the server transmitting the backbone common format data to the client POP; converting at the POP the backbone common format data into a plurality of access common format data for transmission to ones of a plurality of clients. Preferably, wavelet compression is used to convert data to optimally select a predetermined quality. Accordingly, the present invention solves the problem of network congestion by providing a single high quality stream of data to an edge server which may be later filtered to accommodate a client or consumers processing capability.

Description

System and Method for Streaming Media over an Internet Protocol System

The present invention relates to the field of network systems, and more particularly to a system and method for optimizing the bandwidth of Internet systems.

BACKGROUND OF THE INVENTION

The Internet today is defined as the interconnection of Internet protocol (IP) - based networks. The Internet protocol stack diagram is shown in figure 1 and represented in terms of the ISO - 7-layer model, and described in Table I.

Table I.

Various equipment types and products may be associated with the layer functionality that they service, as described in Table II.

Table II.

The Internet may be viewed as a single integrated network in which various access types are interconnected to various backbone types through edge servers and edge equipment (also called remote access servers or network access servers). There are approximately twenty or more different variations on access paths that can be used to connect the backbone services to the customer. There are six basic access types of connection namely, wireless terrestrial, wireless satellite, copper, coaxial-cable, power line carriers and fiber. In the future, additional access types may be created. The resounding success of the Internet and more specifically, the graphical World

Wide Web (the web), may be its undoing. The number of web subscribers, content providers, and requests by those subscribers for content grows exponentially faster than the capability of the network to meet the demand. The majority of current data transfers involve text and graphics. However, the future of the Internet appears to be evolving towards the transfer of full motion video and audio.

As web sites will continue to increase their multimedia content in the near future through the integration of audio, video and data, the ability of the web to effectively deliver this media to Internet end users will yield a congestion problem due to the nature of the web. One of the features that has made the web such a success is the ability of one user to access another user's information, regardless of where that information is stored, what type of computer it is stored on, or what kind of application was used to create it. Unfortunately, the same flexibility and ease of use features result in a serious contention issue, since everyone competes with everyone else for available network resources. Streaming technologies for live audio and video over the web have exacerbated this problem even further.

Normally, hyperlinks point to multimedia files that are downloaded in their entirety to a user's local disk. However, streaming media allows users to watch live video and audio as the file is downloading.

Some of the reasons for these bandwidth restrictions or bottlenecks are described as follows. Since the Internet is IP based, all packets must be evaluated by routers to determine the destination delivery paths, creating traffic congestion, particularly with the increased demand in realtime media, such as video and audio. It has been found that backbone, subnet and router upgrades are not sufficient to increase the Internet throughput to offset the increasing bandwidth requirements of the WWW itself. This problem is further exacerbated by end users having faster access to the ISP POP (Internet Service Provider Point of Presence). Simply providing "bigger pipes" to the POP simply sends bigger chunks of data onto the web. Realtime protocols and specialized backbones have been developed. However, these solutions are suitable only for improving transports for scheduled or premium events, but are unsuitable for the proliferation of multimedia content that is expected in the near future. Although improved compression techniques promise to squeeze multimedia files into smaller and smaller sizes, video and audio will continue to require a "big pipe" as a result of the realtime transport requirements.

A solution to the above problem has been proposed by Oracle Corporation. In this solution, it is proposed that the multimedia data repository is placed closer to the consumer of the multimedia. Thus, servers are deployed at the edge of the WWW and multimedia data is replicated on these edge servers where the users connection terminates at the POP. Hyperlinks on the web pages become pointers to streaming media servers that are physically closest to the consumer. The philosophy behind this implementation is that the POP is the logical termination of the user's access point, and thus packets flowing into or out of the POP are only limited by the access speed of the user's connection. Any data packets that flow behind or through the ISP back channel, for example, router, are affected by bottlenecks. Thus, by placing the media repository at the POP and behind the router, the user is insulated from traffic conditions that exist on the Internet at any given time. It is envisioned that content providers and web publishers use a combination of mirroring or caching techniques to replicate data to the edge servers. A disadvantage of the above scheme is that it requires the content providers and web publishers to themselves stage propagate and update multimedia data to be replicated in the mirroring model, whereas, in the caching model, if the requested data by the user was not already cached, a dialogue box would inform the user with the approximate time the media would be available and might suggest that they visit other sites in the interim. In general, this is unacceptable to most users since most users require instant access to the requested data.

A further improvement on this method, and particularly applicable to streaming media, has been proposed by Real Networks which introduced a distributed multitier broadcast architecture for the internet termed the real broadcast network (RBN). In this solution, access to the RBN server is distributed throughout the Internet backbone. Live feed is transmitted directly to splitters which are located in the major backbone provider's network. This feed is then retransmitted or "split" from the backbone provider to splitters installed at the ISP site, where it is finally streamed to the user's computer. Another solution that Real Networks proposes to counter the problem of providing high quality media (video and audio) to streaming users while accommodating the various physical connection speeds between the user and the ISP, is to create a scalable stream where the server can reduce the amount of data being sent to keep the client from rebuffering. This approach is generally referred to as video "stream-thinning". The limitations of this approach is that a video or audio file designed to play at one data rate and subsequently scaled down to a lower rate results in an inferior quality level when compared to a video optimized specifically for the lower data rates. Furthermore, audio codecs cannot usually dynamically send lower data rates. An approach to address this heterogeneous connection rate environment is to create several files so that when a client connects, the server streams the appropriate file. This has been referred to as "bandwidth negotiation". This process is not dynamic, so if a user's actual throughput changes due to congestion or packet loss, the server cannot adjust. Another difficulty is the increased labour required for encoding and then managing the media clip for different bandwidths. The Real Networks solution to these problems, in its most recent incarnation, is to provide an encoding framework for combining multiple data streams, each at different bit rates, into a single file. A sophisticated client server mechanism is provided for detecting changes in bandwidth and translating those changes into combinations of different streams. While the above attempts to address the solution of bandwidth negotiation and stream thinning, it still suffers from the limitation in that multiple streams corresponding to different bit rates must be composed at the server end. For example, if ten different streams are to be composed at the server to the backbone, where each stream ranges from 1 megabit per second as follows: stream 1 - 1 Mbps, stream 2 - 500 Kbps, stream 3 - 300 Kbps, stream 4 - 200 Kbps, stream 5 - 100 Kbps, stream 6 - 56 Kbps, stream 7 - 33 Kbps, stream 8- 28 Kbps, stream 9 - 14 Kbps, stream 10 - 8 Kbps, then all ten streams must be sent over the backbone from the server to splitters and POPs. Thus, a 3.4 Mbps stream is sent down the backbone. At the POP, ten different caches are now required. The POP then forwards the appropriate bit stream to the user depending on the user's access capability. It may be seen that in this solution, the user is provided with a relatively consistent stream. However, it still does not alleviate the problem of backbone congestion since multiple streams must all be transmitted along the backbone, with these multiple streams requiring increased bandwidth. There is thus a need for a system and method for mitigating at least some of the above disadvantages.

SUMMARY OF THE INVENTION

An advantage of the present invention is to provide a distributed Internet architecture that minimizes congestion on the Internet backbone.

In accordance with this invention there is provided a method of communication between a client and a continuos media server in a data communication backbone network, the method comprising the steps of the server composing data to be transmitted into a backbone common format; the server transmitting the backbone common format data to the client POP; converting at the POP the backbone common format data into a plurality of access common format data for transmission to ones of a plurality of clients. In accordance with a further embodiment of the invention the client includes the step of filtering the received data according to a predetermined parameter of the quality.

A still further embodiment of the invention provides for the conversion of the backbone common format data to wavelet compressed data.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of the preferred embodiments of the invention will become more apparent in the following detailed description in which reference is made to the appended drawings wherein: Figure 1 is schematic diagram of protocol stack diagram;

Figure 2 is schematic diagram of an Internet architecture;

Figure 3 is a schematic diagram of an Internet architecture according to an embodiment of the present invention;

Figure 4 is a schematic diagram of a multiple consumer connection architecture according to an embodiment of the present invention;

Figure 5 is a schematic diagram of an edge server client connection;

Figure 6 is a detailed schematic diagram of a edge server; and

Figure 7 is a schematic diagram of a streaming media server with an edge server transcoding.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description like numeral refer to like structures in the drawings. Referring to figure 2, a general Internet architecture as it currently exists is shown generally by numeral 20. The architecture comprises a backbone network 22 which is defined as the interconnection equipment concerned with connecting local web sites to local POPs and an access network 24 that is defined as the interconnection between the local POPs and the consumers. The Web sites 26 host both digital and analog content from various Content providers 28, which are in-turn connected via a global and national internet infrastructure 30 to the local access internet infrastructure 32. The consumer or viewer 34 connects to the national Internet infrastructure 30 i.e. at the POP by a one or more access links 33 as defined earlier. Cache 36 sites are provided between the global and national Internet infrastructure 30 and the local access Internet infrastructure 32. The cache sites 36 are normally the demarcation between the backbone network 22 and the access network 24. Backbone web sites 26 typically do not consider the needs of various types of access 33 employed by clients 34 and various qualities of access links in their consideration of web content. It is normally the responsibility of the web content provider 28 to customize the web site content for different access links.

Therefore the present invention leverages off the existing architecture, but implements a content compression architecture that uses data link level 6 to application level 16 technologies to optimize the access from the local POP to the customer 34. This architecture relieves the web hosting and web content provider from the burden of customizing the web site content or protocol for different access links and moves the access specific customization to the local POP site 32. This content compression architecture for Internet IP networks uses a concept of backbone common format to access common format (BCS-ACS) Translation.

Referring to Figure 3 a network architecture for streaming media data according to the present invention is shown generally by numeral 60. The architecture 60 includes enhanced data compression 62 to existing or new web sites 26; caching algorithms 64 for the connection of these web sites 26 to existing 36 or new 37 local cache sites; local cache and serving products with compression translation, local compression and access specific translation 66 at the POP; and new hardware and software client technologies for PC and embedded PC consumer sites 68. Each of these enhancements will be discussed in detail below. Referring to Figure 4 a schematic diagram of the access network 24 according to an embodiment of the present invention is shown generally by numeral 70. As described above, the web sites 26 are connected via the backbone 72 to the local access equipment 72. Thus the web site server composes streaming media data to be transmitted into a single stream of backbone common format data. The server transmits this single stream backbone common format data to the client POP. However, at the local access equipment level 74 there are conversions performed that translate the single stream backbone common format data into different common access formats that are suitable for optimized transmission to consumers 78. The data may be transferred via a wireless 3G link 80, a copper DSL link 82, a coaxial cable link 84, or other access links known in the art. These particular three methods are simply exemplary of three different types of access media. The wireless access media 80 is inherently broadcast and simulcast capable, the copper media 82 is inherently point to point, and the cable interconnection 84 is inherently bus broadcast and simulcast capable. Typically multiple users communicate via each link as is also shown in Figure 4. For example, the wireless link 3G 80 has multiple consumers, 1, 2,... N with each consumer receiving the same broadcast or simulcasts information. Each of the consumers may have different client capabilities in terms of processing power and memory resources. Thus a client-filtering device is located at the consumer in order to allow the single stream of the access common data converted from the backbone common format data to be processed according to the capabilities of the respective consumer. This is called client side filtering. This is also valid for coaxial cable links 84. Consider a further example, assume there are one hundred consumers each with different processing power coupled to the POP via a wireless link. The ISP or CLEC service provider wishes to offer the best streaming video picture quality possible to each consumer and further assume that there are three classes of consumers each able to accept 1Mbps streams, 200 kbps streams and 50 Kbps streams respectively. Therefore without client side filtering the conversion equipment must transmit three streams of a combined data rate of 1.25 Mbps. With client side filtering a single 1 Mbps stream can be transmitted to the consumers; the client devices filter the stream to match the capability of the client processing, thus saving bandwidth over the simulcast/broadcast link. Clearly, as the number of client types increases the bandwidth savings achieved by client side filtering increases.

However, for the copper DSL link conversion equipment there is a point-to-point connection between each of the consumers and the equipment. Therefore, it is preferable that client filtering can be performed at the conversion equipment before the copper link is traversed. This approach is referred to as extended client filtering because the capability of client filtering is no longer resident at the consumer 34, but has been extended to the local access equipment 32. In some cases however, a combination of both techniques may be used. For example, in an xDSL connection extended client filtering may be used on the edge server side while client filtering may be employed on each client coupled to the xDSL connection allowing the received data to be tailored to that clients resolution capabilities or other such parameter. Turning now to figure 5 detailed schematic view of the local POP site, access network, and client site is shown. The local equipment level includes transcompression, streaming server capability and caching management if required by the consumer. The transcompression function will take data from the Internet backbone standard compression methods such as MPEG1, MPEG2, JPEG and such like, which are well known and compress it further to access standard compression methods based on wavelet compression. This function will allow the invention to retrofit into the existing Internet connections. The transcompression will be of the streaming variety and therefore, the technology is fast enough to handle multiple simultaneous streams. The local access site also includes a modified congestion algorithm for the access link. It converts the backbone header structure into smaller headers and resizes the IP packet size from backbone size to access size. Standard management information bases (MIB) are accessible over the network by SNMP. The system can be upgraded using software, with a software delivery method such as download available over the web. The system can also interface to the various access networks using OEM equipment from various vendors.

Referring to Figure 6 a schematic diagram of the transcompression module shown in Figure 5 is shown. Content from the backbone websites are received and passed via an HTTP level switching module to a digital CCIR 656 interface. A CCIR 656 interface couples the data to an optimized wavelet compressor. The compressed data from the compressor is sent to a web-site based storage under the control of storage logic. The storage logic is also capable of forwarding received data directly to the storage without it being compressed. The compressed content is made available to a streaming server for eventual serving to a client.

The choice of wavelet compression offers multiple advantages over other compression schemes. At the local site, wavelet technology can be used in the consideration of the speed of the access pipe. Router technology, gateway technology and other Internet infrastructure equipment may benefit from allowing more or less wavelet coefficients through the channel. The number of wavelet coefficients used determines the quality at which the compressed data is uncompressed. This functionality is provided by allowing the wavelet compression stream to have priority tags on the packets which indicate those coefficients that are most important for the data stream and those coefficients that are less important. Wavelet compression is good for this particular function because the primary, secondary and tertiary coefficients are clearly definable. This aspect of the wavelet technology is most useful for point to point connections such as copper DSL links. If the consumer is connected to the Internet via a slow connection, such as a 14k modem, the local equipment can send the compressed data with fewer coefficients. While this may reduce the quality of the data received by the consumer, it will allow the consumer to view the data at perceptively improved rates. Although the actual rate of transmission does not change, since there is less data to be sent the customer perceives the transmission to be faster. If the customer has a faster connection to the Internet, then the local client will send more coefficients and the client will be able to receive better quality data. Therefore, this method improves the speed of transfer of data by reducing the amount of data to be sent. Another benefit of the wavelet technology is that for links with multiple clients accessing the same data stream, only one form of the data needs to be sent. Wavelet compression allows for progressive video. Progressive video occurs when a single compressed bitstream, using wavelets, is sent to various forms of destination capability. Since the data is non-interlaced, the data can be uncompressed linearly with different wavelet coefficients depending on the processing power of the client. If, for example, a single wavelet stream is sent to a mobile user with a Palm Pilot, a PC using a slow processor, and a PC using a fast processor, the stream would be processed differently in each of these environments. For the Palm Pilot™ environment, the screen is small and the processing speed is small and therefore, only small portions of the wavelet coefficients are used to create the picture. For the slow PC, the screen resolution is greater and more wavelet coefficients are used. For the faster PC, the screen resolution is the highest and the processing speed allows all of the wavelet coefficients to be used. Therefore, the encoding source does not need to be concerned about the capability of the end device, and instead, compresses at maximum capability and allows the end device to use whatever portion of the wavelet coefficient it needs. This method does, however, require the client to have the appropriate hardware and software to decompress the wavelet stream.

Other forms of compression may also be used, such as fractal, vector quantization, or wavelet-fractal compression as known in the art.

Local cache is used when fast response times are required for large content requests. The use of local cache assumes that some web sites and content sites are more popular than others, and therefore, local storage of popular content will allow for a faster response time and decreased backbone bandwidth utilization. The extreme end of the local cache would be if all the content on the Internet were sent to each local cache all the time. In this case, the local cache memory size would need to be as large as the rest of the Internet combined, a scenario that is price prohibitive.

Referring to Figure 7, a further embodiment of the invention is shown generally by numeral 700. In this embodiment, the web server at a web site 726 takes the content 728 and applies wavelet compression 730 to the content. Thus, the web site 726, provides streaming media over an IP that is compliant with a wavelet compression. In this embodiment, the web server 726 responds to media requests received over an IP with instructions to compress analog data that is compliant with CCIR recommendation [601 or 656]. The web server 726 will then send wavelet compressed media data that is compliant with the wavelet codec at the POP 732, either in multicast streaming format, unicast streaming format or the file transfer protocol (FTP) format at a scheduled time. The wavelet media server 731 is network manageable with SNMP commands and supports standardized MIB (Management interface base) data.

The wavelet video compression function 730 supports up to 30 fps, both realtime wavelet video compression and non realtime wavelet video compression. The wavelet video compression function 730 receives CCIR compliant analog or digital data from the analog video source 728 and sends wavelet compressed video data to the media database 731. The wavelet video compression function adds a timestamp data to each compressed frame as defined by RTP to support a client audio video lip-synch. The video compression function receives from the client 734 video compression control information to start wavelet compression for IP streaming. The wavelet algorithm compresses 30 fps QCIF/CIF video at bit rates ranging from 50 Kbps to 1 Mbps depending on the subjective video quality to be streamed over the backbone 736.

On the other hand, analog audio data will be compressed in realtime based on the audio compression control information received from the client. The audio compression function also adds a timestamp data per compressed audio packet compliant with RTP to support the client audio video lip-synch algorithm. The MMIP audio compression function sends compressed audio data to the media database 731 for efficient streaming over the backbone network.

The media streaming function 726 sends a wavelet compressed media data in a backbone optimized compressed resolution with the media timestamps compliant with RTP based on the media streaming control information. The media streaming function also supports the sending of data in multicast streaming format , unicast streaming format or FTP at a scheduled time. The media streaming function also supports media multicast over RTP/UDP to multiple destinations to minimize the loading on the backbone network. In addition, support is provided for unicast RTP/TCP for transmissions through firewalls for transmissions to destinations which do not support IP multicasting.

The media streaming function uses an algorithm for coding and timestamping of the audio and video packets read from the media database. Since the time difference between packets is relatively constant, the second order differences generally approaches zero. An algorithm for a variable key packet (video and audio) is used for lossless compression of the time redundancies in the RTP/UDP headers. The algorithm utilizes variable length first and second order differences for the exact reconstruction of the timestamps by the destination and minimization of backbone network loading. In addition, up to 2: 1 lossless compression can be achieved for the packet payload if the web data is not already in an encrypted or compressed format. An algorithm is used to detect if the compression of the web data results in data expansion. Data expansion is typical when the compressed algorithm is applied to encrypted data.

A compression manager function monitors the operational status of the web server and uses COTS hardware and software to support design reuse. The compression manager receives media requests and reads compressed media data from the media database to implement the following functions in maintaining the database, a determination of the newly compressed media that is to be maintained in the server; determining the amount of each media program to be maintained in the server; setting a time for each media program to be kept in a server before it becomes stale; congestion algorithms for the backbone network for unicast and multicast transmissions to support an anytime/anywhere capability; and IP packets segmentation optimizations for the backbone network.

This optimization shall focus as a minimum on the avoidance of IP segmentation and reassembly delays and overhead in the backbone equipment. In the above embodiment, it may be seen that compression is moved to the web server while transcoding, i.e., modification of quantities is done at the caching server at the ISP or POP.

Although the invention has been described with reference to certain specific embodiments, various modifications thereof will be apparent to those skilled in the art without departing from the spirit and scope of the invention as outlined in the claims appended hereto.

Claims

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. A method of communication between a client and a continuos media server in a data communication backbone network, the method comprising the steps of:

(a) the media server composing data to be transmitted into a backbone common format;

(b) the media server transmitting the backbone common format data to the client POP; and (c) converting at the POP the backbone common format data into a plurality of access common format data for transmission to ones of a plurality of clients.

2. A method as defined in claim 1, including the step of filtering said access format data for use by ones of said clients.

3. A method as defined in claim 2, said filtering being the selection of the data compliant with a clients access speed.

4. A method as defined in claim 2, said selection being performed at the client.

5. A method as defined in claim 2, said selection being performed at the POP.

6. A method as defined in claim 1, said converting including applying wavelet compression to said backbone common format data to generate said access common format data.

7. A method as defined in claim 6, including the step of selecting one or more of said wavelet coefficients for transmission to a client.

8. A method as defined in claim 1, said step of converting including applying wavelet compression to said backbone common format data.

9. A method as defined in claim 1, said step of converting including applying fractal compression to said backbone common format data.

10. An edge server comprising:

(a) a transcompression module for converting a streamed backbone common format data to an access common format data; and (b) a memory for caching said converted data.

11. An edge server comprising:

(a) a transcompression module for receiving data from a network backbone in a first format and for converting said received data to a second format; (b) a memory for caching said converted data;

(c) an interface for coupling said cache to a plurality of clients.

12. An edge server as defined in claim 11, said second format data including a plurality of data streams.

13. An edge server as defined in claim 11, said second format data being wavelet compressed data.

14. An edge server as defined in claim 12, including a filter for selecting ones of said plurality of second data in accordance with a predetermined parameter of a client.

15. A method of communication between a client and a continuos media server in a data communication backbone network, the method comprising the steps of:

(a) the media server compressing data to be transmitted into a backbone common format;

(b) the media server transmitting the backbone common format data to the client POP;

(c) transcoding at the POP the backbone common format data into a plurality of access common format data for transmission to ones of a plurality of clients.

16. A method as defined in claim 15, said step of compressing including wavelet compression.

17. A method as defined in claim 15, said step of transcoding including selecting one or more coefficients of said wavelet compressed data for transmission to a client and wherein the selection is determined by the clients access speed.

18. A method as defined in claim 15, said step of transcoding including selecting one or more coefficients of said wavelet compressed data for transmission to a client and wherein the selection is determined by the clients desired display capabilities.