JP3730942B2 - Network moving image distribution system and client device in this system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a network moving image distribution system that enables distribution of high-quality moving images in a system composed of distributedly managed servers, clients, and networks, and a client apparatus in this system.
[0002]
[Prior art]
With the spread of local area networks (hereinafter referred to as LAN), improvement in computer performance, and increase in capacity and cost of disk devices, server / client type database systems have begun to spread widely. In this database system, a database is constructed on the server side, and the client side inquires of the server for information as necessary.
[0003]
In recent database systems, it is necessary to handle not only text-based information but also data such as voice information and moving image information. In particular, although moving image data is compressed and encoded and stored in advance, the size of the data is larger than that of other media, so there is no means to transfer all data to the client once and start playback, such as file transfer. Not very realistic. For this reason, a method of continuously supplying data necessary for the client to decode and reproduce the video in real time from the server is generally used. The server side controls the data supply rate, and the client only needs to receive and play the data.
[0004]
[Problems to be solved by the invention]
However, in order to provide a stable moving image distribution service by a distribution service method in a conventional database system that distributes moving images, a high-performance server that guarantees a stable supply of data to a plurality of clients, data is transferred In other words, it is necessary to construct a system with a network having a guaranteed transmission bandwidth that is not affected by other traffic and a client having a uniform performance for reproducing the supplied data without loss. These are premised on being realized on an ideal system environment without considering other external factors. However, from the viewpoint of efficiency and economy, it is desired to realize a moving image distribution service in an existing environment.
[0005]
The existing environment is defined for each of the server, the network, and the client. The server is assumed to be a general-purpose computer connected to an existing network, and can be used for purposes other than the moving image distribution service. It is assumed that the network is used for purposes other than moving image distribution, and the bandwidth is not guaranteed when moving images are transferred. A situation is also assumed in which a plurality of networks are routed from the server to the client in an environment where a plurality of networks are interconnected. On the client side, resources such as CPU and memory are sometimes used by existing PCs (personal computers) / WSs (workstations), etc., with clients having decoding hardware for dedicated moving image encoded data, depending on internal and external processing conditions. Assume even software-based clients that are susceptible to changes.
[0006]
An object of the present invention is to realize a network moving image distribution system that can be operated without interrupting a moving image distribution service while adapting to the situation in an environment affected by external factors as described above. It is another object of the present invention to provide a client device for realizing such a system.
[0007]
[Means for Solving the Problems]
The present invention is a network moving image distribution system that distributes compressed and encoded moving image data stored on a server side to a client in real time via a network. The server side is designated by a request from the client. In the network moving image distribution system, the client side has a means for selecting and sending the selected moving image data program to the client, and the client side has a means for requesting the server for the desired moving image data program. When the first refresh processing data is received from the server, a request is made in units of packets, the time until the packet arrives for one request is measured , and the number of the first refresh processing data is means for also see contained in the packet, the packet arrives to the one request If the time until the excess of processing time of the moving image data included in one packet, when requesting new refresh process data in order to perform resynchronization, the packet was confirmed at the time of the first refresh data requests And a means for making a request for a plurality of packets to the server based on the server, and the server side has means for continuously transmitting a plurality of packets designated by the request from the client to the client. The network moving image distribution system is a means for achieving the above object.
[0008]
In the network moving image distribution system according to the present invention, the client side further includes a resynchronization suppressing unit that suppresses a request for refresh processing data when processing of refresh processing data is not completed. A network moving image distribution system characterized by the above is used as means for achieving the above object.
[0009]
Further, the present invention is the above-described network moving image distribution system, wherein the client side measures a processing elapsed time from the first first packet arrival time at the start of transfer at the time of arrival of a new packet; The measured processing elapsed time is compared with a time index in the packet recorded in the packet, and when the difference is larger than a preset processing delay allowable value, the decoding process is interrupted, and the packet request and reception The packet that has already arrived at the processing unit is cleared, the server requests new refresh processing data that appears after the next packet waiting for transmission , and is calculated by the transmission processing delay detection unit when the refresh processing data is requested. Means for notifying the server of the difference value, and the server side divides the moving image data into packets. Means for adding a time index to the header of the packet and sending it, and in response to a refresh processing data request from the client, the time index of the next transmission scheduled packet appears at a time after the received difference value A network moving image distribution system having means for detecting a packet including refresh processing data to be transmitted and transmitting the packet from the packet is a means for achieving the above object.
[0010]
In the network moving image distribution system according to the present invention, when the decoding of the video data included in the packet is completed, the client side receives the packet request and the transfer processing from the reception buffer of the reception processing unit. The means for measuring the processing elapsed time from the time when the first first packet is read, and the measured processing elapsed time are compared with the time index in the next packet read from the packet request and reception processing unit, and the difference Is larger than the preset processing delay tolerance, the decoding process is interrupted, the packet that has already arrived at the packet request and reception processing unit is cleared, and a new packet that appears after the next packet waiting for transmission is displayed on the server. with requesting refresh process data, at the time of the refresh process data request, calculated in the decoding processing delay detection unit And means for notifying the value of the difference to the server, the server side may divide the video data into packets, and means for transmitting by adding a time index in the header of the packet, from the client Means for detecting a packet including refresh processing data that appears at a time later than the received difference value in the time index of the next packet to be transmitted in response to the refresh processing data request, and transmitting from the packet A network moving image distribution system characterized by comprising: means for achieving the above object.
[0011]
The present invention also provides a client device of a network moving image distribution system that distributes compressed and encoded moving image data stored on a server side to a client in real time via a network in response to a request from the client. When the refresh processing data is received from the server, the request is made in units of packets, the time until the packet arrives for one request is measured , and the number of the first refresh processing data is included in the number of packets In addition, if the time until the arrival of the packet for the one request exceeds the processing time of the moving image data included in one packet, a new refresh processing data is requested for resynchronization. The packet that was confirmed at the time of the first refresh processing data request. The client devices in the network moving image distribution system, characterized in that it comprises a means for performing a request of a plurality packets to the server, and means for achieving the above object.
[0012]
Further, the present invention is characterized in that the client device in the network moving image distribution system includes a resynchronization suppressing unit that suppresses a request for refresh processing data when the processing of the refresh processing data is not completed. The client apparatus in the network moving image distribution system is a means for achieving the above object.
[0013]
According to the present invention, in the client device in the network moving image distribution system, at the time of arrival of a new packet, means for measuring a processing elapsed time from the first first packet arrival time at the start of transfer, and the measurement The processing elapsed time is compared with the time index in the packet recorded in the packet, and when the difference is larger than a preset processing delay allowable value, the decoding process is interrupted, and the packet request and reception processing unit Clear the packet that has already arrived, request the server for new refresh processing data that appears after the next packet waiting to be sent, and the difference value calculated by the transmission processing delay detector when this refresh processing data is requested Network moving image distribution system comprising: means for notifying the server of The definitive client device, the means for achieving the above object.
[0014]
In addition, the present invention provides a client device in the network moving image distribution system described above, at the time when decoding of video data included in a packet ends, the first request at the start of transfer from the reception buffer of the packet request and reception processing unit. Means for measuring the processing elapsed time from the time when one packet is read, and the measured processing elapsed time are compared with the time index in the next packet read from the packet request and reception processing unit, and the difference is set in advance. If the processing delay tolerance is larger than the specified value, the decoding process is interrupted, the packet request and the packet that has already arrived at the reception processing unit are cleared, and new refresh processing data that appears after the next transmission waiting packet on the server with requesting, at the time of the refresh process data request, calculated in the decoding processing delay detection unit The client devices in the network moving image distribution system, characterized in that it comprises a means for notifying the value of the difference to the server, and means for achieving the above object.
[0015]
The present invention has means for detecting the status of each system, and executes the processing while adapting the processing according to the status of the server, network, and client. In many existing systems, the transfer rate of moving image data is controlled from the server side. In the present invention, control of the transfer rate and detection of the situation are led by the client. The status of the server and the network is intensively understood by the client by observing the arrival status of the data, and the status of the client is monitored by monitoring the progress of decoding. The client sequentially requests moving image data of a size necessary for decoding from the server, and the server divides large data into small packets and transfers them. For video data decoding, processing may be delayed depending on the system conditions. To eliminate the delay, for example, a delay up to the default value is allowed, and the delay exceeds the default value. In the case of an increase, the client clears the moving image data that has been received from the server and accumulated in the temporary buffer and has not been processed, and requests the server to refresh processing data as new moving image data from the time when the delay can be eliminated. Start decryption. Here, when requesting the refresh processing data, when the first refresh processing data is received, the number of packets required to transmit the refresh processing data determined by measuring the packet arrival time for one packet request is also included. Requesting a plurality of packets and suppressing resynchronization during the processing of the refresh processing data. As described above, by detecting the status of each system and executing the processing while adapting the processing according to the status, in a system composed of servers, networks, and clients that are widely used and managed at present, Realize a good video distribution service.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described in detail with reference to the drawings.
[0017]
FIG. 1 is a system configuration diagram illustrating a first embodiment of the present invention. In this embodiment, H.264 used in ISDN videophones and the like. 2 shows an example of a system using a H.261 image coding system. This system is realized by a server 1 and a plurality of clients 2 on a network 3 such as the Internet constructed as a collection of LANs as shown in FIG. The server 1 and the client 2 are connected to each other via a network 3 to which the server 1 and the client 2 are connected.
[0018]
FIG. 2 shows a data transmission form in this embodiment. H. The H.261 coding scheme is a coding scheme using inter-frame differences. In the moving image data distributed to the client 2, as shown in FIG. 2A, an instruction is given to the encoder at the time of encoding, and video frames encoded in the intra-frame encoding mode are periodically inserted. Hereinafter, a video frame encoded in this intra-frame encoding mode is referred to as refresh processing data. Reference numeral 4 in FIG. 2A indicates the head of refresh processing data periodically inserted in the moving image data. H. Since the encoded data by H.261 is represented as a bit string, in order to facilitate the data transfer process on the network 3, an appropriate size for transferring the bit stream of the data on the network as shown in FIG. The packet 5 is divided, and a header 7 is added to the packet 6 as shown in FIG. 2C, and the server 1 holds the aggregate of the packet data 6 as a single moving image data.
[0019]
In the case of a storage service, a collection of packet data 6 is stored as a file, and in the case of real-time delivery, a collection of packet data 6 in a range that goes back from the latest data to a preset time in the past. Hold.
[0020]
In the header 7 of each packet data, a time represented by a decoding process start time in a situation where the decoding of the bit stream up to the bit stream included in the packet is normally performed without being affected by delay or the like. An index 71 and an identifier 72 indicating whether intra-frame encoded data is included in the packet are included.
[0021]
FIG. 3 is a block diagram showing the configuration of the server and client in this embodiment. The server 1 side includes a moving image data accumulation processing unit 11, a refresh processing data detection unit 12, and a server side packet transmission processing unit 13. On the other hand, the client 2 side includes a moving image display processing unit 21, a decoding processing unit 22, a packet request and reception processing unit 23, a decoding processing delay detection unit 24, and a transmission processing delay detection unit 25. A server-side packet transmission processing unit 13 on the server 1 side and a packet request and reception processing unit 23 on the client 2 side are connected to the network 3.
[0022]
The operation of this embodiment will be described based on the configuration of FIG.
[0023]
First, the packet request / reception processing unit 23 connects to the server-side packet transmission processing unit 13 via the network 3 and specifies the content name of the moving image data to be requested first. Next, the packet request and reception processing unit 23 on the client 2 side requests the server-side packet transmission processing unit 13 to transfer data in units of packets from the first refresh processing data that appears, and starts decoding data. Packets received by the packet request / reception processing unit 23 are accumulated in a reception buffer in the packet request / reception processing unit 23, and the decoding processing unit 22 sequentially reads and processes them. On the client 2 side, transmission and decoding processing elapsed time is simultaneously performed in the decoding processing delay detection unit 24 and the transmission processing delay detection unit 25 simultaneously with reception of the first moving image data packet sent by the server side packet transmission processing unit 13. Start measuring.
[0024]
The transmission processing delay detection unit 25 measures the processing elapsed time from the first first packet arrival time at the start of transfer when a new packet is received. The processing elapsed time of the packet measured by the transmission processing delay detection unit 25 is compared with the time index 71 of the bit stream of the data in the packet recorded in the packet.
[0025]
The decoding processing delay detection unit 24, when the decoding processing of the video data included in the packet received from the packet request and reception processing unit 23 is completed, starts from the reception buffer of the packet request and reception processing unit 23 at the beginning of the transfer. The elapsed processing time from the time when the first packet was read is measured. The elapsed processing time measured by the decoding processing delay detection unit 24 is compared with the packet request and the time index 71 of the bit stream of the data in the next packet read from the reception processing unit 23.
[0026]
The processing delay detection performed by the decoding processing delay detection unit 24 and the transmission processing delay detection unit 25 is performed by the following similar procedure. FIG. 5 shows a conceptual diagram thereof.
[0027]
When the processing elapsed time of the decoding process measured by the decoding process delay detection unit 24 is smaller than the time index 71, the processes of the decoding processing unit 22 and the packet request / reception processing unit 23 are suspended until the processing start time is reached. As shown in FIG. 5A, this case corresponds to a case where the transmission / decoding processing time 111 per packet is processed with a processing time 110 or less per packet.
[0028]
As shown in FIG. 5 (b), when the transmission and decoding processing time 112 per packet exceeds the processing time 110 because the processing elapsed time is longer than the processing start time, two operations are performed depending on the size of the difference 113. I do.
[0029]
A processing delay allowable value is set on the client side, and when the difference 113 is smaller than the allowable value, the processing of the decoding processing unit 22 and the packet request / reception processing unit 23 is continued.
[0030]
When the difference 113 is larger than the allowable value, the processing of the decoding processing unit 22 is interrupted, the packet that has already arrived at the packet request and reception processing unit 23 is cleared, and then the server side packet transmission processing unit 12 The refresh processing data that appears after the transmission waiting packet is requested. When the refresh processing data is requested, the value of the difference 113 calculated by the decoding processing delay detection unit 24 and the transmission processing delay detection unit 25 is also notified to the server 1. On the server 1 side, in response to a refresh processing data request from the client 2, a refresh processing that appears at a time after the difference value sent from the time index 71 of the next packet to be transmitted in the refresh processing data detection unit 13. A packet including data is detected and transmitted from the packet. When the client 2 side receives the refresh processing data in response to the request, the packet request and reception processing unit 23 continues to request a packet that is continuous to the packet, and the decoding processing unit 22 continues to the start time recorded in the packet. The delay is eliminated by interrupting the decoding process, and the decoding process is resumed when the start time comes.
[0031]
Next, a second embodiment of the present invention will be described.
[0032]
This embodiment is an embodiment that is effective when the network between the server and the client is significantly below the transmission band for requesting moving image data or when the transmission delay is large via a plurality of networks. FIG. 4 shows a conceptual diagram of transmission processing in this embodiment.
[0033]
In this system, refresh processing data is used as means for eliminating processing delay. H. In the moving image data using the H.261 encoding method, the refresh processing data itself has a larger size than the other data, and is compared with the maximum size of the packet 104 transmitted on the network as shown in FIG. In some cases, it becomes larger and 105 extends over a plurality of packets. In order to eliminate the processing delay, all the refresh processing data must be received, decoded, and displayed. However, when a plurality of packets are required, the request packet 101 is converted into one request packet 101 as shown in FIG. On the other hand, transferring one data packet 102 is inefficient. In particular, when the transmission band is low or the transmission delay is large, the processing delay included in the transmission time 103 of the request packet cannot be ignored. Therefore, the client uses means for requesting a plurality of packets with a single request to the server.
[0034]
First, on the client side, when connecting to the server and receiving the first refresh processing data, a request is made in packet units. At this time, the time until a packet arrives for one request is measured, and it is confirmed whether the transmission time per packet exceeds the processing time of moving image data included in one packet. Also, it is confirmed how many packets the refresh processing data was included in.
[0035]
If the transmission time of one packet is sufficiently smaller than the processing time of one packet, it is determined that the transmission bandwidth on the network is secured, and multiple requests for packets are not made in order to distribute traffic. Transmission is performed every time as in the request packet 106 and the data packet 107 in FIG.
[0036]
In a situation where the transmission time of one packet exceeds the processing time of one packet, the number of packets confirmed at the time of the first refresh processing data request when newly requesting the refresh processing data for resynchronization Based on the above, a request for a plurality of packets is made as indicated by 108 in FIG. In this case, the server continuously transmits a plurality of packets designated by one instruction as indicated by 109. The client side accumulates continuously transmitted moving image packets in the reception buffer. The accumulated moving image packets in the reception buffer are sent from the arrival to the decoding process.
[0037]
If the transmission delay cannot be secured, there is a possibility that the refresh processing data cannot be processed within the range of the processing delay allowable value set in the first embodiment. Therefore, the processing of the refresh processing data is performed even when the allowable value is exceeded. If not completed, it has a function to suppress resynchronization. When the refresh processing data is processed exceeding the allowable value, resynchronization is performed again when the processing is completed. As for other processes, the procedure of the first embodiment is assumed.
[0038]
【The invention's effect】
As described above, according to the present invention, when moving images are distributed over a network, they operate adaptively according to the load situation occurring at each location of the server, network, and client managed in a distributed manner, and within the range of capabilities. The best video playback is possible. In the present invention, since the control subject is on the client side, the load on the server side can be reduced, the upper limit of the number of simultaneous access clients can be increased, and the transmission of unnecessary packets on the network can be suppressed. This is particularly effective when the client side performs decoding using software, or on a network where the transmission band required for moving image data distribution is not guaranteed.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram showing an embodiment of the present invention. FIGS. 2A, 2B, and 2C are diagrams for explaining a data packet transmission form in the embodiment. FIG. FIG. 4A is a block diagram of a server and a client in the above embodiment. FIG. 4A is a diagram for explaining a transmission process when one data packet is transferred for one request packet, and FIG. 4B is a request packet. (C) is a conceptual diagram of transmission processing when sending a plurality of packets after a request packet. FIG. 5 (a) is per packet. FIG. 5B is a conceptual diagram of transmission processing when the transmission and decoding processing time is smaller than the processing time per packet, and FIG. 6B is a diagram of transmission processing when the transmission and decoding processing time per packet is larger than the processing time per packet. Conceptual diagram DESCRIPTION OF SYMBOLS
DESCRIPTION OF SYMBOLS 1 ... Server 11 ... Moving image data storage process part 12 ... Refresh process data detection part 13 ... Server side packet transmission process part 2 ... Client 21 ... Moving image display process part 22 ... Decoding process part 23 ... Packet request and reception process part 24 ... decoding processing delay detection unit 25 ... transmission processing delay detection unit 3 ... network 4 ... heads 5 and 6 of refresh data ... packet 7 ... header 71 ... time index 72 ... identifier

Claims (8)

A network moving image distribution system that distributes compressed and encoded moving image data stored on a server side to a client in real time via a network,
The server side has means for selecting a program of moving image data designated by a request from the client and sending it to the client, and the client side has means for requesting a program of desired moving image data from the server In a network video distribution system having
On the client side,
When the first refresh processing data is received from the server, a request is made in units of packets, the time until the packet arrives for one request is measured , and the number of the first refresh processing data is included in the number of packets Means to check if it was,
When the time until the arrival of a packet with respect to the one request exceeds the processing time of the moving image data included in one packet, the first time when requesting refresh processing data for resynchronization, Means for making a request for a plurality of packets to the server based on the packets confirmed at the time of the refresh processing data request,
On the server side,
A network moving image distribution system comprising means for continuously transmitting a plurality of packets designated by a request from the client to the client. On the client side,
The network moving image distribution system according to claim 1, further comprising resynchronization suppression means for suppressing a request for refresh processing data when processing of the refresh processing data is not completed. On the client side,
Means for measuring the elapsed processing time from the first arrival time of the first packet at the start of transfer at the time of arrival of a new packet;
The measured processing elapsed time is compared with a time index in the packet recorded in the packet, and when the difference is larger than a preset processing delay allowable value, the decoding process is interrupted, and the packet request and reception The packet that has already arrived at the processing unit is cleared, the server requests new refresh processing data that appears after the next packet waiting for transmission , and is calculated by the transmission processing delay detection unit when the refresh processing data is requested. Means for notifying the server of a difference value;
On the server side,
Means for dividing moving image data into packets, adding a time index to the header of the packet, and sending the packet;
In response to a refresh processing data request from the client, a packet including refresh processing data that appears at a time after the received difference value is detected in the time index of the next packet to be transmitted is transmitted from the packet. The network moving image distribution system according to claim 2, further comprising: means. On the client side,
Means for measuring a processing elapsed time from the time when the first first packet at the start of transfer is read from the reception buffer of the packet request and reception processing unit at the time when the decoding process of the video data included in the packet is completed;
The measured processing elapsed time is compared with a time index in the next packet read from the packet request and reception processing unit, and when the difference is larger than a preset processing delay allowable value, the decoding process is interrupted, The packet request and the packet that has already arrived at the reception processing unit are cleared, the server requests new refresh processing data that appears after the next transmission waiting packet, and at the time of this refresh processing data request, the decoding processing delay detection unit Means for notifying the server of the difference value calculated in
On the server side,
Means for dividing moving image data into packets, adding a time index to the header of the packet, and sending the packet;
In response to a refresh processing data request from the client, a packet including refresh processing data that appears at a time after the received difference value is detected in the time index of the next packet to be transmitted is transmitted from the packet. The network moving image distribution system according to claim 2, further comprising: means. In a client device of a network moving image distribution system that distributes compressed and encoded moving image data stored on the server side to a client in real time via a network in response to a request from the client,
When the first refresh processing data is received from the server, a request is made in units of packets and the time until the packet arrives for one request is measured , and the number of the first refresh processing data is included in the number of packets. A means to check if
When the time until the arrival of a packet with respect to the one request exceeds the processing time of the moving image data included in one packet, when the refresh processing data is newly requested for resynchronization, A client apparatus in a network moving image distribution system, comprising: means for requesting a server a plurality of packets based on the packet confirmed at the time of a refresh processing data request. 6. The client apparatus in the network moving image distribution system according to claim 5, further comprising resynchronization suppression means for suppressing a request for refresh processing data when processing of the refresh processing data is not completed. Means for measuring the elapsed processing time from the first arrival time of the first packet at the start of transfer at the time of arrival of a new packet;
The measured processing elapsed time is compared with a time index in the packet recorded in the packet, and when the difference is larger than a preset processing delay allowable value, the decoding process is interrupted, and the packet request and reception The packet that has already arrived at the processing unit is cleared, the server requests new refresh processing data that appears after the next packet waiting for transmission , and is calculated by the transmission processing delay detection unit when the refresh processing data is requested. The client device in the network moving image distribution system according to claim 6, further comprising: a unit that notifies the server of a difference value. Means for measuring a processing elapsed time from the time when the first first packet at the start of transfer is read from the reception buffer of the packet request and reception processing unit at the time when the decoding process of the video data included in the packet is completed;
The measured processing elapsed time is compared with a time index in the next packet read from the packet request and reception processing unit, and when the difference is larger than a preset processing delay allowable value, the decoding process is interrupted, The packet request and the packet that has already arrived at the reception processing unit are cleared, the server requests new refresh processing data that appears after the next transmission waiting packet, and at the time of this refresh processing data request, the decoding processing delay detection unit The client device in the network moving image distribution system according to claim 6, further comprising: a unit for notifying the server of the difference value calculated in step 1.

Images