JP2014003429A - Transmission packet distribution method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for distributing transmission packets in a multicast or unicast capable of compensating losses of transmission packets due to radio wave interferences.SOLUTION: In a method for distributing, by multicast or unicast distribution, content stream data composed of closed GOPs including at least video data and sound data to a terminal device 3 via communication paths 2 and 4, the content stream data is hierarchically configured with a first multiplexed stream packet ST1 derived by multiplexing the data in units of a closed GOP into Mβ pieces sequentially from the beginning, with Nαpieces of closed GOP counted as one group and a second multiplexed stream packet ST2 derived by multiplexing the data into Mβ pieces sequentially from the beginning, with Nαpieces of closed GOP as counted one group. The first multiplexed stream packet and the second multiplexed stream packet are distributed by multicast or unicast distribution with a time difference.

Description

  The present invention relates to a transmission packet distribution method in multicast or unicast.

  In a wireless communication system in which commands from a host computer are transmitted to a plurality of terminals from a plurality of access points connected through a network, identification numbers are assigned to a plurality of channels each composed of an access point and a terminal. It has been proposed to avoid loss of multicast or broadcast transmission packets by providing a time slot assigned according to an identification number assigned to each channel at an access point (Patent Document 1).

JP 2006-245892 A

  However, with the above conventional technology, even if transmission packet loss due to channel interference with an adjacent access point can be avoided, loss of transmission packet due to radio wave interference from other than the access point, such as a microwave oven or amateur radio, should be avoided. I can't.

  The problem to be solved by the present invention is to provide a multicast or unicast transmission packet distribution method that can compensate for the loss of transmission packets due to radio wave interference.

The present invention relates to a method for multicast distribution or unicast distribution of content stream data composed of closed GOP including at least video data and audio data to a terminal via a communication path, and the content stream data is a minimum unit of closed GOP. a manner multiplexed first multiplexed stream packets multiplexed in Mβ number, in order from the head of the _two closed GOP N.alpha to Mβ pieces in the one group from the beginning to _one of the closed GOP N.alpha in one group in order The above-described second multiplexed stream packet is hierarchically configured, and the first multiplexed stream packet and the second multiplexed stream packet are distributed by multicast or unicast with a time difference to solve the above-described problem. .
(However, Nα ₁ , Nα ₂ and Mβ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ )

According to the present invention, multiplexed stream packets having different numbers of closed GOPs Nα ₁ and Nα ₂ (burst error tolerance value) are multicast-distributed with a time difference, so that decoding correction can be performed mutually, and in particular burst error tolerance. Decoding possibility can be improved even for burst errors exceeding the values Nα ₁ and Nα ₂ .

It is a block diagram which shows the multicast delivery apparatus to which one embodiment of invention is applied. It is a figure which shows the data structure of the multicast packet which concerns on one embodiment of this invention. It is a figure which shows the data structure of GOP multiplexing which concerns on one embodiment of this invention. It is a figure which shows the data structure of the hierarchy multiplexing which concerns on one embodiment of this invention. It is a figure which shows the example of correction | amendment of the packet loss by the hierarchy multiplexing which concerns on one embodiment of this invention. It is a data structure figure for demonstrating the effect by the GOP multiplexing decoding which concerns on one embodiment of this invention. It is a data structure figure for demonstrating the effect by the hierarchy multiplexing decoding which concerns on one embodiment of this invention. It is a figure which shows other embodiment which transmits the hierarchy multiplexing packet which concerns on one embodiment of this invention. It is a figure which shows other embodiment which transmits the hierarchy multiplexing packet which concerns on one embodiment of this invention. It is a figure which shows embodiment which provided the hierarchy multiplexing header to the IP packet which concerns on one embodiment of this invention. It is a figure which shows the reproduction | regenerating method of the hierarchy multiplexing packet which concerns on the comparative example of this invention. It is a figure which shows the reproduction | regenerating method of the hierarchy multiplexing packet which concerns on one embodiment of this invention. It is a figure which shows the subtitle reproduction | regeneration method of the hierarchy multiplexing packet which concerns on the comparative example of this invention. It is a figure which shows the subtitle reproduction | regeneration method of the hierarchy multiplexing packet which concerns on one embodiment of this invention. It is a figure which shows the smoothing method of a caption packet which concerns on the comparative example of this invention. It is FIG. (1) which shows the comparative example of the smoothing method of a caption packet which concerns on one embodiment of this invention. It is FIG. (2) which shows the comparative example of the smoothing method of a caption packet which concerns on one embodiment of this invention. It is FIG. (3) which shows the comparative example of the smoothing method of a caption packet which concerns on one embodiment of this invention. It is a figure which shows the reduction / smoothing method of the padding packet which concerns on the comparative example of this invention. It is a figure which shows the comparative example of the reduction / smoothing method of the padding packet which concerns on one embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
In this example, as shown in FIG. 1, video and audio are transmitted from a multicast server 1 to a terminal device 3 such as a smartphone via a WiFi (registered trademark) wireless LAN device 2 (hereinafter also simply referred to as a wireless LAN device 2). In addition, the present invention relates to a method and apparatus for multicast distribution of content including subtitles and the like, and a software program. A data signal is transmitted between the multicast server 1 and the wireless LAN device 2 via the communication line 4, and a data signal is transmitted between the wireless LAN device 2 and the terminal device 3 via the wireless LAN line 5. In the following, the distribution method of the present invention will be described by taking multicast distribution as an example, but it can also be applied to unicast distribution.

  Multicast distribution is the transmission of the same data by designating multiple parties in the network. If data is sent once to multiple parties, the router on the communication path automatically depends on the other party. In this method, data is copied and distributed from one multicast server 1 to a large number of terminal devices 3. In broadcast-type video distribution or the like, a multicast method is often employed in which the same video can be transmitted to a plurality of users in one transmission, and the distribution band on the server side is not compressed even if the number of destinations increases. However, in the transmission method (UDP) based on multicast, since transmission is performed only once in one direction and retransmission control is not performed, transmission packet loss frequently occurs due to the influence of radio wave interference. In particular, in a video distribution system, there are frequent problems that the video is destroyed and cannot be viewed, or that the playback decoder stops with an error.

  The WiFi (registered trademark) wireless LAN is standardized by IEEE 802.11, and the usable radio frequency bands are 2.4 GHz band and 5 GHz band, and some of the current wireless LAN devices support both. In general, compatible devices in the 2.4 GHz band are the mainstream. However, as described above, the WiFi (registered trademark) wireless LAN using the 2.4 GHz band has a problem of radio wave interference with a microwave oven, an amateur radio, or the like. The transmission packet distribution method of the present invention is not limited to using WiFi (registered trademark) wireless LAN, and may be a wired communication path in addition to other wireless communication paths. In addition to such a local area communication network, an Internet line or NGN (Next Generation Network) may be used.

  The current network protocol uses IPv4 and IPv6, and there are various documents in addition to the RFC791 (Ipv4) and RFC2420 (Ipv6) specifications for the respective protocols and formats, and the respective multicast transfer technologies. Has also been established. In this example, H.264 is stored in the MPEG-TS container as multicast video. FIG. 2 shows a packet configuration in which a stream configuration using H.264 video and AAC audio is applied and the above-described IP protocol is applied.

  That is, the original content stored in the multicast server 1 of FIG. 1 is stream data composed of a plurality of IP packets as shown in FIG. 2, and one IP packet includes an IP header and an IP payload, and one IP packet The payload includes a UDP header and a UDP payload, and one UDP payload is configured to include one or a plurality of TS packets. TS packets are composed of MPEG-TS streams. In the terminal device 3, the MPEG-TS stream is finally extracted from the IP packet distributed from the multicast server 1 and reproduced, so that video, audio, subtitles, and the like can be viewed. In the following description, the configuration of the UPD header and the UPD payload is omitted for convenience.

  Various methods have been devised for multicast packet loss, and typical ones are listed below, but each has its own problems.

  1) The packet retransmission request method is a method of re-receiving a lost packet by a retransmission request of a lost packet by unicast when a packet loss is detected. The terminal side for detecting a packet loss and requesting a retransmission by unicast In addition, the processing on the server side becomes complicated, and there is a problem that a delay time occurs from the retransmission request to the reception of the retransmission packet.

  2) The transmission method using a plurality of communication paths is a method in which the same multicast packet is transmitted from a plurality of wireless LAN devices, and when there is a packet loss, correction is performed using normal packets from other wireless LAN devices. However, if multiple wireless LAN devices are required, the connected wireless LAN devices must be monitored and switched from time to time, the system scale will increase, and it may not be suitable for consumer use. There is a problem that the playback image is broken and cannot be viewed.

  3) The transmission method using a plurality of multicast groups (multicast addresses) is a method in which the same multicast packet is transmitted by a plurality of multicast groups, and when there is a packet loss, the normal packets of other multicast groups are corrected. Alternatively, when the same contents of a plurality of multicast groups with different encoding conditions (for example, bit rate) are transmitted and packet loss occurs frequently, for example, a method of switching to a low bit rate multicast group is used. However, there is a problem that processing for detecting a packet loss and switching between multicast groups is required, and a plurality of multicast groups are required, which complicates the processing on the server side and the terminal side. Therefore, if the content is increased, it is necessary to further increase the multicast group. Furthermore, when the packet correction cannot be performed, there is a problem that the reproduced image is broken and cannot be viewed.

  In view of this situation, in this example, a method for dealing with the loss of multicast packets under the following conditions is studied or completed and applied to the multicast video distribution system. That is, a simple system configuration that does not require the use of multiple wireless LAN devices and multiple multicast groups, and a complex processing system that uses a single multicast group without performing unicast switching or retransmission requests And a distribution method and system on the precondition that there is no influence on the reproduction video quality due to packet loss and that no noise or image corruption occurs in the reproduction video.

  As a method for dealing with the loss of the multicast packet described above, it is conceivable to multiplex the multicast packet and transmit a packet with a time difference. However, when an error occurs in a burst, there is still a problem that cannot be corrected. . In addition, when reproduction decoding is performed in a state where there is a packet loss (missing packet), it is conceivable that the image may be corrupted due to noise in the video, or that the reproduction decoder may stop in error in the worst case. In this example, a method of multiplexing IP packets in units of closed GOPs and further multiplexing with another stream to be transmitted with a time difference has been completed (hereinafter also referred to as hierarchical multiplexing method). In the following description, the embodiment of the present invention is described using content including video and audio. However, content including only audio can be distributed in the same configuration, and in the case of content including only audio. The following GOP may be replaced with a frame.

  A GOP (Group of Pictures) is a unit of one reproduction unit, and a closed GOP is a stream configuration in which the first picture of the GOP is always an Idr picture and there is no reference between GOPs. In other words, the closed GOP is reset in relation to the previous GOP at the beginning of the GOP, and normal reproduction can be performed with one GOP alone. FIG. 3 shows the GOP multiplexing configuration of this example, and a case where a stream of 1 GOP = 0.5 seconds will be described. The content described below is exemplified by the content stored in the storage device or the like of the multicast server 1 shown in FIG. 1, but the present invention also includes broadcast-type content such as a television, real-time video captured by a live camera, and the like. Include as.

In the content shown in FIG. 3, one IP packet is composed of, for example, 23 TS packets (188 bytes × 23 = 4324 bytes), and one GOP is composed of an IP packet of 0.5 seconds. And The GOP was extracted _one Nα from the beginning (or may be arbitrary start point GOP) of the content, it generates a Mβ pieces of multiplexed stream packet ST1 _one this Nα as a unit. For example, if Nα ₁ = 10 and Mβ = 2 are set for content consisting of 400 GOPs, this means that 10 GOPs are used as one unit to generate the same data two by two. In the stream packet ST1, two sets of 10 GOPs (= 20 GOPs) are sequentially generated up to 400.

FIG. 4 shows that the first multiplexed stream packet ST1 is set to Nα ₁ = 10, Mβ = 2 for the 800 GOPs (the first to 800th ordinal numbers are indicated by numbers), and the second multiplexed stream The packet ST2 shows a case where Nα ₂ = 400 and Mβ = 2 are set. In the first multiplexed stream packet ST1, 2 sets (= 20 GOPs) of 10 GOPs are sequentially generated up to 800, whereas in the second multiplexed stream packet ST2, 400 GOPs are 2 Each set (= 800 GOPs) is sequentially generated up to 800. Then, at the 400th and 800th GOPs, the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 are multiplexed and synchronized.

In this example, as shown in FIG. 1, in the multicast server 1, for the original content, Nα ₁ = 10 and Mβ = 2 are set to generate the first multiplexed stream packet ST1, and for the same original content, Thus, the second multiplexed stream packet ST2 is generated by setting Nα ₂ = 400 and Mβ = 2. Then, the first multiplexed stream packet ST1 is delayed from the second multiplexed stream packet ST2 by a transmission time of 400 GOPs and transmitted to the wireless LAN device 2 via the communication line 4, and further the terminal Transmit to device 3.

  When the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 are transmitted from the multicast server 1 to the wireless LAN device 2 via the communication line 4, the same multicast group A is used while the multicast address port is used. Change the number and transmit. In the example shown in FIG. 1, the first multiplexed stream packet ST1 is transmitted from the multicast address port 1, and the second multiplexed stream packet ST2 is transmitted from the multicast address port 2. In addition, when applying to unicast delivery, it transmits from one port.

  When transmitting the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 that are hierarchically multiplexed, the first multiplexed stream packet ST1 is transmitted with a time difference from the second multiplexed stream packet ST2, The GOPs affected by the occurrence of the burst error are shifted in each stream, and even if packet loss occurs in either stream, it is possible to correct each other. This effect will be described later.

  When the reception unit receives the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 from the wireless LAN device 2, the stream packets ST1 and ST2 are decoded and packet correction is performed. This process will be described with reference to FIG.

  In FIG. 5, the horizontal axis indicates the time axis, and the left end of the horizontal axis indicates the time t0 when the reproduction is started. It is assumed that the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 are always output, For example, it is assumed that the second multiplexed stream packet ST2 is mainly used. First, in GOP1 to 400 immediately after the start of reproduction on the left side (time t0 to t1), until the first multiplexed stream packet ST1 passes the delay time (corresponding to 400 GOPs), the first multiplexed stream packet ST1 Playback is performed only by GOP multiplexing decoding. This is because the second multiplexed stream packet ST2 has already been distributed at the time of playback activation. In parallel with this, in the GOP multiplexed decoded packet correction X2 of the second multiplexed stream packet ST2, the two packets β1 and β2 of the second multiplexed stream packet ST2 are added to the previously transmitted GOP 401 to 800. Used to perform packet correction in GOP units. Then, packet correction is performed for each GOP, and when there is no packet that cannot be corrected, the second multiplexed stream packet ST2 is used for reproduction. When there is a packet that cannot be corrected in the GOP, GOP As a packet loss, the GOP is discarded.

  At the next time (t1 to t2), the GOP multiplexed decoding packet correction X1 of the first multiplexed stream packet ST1 uses the two packets β1 and β2 of the first multiplexed stream packet ST1 to transfer to the GOPs 401 to 800. Packet correction is performed for each GOP. Then, packet correction is performed for each GOP, and when there is a packet that cannot be corrected in the GOP, the GOP is discarded as a GOP packet loss.

  Next, in the hierarchical multiplexing GOP decoding X3, when there is a GOP packet loss in the output stage (GOP 401 to 800) of the preceding second multiplexed stream packet ST2, the output of the delayed first multiplexed stream packet ST1 A normal GOP in stages (GOP 401 to 800) is applied to obtain a final output. If there is a GOP that cannot be corrected as a result of correction from the first multiplexed stream packet ST1 to the second multiplexed stream packet ST2 in GOP units, the GOP is discarded as a GOP packet loss. In this way, in this example, an uncorrectable GOP is discarded in units of GOPs, so that a video skip occurs in the lost GOP portion, but no malfunction or image destruction occurs during playback. This reproduction operation will be described later.

Here, the effect of the hierarchical multiplexed stream packet of this example will be described.
(1) Effects of GOP Multiplexing Decoding First, as shown in FIG. 6, 10 GOPs are set as one unit (Nα ₁ = 10) for the original content, and two multiplexing (Mβ = 2, β1 and β2) When the first multiplexed stream packet ST1 is generated, it is assumed that 10 or less GOPs have burst errors as shown in the lower left of FIG. If the first to sixth packets are lost, the 7th to 10th GOPs of β1 lost can be corrected by the 7th to 10th GOPs of β2, and 1 of β2 is lost. The sixth to sixth GOPs can be corrected by the first to sixth GOPs of β1.

On the other hand, as shown in the lower right of the figure, if 11 or more GOPs have a burst error, that is, the total of 16th to 20th of β1 and 11th to 17th of β2 is 12 in total. If 18 GOPs have lost packets, the 18th to 20th GOPs of β1 that have lost packets can be corrected by the 18th to 20th GOPs of β2, and 11th to 15th of β2 that has lost packets. The GOP can be corrected by the 11th to 15th GOPs of β1. However, the 16th to 17th GOPs of β1 that have lost packets cannot be corrected because the 16th to 17th GOPs of β2 are also lost. Therefore, even if a closed GOP is simply multiplexed, a burst error of Nα ₁ or less can be handled, but a burst error exceeding that cannot be handled. That is, the value of Nα ₁ is a tolerance value against the burst error.

(2) According to the effect of the layer multiplexing decoding the "(1) the effect of GOP multiplexing decoding", as a countermeasure against burst errors exceeding N.alpha ₁ value, the second multiplexed stream changing N.alpha ₁ value was further granted time difference The effect of hierarchical multiplexing decoding using packet ST2 is shown below. For the sake of convenience, one frame of the GOP in FIG. 7 represents 10 GOPs (for example, GOP = 10 shown in FIG. 7 indicates the 1st to 10th GOPs), and the number indicated is the final GOP. Number.

In the example shown in FIG. 7, the first multiplexed stream packet ST1 in which 10 GOPs are set as one unit (Nα ₁ = 10) and two multiplexing (Mβ = 2, β1 and β2) is performed on the original content. And 400 GOPs as a unit (Nα ₂ = 400) and two multiplexed stream packets ST2 that are multiplexed (Mβ = 2, β1 and β2) are generated for the original content, respectively. Assume that the first multiplexed stream packet ST1 is transmitted with a delay of 400 GOPs with respect to the second multiplexed stream packet ST2. That is, if the first to 400th GOPs of the first multiplexed stream packet ST1 are transmitted at time t0 to t1 shown in FIG. 7, the second multiplexed stream packet ST2 is 400 at the same time t0 to t1. One GOP is transmitted from the preceding 401st GOP (GOP indicated by 410). By leading the second multiplexed stream packet ST2 having a large Nα ₂ (≧ Nα ₁ ), in other words, by leading the stream packet having a higher burst tolerance, the other stream packet for a burst of Nα ₂ or less. This is because there is a high possibility that the correction can be performed with its own stream packet without correction by the above.

  For the first to 400th GOPs of the first multiplexed stream packet ST1, the first multiplexed stream packet ST1 is directly decoded and used for the reproduction process, but the 401st to 800th of the first multiplexed stream packet ST1. For the second GOP, when packet loss occurs in 400 or less GOPs, the GOP is corrected using the 401st to 800th GOPs of the second multiplexed stream packet ST2 transmitted prior to this. For example, when packet loss occurs in the 411st to 780th 360 (≦ 400) GOPs of the first multiplexed stream packet ST1 as shown at times t1 to t2 in the same figure, it is shown in the lower part of the figure. Packet loss is compensated using 360 GOPs of the 411st to 780th of the second multiplexed stream packet ST2. Alternatively, the 401st to 800th GOPs of the second multiplexed stream packet ST2 that have been decoded during the time t0 to t1 and have no packet loss may be directly subjected to the reproduction process.

If the packet loss of the first multiplexed stream packet ST1 is equal to or less than the burst error tolerance value Nα ₁ = 10, 100% decoding is possible with the stream of the first multiplexed stream packet alone. Also, the burst error tolerance values Nα ₁ and Nα ₂ are made different, and the two first multiplexed stream packets ST 1 and ST ₂ having different burst error tolerances are hierarchically multiplexed with a time difference, thereby decoding each other. Correction is possible.

Furthermore, according to the method of this example, the possibility of decoding is increased even for burst errors exceeding the maximum burst error tolerance values Nα ₁ and Nα ₂ . For example, as shown at times t1 to t3 in FIG. 7, it is assumed that the 1181st to 1600th 420 (≧ 400) GOPs of the second multiplexed stream packet ST2 have lost packets.

  In this case, the 1181st to 1600th 420 GOPs of the second multiplexed stream packet ST2 in which the packet is lost are 1181st to 1st of the first multiplexed stream packet ST1 transmitted at the next timing t2 to t3. If the 1600th 420 GOPs can be decoded (if no packet loss occurs), this can be compensated.

Thus, according to the method of this example, the possibility of decoding is increased even for burst errors exceeding the maximum burst error tolerance values Nα ₁ and Nα ₂ . In the example shown in FIG. 7, Nα ₁ = 10 and Nα ₂ = 400 are used as burst error tolerance values. However, when the maximum burst error tolerance value Nα ₂ = 400 and GOP = 0.5 seconds, the actual playback time Therefore, it is possible to decode a burst error of 200 seconds or less.

  Returning to FIG. 1, in the example shown in FIG. 1, the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 that are hierarchically multiplexed are transmitted from two different multicast address port numbers 1 and 2 in the same multicast group. However, the number of ports and the number of channels are not limited, and a plurality of ports and a plurality of channels can be applied. Further, it is possible to cope with two channels shown in FIG. 8 and four channels shown in FIG.

  In the transmission form shown in FIG. 8, two contents of a first channel content and a second channel content are prepared, and the first multiplexed stream packet ST1 and the second multiplexed stream are prepared from each content in the same manner as in the above-described example. A packet ST2 is generated. In the first channel, one of the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 (the first multiplexed stream packet ST1 in the example shown in the figure) is transmitted for 400 GOPs. The data is transmitted from two different multicast address ports 1 and 2 to the communication line 4 with a delay of only a delay. At the same time, in the second channel, one of the first multiplexed stream packet ST1 and the second multiplexed stream packet ST2 (first multiplexed stream packet ST1 in the example shown in the figure) is transferred to 400 GOPs. The transmission is delayed by the transmission time and transmitted from two different multicast address ports 3 and 4 to the communication line 4.

  On the other hand, in the transmission form shown in FIG. 9, four contents of contents for the first to fourth channels are prepared, and the first multiplexed stream packet ST1 and the second multiplexed are prepared from each content in the same manner as in the above-described example. A stream packet ST2 is generated. In this example, four first multiplexed stream packets ST1 and four second multiplexed stream packets ST2 are generated. Then, one multicast port address is used for two channels, and is transmitted at twice the timing of the transmission form shown in FIG.

  That is, from the multicast address port 1, the first multiplexed stream packet ST1 for the first channel and the first multiplexed stream packet ST1 for the second channel are delayed by the transmission time of 400 GOPs. 4 is transmitted. From the multicast address port 2, the second multiplexed stream packet ST2 for the first channel and the second multiplexed stream packet ST2 for the second channel are transmitted to the communication line 4. Also, from the multicast address port 3, the first multiplexed stream packet ST1 for the third channel and the first multiplexed stream packet ST1 for the fourth channel are delayed by the transmission time of 400 GOPs, and the communication line 4 Transmit to. Further, the second multiplexed stream packet ST2 for the third channel and the second multiplexed stream packet ST2 for the fourth channel are transmitted from the multicast address port 4 to the communication line 4.

  Returning to FIG. 1, in order to facilitate the decoding process in the terminal device 3 when performing the hierarchical multiplexing of content in the multicast server 1, a hierarchical multiplexing header is added in the IP packet, and packet loss in units of GOPs You may comprise so that it may be used at the time of correction | amendment and GOP loss correction | amendment. FIG. 10 is a diagram illustrating an example in which a hierarchical multiplexing header is added to an IP packet.

  When the content is MPEG-TS, there is known a method of determining packet loss from the discontinuity of the “continuity counter” of the TS packet header, but this method merely indicates the loss of a TS packet. It is not suitable for packet loss correction and GOP correction. This is because the conventional method uses only the discontinuity information of the continuity counter, and when performing correction in units of GOP as in this example, considerable search processing is required, This is because there is a problem that the information on hierarchization and hierarchy cannot be acquired. For this reason, in this example, header information for hierarchical multiplexing is added to the IP packet as shown in FIG.

  As shown in FIG. 10, a hierarchical multiplexing header is attached to an IP packet, GOP NO indicating GOP number, pck Total indicating the number of IP packets in GOP, pck No indicating IP packet number in GOP, and total packet serial Each information of Pck serNO indicating the number, TS packet number in the IP packet, multicast address port number, and TS sync indicating the multiplexing number Mβ is stored in this order in a predetermined number of bytes. For example, in the IP packet 1 data of multiplexing β1 shown in the figure, since 1,20,1,1001,23,1,2 are recorded in the hierarchical multiplexing header, GOP number 1 is the IP packet. Including 20 packets, the first IP packet among them, the 1001st IP packet, 23 TS packets included in the IP packet, the multicast address port number is 1, and the multiplexing number Mβ is 2 Can be obtained.

  With such a packet configuration, it is possible to easily detect and correct packet loss in units of GOPs, and at the same time, it is possible to easily detect multiplexed streams β in units of multicast groups using multicast address port numbers. Multiplexing of GOP units by address can be realized. Further, by assigning the total packet serial number, it becomes the unique packet number of all the streams (β1 to β2), and it is possible to immediately detect where the loss has occurred.

  Returning to FIG. 1 again, a method of reproducing a stream in the terminal device 3 when packet loss occurs will be described.

  Now, when there is a packet loss during streaming of content, a method of discarding and reproducing the lost packet can be considered easily. However, when the lost packet in the GOP is discarded, the reference at the head of the next GOP There is a problem that the corrupted video is played until is reset. There is also a problem that the reproduction decoder aborts with an error. To deal with these problems, a measure of discarding in units of GOPs is conceivable. When the units are discarded in units of GOPs, the reference is reset in the next GOP, so that video corruption does not occur.

  However, if GOP (4) and GOP (5) are discarded as shown in FIG. 11A, the playback video is paused at the end of GOP (3) when the GOP is discarded, and the normal next This state is kept until the head of GOP (6) comes. For this reason, the viewer pauses at GOP (3) during playback, keeps that state for a while, and the video suddenly jumps when the top of GOP (6) comes and the playback resumes. Will be uncomfortable.

  On the other hand, in this example, when the GOP loss cannot be corrected even by the hierarchical multiplexing method, the GOP is discarded as described above. However, since it is possible to determine which GOP is lost and which next GOP is normal by the hierarchical multiplexing method, when a GOP loss occurs, as shown in FIG. 11B, the next normal GOP is placed at the head. Force the jump and then pause here. Then, the pause state is maintained until the next GOP playback start time is reached, and playback is started as it is when the playback start time is reached. As a result, since the playback is continued without skipping from the paused state (still image), the viewer's uncomfortable feeling is reduced. If the GOP interval is set to 0.5 seconds, for example, one second of video is lost due to two GOP losses, but since it jumps to the beginning of the next GOP just before the pause, Video skipping does not occur even after pausing for 1 second. Therefore, the uncomfortable feeling is reduced.

  Next, a subtitle display method in the terminal device 3 when packet loss occurs will be described. The method of multiplexing the subtitle stream to the MPEG-TS video stream is common, but here again, when packet loss occurs, as with the video stream, the subtitle data is not displayed, the problem is not disappeared, and the subtitle display is corrupted. Problems occur. In order to display and erase the subtitle packet in synchronization with the playback time of the corresponding video stream, the subtitle packet holds In Time (display start time) and Out Time (erasure time) information. Subtitles are displayed and erased in synchronization with the video according to the information. For example, as shown in FIG. 12A, assuming that GOP (4) and GOP (5) are discarded, if the Out Time information of subtitles is stored in GOP (4), the next normal GOP (6) The problem that subtitles do not disappear when playing.

  For this reason, in this example, when the GOP loss cannot be corrected even by the hierarchical multiplexing method, the GOP is discarded, but the following subtitle display control method is further added to the above-described stream reproduction method. Note that subtitle data is compatible with both bitmap format and text format.

  That is, as shown in FIG. 12B, the subtitle information of normal GOP (6) is searched. If there is In Time information of the next subtitle, the subtitle of GOP (3) is deleted, and the next subtitle is displayed in In Time. To do. If there is no next subtitle information, it is cleared by the timeout value.

  Incidentally, when a subtitle stream is multiplexed with an MPEG-TS video stream at the time of content creation, a general encoder and a TS multiplexing device (regardless of hardware and software) have a timing for displaying subtitles. Subtitle streams are intensively arranged for GOP. Therefore, when multicast distribution is performed, as shown in FIG. 13A, only a specific portion (GOP (6)) protrudes to increase the bit rate, and there is a problem that packet loss occurs intensively. In unicast distribution, although there is no influence as much as multicast distribution, depending on the bandwidth of the communication path, an underrun of a reproduction buffer may occur in a portion where a subtitle stream exists, and video reproduction may be delayed.

  In order to solve such a problem, MPEG-TS subtitle packets may be average-multiplexed in this example. That is, as shown in FIG. 13B, subtitle packets arranged in a concentrated manner in a specific GOP are detected, a GOP temporally preceding the GOP of the arrangement is searched, and the bits of each GOP are shown in FIG. 13C. By rearranging the caption packets at a low rate, the overall bit rate is averaged as shown in FIG. 13D. Here, the search range of the rearranged GOP is 10 seconds in this example, but is not limited to this. That is, for each GOP in which a caption packet exists, a GOP in which no caption packet exists between them may be targeted. Further, the number of subtitle packets that need to be rearranged is detected from the GOP in which subtitle packets exist, the TS packet number is searched for each rearrangement destination GOP, and a TS packet number table for each GOP as shown in FIG. create. Based on the TS packet count table for each GOP, subtitle packets that need to be rearranged are smoothed and rearranged for each GOP. When subtitle packets are rearranged, the order of packets needs to be the same as the order of the original subtitle packets.

  In short, for the MPEG-TS standard stream packet including video data, audio data, and caption data, the caption data capacity included in each GOP constituting the stream packet is detected, and this exceeds a predetermined smoothness. Is a method of creating content in which the number of subtitle packets in each GOP is readjusted so as to be less than a predetermined smoothness.

  In addition, when a video, audio, subtitle, or other stream is multiplexed as an MPEG-TS video stream at the time of content creation, a general encoder and TS multiplexer (hardware and software) as described in the previous section are used. In any case, a packet is generated in units of 188 bytes of a TS packet when a TS packet is generated from a PES packet. Therefore, a fraction occurs at the end of the PES packet. In the existing technology, as shown in FIG. 14A, a method is known in which fractional parts are padded (Padding (Null Pack)) and multiplexed in units of 188 bytes.

  However, in multicast distribution where packet loss is a problem, it is important to reduce the bit rate as much as possible and minimize the impact of packet loss. Therefore, it is necessary to remove unnecessary and unnecessary data due to padding as much as possible. There is. Also in unicast delivery, it is desirable to reduce the bit rate as much as possible when the communication band is insufficient.

  For this reason, in this example, when multiplexing TS packets from PES packets, a method in which fractional padding is not generated as much as possible may be employed. That is, as shown in FIG. 14B, in the process of TS multiplexing from the PES packet, the size (P_end_size) of the fractional portion of the PES packet end is detected. Next, P_dest_size is calculated by subtracting the fraction (P_end_size) from the size (P_total_size) of the target PES packet (Audio packet in this example). Next, a bit rate (P_dest_rate) is calculated from P_dest_size and the actual playback time (t) of the PES packet. Next, the stream is re-encoded with P_dest_rate as the target bit rate, and a PES packet is generated again.

  The PES packet obtained as described above is smoothed by TS multiplexing alignment (TS packet payload 184 bytes), and padding is minimized. In this example, the Audio packet is taken as an example, but the present invention can also be applied to video, subtitle packets, and the like.

1 ... multicast server 2 ... WiFi wireless LAN
3 ... Terminal device 4 ... Communication line 5 ... Wireless LAN line

The present invention relates to a content distribution apparatus that multicasts or unicasts content stream data composed of closed GOPs including at least video data and audio data to a terminal via a communication path, and minimizes content stream data and closed GOPs. in units, beginning from the first multiplexed stream packets multiplexed N.alpha ₁ single closed GOP in order to a plurality of GOP groups in the one group, the head of a plurality of the _two closed GOP N.alpha in one group in order By hierarchically configuring with the second multiplexed stream packet multiplexed in the GOP group, by providing a time difference between the first multiplexed stream packet and the second multiplexed stream packet, multicast distribution or unicast distribution, Solve the above problems.
(Where Nα ₁ and Nα ₂ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ )

Claims (17)

In a method of multicast distribution or unicast distribution of content stream data composed of closed GOP including at least video data and audio data to a terminal via a communication path,
Converting said content stream data, and the closed GOP in one unit, in order from the beginning of _one of the closed GOP N.alpha to Mβ number of first multiplexed stream packets in one group,
Converting the content stream data into Mβ second multiplexed stream packets in which Nα ₂ closed GOPs are grouped in order from the top with closed GOP as one unit, and a transmission packet distribution method comprising: .
(However, Nα ₁ , Nα ₂ and Mβ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ ) 2. The transmission packet distribution method according to claim 1, wherein the first multiplexed stream packet and the second multiplexed stream packet are distributed with a time difference corresponding to the number of Nα ₂ closed GOPs. Distribution of the first multiplexed stream packet and the second multiplexed stream packet with a smaller number of closed GOPs (Nα ₁ , Nα ₂ ) in one group delayed by the number of GOPs than the other. The transmission packet delivery method according to claim 2. When multicasting the content stream data,
The transmission packet distribution method according to claim 2 or 3, wherein the first multiplexed stream packet and the second multiplexed stream packet are distributed with different multicast address port numbers through the same multicast group.   The transmission packet distribution method according to claim 4, wherein the first multiplexed stream packet and the second multiplexed stream packet are distributed over a plurality of channels. Decodes the first multiplexed stream packet transmitted to the terminal, if the GOP lost of _one of the closed GOP N.alpha is detected, a first correction step of correcting at another GOP of Mβ pieces ,
A second correction step of decoding the second multiplexed stream packet transmitted to the terminal and correcting with a different GOP of Mβ when a lost GOP is detected among Nα ₂ closed GOPs; ,
When an uncorrectable GOP is detected in the first correction step and the second correction step, the corresponding GOP of the first multiplexed stream packet or the second multiplexed stream packet distributed in advance in time is determined. The transmission packet delivery method according to claim 1, further comprising: a third correction step that uses and corrects the uncorrectable GOP.   7. The transmission packet distribution method according to claim 6, wherein when an uncorrectable GOP is detected in the third correction step, the GOP is discarded as a loss.   When an uncorrectable GOP is detected in the third correction step and the content stream data in which the GOP is discarded as a loss is reproduced, the GOP discarded after the GOP before the discarded GOP has been reproduced. The distribution method according to claim 7, further comprising a step of shifting to a head of a GOP next to the GOP and pausing until reaching a reproduction start time. The content stream data includes subtitle data and subtitle header information consisting of text data or image data,
Retrieving subtitle header information of the previous GOP and the next GOP of the discarded GOP,
When subtitle insertion information is detected in both the preceding and following GOPs, the subtitle data of the GOP before the discarded GOP is deleted and the subtitle data of the next GOP is displayed according to the subtitle header information.
The transmission according to claim 7 or 8, wherein subtitle insertion information is detected in a GOP before the discarded GOP, and if no subtitle insertion information is detected in a GOP next to the discarded GOP, the transmission is cleared according to a timeout value. Packet delivery method.   In each GOP of the first multiplexed stream packet and the second multiplexed stream packet, the GOP order, the number and order of IP packets included in the GOP, and the total IP packets included in the content stream data The transmission packet delivery method according to any one of claims 1 to 9, further comprising a step of adding a header including an order, the number of TS packets included in the GOP, a multicast address port number, and a multiplexing number Mβ. In a distribution server that multicasts or unicasts content stream data composed of closed GOP including at least video data and audio data to a terminal via a communication path,
The content stream data, and a closed GOP to one unit, a first converting means for converting the N.alpha ₁ single closed GOP in order to Mβ number of first multiplexed stream packets in a group from the beginning,
A transmission packet comprising: second conversion means for converting the content stream data into Mβ second multiplexed stream packets in which Nα ₂ closed GOPs are grouped in order from the top with closed GOP as one unit Distribution server.
(However, Nα ₁ , Nα ₂ and Mβ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ ) In a program for multicasting or unicasting content stream data composed of closed GOP including at least video data and audio data to a terminal via a communication path,
Converting said content stream data, and the closed GOP in one unit, in order from the beginning of _one of the closed GOP N.alpha to Mβ number of first multiplexed stream packets in one group,
Converting the content stream data into Mβ second multiplexed stream packets in which Nα ₂ closed GOPs are grouped in order from the top with closed GOP as a unit, and causing the computer to execute Transmission packet distribution program.
(However, Nα ₁ , Nα ₂ and Mβ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ ) The content stream data composed of a closed GOP including at least video data and audio data transmitted to the terminal device is converted, and the closed GOP is set as one unit, and Nβ1 closed GOPs are grouped in order from the head. Decoding first first multiplexed stream packets, and if a lost GOP is detected among Nα1 closed GOPs, a first correction step of correcting with other GOPs of Mβ,
The content stream data transmitted to the terminal device is converted, and Mβ second multiplexed stream packets in which Nα2 closed GOPs are grouped in order from the top in units of closed GOPs as a unit are decoded, If a lost GOP is detected among Nα2 closed GOPs, a second correction step of correcting with other GOPs of Mβ;
When an uncorrectable GOP is detected in the first correction step and the second correction step, the corresponding GOP of the first multiplexed stream packet or the second multiplexed stream packet distributed in advance in time is determined. A transmission packet decoding correction program for causing a computer to execute a third correction step for correcting the uncorrectable GOP. In a method for multicast distribution or unicast distribution of content stream data composed of frames including audio data to a terminal via a communication path,
And converting the content stream data, and the frame in one unit, in order from the beginning of _one frame Nα the Mβ number of first multiplexed stream packets in one group,
Wherein the content stream data, and the frame in one unit, distribution method of the transmission packet comprising the step of converting from the head in order to N.alpha ₂ frames to Mβ number of the second multiplexed stream packets in one group, the.
(However, Nα ₁ , Nα ₂ and Mβ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ ) In a distribution server that multicasts or unicasts content stream data composed of frames including audio data to a terminal via a communication path,
The content stream data, a first conversion means for converting the frame into one unit, in order from the beginning of _one frame Nα the Mβ number of first multiplexed stream packets in one group,
Transmission packet comprising: second conversion means for converting the content stream data into Mβ second multiplexed stream packets in which Nα ₂ frames are grouped in order from the top in units of frames as one unit. server.
(However, Nα ₁ , Nα ₂ and Mβ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ ) In a program for multicasting or unicasting content stream data composed of frames including audio data to a terminal via a communication path,
And converting the content stream data, and the frame in one unit, in order from the beginning of _one frame Nα the Mβ number of first multiplexed stream packets in one group,
A transmission packet for causing the computer to execute the step of converting the content stream data into Mβ second multiplexed stream packets in which Nα ₂ frames are grouped in order from the top in units of frames. Delivery program.
(However, Nα ₁ , Nα ₂ and Mβ are natural numbers, Nα ₁ <Nα ₂ and Nα ₂ is a multiple of Nα ₁ ) The content stream data composed of frames including audio data transmitted to the terminal device is converted, and Mβ first multiplexed streams in which Nα1 frames are grouped in order from the top in units of frames. A first correction step of decoding the packet and correcting for other frames of Mβ if a lost frame is detected among the Nα1 frames;
The content stream data transmitted to the terminal device is converted, and Mβ second multiplexed stream packets in which Nα2 frames are grouped in order from the beginning in units of frames are decoded, and Nα2 pieces are decoded. When a lost frame is detected among the frames, a second correction step of correcting with other frames of Mβ,
When an uncorrectable frame is detected in the first correction step and the second correction step, the corresponding frame of the first multiplexed stream packet or the second multiplexed stream packet distributed in advance in time is selected. A transmission packet decoding correction program for causing a computer to execute a third correction step using and correcting the uncorrectable frame.

Images