JP7569195B2 - Transmitting device and receiving device - Google Patents

Description

The present invention relates to a transmitting device and a receiving device, and in particular to a transmitting device and a receiving device that transmit and receive video and audio content in a broadcasting system using MMT (MPEG Media Transport).

In MMT defined in Non-Patent Document 1, the processing unit of video and audio codec is called MPU (Media Processing Unit). The first data of MPU needs to be a random access point that can be processed without depending on data transmitted in the past. In Non-Patent Document 2, which defines the use of MMT in broadcasting, a GOP (Group Of Picture) in which an intra frame (a frame that performs intraframe coding) of video coding is the first in the decoding order is treated as an MPU.
The term GOP is not used in the High Efficiency Video Coding (HEVC) standard, which is used as an example of a video coding method. However, following the example of conventional methods such as MPEG-2 Video, a set of frames starting with an intraframe may be referred to as a GOP for convenience.

According to Non-Patent Document 1, MPU is originally defined as being based on the ISOBMFF (ISO Base Media File Format). Non-Patent Document 1 also defines a method of transmitting the metadata portion of ISOBMFF as MPU metadata and movie fragment metadata using MMTP (MMT Protocol). However, in the broadcasting applications defined in Non-Patent Document 2, the generation and transmission of MPU metadata and movie fragment metadata is omitted in order to reduce processing latency, and the NAL (Network Abstraction Layer) units generated by the HEVC encoder are multiplexed directly into MMTP/UDP (User Diagram Protocol)/IP (Internet Protocol) packets and transmitted.

The basic data structure of ISOBMFF is specified in Non-Patent Document 3, and as an extension of that, for example, Non-Patent Document 4 specifies the operation method and extended data structure of MPEG-DASH (Dynamic Adaptive Streaming over HTTP), a video streaming delivery method using HTTP (Hypertext Transfer Protocol)/TCP (Transmission Control Protocol) communication. HLS (HTTP Live Streaming), specified in Non-Patent Document 5, is used as a video streaming delivery method that uses HTTP/TCP like MPEG-DASH. The ISOBMFF-based segment format that can be used in common with MPEG-DASH and HLS is specified as CMAF in Non-Patent Document 6. CMAF is a new standard that was formulated more recently than Non-Patent Document 1 and Non-Patent Document 2. In addition to standardizing the segment format, CMAF defines a chunk structure that further subdivides the segment structure as a technology to reduce the delay of video streaming delivery.

In Non-Patent Document 2, in order to reduce the delay of video and audio content transmission in broadcasting, the generation of metadata in the ISOBMFF structure and the transmission as movie fragment metadata are omitted. However, if the transmission of the movie fragment metadata is omitted, there is a problem that the DTS-PTS difference information (dts_pts_offset) cannot be transmitted to the decoder of the receiving device. This DTS-PTS difference information is information indicating a difference value between a DTS (Decoding Timestamp) that indicates the decoding timing of a video frame and a PTS (Presentation Timestamp) that indicates the presentation timing of a video frame, which is associated with the inter-frame reference structure of video coding.
Therefore, Non-Patent Document 2 defines an "MPU extended time stamp descriptor" that separately describes information such as this DTS-PTS differential information, and specifies that it is transmitted as a descriptor in the MMT Package Table (MP table), which is control information. The MPU extended time stamp descriptor lists the DTS-PTS differential information of all frames in the MPU (GOP).
In addition, in the bit stream of encoded video, video frames are encoded in the order in which the decoding process is performed in the receiving device. Therefore, the DTS indicating the decoding timing can be easily calculated by adding up the reciprocal of the frame rate in the receiving device, even if it is not explicitly transmitted from the transmitting device to the receiving device. For example, for a video of 60 frames per second, the DTS of each video frame can be calculated by adding 1/60th of a second at regular intervals in the order of reception.

On the other hand, Non-Patent Document 7 specifies a more detailed operation method for the use of MMT specified in Non-Patent Document 2 for actual services (advanced wideband digital satellite broadcasting). According to Non-Patent Document 7, it is required that the MPU extended time stamp descriptor be transmitted earlier than the corresponding MPU. However, the MPU extended time stamp descriptor can only be generated after the video coding of the entire MPU is completed. In order to avoid such video delay, it is possible to use ISOBMFF metadata corresponding to the chunk structure newly specified in CMAF in MMT as well. Since the CMAF chunk has a structure based on ISOBMFF, it can be multiplexed in MMTP. At this time, the MPU (GOP) is further divided into several frames, which are configured as chunks. Then, by describing the DTS-PTS differential information by ISOBMFF metadata on a chunk-by-chunk basis and transmitting it as MMTP movie fragment metadata, the necessary DTS-PTS differential information can be transmitted to the decoder without using the MPU extended time stamp descriptor. Specifically, the sample_composition_time_offset in TrackFragmentRunBox ('trun') can be used to describe the DTS-PTS differential information for each frame in the chunk.

In the existing technology, the presentation timing of each MPU is transmitted by the MPU timestamp descriptor (Non-Patent Document 1) as the absolute time of the presentation timing of the head of the MPU, and this time is used for video and audio presentation control in the receiving device. The MPU timestamp (mpu_presentation_time) described in the MPU timestamp descriptor indicates the presentation time of the MPU in 64-bit UTC (Coordinated Universal Time) time in NTP (Network Time Protocol) format. Therefore, when a leap second is inserted or deleted, discontinuity occurs such as rewinding the MPU timestamp. As a countermeasure against this, it has been decided to transmit a 2-bit leap second indicator (mpu_presentation_time_leap_indicator; hereinafter abbreviated as Leap Indicator) in the MPU extended timestamp descriptor. The leap second indicator (Leap Indicator) indicates which of the following (1) to (3) the MPU timestamp (UTC time) of the MPU associated with the same MPU sequence number described in the MPU timestamp descriptor is.
(1) The time before the leap second is inserted on the day (UTC standard) when the leap second is inserted is used as the base (Leap Indicator = 1),
(2) The time before the leap second is deleted on the day (UTC basis) on which the leap second is deleted is used as the base (Leap Indicator = 2),
(3) The time is based on a day without a leap second (UTC standard) or on the time after a leap second has been inserted or deleted (Leap Indicator = 0).

A day according to UTC corresponds to the period from 9:00:00 a.m. Japan time to 8:59:59 a.m. the following day. When a leap second is inserted, 8:59:60 a.m. is inserted after 8:59:59 a.m. Japan time, lengthening the day by one second. On the other hand, when a leap second is deleted, 8:59:59 a.m. is deleted, shortening the day by one second.

“Information technology - High efficiency coding and media delivery in heterogeneous environments: MPEG media transport”, ISO/IEC 23008-1 "Media transport method using MMT in digital broadcasting (ARIB STD-B60)" “ISO/IEC base media file format”, ISO/IEC 14496-12 “Dynamic adaptive streaming over HTTP (DASH) - Part 1: Media presentation description and segment formats”, ISO/IEC 23009-1 “HTTP Live Streaming”, [online], [searched on July 7, 2020], Internet <URL: https://developer.apple.com/streaming/> “Common media application format (CMAF) for segmented media”, ISO/IEC23000-19 "Advanced Wideband Satellite Digital Broadcasting Operational Standards (Volume 3) (ARIB TR-B39)"

However, if DTS-PTS differential information is sent using ISOBMFF metadata and the MPU extended timestamp descriptor is not used, it will not be possible to transmit auxiliary information (leap second indicator) regarding the "leap second" in the MPU extended timestamp descriptor.

The present invention was made in consideration of the above problems, and aims to provide a transmitting device and a receiving device that can deal with leap seconds without using the MPU extended timestamp descriptor.

In order to solve the above problem, the transmission device according to the present invention is a transmission device that transmits an MMT (MPEG Media Transport) signal in which an encoded video signal, a timestamp descriptor assigned to an MPU (Media Processing Unit), and metadata are multiplexed, and includes a TAI clock reproduction unit that communicates with a time server that provides TAI (International Atomic Time) time and reproduces a counter value that represents the TAI time, a video encoder unit that performs encoding of video frames and outputs an encoded video signal and a GOP (Group Of Picture) start trigger signal, a PTS determination unit that determines an MPU timestamp expressed in TAI time as a PTS (Presentation Timestamp) to be assigned to an MPU in accordance with the GOP start trigger signal from the video encoder unit based on the counter value that represents the TAI time reproduced by the TAI clock reproduction unit, and a UTC (Coordinated Universal Time) timestamp expressed in seconds at the same timing as when the MPU timestamp expressed in TAI time was assigned to the MPU. a descriptor generation unit that generates, as the timestamp descriptor, a descriptor including a UTC-TAI offset indicating the difference between the UTC (Decoding Timestamp) time and the TAI time; an ISOBMFF metadata generation unit that acquires DTS-PTS differential information, which is information indicating the differential value between the DTS (Decoding Timestamp) and the PTS, as picture reorder information for each frame from the video encoder unit, and generates ISO Base Media File Format (ISOBMFF) metadata including the DTS-PTS differential information as the metadata, in units of the MPU or in units of chunks obtained by dividing the MPU; and an MMT multiplexing unit that multiplexes the encoded video signal, the timestamp descriptor assigned to the MPU, and the metadata including the DTS-PTS differential information to generate the MMT signal.

The receiving device according to the present invention is a receiving device that receives an MMT signal transmitted from a transmitting device, and includes a UTC clock reproduction unit that communicates with a time server that provides UTC time to reproduce a counter value representing UTC time, a descriptor analysis unit that extracts an MPU timestamp expressed in TAI time in a specified MPU and a UTC-TAI offset from a timestamp descriptor multiplexed and transmitted on the MMT signal, a TAI offset addition unit that converts the counter value into a counter value representing TAI time by adding the UTC-TAI offset to the counter value representing UTC time reproduced by the UTC clock reproduction unit, and a picture of each frame from metadata multiplexed and transmitted on the MMT signal. The system is configured to include an ISOBMFF metadata analysis unit that extracts DTS-PTS differential information, which is information indicating the difference between the DTS and PTS as picture reorder information; a PTS calculation unit that calculates the PTS expressed in the TAI time of each frame from the MPU timestamp expressed in the TAI time and the DTS-PTS differential information of each frame extracted from the ISOBMFF metadata analysis unit; and a picture reorder buffer that receives as input a video frame obtained by decoded the video signal multiplexed and transmitted in the MMT signal, compares the counter value representing the TAI time converted by the TAI offset addition unit with the PTS expressed in the TAI time of each frame, and rearranges and outputs the video frames as necessary.

The present invention provides the following excellent effects.
The transmitting device can transmit the presentation time of the MPU by using the MPU timestamp expressed in TAI time. In addition, according to the transmitting device, the UTC-TAI offset indicating the difference between the UTC time and the TAI time at the same timing as the MPU timestamp expressed in TAI time is given to the MPU can be transmitted by the MMT signal. Since the MPU timestamp expressed in TAI time simply increases regardless of the insertion/deletion of leap seconds, the receiving device receiving this MMT signal can control the presentation of video, audio, etc. by using the TAI time as the reference clock without necessarily having to perform control that takes into special consideration the insertion/deletion of leap seconds. In addition, the receiving device can convert the UTC time acquired from a time server that provides the UTC time into TAI time by utilizing the UTC-TAI offset extracted from the timestamp descriptor that has been multiplexed and transmitted in the MMT signal, and use it for the presentation control of video, audio, etc.

1 is a block diagram showing an overall configuration of a transmission system according to a first embodiment of the present invention. A data structure diagram showing the structure of an MPU timestamp descriptor in a comparative example. A data structure diagram showing the structure of a TAI-MPU timestamp descriptor in an embodiment. 1 is an explanatory diagram showing the relationship between UTC time (NTP format) and TAI time (PTP format) and the insertion of leap seconds. 1 is a block diagram illustrating a configuration of a transmission device according to a first embodiment of the present invention. 4 is a flowchart showing an operation of the transmitting device according to the first embodiment of the present invention. 1 is a block diagram illustrating a configuration of a receiving device according to a first embodiment of the present invention. 4 is a flowchart showing an operation of the receiving device according to the first embodiment of the present invention. FIG. 11 is a block diagram illustrating a configuration of a transmitting device according to a second embodiment of the present invention.

Hereinafter, an embodiment of a transmitting device and a receiving device according to the present invention will be described.
First Embodiment
For convenience of explanation, the following sections will be described: 1. Overall configuration of a transmission system, 2. Time stamp descriptor assigned to an MPU, 3. UTC-TAI offset, 4. Configuration of a transmitting device, 5. Operation of a transmitting device, 6. Configuration of a receiving device, and 7. Operation of a receiving device.

[1. Overall configuration of the transmission system]
First, the overall configuration of a transmission system according to the first embodiment will be described with reference to FIG.
The transmission system 1 includes a transmitting device 300 and a receiving device 500 .
The transmitting device 300 encodes and transmits the video and audio signals input from the video and audio signal source 100. As an example, the transmitting device 300 encodes the video using HEVC, which is a video encoding method, and encodes the audio using AAC, which is an audio encoding method, and multiplexes and transmits them using MMT. In MMT, an MPU timestamp of absolute time is assigned to the MPU, which is a processing unit of encoded video and audio data, and is transmitted. The transmitting device 300 uses the TAI time server 200 as a reference clock to refer to TAI (International Atomic Time) time, and transmits the presentation time of the MPU of video and audio, etc. based on the TAI time, and the difference information between the UTC time and the TAI time at the same timing as the presentation time, in a descriptor of the MMT control information. In the following, this descriptor will be referred to as a TAI-MPU timestamp descriptor.

The video/audio signal source 100 is an electronic device, a server, etc. that stores contents such as video signals and audio signals. For example, a video signal source that stores video signals is a video camera, a video server, etc.
The TAI time server 200 is a time server that acquires time information from a GPS or the like and distributes the TAI time to various devices via a network. In this embodiment, the TAI time server 200 is, for example, a PTP (Precision Time Protocol) server. The PTP server also distributes difference information between UTC time and TAI time, so that various devices can also obtain UTC time. Note that there are two types of PTP servers: a grandmaster that directly references a signal source such as a GPS, and a slave that synchronizes with the grandmaster, but either type may be used.

The transmission path 400 shown in FIG. 1 is a schematic representation of broadcasting, the Internet, and the like. Here, broadcasting refers to, for example, satellite broadcasting, terrestrial broadcasting, cable broadcasting, and IP (Internet Protocol) multicast broadcasting. In other words, the transmitting device 300 transmits content using MMT (MPEG Media Transport), for example, satellite broadcasting, terrestrial broadcasting, cable broadcasting, and IP multicast broadcasting.

The receiving device 500 receives the MMTP packet sequence (MMT signal) sent by the transmitting device 300 through the transmission path 400, demultiplexes it, and decodes and plays back the video and audio of the broadcast program. The sent MMT signal multiplexes video and audio data and timestamp information of the TAI time. The receiving device 500 decodes the video and outputs it to the monitor 600, and also decodes the audio and outputs it to the speaker 700. In FIG. 1, the monitor 600 and the speaker 700 are illustrated outside the receiving device 500, but the monitor 600 and the speaker 700 may be built into the receiving device 500. Examples of the receiving device 500 include a general television, a smartphone, and a tablet. In FIG. 1, only one receiving device 500 is illustrated to make the drawing easier to see, but usually there are multiple receiving devices 500.

The receiving device 500 obtains the UTC time from a UTC time server 800 that exists on the Internet, etc. In this embodiment, the UTC time server 800 is, for example, an NTP server. Note that while a PTP server on the transmitting side is used as broadcasting station equipment or as a business server, PTP servers are not yet widespread on the receiving side, and NTP servers are generally used to set the time.

The receiving device 500 plays video, audio, etc. by referring to the UTC (Coordinated Universal Time, Universal Time) time synchronized with the UTC time server 800, the presentation time of the video and audio MPU based on the TAI time of the descriptor (TAI-MPU timestamp descriptor) received from the transmitting device 300, and the difference information between the UTC time and TAI time at the same timing as the presentation time.

The difference between TAI time and UTC time is that UTC time may repeat the same value due to the insertion of leap seconds, whereas TAI time is counted up regardless of the insertion of leap seconds. As described below, in this embodiment, the transmitting device 300 transmits a descriptor in which the presentation time of the MPU of video, audio, etc. is expressed in TAI time. On the other hand, the receiving device 500 generally refers to the UTC time provided by a UTC time server 800 such as an NTP server via a network, and thus could not obtain the TAI time in the past. In contrast, as described below, in this embodiment, the transmitting device 300 transmits the TAI-MPU timestamp descriptor including the UTC-TAI offset as difference information between the UTC time and the TAI time. This allows the receiving device 500 to convert the UTC time count value provided by the UTC time server 800 into a TAI time count value and use it as a reference clock, eliminating the need to perform control in the TAI offset addition unit that takes into account the existence of leap seconds.

In addition to UTC time, the UTC time server 800 provides the receiving device 500 with a leap second indicator, which notifies the user of the insertion or deletion of a leap second. The leap second indicator takes the value of 0, 1, or 2. A leap second indicator of 0 indicates that no leap second is planned to be inserted or deleted. A leap second indicator of 1 indicates that a leap second is planned to be inserted after the next 8:59:59 a.m. (Japan time). A leap second indicator of 2 indicates that a leap second is planned to be deleted after the next 8:59:58 a.m. (Japan time). The leap second indicator allows the receiving device 500 to know in advance the timing of the addition or deletion of a leap second.

The transmitting device 300 can obtain difference information between UTC time and TAI time from the TAI time server 200 such as a PTP server, and can easily write this information in the descriptor.
In the case of a PTP server, the difference between UTC time and TAI time resulting from the accumulation of leap second insertions and deletions made in the past is distributed as currentUTCOffset by an Announce Message. The value of this currentUTCOffset is 37 seconds as of June 2020. Furthermore, 10 seconds of this 37 seconds are the initial value when UTC was established in 1972. In other words, 27 seconds of the 37 seconds are the accumulated value resulting from the insertion of leap seconds 27 times in the past.

[2. Time stamp descriptor assigned to MPU]
Next, the TAI-MPU timestamp descriptor (see FIG. 3) generated by the transmitting device 300 according to this embodiment as a timestamp descriptor to be assigned to the MPU will be described in comparison with the existing MPU timestamp descriptor (see FIG. 2). FIG. 2 shows the data structure of the MPU timestamp descriptor defined in Non-Patent Document 1. Also, FIG. 3 shows an example of the data structure of the TAI-MPU timestamp descriptor according to this embodiment.

In the "Data Representation" section of Figures 2 and 3, uimsbf is an abbreviation for "unsigned integer most significant bit first," meaning "unsigned integer, most significant bit first." Also, simsbf is an abbreviation for "signed integer most significant bit first," meaning "signed integer, most significant bit first."

In the existing MPU timestamp descriptor shown in FIG. 2, "descriptor_tag" stores an identification value for identifying the type of descriptor.
"descriptor_length" stores a number indicating the byte size of the descriptor.
In the for loop, multiple combinations of "mpu_sequence_number" and "mpu_presentation_time" can be stored.
"mpu_sequence_number" stores the MPU sequence number that specifies the target MPU.
"mpu_presentation_time" stores the MPU timestamp in UTC time in NTP format.

The TAI-MPU timestamp descriptor shown in Fig. 3 is different from the MPU timestamp descriptor shown in Fig. 2 in that the for loop is partially different. The TAI-MPU timestamp descriptor has "tai_mpu_presentation_time" instead of "mpu_presentation_time". The MPU timestamp of the TAI time is stored in "tai_mpu_presentation_time". The TAI time can have various description formats, but in Fig. 3, the number of bits of "tai_mpu_presentation_time" is 64 bits as an example. When this 64-bit timestamp is used in the same way as the NTP format, the first 32 bits represent the number of seconds, and the latter 32 bits represent the number of digits below the decimal point (binary decimal).
As an example of another description format, a timestamp similar to the PTP format can be used, in which case it is 80 bits in total, with the first 48 bits representing the number of seconds and the latter 32 bits representing the number of decimal points (nanoseconds).

In addition, the TAI-MPU timestamp descriptor shown in FIG. 3 includes "utc_tai_offset" which was not included in the MPU timestamp descriptor.
In "utc_tai_offset", the difference between the UTC time and the TAI time expressed in seconds at the same timing when "tai_mpu_presentation_time" is added to the MPU is stored. In the following, this difference between the UTC time and the TAI time is called the UTC-TAI offset and will be described in detail.

[3. UTC-TAI Offset]
Next, the UTC-TAI offset will be described with reference to Figure 4. Figure 4 is a conceptual diagram of the insertion of a "leap second". On the horizontal axis of Figure 4, time T1 is 8:59:59 AM on the leap second insertion date, Japan time. Time T2 is 8:59:60 AM (leap second), Japan time, on the leap second insertion date. Time T3 is 9:00:00 AM on the leap second insertion date, Japan time. The vertical axis of Figure 4 shows the counter value from the starting date and time of UTC time (NTP format) and the counter value from the starting date and time of TAI time (PTP format).

As shown in FIG. 4, UTC time (NTP format) and TAI time (PTP format) each indicate time information based on a counter value from a certain starting date and time. Specifically, UTC time (NTP format) has a starting date and time of 0:00:00 (UTC) on January 1, 1900. The counter value of UTC time (NTP format) is the count of the elapsed time from 0:00:00 (UTC) on January 1, 1900.

The starting date and time for TAI time (PTP format) is 0:00:00 (UTC) on January 1, 1970. The counter value for TAI time (PTP format) is the count of the time elapsed since 0:00:00 (UTC) on January 1, 1970. Note that in this specification, the notation (UTC) indicates the absolute time of Coordinated Universal Time (Universal Time).

Unless otherwise specified, the term UTC time below refers to NTP format time information with the origin (epoch) being January 1, 1900, 0:00:00 (UTC). Also, the term TAI time below refers to PTP format time information with the origin (epoch) being January 1, 1970, 0:00:00 (UTC).

Regarding the insertion and deletion of a "leap second" applied to UTC time, deletion is also considered, but in the past, only insertion has been performed. In the case of inserting a "leap second", as shown by the thick dashed line in Figure 4, the counter value of UTC time is discontinuous at time T2, and rewinds by one second. On the other hand, as shown by the thick solid line in Figure 4, the counter value of TAI time does not rewind at time T2. Therefore, as shown in Figure 4, the UTC-TAI offset Δ2 after the leap second is inserted is one second smaller than the UTC-TAI offset Δ1 before the leap second is inserted.
In the following, for ease of understanding, the UTC-TAI offsets Δ1 and Δ2 before and after the insertion of the leap second will not be distinguished, and will simply be referred to as the UTC-TAI offset.

The UTC-TAI offset can be expressed as the difference in seconds between the integer part of the counter value of the UTC time (the most significant 32 bits of a 64-bit timestamp in an NTP format) and the integer part of the counter value of the TAI time (the most significant 48 bits of an 80-bit timestamp in a PTP format).
The UTC-TAI offset can be defined, for example, by the sum of (i) an offset value of 70 years due to the difference between the origin of UTC time and the origin of TAI time, and (ii) an offset value indicating the number of seconds accumulated by the insertion of leap seconds in the past. When defined as the sum of the offset value (i) and the offset value (ii) in this way, the UTC-TAI offset data can be expressed, for example, as a 32-bit integer. As a specific example, if (i) is -2208988800 (seconds) and (ii) is 37 (seconds), then the UTC-TAI offset transmitted is -2208988763 (seconds), which is the sum of (i) and (ii).

Note that since the offset value for 70 years in (i) above is a known fixed value, it is acceptable for the UTC-TAI offset value transmitted from the transmitting device 300 to the receiving device 500 to be only the offset value in (ii) above indicating the accumulated number of seconds of past leap second insertions, with the offset value in (i) being assumed to be already taken into account. As a specific example, the UTC-TAI offset value transmitted is 37 (seconds) in (ii) above.

The UTC-TAI offset must have a bit width sufficient to represent the difference, but is not limited to the 32-bit width mentioned above. If the UTC-TAI offset does not include the 70-year offset value in (i) above, but is only the offset value in (ii) above indicating the number of seconds accumulated from the insertion of past leap seconds, then a bit width of, for example, 8 or 16 bits is sufficient.

Furthermore, the positive or negative offset changes depending on whether UTC time or TAI time is used as the base time, but this can be determined by a common standard/operation so that the recognition is consistent between the transmitting device 300 and the receiving device 500, and the application of this embodiment is not limited to the positive or negative offset. For example, to convert UTC time to TAI time based on UTC time, it is necessary to add a negative offset to the UTC time, but it is also possible to transmit the absolute value of 2208988763 (seconds) as the UTC-TAI offset value, taking into account the possibility that the offset will be negative.

In addition, in one example format, the TAI time written in the transmitted descriptor is expressed as a counter value starting from 0:00:00 (UTC) on January 1, 1970, similar to the PTP format, but the applicability of this embodiment is not impaired in any way even if an expression starting from another date and time is used.

[4. Configuration of the transmitting device]
Next, a configuration example of the transmission device 300 will be described with reference to Fig. 5. In order to simplify the description, the description will be focused on video processing that requires picture reordering (rearrangement of the decoding order and display order of frames), and the description of audio processing will be omitted. It is assumed that the transmission device 300 is a HEVC/MMT encoder.

The transmitting device 300 transmits an MMT (MPEG Media Transport) signal that is a multiplexed combination of an encoded video signal, a timestamp descriptor assigned to an MPU (Media Processing Unit), and metadata, and includes a TAI clock recovery unit 310, a frame buffer unit 320, a HEVC encoder unit (video encoder unit) 330, an MMT multiplexing unit 340, a PTS determination unit 350, an MPU sequence number counter unit 360, a descriptor generation unit 370, and an ISOBMFF metadata generation unit 380.

The TAI clock regeneration unit 310 communicates with the TAI time server 200 (e.g., a PTP server) that provides TAI (International Atomic Time) time, and regenerates a counter value that represents the TAI time. The TAI clock regeneration unit 310 regenerates the counter value of the TAI time of the PTP packets that are discretely sent from the TAI time server 200 as a continuous one, and outputs this regenerated absolute value of the TAI time (reference clock within the device) to the PTS determination unit 350. In this embodiment, the TAI clock regeneration unit 310 also acquires and holds the UTC-TAI offset (currentUTCOffset) from the TAI time server 200.

The UTC-TAI offset used in the receiving device 500 is defined, for example, as the sum of the offset value for 70 years in (i) and the offset value indicating the number of seconds accumulated by the insertion of past leap seconds in (ii), and has a bit width of 32 bits. In this case, the TAI clock recovery unit 310 outputs to the PTS determination unit 350 a UTC-TAI offset having a value obtained by adding the offset value for 70 years in (i) to the number of seconds accumulated by the insertion of past leap seconds (currentUTCOffset) in (ii) obtained from the TAI time server 200. In addition, when the UTC-TAI offset is defined, for example, as an offset value indicating the number of seconds resulting from the insertion of past leap seconds (ii) above, the TAI clock recovery unit 310 outputs the number of seconds resulting from the insertion of past leap seconds (ii) (currentUTCOffset) obtained from the TAI time server 200 as the UTC-TAI offset directly to the PTS determination unit 350.

The frame buffer unit 320 is a memory that holds the video signal input from the video and audio signal source 100, and the held video signal is input to the HEVC encoder unit 330. In other words, the frame buffer unit 320 temporarily holds the video signal before it is input to the HEVC encoder unit 330.

The HEVC encoder unit 330 encodes a video frame and outputs an encoded video signal and a GOP (Group Of Picture) start trigger signal. The HEVC encoder unit 330 obtains video frames from the frame buffer unit 320 and encodes them. The HEVC encoder unit 330 determines various encoding modes, such as intraframe encoding and interframe reference encoding, for each frame and performs the encoding mode. For broadcasting purposes, intraframe encoding is performed at a fixed cycle of approximately 0.5 seconds.
The HEVC encoder unit 330 outputs the encoded video signal (HEVC encoded data) to the MMT multiplexing unit 340. In the MMT multiplexing unit 340, a GOP starting from an intra frame is treated as an MPU, so that the HEVC encoder unit 330 transmits a GOP start trigger to the PTS determination unit 350 and the MPU sequence number counter unit 360 at the timing of encoding the intra frame. The HEVC encoder unit 330 outputs DTS-PTS difference information to the ISOBMFF metadata generation unit 380 as picture reorder information of each frame.
The GOP start trigger signal is a trigger signal that the HEVC encoder unit 330 encodes a certain frame as an intraframe using only intraframe encoding, and notifies each unit in the transmission device 300 that a new MPU (GOP) will start. Note that the interface and data format are not limited for realizing the GOP start trigger signal. As an example, the HEVC encoder unit 330 may output the GOP start trigger signal at the time when the encoding mode of the frame being encoded is determined to be an intraframe. Also, the GOP start trigger signal can be realized by adding a minor function to an existing encoder. For example, the HEVC encoder unit 330 may be configured to include an existing encoder and an additional function unit that generates a GOP start trigger signal when it detects that the encoded data actually encoded as an intraframe has been output from the encoder. In this way, various implementations are possible for realizing the GOP start trigger signal.

The PTS determination unit 350 determines an MPU timestamp expressed in TAI time (hereinafter referred to as TAI-MPU timestamp) as a PTS (Presentation Timestamp) to be assigned to an MPU, based on a counter value representing the TAI time reproduced by the TAI clock reproduction unit 310, in accordance with a GOP start trigger signal from the HEVC encoder unit 330. The PTS determination unit 350 outputs the TAI-MPU timestamp to the descriptor generation unit 370. The PTS determination unit 350 also outputs UTC-TAI offset information of the same timing as that for determining the TAI-MPU timestamp to the descriptor generation unit 370.
If the PTS determination unit 350 uses the UTC time at which the GOP start trigger signal is received as the PTS of the MPU, an error may occur due to processing timing. In order to prevent such an error, in this embodiment, the PTS determination unit 350 determines the PTS to be assigned to the newly started MPU as follows. That is, the PTS determination unit 350 calculates the duration (time length) of the MPU immediately before the newly started MPU based on the number of frames in the MPU immediately before the newly started MPU. Then, the PTS determination unit 350 determines the value (UTC time) obtained by adding the duration (time length) of the immediately previous MPU to the PTS (UTC time) assigned to the immediately previous MPU as the PTS of the newly started MPU. That is, the time obtained by adding the duration of the previous MPU to the start time of the previous MPU is recursively repeated as the start time of the next MPU. In this way, the PTS determination unit 350 can calculate an accurate PTS and assign it to the MPU.

The MPU sequence number counter unit 360 increments the MPU sequence number for identifying each MPU by one. The MPU sequence number counter unit 360 holds the current value of the MPU sequence number, and in accordance with a GOP start trigger signal from the HEVC encoder unit 330, counts up the MPU sequence number to determine the MPU sequence number of a new MPU, and outputs the current value of the MPU sequence number to the descriptor generation unit 370.

The descriptor generation unit 370 generates a descriptor containing an MPU sequence number, an MPU timestamp (TAI-MPU timestamp) expressed in TAI time, and a UTC-TAI offset as a TAI-MPU timestamp descriptor (Figure 3). Here, the UTC-TAI offset indicates the difference between the TAI time and the UTC (Coordinated Universal Time) time expressed in seconds at the same timing when the TAI-MPU timestamp is assigned to the MPU. The TAI-MPU timestamp descriptor is multiplexed into the MMT control information and output to the MMT multiplexing unit 340.

The ISOBMFF metadata generating unit 380 acquires DTS-PTS differential information, which is information indicating a differential value between DTS (Decoding Timestamp) and PTS as picture reorder information of each frame, from the HEVC encoder unit 330, and generates ISOBMFF metadata including the DTS-PTS differential information as metadata for MPU units or chunk units obtained by dividing an MPU. Since reordering (reordering of the decoding order and the display order) occurs in the time direction for I (intra) frames, P (Predictive) frames, and B (Bidirectionally Predictive) frames whose order has been changed for encoding processing using inter-frame reference, the ISOBMFF metadata generating unit 380 generates ISOBMFF metadata of the DTS-PTS differential information as the reorder information. The generated metadata is multiplexed into a packet of movie fragment metadata and output to the MMT multiplexing unit 340.
Note that ISOBMFF metadata including DTS-PTS difference information can also be generated and transmitted in chunk units obtained by dividing an MPU (GOP). For example, when an MPU (GOP) is composed of 32 frames, 8 frames are treated as a chunk, and ISOBMFF metadata including DTS-PTS difference information for 8 frames can be generated and transmitted as movie fragment metadata.

The MMT multiplexing unit 340 multiplexes the video signal and various control information, and transmits the result as an MMT signal to the receiving device 500 via the transmission path 400. The MMT multiplexing unit 340 receives HEVC encoded data from the HEVC encoder unit 330, a TAI-MPU timestamp descriptor from the descriptor generating unit 370, and metadata from the ISOBMFF metadata generating unit 380. The MMT multiplexing unit 340 multiplexes the encoded video signal (HEVC encoded data), a timestamp descriptor (TAI-MPU timestamp descriptor) assigned to the MPU, and metadata including DTS-PTS difference information to generate an MMT signal.

5. Operation of the Transmitter
Next, with reference to FIG. 6 (and also with reference to FIGS. 1 and 5 as appropriate), the operation of the transmitting device according to the first embodiment of the present invention will be described.
The TAI clock reproducing unit 310 of the transmitting device 300 communicates with the TAI time server 200 to reproduce a counter value representing the TAI time (step S101). Meanwhile, the HEVC encoder unit 330 encodes an input video signal and outputs HEVC encoded data and a GOP start trigger signal.
Then, the PTS determination unit 350 determines a TAI-MPU timestamp based on the counter value representing the TAI time reproduced by the TAI clock reproduction unit 310, in accordance with the GOP start trigger signal from the HEVC encoder unit 330 (step S102).

Then, the descriptor generation unit 370 generates a TAI-MPU timestamp descriptor including the MPU sequence number, the TAI-MPU timestamp, and the UTC-TAI offset (step S103). Meanwhile, the ISOBMFF metadata generation unit 380 acquires DTS-PTS difference information as picture reorder information for each frame from the HEVC encoder unit 330, and generates ISOBMFF metadata including the DTS-PTS difference information.
Then, the MMT multiplexing unit 340 multiplexes the HEVC encoded data from the HEVC encoder unit 330, the TAI-MPU timestamp descriptor from the descriptor generation unit 370, and metadata including the DTS-PTS difference to generate an MMT signal, and transmits it from the transmission path 400 to the receiving device 500 (step S104).

[6. Configuration of Receiving Device]
Next, a configuration example of the receiving device 500 will be described with reference to Fig. 7. Note that, in order to simplify the description, the description will be focused on video processing that requires picture reordering (rearrangement of the decoding order and the display order), and a description of audio processing will be omitted.

The receiving device 500 is a receiving device that receives the MMT signal transmitted from the transmitting device 300, and includes a UTC clock recovery unit 510, an MMT demultiplexing unit 520, a descriptor analysis unit 530, a TAI offset addition unit 540, an ISOBMFF metadata analysis unit 550, a PTS calculation unit 560, a HEVC decoder unit (video decoder unit) 570, and a picture reorder buffer 580.

The UTC clock regeneration unit 510 communicates with a UTC time server 800 (e.g., an NTP server) that provides UTC time, and regenerates a counter value that represents UTC time. The UTC clock regeneration unit 510 regenerates the UTC time counter value as a clock and outputs it to the TAI offset addition unit 540. The UTC clock regeneration unit 510 also receives a leap second indicator provided by the UTC time server 800, which notifies the user of the insertion or deletion of a leap second.

The MMT demultiplexing unit 520 separates, for example, control information, MPU sequence number, metadata, and HEVC encoded data (encoded video signal) from the MMT signal input to this receiving device 500, and extracts the TAI-MPU timestamp descriptor from the control information. The MMT demultiplexing unit 520 outputs control information such as the TAI-MPU timestamp descriptor and the MPU sequence number to the descriptor analysis unit 530. The MMT demultiplexing unit 520 outputs the metadata to the ISOBMFF metadata analysis unit 550, and outputs the HEVC encoded data to the HEVC decoder unit 570.

The descriptor analysis unit 530 extracts an MPU timestamp (TAI-MPU timestamp) expressed in TAI time for a specific MPU, and a UTC-TAI offset from the TAI-MPU timestamp descriptor multiplexed and transmitted in the MMT signal. The descriptor analysis unit 530 analyzes the TAI-MPU timestamp descriptor obtained from the MMT multiplexing/demultiplexing unit 520 to identify the TAI-MPU timestamp and UTC-TAI offset of the current MPU. The descriptor analysis unit 530 outputs the UTC-TAI offset to the TAI offset addition unit 540. The descriptor analysis unit 530 outputs the TAI-MPU timestamp to the PTS calculation unit 560. The TAI-MPU timestamp stores the presentation time based on the absolute time of the first picture in the MPU.

The TAI offset addition unit 540 converts the counter value representing the UTC time reproduced by the UTC clock reproduction unit 510 into a counter value representing the TAI time by adding a UTC-TAI offset to the counter value. The TAI offset addition unit 540 adds the UTC-TAI offset obtained from the descriptor analysis unit 530 to the UTC time obtained from the UTC clock reproduction unit 510, and converts the clock into a clock expressed in absolute time of the TAI time. The TAI offset addition unit 540 outputs the TAI time clock to the picture reorder buffer 580.

Here, the UTC-TAI offset transmitted in the TAI-MPU timestamp descriptor is the UTC-TAI offset at the UTC time when the transmitting device 300 determines the PTS. Therefore, if there is a transmission delay between the transmitting device 300 and the receiving device 500, the following precautions must be taken. As shown in Figure 4, the UTC-TAI offset decreases when a leap second is inserted into the UTC time, while the UTC-TAI offset increases when a leap second is deleted (not shown). If there is no transmission delay, the timing of the insertion and deletion of a leap second into the UTC time in the receiving device 500 is equal to the timing at which the UTC-TAI offset transmitted in the TAI-MPU timestamp descriptor increases or decreases, for example. On the other hand, if there is a transmission delay, the timing of the increase or decrease of the UTC-TAI offset transmitted in the TAI-MPU timestamp descriptor will be delayed by the amount of the transmission delay relative to the timing of the insertion or deletion of leap seconds into the UTC time that the receiving device 500 directly references from the UTC time server 800, and a timing discrepancy may occur, resulting in incorrect conversion to TAI time.

To avoid this, the following method may be used in this embodiment. In this method, the UTC clock regeneration unit 510 of the receiving device 500 uses the absolute value of the UTC time acquired from the UTC time server 800 (e.g., an NTP server) only once when the reception of the MMT signal from the transmitting device 300 starts, and thereafter uses the time information acquired from the UTC time server 800 simply as a speed reference for the passage of time to output a time that increases simply. When using the time information of the UTC time server 800 (e.g., an NTP server) as a speed reference for the passage of time after the reception of the MMT signal starts, the UTC clock regeneration unit 510 also refers to the leap second indicator, and even if a leap second is inserted or deleted, it continues to output a time that increases simply like the TAI time as a control that takes into consideration the effect of the leap second. By doing this, the TAI offset addition unit 540 downstream of the UTC clock recovery unit 510 does not need to perform control that takes into special consideration the insertion and deletion of leap seconds. In this case, the TAI offset addition unit 540 holds the UTC-TAI offset (initial value) that is first received after the start of reception of the MMT signal, and thereafter, even if the UTC-TAI offset transmitted in the TAI-MPU timestamp descriptor increases or decreases, the initial value of the UTC-TAI offset held at the start of reception is used as is.

Note that, depending on the needs of the common standards and operational agreements between the sending and receiving sides, the TAI offset addition unit 540 does not simply add the value of the UTC-TAI offset transmitted in the TAI-MPU timestamp descriptor to UTC time, but instead converts it to TAI time by inverting the positive/negative sign of the transmitted UTC-TAI offset or adding a fixed offset value that has already been incorporated, and then adding it to UTC time.

The ISOBMFF metadata analysis unit 550 extracts DTS-PTS differential information from metadata multiplexed and transmitted in an MMT signal. The ISOBMFF metadata analysis unit 550 extracts DTS-PTS differential information of, for example, each frame belonging to the current MPU from the metadata separated by the MMT demultiplexing unit 520, and outputs the DTS-PTS differential information to the PTS calculation unit 560. The DTS-PTS differential information is expressed as a relative value rather than an absolute time. Note that, when the ISOBMFF metadata including the DTS-PTS differential information is received in chunk units obtained by dividing an MPU (GOP), the ISOBMFF metadata analysis unit 550 outputs the DTS-PTS differential information to the PTS calculation unit 560 in chunk units of, for example, 8 frames.

The PTS calculation unit 560 calculates the PTS expressed in TAI time for each frame in the MPU from the MPU timestamp (TAI-MPU timestamp) expressed in TAI time and the DTS-PTS difference information for each frame extracted from the ISOBMFF metadata analysis unit 550. The PTS calculation unit 560 calculates the PTS expressed in TAI time for each frame from the TAI-MPU timestamp acquired from the descriptor analysis unit 530 and the DTS-PTS difference information (relative value) output from the ISOBMFF metadata analysis unit 550. The TAI offset addition unit 540 outputs the PTS of the TAI time for each frame to the picture reorder buffer 580.

The HEVC decoder unit 570 decodes the HEVC encoded data separated by the MMT demultiplexing unit 520 and outputs the decoded video frames (each picture) to the picture reorder buffer 580.

The picture reorder buffer 580 receives video frames decoded from the video signal multiplexed into the MMT signal and transmitted, compares the counter value representing the TAI time converted by the TAI offset addition unit 540 with the PTS expressed in the TAI time of each frame calculated by the PTS calculation unit 560, reorders the video frames as necessary, and outputs them to a monitor 600, etc.
In addition, when there is a transmission delay between the transmitting device 300 and the receiving device 500, the TAI time indicated by the PTS may be a time in the past with respect to the TAI time converted by the TAI offset adding unit 540. In this case, the picture reorder buffer 580 adjusts the offset time to be added to the PTS, and performs presentation control by comparing the TAI time obtained by adding this offset time to the PTS of each frame with the TAI time converted by the TAI offset adding unit 540. Stable playback is possible by controlling the offset time so that the offset time is increased when a buffer underflow occurs and is decreased when a buffer overflow occurs.

7. Operation of the Receiving Device
Next, the operation of the receiving device according to the first embodiment of the present invention will be described with reference to FIG. 8 (and also with reference to FIGS. 1 and 7 as appropriate).
The UTC clock regeneration unit 510 of the receiving device 500 communicates with the UTC time server 800 to regenerate a counter value representing the UTC time (step S201). Meanwhile, the MMT demultiplexing unit 520 separates the control information, the MPU sequence number, the metadata, and the HEVC encoded data from the transmitted MMT signal, and extracts the TAI-MPU timestamp descriptor from the control information.
Then, the descriptor analyzer 530 extracts the TAI-MPU timestamp and the UTC-TAI offset from the TAI-MPU timestamp descriptor demultiplexed from the MMT signal (step S202).Then, the TAI offset adder 540 adds the UTC-TAI offset to the counter value representing the UTC time reproduced by the UTC clock regenerator 510, and converts it to TAI time (step S203).

Then, the ISOBMFF metadata analysis unit 550 extracts the DTS-PTS difference from the metadata demultiplexed from the MMT signal (step S204).Then, the PTS calculation unit 560 calculates the PTS expressed by the TAI time of each frame in the MPU from the TAI-MPU timestamp and the DTS-PTS difference of each frame belonging to the MPU, for example (step S205).On the other hand, the HEVC decoder unit 570 decodes the HEVC encoded data demultiplexed from the MMT signal, and outputs the decoded video frame (each picture) to the picture reorder buffer 580.
Then, the picture reorder buffer 580 compares the counter value representing the TAI time with the PTS represented by the TAI time of each frame (adding an offset time if necessary), and reorders the video frames as necessary and outputs them to the monitor 600 (step S206).

Second Embodiment
Next, a transmission system according to a second embodiment will be described. The overall configuration of the transmission system according to the second embodiment is the same as that of the first embodiment (see FIG. 1), but the configuration of the transmission device is different. FIG. 9 is a block diagram showing a schematic diagram of the transmission device according to the second embodiment. In FIG. 9, the same components as those of the transmission device shown in FIG. 5 are denoted by the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 9, the transmitting device 300B includes a TAI clock recovery unit 310, a frame buffer unit 320, a HEVC encoder unit 330, an MMT multiplexing unit 340, a PTS determination unit 350, an MPU sequence number counter unit 360, a descriptor generation unit 370, and an ISOBMFF metadata generation unit 380, and further includes a starting point offset addition unit 390 for changing the starting point (epoch).

In this embodiment, the TAI clock regeneration unit 310 regenerates the counter value of the TAI time acquired from the TAI time server 200 via communication as a reference clock within the device, and outputs it to the origin offset addition unit 390. The TAI clock regeneration unit 310 also acquires and holds the UTC-TAI offset (currentUTCOffset) from the TAI time server 200, and outputs it to the origin offset addition unit 390.

The starting point offset addition unit 390 adds a predetermined fixed offset expressed in seconds to the counter value representing the TAI time reproduced by the TAI clock reproduction unit 310, thereby changing the starting point of the counter value representing the TAI time. The starting point offset addition unit 390 outputs the counter value representing the TAI time to which the offset has been added and the UTC-TAI offset to the PTS determination unit 350. In this embodiment, the PTS determination unit 350 determines the MPU timestamp (TAI-MPU timestamp) expressed in TAI time according to the counter value representing the TAI time to which the offset has been added by the starting point offset addition unit 390 and the GOP start trigger signal.

When the TAI time server 200 that provides the TAI time is a PTP server, the counter value representing the TAI time reproduced by the TAI clock reproduction unit 310 indicates time information whose starting point (epoch) is 0:00:00 (UTC) on January 1, 1970. The starting point offset addition unit 390 changes this starting point (epoch). The starting point offset addition unit 390 can add a positive offset or a negative offset to the counter value representing the reproduced TAI time. When a positive offset is added, this corresponds to the starting point (epoch) going back in time, so the starting point can be changed to a date and time earlier than 0:00:00 (UTC) on January 1, 1970, for example, the same date and time as the starting point of UTC time. On the other hand, when a positive offset is added, the starting point (epoch) is postponed, so the starting point can be changed to a date and time later than January 1, 1970, 0:00:00 (UTC), for example, January 1, 2000, 0:00:00 (UTC).

When the TAI time server 200 that provides the TAI time is a PTP server, the starting point offset addition unit 390 may, for example, change the starting point of the counter value that represents the TAI time to the same date and time as the starting point of the UTC time in the NTP format as follows. That is, the starting point offset addition unit 390 adds a predetermined offset expressed in seconds to the counter value that represents the TAI time in the PTP format, which has the starting point of the counter value that represents the TAI time as 0:00:00 (UTC) on January 1, 1970 (UTC), to generate a counter value that represents the TAI time that has the starting point of 0:00:00 (UTC) on January 1, 1900, similar to the UTC time in the NTP (Network Time Protocol) format. Here, the offset added to the counter value (seconds portion) that represents the TAI time in the PTP format is, specifically, the counter value (seconds portion) that represents 0:00:00 (UTC) on January 1, 1970 in the UTC time in the NTP format.

In this way, if the starting point of the counter value expressing the TAI time is changed to the same date and time as the starting point of UTC time in the NTP format, there is no need to take into account the offset value of 70 years in (i) above in the UTC-TAI offset that the starting point offset adder 390 outputs to the PTS determination unit 350. Therefore, in this case, the UTC-TAI offset that the starting point offset adder 390 outputs to the PTS determination unit 350 can be the offset value in (ii) above that indicates the number of seconds accumulated by the insertion of past leap seconds.

In the first embodiment, the TAI clock regeneration unit 310 can output a UTC-TAI offset having a value obtained by adding, for example, the 70-year offset value (i) to the currentUTCOffset acquired from the TAI time server 200, but such a function may be integrated into the origin offset addition unit 390. In other words, in the second embodiment, the TAI clock regeneration unit 310 may output the currentUTCOffset as is to the origin offset addition unit 390, and the origin offset addition unit 390 may perform both the function of adding an offset to the counter value representing the regenerated TAI time and the function of determining the value of the UTC-TAI offset to be output to the PTS determination unit 350.

(Modification)
In the transmitting device and the receiving device according to each embodiment, the video codec is HEVC, but the video codec is not limited to HEVC. For example, other video encoding methods such as AVC (Advanced Video Coding), VVC (Versatile Video Coding), and AV1 (AOMedia Video 1) can also be used.

This can be applied not only to video codecs, but also to audio codecs such as AAC (Advanced Audio Codec). Note that in the case of audio codecs, the MPU, which is the coding unit equivalent to a GOP in video, can be bounded by any AU (Access Unit). This means that MPUs can be configured at a fixed time period or in synchronization with video GOPs.

The transmitting device may also transmit a combination of a TAI-MPU timestamp descriptor including a TAI-MPU timestamp based on TAI time (including cases where the starting date and time has been changed) and an MPU timestamp descriptor including an MPU timestamp based on UTC time. In this case, since the fractional part of the MPU timestamp (UTC) that is less than a second and the fractional part of the TAI-MPU timestamp that is less than a second are the same regardless of leap seconds, it is also possible to omit the fractional part of the TAI-MPU timestamp that is less than a second (the bit portion below the decimal point) in the TAI-MPU timestamp descriptor and transmit only the integer portion (the bit portion representing the number of seconds).

In addition, as a method for dealing with the possibility that the receiving device 500 may not convert to TAI time correctly when there is a transmission delay between the sending side and the receiving side, the following method may be used instead of the method described in the above embodiment. In the receiving device 500 of this modified example, the UTC clock recovery unit 510 always uses the absolute value of the UTC time acquired from the UTC time server 800 and outputs it to the TAI offset addition unit 540, and when a leap second is inserted or deleted, it outputs the absolute value of the UTC time that reflects it. Then, the TAI offset addition unit 540 does not wait for the UTC-TAI offset transmitted in the TAI-MPU timestamp descriptor to increase or decrease, but increases or decreases the UTC-TAI offset in advance in accordance with the timing of the insertion or deletion of a leap second into the UTC time announced by the leap second indicator. Although the modified receiving device 500 needs to receive the TAI-MPU timestamp descriptor at least once to know the absolute value of the UTC-TAI offset, how the UTC-TAI offset increases or decreases due to the deletion or insertion of leap seconds into UTC time can be easily predicted by common standards and operational agreements between the sending and receiving sides.

The transmitting device and receiving device according to the embodiment of the present invention have been described above, but the spirit of the present invention is not limited to these descriptions and must be interpreted broadly based on the claims. Furthermore, it goes without saying that various changes and modifications based on these descriptions are also included in the spirit of the present invention.

The transmitting device and receiving device according to this embodiment can be used for content transmission using MMT in satellite broadcasting, terrestrial broadcasting, cable broadcasting, IP multicast broadcasting, etc.

1 Transmission system 100 Video/audio signal source 200 TAI time server (time server)
300, 300B Transmitting device 310 TAI clock recovery unit 320 Frame buffer unit 330 HEVC encoder unit (video encoder unit)
340 MMT multiplexing unit 350 PTS determination unit 360 MPU sequence number counter unit 370 Descriptor generation unit 380 ISOBMFF metadata generation unit 390 Starting point offset addition unit 400 Transmission path 500 Receiving device 510 UTC clock recovery unit 520 MMT demultiplexing unit 530 Descriptor analysis unit 540 TAI offset addition unit 550 ISOBMFF metadata analysis unit 560 PTS calculation unit 570 HEVC decoder unit 580 Picture reorder buffer 600 Monitor 700 Speaker 800 UTC time server (time server)

Claims (4)

A transmitting device for transmitting an MMT (MPEG Media Transport) signal in which an encoded video signal, a time stamp descriptor assigned to an MPU (Media Processing Unit), and metadata are multiplexed, the transmitting device comprising:
a TAI clock regeneration unit that communicates with a time server that provides TAI (International Atomic Time) time and regenerates a counter value that represents the TAI time;
a video encoder unit that encodes video frames and outputs an encoded video signal and a GOP (Group Of Picture) start trigger signal;
a PTS determination unit that determines an MPU timestamp represented by the TAI time as a PTS (Presentation Timestamp) to be assigned to an MPU in accordance with the GOP start trigger signal from the video encoder unit, based on a counter value representing the TAI time reproduced by the TAI clock reproduction unit;
a descriptor generation unit that generates, as the timestamp descriptor, a descriptor including an MPU sequence number, an MPU timestamp expressed in the TAI time, and a UTC-TAI offset indicating the difference between the UTC (Coordinated Universal Time) time and the TAI time expressed in seconds at the same timing as the MPU timestamp expressed in the TAI time is assigned to the MPU;
an ISOBMFF metadata generation unit that acquires DTS-PTS differential information, which is information indicating a differential value between a DTS (Decoding Timestamp) and a PTS, as picture reorder information of each frame from the video encoder unit, and generates, as the metadata, metadata of ISO Base Media File Format (ISOBMFF) including the DTS-PTS differential information in units of MPUs or in units of chunks obtained by dividing the MPUs;
A transmitting device characterized by comprising an MMT multiplexing unit that multiplexes the encoded video signal, a timestamp descriptor assigned to the MPU, and the metadata including the DTS-PTS differential information to generate the MMT signal. a start point offset adding unit that adds a predetermined offset expressed in seconds to a counter value representing the TAI time reproduced by the TAI clock reproducing unit to change a start point of the counter value representing the TAI time;
The PTS determination unit determines an MPU time stamp expressed by the TAI time according to a counter value representing the TAI time to which the offset is added by the starting point offset addition unit and the GOP start trigger signal.
2. The transmitting device according to claim 1 . The time server that provides the TAI time is a PTP (Precision Time Protocol) server,
The origin offset addition unit generates a counter value representing a TAI time having a start point earlier or later than January 1, 1970, 0:00:00 (UTC) by adding a predetermined positive offset or negative offset expressed in seconds to a counter value representing a TAI time in a PTP format in which the start point of the counter value representing the TAI time is January 1, 1970, 0:00:00 (UTC).
3. The transmitting device according to claim 2. A receiving device that receives an MMT signal transmitted from a transmitting device,
a UTC clock regenerator that communicates with a time server that provides UTC time and regenerates a counter value that represents the UTC time;
A descriptor analysis unit that extracts an MPU timestamp expressed in a TAI time in a specific MPU and a UTC-TAI offset from the timestamp descriptor multiplexed and transmitted in the MMT signal;
a TAI offset adding unit that converts a counter value representing a UTC time reproduced by the UTC clock reproducing unit into a counter value representing a TAI time by adding the UTC-TAI offset to the counter value;
an ISOBMFF metadata analysis unit that extracts DTS-PTS difference information, which is information indicating a difference value between a DTS and a PTS as picture reorder information of each frame, from the metadata multiplexed and transmitted in the MMT signal;
a PTS calculation unit that calculates a PTS expressed in TAI time of each frame from an MPU time stamp expressed in TAI time and DTS-PTS difference information of each frame extracted from the ISOBMFF metadata analysis unit;
A receiving device characterized by comprising a picture reorder buffer which receives as input a video frame decoded from a video signal multiplexed and transmitted onto the MMT signal, compares a counter value representing the TAI time converted by the TAI offset addition unit with a PTS represented by the TAI time of each frame, and rearranges and outputs the video frames as necessary.

Images