KR20220126534A - Device for outputting content and delay compensation method of content output device - Google Patents

Abstract

A content output device is disclosed. The content output device according to the present invention comprises: an external device interface; a memory for storing a plurality of delay time information respectively mapped to a plurality of compensation sets; and a control unit. The control unit receives identification data of a display device through the external device interface, when the display device is connected to the external device interface, generates an identification ID of the display device using the identification data, outputs content including video data and audio data to the display device through the external device interface, obtains information about a content format of the content, selects a compensation set corresponding to the identification ID of the display device and the content format, among the plurality of compensation sets, and adjusts a time gap between the video data and the audio data based on delay time information mapped to the selected compensation set. The present invention can optimize the synchronization of content actually viewed by a user.

Description

DEVICE FOR OUTPUTTING CONTENT AND DELAY COMPENSATION METHOD OF CONTENT OUTPUT DEVICE

The present invention relates to a content output apparatus capable of supporting sophisticated video/audio sync of content output from a display apparatus by applying delay time information mapped to a display apparatus and a format of the content, and a method for compensating for delay of the content output apparatus. .

Recently, TVs have become larger and support high quality such as 4K, 8K, and HDR, and in line with this, various OTT (Over The Top) services are also providing high-definition content.

On the other hand, a user who watches TV on a clearer and larger screen may more easily discover that the synchronization between video and audio is out of sync. Therefore, pay TV broadcasting (Pay TV), premium OTT service, etc. require more enhanced video and audio sync standards, and accordingly, content output devices must satisfy the sync standards as much as possible.

Accordingly, a technology for synchronizing video and audio is applied to a content output device (eg, a set-top box) that outputs content to a display device.

However, in the case of a display device (eg, a TV) that is connected to a set-top box and outputs content to a user (display of an image, output of sound), a delay element of video or delay element of audio exists in the display device itself.

In addition, the delay factor may be different depending on the manufacturer and model of the display device, and even with the same delay factor, the delay time may be different depending on the format of the content.

Therefore, even when a content output device such as a set-top box performs precise synchronization, an additional delay occurs in the actual display device, resulting in a problem in which out-of-sync content is provided to the user.

SUMMARY OF THE INVENTION The present invention is to solve the above problems, and an object of the present invention is to apply delay time information mapped to a display device and a format of the content to support sophisticated video/audio sync of content output from the display device. An object of the present invention is to provide an output device and a method for compensating for delay of a content output device.

The content output device according to the present invention includes an external device interface, a memory for storing a plurality of delay time information mapped to a plurality of compensation sets, and, when a display device is connected to the external device interface, through the external device interface receiving identification data of the display device, generating an identification ID of the display device using the identification data, and outputting content including video data and audio data to the display device through the external device interface; Acquire information on the content format of the content, select an identification ID of the display device and a reward set corresponding to the content format among the plurality of reward sets, and based on the delay time information mapped to the selected reward set and a controller for adjusting a time gap between the video data and the audio data.

In this case, the external device interface may be an HDMI port, and the identification data may be EDID.

Meanwhile, each of the plurality of reward sets may be different from the other reward sets in at least one of an identification ID and a content format.

Meanwhile, the content format may be distinguished from other content formats by the video format, audio format, and frame rate of the content.

Meanwhile, the control unit may repeat or repeat the frame of the video data in the buffer so that a time gap between the video data and the audio data output from the display device approaches 0 based on the mapped delay time information. A discard can be performed.

In this case, the control unit repeats the frame of the video data in the buffer so that a time gap between the video data and the audio data output from the buffer approaches a value having a negative sign in the mapped delay time information. ) or discard.

Meanwhile, the controller remaps the video PTS of the video data or the audio PTS of the audio data so that a time gap between the video data and the audio data output from the display device approaches 0 based on the mapped delay time information. can do.

In this case, the controller adds the negative sign to the video PTS of the video data in the buffer so that a time gap between the video data and the audio data output from the buffer approaches a value having a negative sign in the mapped delay time information. value, or the delay time information may be added to the audio PTS of the audio data in the buffer.

Meanwhile, the method for compensating for delay of a content output device according to the present invention includes the steps of storing a plurality of delay time information mapped to a plurality of compensation sets, respectively, and when a display device is connected to an external device interface, the display through the external device interface Receiving identification data of a device, generating an identification ID of the display device using the identification data, outputting content including video data and audio data to the display device through the external device interface; obtaining information on the content format of the content, and selecting a reward set corresponding to an identification ID of the display device and the content format from among the plurality of reward sets, and a delay time mapped to the selected reward set and adjusting a time gap between the video data and the audio data based on the information.

In this case, the external device interface may be an HDMI port, and the identification data may be EDID.

Meanwhile, each of the plurality of reward sets may be different from the other reward sets in at least one of an identification ID and a content format.

Meanwhile, the content format may be distinguished from other content formats by the video format, audio format, and frame rate of the content.

Meanwhile, the adjusting of the time gap between the video data and the audio data may include: based on the mapped delay time information, the time gap between the video data and the audio data output from the display device approaches zero in the buffer. The method may include repeating or discarding the frame of the video data.

In this case, in the step of repeating or discarding the frame of the video data in the buffer, the time gap between the video data and the audio data output from the buffer is negative in the mapped delay time information. To access a value having a sign, it is possible to repeat or discard the frame of the video data in the buffer.

Meanwhile, the adjusting the time gap between the video data and the audio data may include adjusting the time gap between the video data and the audio data output from the display device based on the mapped delay time information so that the time gap between the video data and the audio data approaches zero. and remapping the video PTS of , or the audio PTS of the audio data.

In this case, the step of remapping the video PTS of the video data or the audio PTS of the audio data may include such that a time gap between the video data and the audio data output from the buffer approaches a value having a negative sign in the mapped delay time information. , the negative value may be added to the video PTS of the video data in the buffer, or the delay time information may be added to the audio PTS of the audio data in the buffer.

Meanwhile, in the method for generating delay time information according to the present invention, a first step of outputting test content to a display device through an external device interface of a content output device, and video and audio generated while the test content is output by the display device A second step of obtaining delay time information, and a third step of mapping and storing the delay time information in a compensation set including an identification ID of the display device and a content format of the test content are included.

In this case, by repeating the first to third steps for different display devices using a plurality of test content having different content formats, at least one of an identification ID and a content format corresponds to each of a plurality of reward sets different from each other. It may further include the step of mapping the delay time information.

In this case, the content format may be distinguished from other content formats by the video format, audio format, and frame rate of the content.

According to the present invention, since the sync between the video and audio is adjusted in consideration of the characteristics of the display device and the content format, there is an advantage in that the sync of the content actually viewed by the user can be optimized.

In addition, according to the present invention, by generating a plurality of compensation sets reflecting various display devices and various content formats, and performing compensation using a plurality of delay time information mapped to the plurality of compensation sets, a certain display is displayed on the content output device. It has the advantage of being able to precisely synchronize regardless of whether the device is connected or what format the content is output in.

Also, since HDMI is a one-way interface, the display device cannot transmit actual delay information generated by the display device to the content output device. However, the present invention has the advantage of compensating for the delay occurring in the display device by collecting the time delay information in advance, mapping it with the compensation set, and storing the time delay information mapped to the compensation set in the content output device.

In addition, according to the present invention, when a new content format or a new display device appears, it is possible to measure a new compensation set and corresponding delay time information and distribute it to the content output devices. It has the advantage of being done in a simple way. Therefore, the present invention has an advantage that can be easily applied to content output devices that are already in use.

1 is a block diagram illustrating a content output device according to the present invention.
2 is a block diagram for explaining a detailed configuration of a control unit, a configuration of a display device, and a configuration of a delay measuring apparatus according to the present invention.
3 is a diagram illustrating a video decoder output buffer (VDEC Output Buffer) 148 and an audio decoder output buffer (ADEC Output Buffer) 149 according to the present invention.
4 is a diagram for explaining a method of generating delay time information according to the present invention.
5 is a diagram illustrating various content formats.
6 is a diagram illustrating a plurality of compensation sets.
7 is a diagram for explaining a delay compensation method of a content output device according to the present invention.
8 is a diagram for explaining a method of performing compensation based on delay time information according to an embodiment of the present invention.
9 is a diagram for explaining a method of performing compensation based on delay time information according to another embodiment of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

In implementing the present invention, components may be subdivided for convenience of description, but these components may be implemented in one device or module, or one component may be divided into a plurality of devices or modules. It can also be implemented in

Prior to the description of the invention, a delay element that may occur in the display device will be described.

Various delay elements exist in a display device (eg, a TV), and thus the delay may be different for each display device.

Specifically, the delay may be different according to manufacturing methods of display device manufacturers. Also, due to changes in the chipset used or applied technology, delays may be different for each model of display devices manufactured by the same manufacturer.

In addition, the biggest factor causing the delay in the display device may be the image quality and sound quality improvement algorithm that the display device owns.

In the case of image quality, the improvement algorithm has various modes such as standard, sharp, game, and AI. For example, in the case of game mode, the screen is usually output to the panel by minimizing the delay.

In addition, in the case of the sound quality improvement algorithm, the original voice data is processed according to a specific genre, such as a movie or music, and outputted through a speaker of the display device.

In addition, delay occurs whenever video processing and audio processing are performed in order to improve picture quality or sound quality.

Also, even if a display device of the same model of the same manufacturer goes through the same improvement algorithm, different delays may occur depending on the format of the content. For example, video processing and audio processing tasks use a buffer in each step, and each time the buffer is passed, a delay of 1/frame rate (16.67 ms for 60 fps and 33.3 ms for 30 fps) occurs.

On the other hand, delays may also occur with respect to the video color space. Specifically, in the case of a TV panel, an image is finally output in an RGB color space, but data transmitted to the display device through HDMI is YUV420, YUV444, or the like, and data in a different color space may be transmitted to the display device. Also, when the color spaces are different as described above, a delay occurs while performing color space conversion.

A delay may also occur in the relationship between HDR (High Dynamic Range) and SDR (Standard Dynamic Range). Specifically, in the case of HDR (High Dynamic Range), remapping of RGB values is required to match the characteristics of the TV panel. Also, if the TV panel does not support HDR, it is necessary to go through the process of converting HDR to SDR.

There may also be delays with respect to the video frame rate. Specifically, in the case of a TV, there is a function of reading the frame rate of the original content to generate and output high frame rate content. In addition, since the process of changing the picture quality goes through the buffer and the delay in the buffer is related to the frame rate, a delay occurs.

Also, in the case of streaming services or VoD, delays may occur due to operations such as fast forward, rewind, skip, and play after pause. Specifically, in the case of operations such as fast forward, rewind, skip, and playback after pause, the screen temporarily stops and then resumes. Different delays may occur depending on output.

Also, in the case of audio, a different delay may occur depending on whether data transmitted through HDMI is a bitstream or pulse code modulation (PCM). The reason is that in the case of PCM (pulse code modulation), the audio codec chip recognizes it and outputs it directly to the TV speaker through the DAC (Digital to Analog Converter) and amplifier, but the bitstream such as Dolby Digital This is because, in the case of , the process of performing audio decoding on the TV is added.

In addition to the above, there are various factors that can cause video or audio delay in display devices (for example, TVs), and the delay varies depending on the manufacturer of the display device, and each manufacturer’s model despite being made by the same manufacturer. Although it is the same model, it may vary depending on the format of the outputted content.

Accordingly, the present invention supports sophisticated video/audio sync of content output from the display device by applying the delay compensation value mapped to the display device and the format of the content in the content output device.

Meanwhile, as digital broadcasting or online streaming services become popular, the chipset (multimedia processor) of the content output device already has a built-in function to synchronize between video and audio.

In addition, in the transport stream or the program stream, in digital broadcasting and streaming services, video output time (Video Presentation Time Stamp, hereinafter referred to as video PTS) and audio output time (Audio Presentation Time) Stamp, hereinafter referred to as audio PTS) will be included.

And, when the PTS gap is within a certain range, the chipsets of the content output device do not perform a video-audio sync operation. In this specification, a range in which a video-audio sync operation is not performed is referred to as a margin range.

A chipset (multimedia processor) currently used in a content output device such as a set-top box is designed to perform a sync operation so that the PTS gap falls within the margin range when the PTS gap is outside the margin range.

1 is a block diagram illustrating a content output device according to the present invention.

The content output device 100 may refer to devices that receive and decode a broadcast stream and transmit decoded video data and audio data to a display device.

For example, the content output device 100 may be an image processing device such as a set-top box. In this case, the image processing apparatus may receive a content stream for a selected broadcast channel, decode the received content stream, and output video data and audio data. In this case, the output video data and audio data may be transmitted to a display device such as a TV and displayed or output by the display device.

The content output device 100 may include a communication unit 110 , an external device interface 120 , a memory 130 , and a control unit 140 .

The communication unit 110 may support wired or wireless communication between the content output device 100 and another device.

Specifically, the communication unit 110 may provide an interface for connecting the content output device 100 to a wired/wireless network including an Internet network. The communication unit 110 may transmit or receive data to or from the server 100 through the connected network or another network linked to the connected network.

Meanwhile, the communication unit 110 may receive a content stream. Here, the content stream may include a transport stream and a program stream, and broadcast content, VOD, streaming content, etc. may be transmitted through the content stream.

The external device interface 120 may include an interface for transmitting content including video data and audio data to the display device.

Specifically, the external device interface 120 may be an HDMI port. In this case, the content output device 100 may be connected to the display device by an HDMI cable to transmit content to the display device.

Meanwhile, the memory 130 may store programs, contents, decoded contents, and other data for processing and controlling various signals of the controller 140 .

Also, the memory 230 may store an application program or data for the operation of the content output device 100 .

Meanwhile, the controller 140 may control the overall operation of the content output apparatus 100 .

In addition, the controller 140 decodes the content stream to generate video data and audio data, obtains a time gap of the video data and audio data based on the video PTS of the video data and the audio PTS of the audio data, and the time gap is a margin If it is out of range, a sync operation can be performed.

2 is a block diagram for explaining a detailed configuration of a control unit, a configuration of a display device, and a configuration of a delay measuring apparatus according to the present invention.

The detailed configuration of the control unit 140 will be described with the MPEG2 System standard among various standard sets of MPEG as an example. However, the scope of the present invention is not limited to the MPEG2 System because the basic operation of the present invention can be sufficiently modified to the domestic and international standards or de-facto standards of other moving pictures.

The demux 141 may analyze content (broadcast content, VOD, streaming content, etc.) received through the content stream to classify video content, audio content, and data.

The descrambler 142 can classify content restricted by broadcast access control system (CAS) or digital rights management (DRM).

The video decoder input buffer (VDEC Input Buffer) 143 and the audio decoder input buffer (ADEC Input Buffer) may be buffers existing in front of the decoder. In addition, the video decoder input buffer (VDEC Input Buffer) 143 may transmit video content to the video decoder (VDEC) 146 according to a decoding time stamp (DTS) included in the content stream. Also, the audio decoder input buffer (ADEC Input Buffer) 144 may transmit audio content to the audio decoder (ADEC) 147 according to a decoding time stamp (DTS) included in the content stream.

The system time clock 145 may generate a reference clock by parsing a program clock reference (PCR) from the content stream, and may provide the reference clock to each component of the content output device 100 . In particular, the reference clock may provide a time reference value for a decoding time stamp (Decoding Time Stamp, DTS) and presentation time designation information (Presentation Time Stamp, PTS).

The video decoder (VDEC) 146 may decode video content to generate video data, and transmit the generated video data to a video decoder output buffer (VDEC Output Buffer) 148 .

Also, the audio decoder (ADEC) 147 may generate audio data by decoding the audio content, and transmit the generated audio data to an audio decoder output buffer (ADEC Output Buffer) 149 .

The video decoder output buffer (VDEC Output Buffer) 148 may transmit video data to the output unit 120 under the control of the AV output controller 150 .

Also, the audio decoder output buffer (ADEC Output Buffer) 149 may transmit video data to the output unit 120 under the control of the AV output controller 150 .

The AV output controller 150 obtains a time gap of video data and audio data based on a video PTS of video data and an audio PTS of audio data, and performs a sync operation when the time gap is out of a margin range. can do.

Meanwhile, in this specification, the term PTS gap may be used interchangeably with the term time gap.

3 is a diagram illustrating a video decoder output buffer (VDEC Output Buffer) 148 and an audio decoder output buffer (ADEC Output Buffer) 149 according to the present invention.

The video decoder output buffer (VDEC Output Buffer) 148 may store a plurality of frames of video data.

For example, referring to FIG. 3A , the video decoder output buffer (VDEC Output Buffer) 148 may store a plurality of frames from the first frame to the N-th frame, and the plurality of frames sequentially from the first frame to the output unit ( 120) can be output.

Also, a plurality of frames may each have a corresponding video PTS.

Meanwhile, the audio decoder output buffer (ADEC Output Buffer) 149 may store a plurality of partial audio data of the audio data.

For example, referring to FIG. 3B , the audio decoder output buffer (ADEC Output Buffer) 149 may store a plurality of partial audio data from the first partial audio data to the Nth partial audio data, and the plurality of partial audio data The first partial audio data may be sequentially output to the output unit 120 .

Also, each of the plurality of partial audio data may have a corresponding audio PTS.

Meanwhile, the AV output controller 150 includes a plurality of video PTSs respectively allocated to a plurality of frames in a video decoder output buffer 148 , and an audio decoder output buffer 149 . ), a plurality of audio PTSs each assigned to a plurality of partial audio data can be monitored in real time.

Meanwhile, the AV output controller 150 may acquire a time gap between video data and audio data.

Specifically, the AV output controller 150 may acquire a time gap between video data and audio data corresponding to each other. For example, the AV output controller 150 compares the first frame of the video data with the first partial audio data of the audio data to obtain a time gap, and the second frame of the video data and the audio data A time gap may be obtained by comparing the second partial audio data.

In this case, the AV output controller 150 may acquire a time gap between the video data and the audio data based on the video PTS of the video data and the audio PTS of the audio data.

For example, the AV output controller 150 may configure a time gap (video PTS - audio PTS) can be obtained.

In addition, the AV output controller 150 may perform a sync operation when the time gap is out of a margin range based on a time gap between the video data and the audio data.

Referring back to FIG. 2 , the AV output controller 150 may adjust a time gap between the video data and the audio data based on the compensation value received from the sync manager 152 .

Meanwhile, when the display device 200 is connected to the content output device 100 , the external device interface 120 may receive identification data of the display device 200 from the display device 200 . Here, the identification data may be EDID (Extended Display Identification Data).

In this case, the external device interface 120 may transmit the identification data to the sink detector 151 .

Meanwhile, the sink detector 151 may generate an identification ID of the display apparatus 200 connected to the content output apparatus 100 based on the identification data. In this case, the sink detector 151 may transmit the identification ID of the display apparatus 200 to the sync manager 152 .

Also, the sync manager 152 may detect the format of the content currently output to the external device interface 120 .

Meanwhile, in the process of acquiring the plurality of delay time information, the sync manager 152 communicates with the delay specifying device 300 through the first communication unit 110 , and delay time information from the delay specifying device 300 . can receive In this case, the sync manager 152 may map the delay time information to a compensation set including the identification ID of the display device 200 and the format of the content and store the mapping.

Also, in the process of performing compensation based on the delay time information, the sync manager 152 acquires specific delay time information mapped to a specific compensation set including the identification ID of the display device 200 and the format of the content. and a compensation value based on the obtained delay time information may be transmitted to the AV output controller 150 .

Meanwhile, the display device 200 may refer to a device that displays video data included in content, such as a TV, and outputs audio data included in the content through a speaker.

The display apparatus 200 may include a control unit 220 including an input unit 210 , a video processing unit 225 , and an audio processing unit 230 , a video output unit 240 , and an audio output unit 250 .

The input unit 210 may be connected to the external device interface 120 of the content output apparatus 100 to receive content including video data and audio data from the content output apparatus 100 .

Meanwhile, the input unit 210 may be connected to the external device interface 120 through an HDMI cable, and in this case, the input unit 210 may include an HDMI port.

Also, the input unit 210 may transmit identification data (eg, EDID) to the external device interface 120 .

Meanwhile, the controller 220 may process the received video data and transmit it to the video output unit 240 , and may process the received audio data and transmit it to the audio output unit 250 .

Meanwhile, the delay in the display apparatus 200 may occur in the controller 220 , that is, at least one of the video processor 225 and the audio processor 230 .

That is, the control unit 220 performs processing for outputting the received video data from the video output unit 240 , processing for outputting the received audio data from the audio output unit 250 , an image quality improvement algorithm, a sound quality improvement algorithm, and various other processing methods. processing is performed, and accordingly, a delay may occur in the controller 220 .

Meanwhile, the video output unit 240 may include a display for displaying an image, and may display data transmitted from the control unit 220 .

Also, the audio output unit 250 may include a speaker for outputting audio, and may output data transmitted from the control unit 220 .

On the other hand, the delay measurement apparatus 300 may calculate a delay time between the video and audio output from the display apparatus 200 (video output time - audio output time) when the test content is reproduced on the display apparatus 200 . have.

In more detail, the delay measurement apparatus 300 may include a video detector 310 , an audio detector 320 , and a delay calculator 330 .

The video detector 310 may include a sensor that detects light, and the audio detector 320 may include a sensor that detects sound.

In addition, the delay calculator 330 uses the value detected by the video detector 310 and the value detected by the audio detector 320 , the delay time between the video output from the display device 200 and the audio ( output time - output time of audio) can be calculated.

Meanwhile, the test content may be produced so that the video detector 310 detects light and the audio detector 320 easily detects sound.

For example, when test content is produced such that a video gradually changes from a black level to a white level and a certain tone is generated only when the luminance of the audio is changed, the delay calculator 330 calculates the video from the black level. It is possible to calculate the time of the midpoint of the change to the white level, and the time at the midpoint of the time for outputting a total tone for audio. In addition, the delay calculator 330 may calculate delay time information between video and audio (output time of video - output time of audio) by calculating the difference between the two times.

Meanwhile, the delay calculator 330 may communicate with the sync manager 152 of the content output device 100 and transmit delay time information to the sync manager 152 .

4 is a diagram for explaining a method of generating delay time information according to the present invention.

Hereinafter, it will be described that the subject receiving the delay time information from the delay measurement device 300 and matching the delay compensation information to the compensation set is the content output device 100, but is not limited thereto, and receiving the delay time information and adding the delay compensation information to the compensation set. The subject matching the delay compensation information may be a device other than the content output device 100 (eg, a server).

The external device interface 120 of the content output device 100 may be connected to the input unit 210 of the display device 200 .

In this case, the external device interface 120 of the content output device 100 may receive identification data from the input unit 210 of the display device 200 (S410).

Here, the identification data may be EDID (Extended Display Identification Data).

The content output device 100, such as a set-top box, and a display device, such as a TV, may be connected by HDMI.

Meanwhile, EDID (Extended Display Identification Data) is used in order for a source device (content output device) connected through HDMI to acquire information on a sink device (display device).

Specifically, in the case of EDID information transmitted from the sink device (display device) to the source device (content output device), basic information (manufacturer, product code, serial number, HDMI version information, etc.), video information (supported Video resolution, color space, bit depth, dynamic range, etc.), audio information (supported audio bit depth, sampling rate, Dolby support, etc.) may include

Meanwhile, the external device interface 120 may transmit identification data to the sync detector 151 (S420).

In addition, the sync detector 151 may generate an identification ID of the display apparatus 200 using the identification data.

In this case, the sync detector 151 may generate an identification ID of the display apparatus 200 by combining one or more pieces of information included in the identification data.

For example, the sync detector 151 may generate an identification ID by combining a manufacturer, a product code, and a serial number included in the identification data. In this case, the identification ID may have a format of uniqueID=”$ManufacturerName_$ProductCode_$SerialNumber”.

And when the manufacturer (ManufacturerName) is GSM, the product code (ProductCode)=1, and the serial number (SerialNumber) is 16843009, the identification ID (uniqueID) of the display apparatus 200 may be GSM_1_16843009.

Meanwhile, the sync detector 151 may transmit the generated identification ID to the sync manager 152 (S440).

Meanwhile, the sync manager 152 may create a database including the identification ID (S450).

Meanwhile, the content output device 100 may output the test content to the display device 200 through the external device interface 120 .

Also, the sync manager 152 may acquire information on the content format of the test content output to the display device 200 ( S460 ).

Meanwhile, the delay measurement device 300 calculates delay time information (video output time - audio output time) between the video and audio output from the display device 200 , and sends the calculated delay time information to the sync manager 152 . can be transmitted

In this case, the delay measuring apparatus 300 may transmit the average value of the plurality of delay time information calculated based on the plurality of measurement results to the sink manager 152 .

Also, the delay measurement apparatus 300 may transmit a plurality of delay time information calculated based on the plurality of measurement results to the sink manager 152 . In this case, the sync manager 152 may calculate an average value by averaging a plurality of delay time information. In addition, the average value may be used as delay time information (video output time - audio output time).

Meanwhile, the sync manager 152 may acquire delay time information between video and audio generated while test content is output by the display device 200 by receiving the delay time information from the delay measuring apparatus 300 .

In this way, the identification ID of the display device, the content format of the test content, and the delay time information between video and audio can be obtained.

In addition, the sync manager 152 may generate a reward set including an identification ID and a content format. In addition, the sync manager 152 may map the delay time information to the reward set including the identification ID and the content format and store it in the database ( S490 ).

Meanwhile, by repeating the above process for different display devices using a plurality of test content having different content formats, delay time information corresponding to each of a plurality of compensation sets having different at least one of an identification ID and a content format is mapped. can be

This will be described with reference to FIGS. 5 to 6 .

5 is a diagram illustrating various content formats.

The content format may include a video format, an audio format, and a frame rate.

For example, the first content format (H264_30_PCM) may include a video format of H264 YUV420, a frame rate of 30, and an audio format of PCM_16BIT_48KHZ.

For another example, the second content format (H264_30_DDP) may include a video format of H264 YUV420, a frame rate of 30, and an audio format of DDP_16BIT_384KBPS.

In addition, the content format may be distinguished from other content by a video format (eg, H264/HEVC/HDR), an audio format (eg, PCM/DDP), and a frame rate (eg, 30FPS/60FPS).

That is, the specific content format may be different from other content formats in at least one of a video format, an audio format, and a frame rate.

For example, the first content format H264_30_PCM has a different audio format from the second content format H264_30_DDP. That is, since the audio format is different, the first content format H264_30_PCM may be distinguished from the second content format H264_30_DDP.

For another example, the first content format (H264_30_PCM) is different from the sixth content format (HEVC_30_DDP) in a video format and an audio format. That is, since the video format and the audio format are different, the first content format H264_30_PCM may be distinguished from the sixth content format HEVC_30_DDP.

Although 12 content formats are exemplified in FIG. 5 , the present invention is not limited thereto, and various content formats in which at least one of a video format, an audio format, and a frame rate are different from other content formats may exist.

Meanwhile, while the display apparatus 200 is connected to the content output apparatus 100 , the process described in FIG. 4 may be repeated using a plurality of test content having different content formats.

Accordingly, a plurality of reward sets including the same identification ID and various content formats may be generated, and the sync manager 152 may map and store delay time information corresponding to each of the plurality of reward sets.

In addition, the process described with reference to FIGS. 4 and 5 may be repeated while various display devices are connected to the content output device 100 .

For example, by measuring delay time information using a plurality of test content having different content formats while the first display device is connected to the content output device 100, the identification ID of the first display device and various content formats are determined. A plurality of compensation sets including the plurality of compensation sets may be generated, and the sync manager 152 may map and store delay time information corresponding to each of the plurality of compensation sets.

Then, in a state in which the second display device is connected to the content output device 100 , the delay time information is measured using a plurality of test contents having different content formats, thereby including the identification ID of the second display device and various content formats. A plurality of compensation sets may be generated, and the sync manager 152 may map and store delay time information corresponding to each of the plurality of compensation sets.

And by repeating this process, a plurality of compensation sets as shown in FIG. 6 may be generated.

6 is a diagram illustrating a plurality of compensation sets.

One reward set may include an identification ID of a display device and a content format.

For example, the first compensation set 610 may include an identification ID of GSM_1_16843009 and a content format of H264_30_PCM.

As another example, the second compensation set 620 may include an identification ID of GSM_1_16843009 and a content format of H264_30_DDP.

As another example, the third compensation set 630 may be configured to include an identification ID of SEC_15360_3355 and content information of HDR_60_DDP.

Meanwhile, each of the plurality of reward sets may be different from other reward sets in at least one of an identification ID and a content format.

For example, the first compensation set 610 may have a different content format from the second compensation set 620 .

As another example, the first compensation set 610 may have a different identification ID and content format from the third compensation set 630 .

Meanwhile, a plurality of delay time information may be respectively mapped to different compensation sets having different at least one of an identification ID and a content format.

For example, delay time information of 100 ms may be mapped to the first compensation set 610 . As another example, delay time information of 50 ms may be mapped to the second compensation set 620 . As another example, delay time information of -20 ms may be mapped to the third compensation set 630 .

Meanwhile, as described above, the method of generating the delay time information may be performed by the server. In this case, the server may transmit a plurality of delay time information mapped to a plurality of compensation sets, respectively, to a plurality of content output devices.

Also, the method of generating the delay time information may be performed by the content output device. In this case, the content output device may transmit delay time information mapped to the compensation set to the server. In this case, the server collects delay time information mapped to the compensation set from a plurality of content output devices to obtain a plurality of delay time information mapped to the plurality of compensation sets, respectively, and a plurality of delay times mapped to the plurality of compensation sets, respectively Information may be distributed to content output devices.

In addition, when a new content format or a new display device is introduced, the server may generate a new reward set and delay time information mapped to the new reward set, and distribute the delay time information mapped to the new reward set to content output devices. .

7 is a diagram for explaining a delay compensation method of a content output device according to the present invention.

The memory 130 of the content output apparatus 100 may store a plurality of delay time information each mapped to a plurality of compensation sets ( S710 ).

Also, the controller 140 of the content output device 100 may receive delay time information mapped to a new reward set from the server, and in this case, the controller 140 stores the delay time information mapped to the new reward set in the memory 130 . ) can be stored in

Meanwhile, when the display device 200 is connected to the external device interface 120 , the controller 140 may receive identification data of the display device 200 through the external device interface 120 . Here, the external device interface 120 may be an HDMI port, and the identification data may be Extended Display Identification Data (EDID).

In this case, the controller 140 may generate an identification ID of the display apparatus 200 connected to the content output apparatus 100 based on the identification data (S730).

In more detail, the controller 140 may generate an identification ID of the display apparatus 200 by combining one or more pieces of information included in the identification data.

Also, the controller 140 may output content including video data and audio data to the display apparatus 200 through an external device interface (S740). Here, the content may include broadcast content, VOD, streaming content, and the like.

In addition, the controller 140 may obtain information on the content format of the content output to the display apparatus 200 (S750).

In addition, the controller 140 may select a compensation set corresponding to the identification ID and content format of the display device from among the plurality of compensation sets, and obtain delay time information mapped to the selected compensation set ( S760 ).

Specifically, the controller 140 may select a compensation set having an identification ID of the display apparatus 200 and a content format of the content output to the display apparatus 200 from among the plurality of compensation sets. In addition, the controller 140 may read the delay time information mapped to the selected compensation set from the memory 130 .

Meanwhile, the controller 140 may adjust a time gap between the video data and the audio data based on the delay time information mapped to the selected compensation set ( S770 ).

Specifically, the sync manager 152 may generate a compensation value (delay time information or a control value generated based on delay time information) and transmit the value to the AV output controller 150 .

In this case, the AV output controller 150 may control the buffers 148 and 149 to apply a compensation value to the video data and the audio data.

A first method of performing compensation based on delay time information will be described with reference to FIG. 8 .

8 is a diagram for explaining a method of performing compensation based on delay time information according to an embodiment of the present invention.

In order not to cause a delay in the display device 200, the video data and audio data output from the buffers 148 and 149 are not output from the actual display device 200 (video display and audio output through the speaker) Compensation should be performed so that the time gap between the video data and the audio data to be used approaches zero.

Accordingly, the AV output controller 150 performs a buffer operation such that a time gap between the video data and the audio data output from the display device 200 approaches 0 based on the delay time information mapped to the selected compensation set. I can repeat or discard a frame of my video data.

Meanwhile, in the content output device 100 , only video data and audio data output from the buffers 148 and 149 can be controlled.

Accordingly, the AV output controller 150 determines that the time gap between the video data and the audio data output from the buffers 148 and 149 is negative in the delay time information based on the delay time information mapped to the selected compensation set. To access a value having a sign of , repeat or discard of a frame of video data in the buffer may be performed.

More specifically, when the delay time information mapped to the selected compensation set is 50 ms, the display apparatus 200 is in a state in which the video is output faster than the audio by 50 ms.

Therefore, referring to FIG. 8B , the AV output controller 150 provides a negative sign to delay time information (50ms) to which a time gap between the video data and the audio data output from the buffers 148 and 149 is mapped. To access a value (-50 ms) having a value of , it is possible to repeat the frame of the video data in the buffer. That is, the AV output controller 150 outputs audio data as it is and outputs the current frame for video data, but does not delete the output frame from the buffer, but keeps it as it is and then outputs it again. It can be done more than once.

Also, when the delay time information mapped to the selected compensation set is -50 ms, the display apparatus 200 is in a state in which the video is output later than the audio by 50 ms.

Accordingly, referring to FIG. 8C , the AV output controller 150 transmits a negative value to the delay time information (-50ms) in which the time gap between the video data and the audio data output from the buffers 148 and 149 is mapped. To access a signed value (50 ms), it is possible to repeat the frame of the video data in the buffer. That is, the AV output controller 150, that is, the AV output controller 150 outputs audio data as it is, and discards the current frame for video data and outputs the next frame. can be performed more than once.

Meanwhile, when the delay time information mapped to the selected compensation set is 0 ms, the display apparatus 200 is in a state in which video and audio are output in synchronization.

Therefore, referring to FIG. 8A , the AV output controller 150 repeats or discards (repeat or discard) so that the time gap between the video data and the audio data output from the buffers 148 and 149 is maintained in the current state. discard) may not be performed.

Next, a second method of performing compensation based on delay time information will be described with reference to FIG. 9 .

9 is a diagram for explaining a method of performing compensation based on delay time information according to another embodiment of the present invention.

In order not to cause a delay in the display device 200, the video data and audio data output from the buffers 148 and 149 are not output from the actual display device 200 (video display and audio output through the speaker) Compensation should be performed so that the time gap between the video data and the audio data to be used approaches zero.

Accordingly, the AV output controller 150 controls the video data so that the time gap between the video data and the audio data output from the display device 200 approaches 0 based on the delay time information mapped to the selected compensation set. You can remap the video PTS of data or the audio PTS of audio data.

Meanwhile, in the content output device 100 , only video data and audio data output from the buffers 148 and 149 can be controlled.

Accordingly, the AV output controller 150 determines that the time gap between the video data and the audio data output from the buffers 148 and 149 is negative in the delay time information based on the delay time information mapped to the selected compensation set. Add the negative sign value to the video PTS of the video data in the buffers 148, 149, or add delay time information to the audio PTS of the audio data in the buffers 148, 149 to access the signed value of can

Specifically, when the delay time information mapped to the selected compensation set is 50 ms, in the display apparatus 200, the video is output faster than the audio by 50 ms.

Therefore, the video data output from the buffers 148 and 149 should be output later than the audio data output from the buffers 148 and 149 by 50 ms.

Therefore, referring to FIG. 9A , the AV output controller 150 is a value having a negative sign in delay time information mapped to the video PTS of the video data in the video decoder output buffer 911 (-50 ms) can be added

In this case, since the time gap (video PTS - audio PTS) between the video data and audio data output from the buffers 148 and 149 becomes -50 ms, the delay (50 ms) occurring in the display device 200 can be compensated. do.

Also, referring to FIG. 9B , the AV output controller 150 may add mapped delay time information (50 ms) to the audio PTS of audio data in the audio decoder output buffer 922 .

In this case, since the time gap (video PTS - audio PTS) between the video data and audio data output from the buffers 148 and 149 becomes -50 ms, the delay (50 ms) occurring in the display device 200 can be compensated. do.

Meanwhile, the description in FIG. 9 is only an example of compensating for a delay, and a method of remapping both the video PTS and the audio PTS is also possible.

For example, the AV output controller 150 may add -25 ms to the video PTS of the video data in the video decoder output buffer and 25 ms to the audio PTS of the audio data in the audio decoder output buffer.

In this case, since the time gap (video PTS - audio PTS) between the video data and audio data output from the buffers 148 and 149 becomes -50 ms, the delay (50 ms) occurring in the display device 200 can be compensated. do.

As described above, according to the present invention, since the sync between the video and audio is adjusted in consideration of the characteristics of the display device and the content format, there is an advantage in that the sync of the content actually viewed by the user can be optimized.

In addition, according to the present invention, by generating a plurality of compensation sets reflecting various display devices and various content formats, and performing compensation using a plurality of delay time information mapped to the plurality of compensation sets, a certain display is displayed on the content output device. It has the advantage of being able to precisely synchronize regardless of whether the device is connected or what format the content is output in.

Also, since HDMI is a one-way interface, the display device cannot transmit actual delay information generated by the display device to the content output device. However, the present invention has the advantage of compensating for the delay occurring in the display device by collecting the time delay information in advance, mapping it with the compensation set, and storing the time delay information mapped to the compensation set in the content output device.

In addition, according to the present invention, when a new content format or a new display device appears, it is possible to measure a new compensation set and corresponding delay time information and distribute it to the content output devices. It has the advantage of being done in a simple way. Therefore, the present invention has an advantage that can be easily applied to content output devices that are already in use.

The present invention described above can be implemented as computer-readable code on a medium in which a program is recorded. The computer readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is this. In addition, the computer may include a control unit. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

100: content output device 110: communication unit
120: external device interface 130: memory
140: control unit

Claims (19)

external device interface;
a memory for storing a plurality of delay time information mapped to a plurality of compensation sets, respectively; and
When the display device is connected to the external device interface, the identification data of the display device is received through the external device interface,
generating an identification ID of the display device using the identification data;
output content including video data and audio data to the display device through the external device interface;
obtaining information about the content format of the content;
A compensation set corresponding to an identification ID of the display device and the content format is selected from among the plurality of compensation sets, and a time gap between the video data and the audio data is calculated based on delay time information mapped to the selected compensation set. a control unit to control; including
content output device. The method of claim 1,
The external device interface is an HDMI port,
The identification data is EDID
content output device. The method of claim 1,
Each of the plurality of reward sets,
At least one of the identification ID and the content format is different from the other reward set.
content output device. The method of claim 1,
The content format is
It is distinguished from other content formats by the video format, audio format, and frame rate of the content.
content output device. The method of claim 1,
The control unit is
Based on the mapped delay time information, repeat or discard the frame of the video data in the buffer so that the time gap between the video data and the audio data output from the display device approaches 0 performing
content output device. 6. The method of claim 5,
The control unit is
Repeat or discard the frame of the video data in the buffer so that the time gap between the video data and the audio data output from the buffer approaches a value having a negative sign in the mapped delay time information to do
content output device. The method of claim 1,
The control unit is
remapping the video PTS of the video data or the audio PTS of the audio data so that a time gap between the video data and the audio data output from the display device approaches 0 based on the mapped delay time information
content output device. 8. The method of claim 7,
The control unit is
The negative value is added to the video PTS of the video data in the buffer so that a time gap between the video data and the audio data output from the buffer approaches a negative value to the mapped delay time information, or adding the delay time information to the audio PTS of the audio data in the buffer
content output device. storing a plurality of delay time information mapped to a plurality of compensation sets, respectively;
receiving identification data of the display device through the external device interface when the display device is connected to the external device interface;
generating an identification ID of the display device using the identification data;
outputting content including video data and audio data to the display device through the external device interface;
obtaining information on a content format of the content; and
A compensation set corresponding to an identification ID of the display device and the content format is selected from among the plurality of compensation sets, and a time gap between the video data and the audio data is calculated based on delay time information mapped to the selected compensation set. adjusting; including
A method of compensating for delay in a content output device. 10. The method of claim 9,
The external device interface is an HDMI port,
The identification data is EDID
A method of compensating for delay in a content output device. 10. The method of claim 9,
Each of the plurality of reward sets,
At least one of the identification ID and the content format is different from the other reward set.
A method of compensating for delay in a content output device. 10. The method of claim 9,
The content format is
It is distinguished from other content formats by the video format, audio format, and frame rate of the content.
A method of compensating for delay in a content output device. 10. The method of claim 9,
Adjusting the time gap between the video data and the audio data comprises:
Based on the mapped delay time information, repeat or discard the frame of the video data in the buffer so that the time gap between the video data and the audio data output from the display device approaches 0 performing; including
A method of compensating for delay in a content output device. 14. The method of claim 13,
Repeating or discarding the frame of the video data in the buffer comprises:
Repeat or discard the frame of the video data in the buffer so that the time gap between the video data and the audio data output from the buffer approaches a value having a negative sign in the mapped delay time information to do
A method of compensating for delay in a content output device. 10. The method of claim 9,
Adjusting the time gap between the video data and the audio data comprises:
Remapping the video PTS of the video data or the audio PTS of the audio data so that a time gap between the video data and the audio data output from the display device approaches 0 based on the mapped delay time information; doing
A method of compensating for delay in a content output device. 16. The method of claim 15,
Remapping the video PTS of the video data or the audio PTS of the audio data comprises:
The negative value is added to the video PTS of the video data in the buffer so that a time gap between the video data and the audio data output from the buffer approaches a negative value to the mapped delay time information, or adding the delay time information to the audio PTS of the audio data in the buffer
A method of compensating for delay in a content output device. a first step of outputting test content to a display device through an external device interface of the content output device;
a second step of obtaining delay time information between video and audio generated while the test content is output by the display device; and
A third step of mapping and storing the delay time information to a compensation set including the identification ID of the display device and the content format of the test content;
How to generate latency information. 18. The method of claim 17,
By repeating the first to third steps for different display devices using a plurality of test content having different content formats, the delay corresponding to each of the plurality of reward sets having different at least one of an identification ID and a content format mapping time information; further comprising
How to generate latency information. 19. The method of claim 18,
The content format is
It is distinguished from other content formats by the video format, audio format, and frame rate of the content.
How to generate latency information.

Images