JP2006191463A - Coder, decoder, and coding transmission system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a coding transmission system in which AV synchronization of a video signal and an audio signal with each other is accurately established. <P>SOLUTION: A coder 100 generates video data and audio data for AV synchronization by a video signal processing part 101 and an audio signal processing part 102 and codes them and multiplexes them by a media multiplexing part 106 and sends them as coded multiplex data. A decoder 200 separates coded video data and coded audio data by a media separation part 201 and decodes them. An AV synchronization error detection part 204 detects an error between video data and audio data for AV synchronization and informs an AV delay control part 207 of the value of the error. The AV delay control part 207 controls delay of output timings of images and sounds so that the output timings are synchronized with each other. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an encoding transmission system in which a digital video signal and an audio signal are each encoded by an encoding device, and the transmitted encoded data is decoded by a decoding device. The present invention relates to an apparatus for controlling (AV synchronization error).

  In the conventional coded transmission system, the progress of the expansion process as a result of comparing the ideal number of frames related to the video component signal that should be expanded and displayed with the actual number of frames related to the video component signal that is actually expanded and displayed. Based on the situation, frame dropping of the frame of the video component signal or the like is performed (for example, see Patent Document 1).

JP-A-10-136308

  In the conventional encoding transmission system, when the video and audio time stamp values cannot be accurately added due to lack of control accuracy of the processing of the video encoding unit, audio encoding unit and video decoding unit, audio decoding unit, etc. When the reproduction and output cannot be performed accurately at the timing, there is a problem that the AV synchronization with the video signal and the audio signal cannot be accurately performed.

  The present invention has been made to solve the above-described problems. When the time stamp value of video or audio cannot be accurately added due to lack of control accuracy of encoding or decoding processing, or accurately at the timing of the time stamp. It is an object of the present invention to obtain an encoding device, a decoding device, and an encoded transmission system that can automatically adjust AV synchronization to a state in which reproduction is not possible.

  The encoding apparatus according to the present invention includes a video signal processing unit for sending AV synchronization video data for indicating a video and audio synchronization position at an arbitrary timing, and the same timing as the timing of the video data for AV synchronization. And an audio signal processing unit for transmitting audio data for AV synchronization, a multiplexing unit for multiplexing the encoded video data and the encoded audio data, and transmitting the encoded data as encoded multiplexed data. Is.

  In the encoding apparatus according to the present invention, the video data and the audio data each include the AV synchronization data for indicating the synchronization position of the video and the audio. In addition, it is possible to automatically adjust the AV synchronization.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an encoded transmission system according to Embodiment 1 of the present invention.
In the figure, the encoding transmission system includes an encoding device 100 and a decoding device 200. The encoding apparatus 100 is an apparatus that encodes and outputs a video signal and an audio signal, and includes a video signal processing unit 101, an audio signal processing unit 102, an AV synchronization adjustment trigger generation unit 103, a video encoding unit 104, an audio An encoding unit 105 and a media multiplexing unit 106 are provided.

  The video signal processing unit 101 converts an externally input video signal into an encoding format, and when an AV synchronization adjustment trigger is given from the AV synchronization adjustment trigger generation unit 103, the video signal processing unit 101 performs AV synchronization. It is configured to send video data. The audio signal processing unit 102 converts an externally input audio signal into an encoding format, and when an AV synchronization adjustment trigger is given from the AV synchronization adjustment trigger generation unit 103, the audio signal processing unit 102 It is configured to send audio data. The AV synchronization adjustment trigger generation unit 103 is a control signal for the AV synchronization reference signal at an arbitrary timing by external control such as an operation panel or a set automatic timing such as immediately after a parameter change of the encoding device 100. Is transmitted to the video signal processing unit 101 and the audio signal processing unit 102.

  The video encoding unit 104 has a function of encoding video data in an encoding format output from the video signal processing unit 101 and encoding video data for AV synchronization. The audio encoding unit 105 has a function of encoding audio data in the encoding format output from the audio signal processing unit 102 and encoding audio data for AV synchronization. The media multiplexing unit 106 multiplexes the video data encoded by the video encoding unit 104, the audio data encoded by the audio encoding unit 105, and data notifying that an AV synchronization reference signal has been generated. For example, it is a functional unit for transmitting as an MPEG-2 transport stream.

  The decoding device 200 is a device that decodes an encoded video signal and audio signal and outputs them as a video signal and an audio signal, and includes a media separation unit 201, a video decoding unit 202, an audio decoding unit 203, and AV synchronization error detection. Unit 204, video delay unit 205, audio delay unit 206, and AV delay control unit 207. The media separation unit 201 receives the encoded multiplexed data transmitted from the media multiplexing unit 106, for example, an MPEG-2 transport stream, and the received encoded multiplexed data is converted into video encoded data, audio encoded data, etc. This is a functional unit that separates each piece of media and sends the separated data to the video decoding unit 202 and the audio decoding unit 203. Further, the media separation unit 201 has a function of notifying the AV synchronization error detection unit 204 of data that notifies that the AV synchronization reference signal has been generated in the encoded multiplexed data. .

  The video decoding unit 202 has a function of decoding the encoded video data transmitted from the media separation unit 201 and transmitting the decoded data to the video delay unit 205 and the AV synchronization error detection unit 204. The audio decoding unit 203 has a function of decoding the audio encoded data transmitted from the media separation unit 201 and transmitting the decoded data to the audio delay unit 206 and the AV synchronization error detection unit 204. The AV synchronization error detection unit 204 calculates an AV synchronization error based on the video data transmitted from the video decoding unit 202 and the audio data transmitted from the audio decoding unit 203, and calculates the calculated error as an AV delay control unit 207. It has a function to output to. The video delay unit 205 temporarily stores the video signal output from the video decoding unit 202, controls the timing based on the delay control signal from the AV delay control unit 207, and outputs the video signal to the outside. have. The audio delay unit 206 temporarily holds the audio signal output from the audio decoding unit 203, controls the timing based on the delay control signal from the AV delay control unit 207, and outputs the audio signal to the outside. It has a function to do. The AV delay control unit 207 controls the delay amounts of the video delay unit 205 and the audio delay unit 206 so that the AV synchronization of the video and audio is matched based on the calculation result of the AV synchronization error transmitted from the AV synchronization error detection unit 204. It has a function of sending out a control signal for this purpose.

  The encoding device 100 and the decoding device 200 are configured by a computer, and each component is configured by software corresponding to each function and hardware such as a CPU and a memory for executing the software. Yes.

Next, the operation of the first embodiment will be described.
First, in a state where the encoding transmission system is in operation, AV control is performed at an arbitrary timing from the outside such as an operation panel or at a set automatic timing such as immediately after a parameter change of the encoding device 100. An AV synchronization adjustment trigger signal is simultaneously transmitted from the synchronization adjustment trigger generation unit 103 to the video signal processing unit 101 and the audio signal processing unit 102. At this time, the media multiplexing unit 106 is notified that an AV synchronization reference signal has been generated.

FIG. 2 is an explanatory diagram showing video data and audio data when an AV synchronization reference signal is generated.
The operation when the AV synchronization reference signal is generated will be described below with reference to FIG.
The video signal processing unit 101 normally converts a video signal input from the outside into a data format for encoding and transmits the data to the video encoding unit 104. On the other hand, when the AV synchronization adjustment trigger signal sent from the AV synchronization adjustment trigger generator 103 is input, first, as shown in FIG. 2A, the black screen is displayed as the video data for detecting the reference position of the video signal. The video frame data is transmitted for several frames, and then the video data switched to the video frame of the color bar for several frames is replaced with the video input signal and transmitted to the video encoding unit 104. The position where the black frame video frame is switched to the color bar video frame is the reference position of the video signal during AV synchronization adjustment.

  The audio signal processing unit 102 normally converts an audio signal input from the outside into a data format for encoding and transmits the data to the audio encoding unit 105. On the other hand, when the AV synchronization adjustment trigger signal output from the AV synchronization adjustment trigger generator 103 is input, first, as shown in FIG. 2B, silent sound is generated as sound data for detecting the reference position of the sound signal. The frame data is transmitted for several frames, and then the audio frame data for switching from silence to sine wave at the timing synchronized with the reference position of the video signal at the time of AV synchronization adjustment is transmitted to the audio encoding unit 105.

The video encoding unit 104 encodes the video data for AV synchronization error detection transmitted from the video signal processing unit 101 without changing from the normal operation, and transmits the video encoded data to the media multiplexing unit 106. Similarly, the audio encoding unit 105 encodes the audio data for AV synchronization error detection transmitted from the audio signal processing unit 102 without changing from the normal operation, and transmits the audio encoded data to the media multiplexing unit 106. To do.
The media multiplexing unit 106 multiplexes the encoded data transmitted from the video encoding unit 104 and the audio encoding unit 105 and data notifying that the AV synchronization reference signal has been generated, and passes through a communication medium such as a satellite. Then, the encoded multiplexed data, for example, an MPEG transport stream is transmitted to the decoding device 200.

When the decoding apparatus 200 receives the encoded data transmitted from the media multiplexing unit 106, for example, an MPEG transport stream, first, the received encoded multiplexed data is converted into video encoded data, audio by the media separation unit 201. The data is separated for each medium such as encoded data, and each separated data is transmitted to the video decoding unit 202 and the audio decoding unit 203. Also, at this time, the AV synchronization error detection unit 204 is notified that the AV synchronization reference signal has been generated.
The video decoding unit 202 decodes the encoded video data transmitted from the media separation unit 201 and transmits the decoded data to the video delay unit 205 and the AV synchronization error detection unit 204. The audio decoding unit 203 decodes the audio encoded data transmitted from the media separation unit 201 and transmits the decoded data to the audio delay unit 206 and the AV synchronization error detection unit 204.

When the AV synchronization error detection unit 204 receives the AV synchronization reference signal generation notification from the media separation unit 201 and detects video data that becomes a black screen for several frames from the video data transmitted from the video decoding unit 202, the AV synchronization error detection unit 204 The video signal reference position (timing to switch to the color bar) is waited for at the time of synchronization adjustment. As for the audio data, when audio data that becomes silent audio for several frames is detected from the audio data transmitted from the audio decoding unit 203, the audio signal is waiting for a reference position (timing to switch to a sine wave) during AV synchronization adjustment. It becomes. When switching to the reference position occurs in the data of one of the media from the state of waiting for the reference position, the difference from that time until the other medium is switched to the reference position is calculated, and the calculation result is used as the AV delay control unit. To 207.
The AV delay control unit 207 determines the delay amounts of the video delay unit 205 and the audio delay unit 206 so that the AV synchronization of the video signal and the audio signal is matched based on the calculation result of the AV synchronization error transmitted from the AV synchronization error detection unit 204. To control.
The video delay unit 205 and the audio delay unit 206 temporarily hold the video data and audio data transmitted from the video decoding unit 202 and the audio decoding unit 203, and timing based on the control signal transmitted from the AV delay control unit 207. To output video and audio signals to the outside.

  Through the above processing, video and audio time stamp values cannot be accurately added due to insufficient processing of the video encoding unit, audio encoding unit, video decoding unit, and audio decoding unit, or playback and output cannot be performed accurately according to the time stamp. Even in the case of an encoded transmission system that needs to output video and audio signals at a timing different from the time stamp value, such as with a frame synchronizer function, the encoded transmission with AV synchronization matched with the decoded output Can be realized.

  In the first embodiment, data for switching from a black screen to a color bar is used as video data for detecting the reference position of a video signal, and data for switching from silence to sine wave is used as audio data for detecting the reference position of an audio signal. However, the present invention is not limited to these data, and may be signals having other contents such as a specific pattern of video and audio frames.

  As described above, according to the encoding apparatus of the first embodiment, the video signal processing unit that transmits the video data for AV synchronization for indicating the synchronization position of the video and audio at an arbitrary timing, and the video signal processing A video encoding unit that encodes video data from the audio signal processing unit, an audio signal processing unit that transmits audio data for AV synchronization at the same timing as the video data for AV synchronization, and audio from the audio signal processing unit An audio encoding unit that encodes data, and a multiplexing unit that multiplexes the video data encoded by the video encoding unit and the audio data encoded by the audio encoding unit, and transmits the multiplexed data as encoded multiplexed data Therefore, on the receiving side, encoded multiplexed data that can be automatically adjusted to a state in which AV synchronization is achieved can be transmitted.

  Further, according to the decoding apparatus of the first embodiment, the video decoding unit that decodes the encoded video data, the audio decoding unit that decodes the encoded audio data, and the AV included in the video data decoded by the video decoding unit. An AV synchronization error detecting unit for detecting a synchronization error between the video data and the audio data based on the video data for synchronization and the audio data for AV synchronization included in the audio data decoded by the audio decoding unit; Since the AV delay control unit that controls these delay amounts is provided so that the output timing of the video data and the audio data is synchronized based on the synchronization error detected by the detection unit, the time stamp values of the video and audio are Even if it is not accurately added, it can be automatically adjusted to a state in which the AV synchronization is suitable.

  Moreover, according to the encoding transmission system of Embodiment 1, it is an encoding transmission system which consists of an encoding apparatus and a decoding apparatus, and an encoding apparatus shows the synchronous position of a video and an audio | voice at arbitrary timings. A video signal processing unit for transmitting video data for AV synchronization, a video encoding unit for encoding video data from the video signal processing unit, and AV synchronization at the same timing as the video data for AV synchronization Audio signal processing unit for transmitting audio data, audio encoding unit for encoding audio data from the audio signal processing unit, video data encoded by the video encoding unit, and encoding by the audio encoding unit And a multiplexing unit that multiplexes the encoded audio data and transmits the encoded audio data as encoded multiplexed data. The decoding device includes a video decoding unit that decodes the encoded video data, and an audio decoding unit that decodes the encoded audio data The video data and the audio data are synchronized based on the AV synchronization video data included in the video data decoded by the video decoding unit and the AV synchronization audio data included in the audio data decoded by the audio decoding unit. An AV synchronization error detection unit that detects an error, and an AV delay control unit that controls the delay amounts of the video data and the audio data so that the output timings of the video data and the audio data are synchronized based on the synchronization error detected by the AV synchronization error detection unit AV synchronization is automatically performed even when the time stamp value of video and audio cannot be accurately added due to lack of control accuracy of encoding or decoding processing, or when it cannot be accurately reproduced and output at the timing of the time stamp. Can be adjusted to the correct state.

Embodiment 2. FIG.
FIG. 3 is a block diagram showing an encoded transmission system according to Embodiment 2 of the present invention.
In the figure, the encoding transmission system includes an encoding device 100a and a decoding device 200a. The encoding apparatus 100a includes a video signal processing unit 101, an audio signal processing unit 102, an AV synchronization adjustment trigger generation unit 103, a video encoding unit 104, an audio encoding unit 105, a media multiplexing unit 106, and an auxiliary data transmission unit 107. It has. Here, the basic functions of the video signal processing unit 101 to the audio encoding unit 105 are the same as those in the first embodiment shown in FIG. The AV synchronization adjustment trigger generation unit 103 is configured to output the generated AV synchronization adjustment trigger to the audio signal processing unit 102 and the auxiliary data transmission unit 107.

  The auxiliary data transmission unit 107 is configured by adding data to be superimposed on the blanking area of the video and reproduction information synchronized with the video when the AV synchronization adjustment trigger is sent from the AV synchronization adjustment trigger generating unit 103. And a function of transmitting the data to the media multiplexing unit 106. The media multiplexing unit 106 a multiplexes the video encoded data output from the video encoding unit 104 and the audio encoded data output from the audio encoding unit 105, and outputs the multiplexed data from the auxiliary data transmission unit 107. A function of superimposing the multiplexed auxiliary data on the blanking area of the video and multiplexing it.

  The decoding device 200a includes a media separation unit 201a, video decoding unit 202, audio decoding unit 203, AV synchronization error detection unit 204, video delay unit 205, audio delay unit 206, AV delay control unit 207, auxiliary data reception unit 208, auxiliary data A data superimposing unit 209 is provided. The media separation unit 201a receives the encoded data transmitted from the media multiplexing unit 106a, for example, an MPEG-2 transport stream, and converts the received encoded multiplexed data into video encoded data, audio encoded data, and the like. It is a functional unit that separates each medium and separates auxiliary data superimposed on a video blanking area and sends the separated data to the video decoding unit 202, the audio decoding unit 203, and the auxiliary data receiving unit 208. The auxiliary data receiving unit 208 has a function of extracting data to be superimposed on the blanking area from the received auxiliary data and transmitting the data to the auxiliary data superimposing unit 209.

  The auxiliary data superimposing unit 209 superimposes the auxiliary data transmitted from the auxiliary data receiving unit 208 on the blanking area of the video data transmitted from the video decoding unit 202, and transmits this video data to the AV synchronization error detecting unit 204a. It has a function. The AV synchronization error detection unit 204a is configured to detect an AV synchronization error based on the auxiliary data superimposed on the video data output from the auxiliary data superimposing unit 209 and the audio data transmitted from the audio decoding unit 203. ing. Since the configuration other than this is the same as that of the first embodiment shown in FIG. 1, the same reference numerals are given to corresponding portions, and the description thereof is omitted.

Next, the operation of the second embodiment will be described.
First, in a state where the encoding transmission system is in operation, AV control is performed at an arbitrary timing from the outside such as an operation panel or at a set automatic timing such as immediately after a parameter change of the encoding device 100. An AV synchronization adjustment trigger signal is simultaneously transmitted from the synchronization adjustment trigger generation unit 103 to the audio signal processing unit 102 and the auxiliary data transmission unit 107. Also, the AV synchronization adjustment trigger generation unit 103 notifies the media multiplexing unit 106a that an AV synchronization reference signal has been generated.

FIG. 4 is an explanatory diagram showing video data, audio data, and auxiliary data when an AV synchronization reference signal is generated.
The operation when the AV synchronization reference signal is generated will be described below with reference to FIG.
The auxiliary data transmission unit 107 does not normally need to operate, but when the AV synchronization adjustment trigger signal output from the AV synchronization adjustment trigger generation unit 103 is input, data for detecting the reference position of the auxiliary data Is sent out. As this data, as shown in FIG. 4A, first, fixed data in which all the data to be superimposed on the blanking area of the video are set to 0 is transmitted, and then the reference position of the video at the time of AV synchronization adjustment. The fixed data in which all the data to be superimposed on the blanking area are set to 1 at the timing of the video frame to be transmitted to the media multiplexing unit 106a.

  The video signal processing unit 101 does not change the operation when the AV synchronization reference signal is generated, and converts the video signal input from the outside into a data format for encoding and transmits the data to the video encoding unit 104. The audio signal processing unit 102 normally converts an audio signal input from the outside into a data format for encoding and transmits the data to the audio encoding unit 105. On the other hand, when the AV synchronization adjustment trigger signal is input from the AV synchronization adjustment trigger generation unit 103, the audio signal processing unit 102, as shown in FIG. First, silent audio frame data is transmitted for several frames, and then audio frame data for switching from silence to sine wave is transmitted to the audio encoding unit 105 at a timing synchronized with the reference position of the video data at the time of AV synchronization adjustment.

  The video encoding unit 104 encodes the video data transmitted from the video signal processing unit 101 without changing from the normal operation, and transmits the encoded video data to the media multiplexing unit 106a. Similarly to Embodiment 1, speech encoding section 105 encodes the AV synchronization error detection speech data transmitted from speech signal processing section 102, and stores the speech encoded data in media multiplexing section 106a. Send. The media multiplexing unit 106a notifies that the auxiliary data transmitted from the auxiliary data transmitting unit 107, the encoded data transmitted from the video encoding unit 104 and the audio encoding unit 105, and the AV synchronization reference signal have been generated. The data is multiplexed, and encoded multiplexed data, for example, an MPEG transport stream is transmitted to the decoding apparatus 200a via a communication medium such as a satellite.

  When the decoding apparatus 200a receives the encoded data transmitted from the media multiplexing unit 106a, for example, an MPEG transport stream, first, the received encoded multiplexed data is converted into video encoded data, audio code by the media separating unit 201a. The data is separated for each medium such as digitized data and auxiliary data, and the separated data is transmitted to the video decoding unit 202, the audio decoding unit 203, and the auxiliary data receiving unit 208, respectively. At this time, the AV synchronization error detection unit 204a is notified that the AV synchronization adjustment signal has been generated.

The video decoding unit 202 decodes the encoded video data transmitted from the media separation unit 201a, and transmits the decoded data to the video delay unit 205 and the auxiliary data multiplexing unit 209. The audio decoding unit 203 decodes the audio encoded data transmitted from the media separation unit 201a, and transmits the decoded data to the audio delay unit 206 and the auxiliary data superimposing unit 209.
The auxiliary data receiving unit 208 extracts data to be superimposed on the blanking area from the auxiliary data transmitted from the media separation unit 201a, and transmits the data to the auxiliary data superimposing unit 209.
The auxiliary data superimposing unit 209 superimposes the auxiliary data transmitted from the auxiliary data receiving unit 208 on the blanking region of the video frame that matches the reproduction information of the video data transmitted from the video decoding unit 202. The video data on which the auxiliary data is superimposed is transmitted to the AV synchronization error detection unit 204a.

When the AV synchronization error detection unit 204a receives the AV synchronization adjustment signal generation notification from the media separation unit 201a, the auxiliary data for several frames becomes 0 from the blanking area in the video data transmitted from the auxiliary data superimposing unit 209. Detect video frames. As a result, the video signal reference position (timing at which auxiliary data is switched from 0 to 1) is waited for during AV synchronization adjustment. As for the audio data, when audio data that becomes silent audio for several frames is detected from the audio data transmitted from the audio decoding unit 203, the audio signal waits for a reference position (timing to switch to a sine wave) at the time of AV synchronization adjustment. It becomes a state. When switching to the reference position occurs in the data of one of the media from the state of waiting for the reference position, the difference from that time until the other medium is switched to the reference position is calculated, and the calculation result is used as the AV delay control unit. To 207.
The AV delay control unit 207 determines the delay amounts of the video delay unit 205 and the audio delay unit 206 so that the AV synchronization of the video signal and the audio signal matches from the calculation result of the AV synchronization error transmitted from the AV synchronization error detection unit 204a. Control.

  The video delay unit 205 and the audio delay unit 206 temporarily hold the video data and audio data transmitted from the video decoding unit 202 and the audio decoding unit 203, and the video signal and the timing are controlled by the AV delay control unit 207. Output audio signals to the outside.

  Through the above processing, video and audio time stamp values cannot be accurately added due to insufficient processing of the video encoding unit, audio encoding unit, video decoding unit, and audio decoding unit, or playback and output cannot be performed accurately according to the time stamp. Even in this case, encoded transmission with AV synchronization can be realized.

  In the second embodiment, the data for switching the auxiliary data from 0 to 1 is superimposed on the blanking area of the video, and the data for switching from silence to sine wave is used as the audio data for detecting the reference position of the audio signal. Although used, the present invention is not limited to these data, and may be a signal having other contents as long as auxiliary data indicates a position for AV synchronization.

  As described above, according to the encoding apparatus of the second embodiment, the auxiliary data transmission unit that transmits the auxiliary data for AV synchronization for indicating the synchronization position of the video and audio at any timing, and the AV synchronization An audio signal processing unit for transmitting audio data for AV synchronization at the same timing as the auxiliary data, an audio encoding unit for encoding audio data from the audio signal processing unit, auxiliary data, and audio encoding And a multiplexing unit that multiplexes the audio data encoded by the unit and transmits the encoded multiplexed data as encoded multiplexed data, so that the receiving side can automatically adjust to the state in which AV synchronization is achieved. Data can be sent out.

  In addition, according to the decoding apparatus of the second embodiment, the auxiliary data receiving unit that receives the auxiliary data included in the encoded multiplexed data, the audio decoding unit that decodes the encoded audio data, and the AV synchronization included in the auxiliary data. AV synchronization error detecting unit for detecting a synchronization error between video data and audio data based on data for use and audio data for AV synchronization included in the audio data decoded by the audio decoding unit, and AV synchronization error detecting unit In order to synchronize the output timing of the video data and the audio data based on the synchronization error detected in step 1, the AV delay control unit for controlling these delay amounts is provided. Even if it is not added, it is possible to automatically adjust to a state in which AV synchronization is suitable.

  Further, according to the encoding apparatus of the second embodiment, the multiplexing unit superimposes the auxiliary data on the video blanking region and multiplexes the video data and the audio data. Therefore, the degree of freedom of AV synchronization processing can be improved.

  Further, according to the decoding apparatus of the second embodiment, the video decoding unit that decodes the encoded video data, and the auxiliary that superimposes the auxiliary data on the video blanking region with respect to the video data decoded by the video decoding unit A data superimposing unit, and the AV synchronization error detecting unit detects a synchronization error between the video data and the audio data based on the data for AV synchronization output from the auxiliary data superimposing unit. Since the process and the process for the AV synchronization process can be performed separately, the degree of freedom of the AV synchronization process can be improved.

  In addition, according to the encoding transmission system of the second embodiment, the encoding transmission system includes an encoding device and a decoding device, and the encoding device indicates the synchronization position of video and audio at an arbitrary timing. From the auxiliary data transmitting unit for transmitting auxiliary data for AV synchronization, the audio signal processing unit for transmitting audio data for AV synchronization at the same timing as the auxiliary data for AV synchronization, and the audio signal processing unit A speech encoding unit that encodes speech data, an auxiliary data, and a multiplexing unit that multiplexes the speech data encoded by the speech encoding unit and transmits the multiplexed data as encoded multiplexed data. Auxiliary data receiving unit for receiving auxiliary data included in encoded multiplexed data, an audio decoding unit for decoding encoded audio data, AV synchronization data included in auxiliary data, and an audio decoding unit Based on AV synchronization audio data included in the decoded audio data, an AV synchronization error detection unit that detects a synchronization error between the video data and the audio data, and a synchronization error detected by the AV synchronization error detection unit In order to synchronize the output timing of video data and audio data, an AV delay control unit for controlling these delay amounts is provided, so that time stamp values of video and audio can be set due to insufficient control accuracy of encoding or decoding processing. Even when it cannot be accurately added or when it cannot be accurately reproduced and output at the timing of the time stamp, it can be automatically adjusted to a state in which AV synchronization is achieved.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the encoding transmission system by Embodiment 1 of this invention. It is explanatory drawing which shows the video data and audio | voice data when the AV synchronous reference signal of the encoding transmission system by Embodiment 1 of this invention is generated. It is a block diagram which shows the encoding transmission system by Embodiment 2 of this invention. It is explanatory drawing which shows the video data, auxiliary data, and audio | voice data when the AV synchronous reference signal of the encoding transmission system by Embodiment 2 of this invention is generated.

Explanation of symbols

  100, 100a encoding apparatus, 101 video signal processing unit, 102 audio signal processing unit, 103 AV synchronization adjustment trigger generating unit, 104 video encoding unit, 105 audio encoding unit, 106, 106a media multiplexing unit, 107 auxiliary Data transmission unit, 107, 200, 200a decoding device, 201, 201a media separation unit, 202 video decoding unit, 203 audio decoding unit, 204, 204a AV synchronization error detection unit, 207 AV delay control unit, 208 auxiliary data reception unit, 209 Auxiliary data superimposing unit.

Claims (8)

A video signal processing unit for sending video data for AV synchronization for indicating a video and audio synchronization position at an arbitrary timing;
A video encoding unit for encoding video data from the video signal processing unit;
An audio signal processing unit for transmitting audio data for AV synchronization at the same timing as the video data for AV synchronization;
An audio encoding unit for encoding audio data from the audio signal processing unit;
An encoding apparatus comprising: a multiplexing unit that multiplexes video data encoded by the video encoding unit and audio data encoded by the audio encoding unit and transmits the multiplexed data as encoded multiplexed data. A video decoding unit for decoding the encoded video data;
A voice decoding unit for decoding the encoded voice data;
Based on the video data for AV synchronization included in the video data decoded by the video decoding unit and the audio data for AV synchronization included in the audio data decoded by the audio decoding unit, the video data and the audio data An AV synchronization error detection unit for detecting a synchronization error;
A decoding apparatus comprising: an AV delay control unit that controls a delay amount of the video data and the audio data based on a synchronization error detected by the AV synchronization error detection unit so that output timings of the video data and the audio data are synchronized. An auxiliary data transmitting unit for transmitting auxiliary data for AV synchronization for indicating a synchronization position of video and audio at an arbitrary timing;
An audio signal processing unit for sending audio data for AV synchronization at the same timing as the auxiliary data timing for AV synchronization;
An audio encoding unit for encoding audio data from the audio signal processing unit;
An encoding apparatus comprising: a multiplexing unit that multiplexes the auxiliary data and the audio data encoded by the audio encoding unit and transmits the multiplexed data as encoded multiplexed data.   4. The encoding apparatus according to claim 3, wherein the multiplexing unit multiplexes the video data and the audio data by superimposing the auxiliary data on the blanking area of the video. An auxiliary data receiving unit for receiving auxiliary data included in the encoded multiplexed data;
A voice decoding unit for decoding the encoded voice data;
AV synchronization error for detecting a synchronization error between video data and audio data based on AV synchronization data included in the auxiliary data and AV synchronization audio data included in the audio data decoded by the audio decoding unit A detection unit;
A decoding apparatus comprising: an AV delay control unit that controls a delay amount of the video data and the audio data based on a synchronization error detected by the AV synchronization error detection unit so that output timings of the video data and the audio data are synchronized. A video decoding unit for decoding the encoded video data;
An auxiliary data superimposing unit that superimposes auxiliary data on a blanking region of the video with respect to the video data decoded by the video decoding unit,
6. The decoding apparatus according to claim 5, wherein the AV synchronization error detection unit detects a synchronization error between the video data and the audio data based on the AV synchronization data output from the auxiliary data superimposing unit. An encoding transmission system comprising an encoding device and a decoding device,
The encoding device includes:
A video signal processing unit for sending video data for AV synchronization for indicating a video and audio synchronization position at an arbitrary timing;
A video encoding unit for encoding video data from the video signal processing unit;
An audio signal processing unit for transmitting audio data for AV synchronization at the same timing as the video data for AV synchronization;
An audio encoding unit for encoding audio data from the audio signal processing unit;
A multiplexing unit that multiplexes the video data encoded by the video encoding unit and the audio data encoded by the audio encoding unit, and transmits the multiplexed data as encoded multiplexed data;
The decoding device
A video decoding unit for decoding the encoded video data;
A voice decoding unit for decoding the encoded voice data;
Based on the video data for AV synchronization included in the video data decoded by the video decoding unit and the audio data for AV synchronization included in the audio data decoded by the audio decoding unit, the video data and the audio data An AV synchronization error detection unit for detecting a synchronization error;
An encoding transmission system including an AV delay control unit that controls the delay amount so that the output timing of the video data and the audio data is synchronized based on the synchronization error detected by the AV synchronization error detection unit . An encoding transmission system comprising an encoding device and a decoding device,
The encoding device includes:
An auxiliary data transmitting unit for transmitting auxiliary data for AV synchronization for indicating a synchronization position of video and audio at an arbitrary timing;
An audio signal processing unit for sending audio data for AV synchronization at the same timing as the auxiliary data timing for AV synchronization;
An audio encoding unit for encoding audio data from the audio signal processing unit;
A multiplexing unit that multiplexes the auxiliary data and the audio data encoded by the audio encoding unit and transmits the multiplexed data as encoded multiplexed data;
The decoding device
An auxiliary data receiving unit for receiving auxiliary data included in the encoded multiplexed data;
A voice decoding unit for decoding the encoded voice data;
AV synchronization error for detecting a synchronization error between video data and audio data based on AV synchronization data included in the auxiliary data and AV synchronization audio data included in the audio data decoded by the audio decoding unit A detection unit;
An encoding transmission system including an AV delay control unit that controls the delay amount so that the output timing of the video data and the audio data is synchronized based on the synchronization error detected by the AV synchronization error detection unit .

Images