KR100392251B1 - Digital Watermark Insertion Apparatus for Copyright Protection of Digital Audio/Video/Multimedia - Google Patents

Abstract

The present invention relates to an apparatus for inserting a digital watermark including copyright information into digital audio / video and multimedia, wherein the audio / video having a watermark inserted therein without degrading the quality of audio and video even after the watermark is inserted; Ensure that the watermark is robust even if any attack is made against the multimedia.

As the use of computer networks and computers such as the Internet is rapidly increasing, the use and dissemination of digital multimedia data is becoming common. However, such multimedia data can be easily copied due to the digital property, and anyone can easily obtain digital data through illegal copying. As a result, there is a demand to prevent illegal copying and to protect the copyright and ownership of the author who produced the multimedia contents.

In the watermark embedding technology, which is one of the data protection technologies, the present invention uses a perceptual model in consideration of human visual and auditory characteristics, so that the quality of audio, video, and multimedia does not deteriorate even after the watermark is inserted. The watermark does not disappear even if an arbitrary attack is made against the embedded audio / video and multimedia.

Description

Digital Watermark Insertion Apparatus for Copyright Protection of Digital Audio / Video / Multimedia}

The present invention relates to an apparatus for inserting a digital watermark including copyright information into digital audio / video / multimedia. More specifically, the present invention relates to a digital audio / video / multimedia using a perceptual model considering human visual and auditory characteristics. A digital watermark embedding apparatus.

As the use of computer networks and computers such as the Internet is rapidly increasing, the use and dissemination of digital multimedia data is becoming common. However, such multimedia data can be easily copied due to the digital property, and anyone can easily obtain digital data through illegal copying. As a result, there has been a demand to prevent illegal copying and to protect copyrights and ownership of writers producing multimedia contents.

To date, the most widely used data protection method is the data encryption method. This data encryption method is impossible to access the data without knowing the password, but once decrypted, there is a problem that can be illegally copied and distributed without any sanction.

To solve this problem, a method of inserting a watermark, which is a kind of digital signature, into the original signal has been proposed. A digital watermark is a kind of copyright protection technology that solves a copyright problem by inserting a signal of an author unrecognizable in a multimedia and extracting a watermark embedded in the multimedia later. Watermark may be inserted in the form of a logo that can be seen by humans, but due to the ease of deletion of watermarks and the deterioration of the multimedia quality, there are many ways to insert a watermark in a form that cannot be seen in recent years. Research is used.

There are some requirements for embedding invisible watermarks. First, the inserted watermark must have a non-perceptibility that cannot be distinguished by the human eye. That is, the degradation of the quality of the original multimedia caused by the insertion of the watermark should not occur, and should almost never occur. Second, watermarks must be robust. In other words, the inserted watermark must have robustness that is not deleted even when subjected to various data processing or deliberate attacks by others who want to delete the watermark. Third, the watermark must be unambiguous. Inhomogeneity is the property of being able to identify exactly who the inserted watermark is. That is, there should be no problem such as determining that an uninserted watermark is inserted or misinterpreting the inserted watermark as a watermark of another person.

However, copyright protection methods for audio and video using watermarks that have been published so far may reduce the quality of the audio or video after inserting the watermark, or cause arbitrary attacks such as compression or filtering on the audio or video embedded in the watermark. In this case, the inserted watermark disappears.

Accordingly, an object of the present invention for solving the above problems of the prior art is that the quality of audio / video and multimedia does not deteriorate even after a watermark signal is inserted using a perceptual model considering human visual and auditory characteristics. Rather, it is to provide a watermark embedding apparatus in which a watermark signal including copyright information does not disappear even when various attacks are applied to the audio / video and multimedia in which the watermark is embedded.

1 is a block diagram showing a digital watermark embedding apparatus of digital multimedia according to an embodiment of the present invention;

2 is a detailed block diagram of the audio watermarking unit shown in FIG. 1;

FIG. 3 is a detailed block diagram illustrating the video watermarking unit shown in FIG. 1.

※ Explanation of code about main part of drawing ※

100: audio water marking unit

101: frequency converter 102: perceptual modeling unit

103: error correction encoder 104: pseudo noise sequence generator

105: frequency converter 106: time-varying adaptive filter unit

107: frequency inverse transform unit 108: the adder

200: video watermarking unit

201: frequency conversion unit 202: perceptual modeling unit

203 error correction encoder 204 pseudo noise sequence generator

205: frequency converter 206: time-varying adaptive filter unit

207: adder 208: frequency inverse converter

300: audio encoder 400: video encoder

500: multiplexer

Watermark embedding apparatus of the digital multimedia according to the present invention for achieving the above object, the audio watermarking unit for inserting an audio watermark including audio copyright information in the digital audio;

An audio encoder which compresses the watermark-embedded audio signal output from the audio watermarking unit;

A video watermarking unit for inserting a video watermark including video copyright information into digital video,

A video encoder for compressing the watermark-embedded video signal output from the video watermarking unit, and

And a multiplexer for multiplexing the watermark-embedded compressed audio and video signals outputted from the audio encoder and the video encoder to output the watermark-embedded multimedia.

In addition, the digital watermark embedding apparatus for digital audio according to the present invention includes perceptual modeling means for converting original audio into a frequency domain and extracting a masking threshold value considering human hearing characteristics in the frequency domain;

Audio copyright information modulation means for multiplying audio copyright information with pseudo noise,

Frequency converting means for converting the modulated audio copyright information into a frequency domain;

Time-varying adaptive filter means for filtering audio copyright information output from the frequency converting means by using a masking threshold value extracted from the perceptual modeling means, and

And an output means for converting a signal output from the time-varying adaptive filter means into a time domain, adding the original audio signal, and outputting a watermark-embedded audio signal.

In addition, the digital watermark embedding apparatus for digital video according to the present invention comprises: perceptual modeling means for receiving a raw video and extracting visual weights in consideration of human visual characteristics;

Video copyright information modulation means for multiplying video copyright information by pseudo noise,

Time-varying adaptive filter means for filtering the modulated video copyright information by using a masking threshold value extracted from the perceptual modeling means, and

And adding means for adding the signal output from the time-varying adaptive filter means and the original video to output the watermark-embedded audio signal.

Hereinafter, a digital watermark embedding apparatus of digital audio / video / multimedia according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a digital watermark embedding apparatus for digital multimedia according to the present invention.

Referring to FIG. 1, the present invention provides an audio watermarking unit 100 that receives audio and audio copyright information and inserts a watermark into audio, and a video water that receives a video and video copyright information and inserts a watermark into the video. The marking unit 200, the audio encoding unit 300 for compressing the watermarking inserted audio, the video encoding unit 400 for compressing the watermarking inserted video, and the watermark is inserted and multiplexed audio and video It includes a multiplexer 500.

Audio and video copyright information is repeatedly inserted into the audio and video input to the audio watermarking unit 100 and the video watermarking unit 200 at regular time intervals. At this time, the amount of data to be inserted can be arbitrarily adjusted. That is, the amount of information to be inserted can be adjusted by adjusting the size of the audio and video section into which the copyright information is to be inserted. In addition, the copyright information to be inserted into the audio and video signals is not limited in its form (image, video, audio, voice, text, etc.).

2 is a detailed block diagram of the audio watermarking unit 100 according to the present invention.

Referring to FIG. 2, the audio watermarking unit 100 receives a frequency conversion unit 101 for converting input audio into a frequency domain and audio frequency converted by the frequency conversion unit 101 to receive human hearing characteristics. Perceptual modeling unit 102 for extracting the masking threshold value considered, error correction encoder 103 for correcting an error that may occur when watermark is detected by receiving audio copyright information, pseudo noise to modulate error corrected audio copyright information A pseudo-noise sequence generation unit 104 for generating a signal, a frequency converter 105 for converting the modulated audio copyright information into a frequency domain, and a frequency converter 105 using a masking threshold extracted from the perceptual modeling unit 102. A time-varying adaptive filter unit 106 for filtering a signal output from the frequency converter, a frequency inverse transformer 107 for converting a signal output from the time-varying adaptive filter unit 106 into a time domain, and Adding the signal output from the video and the frequency inverse conversion unit 107 and an adder 108 for outputting the watermark audio signal.

Here, the frequency converters 101 and 105 convert the audio into the frequency domain through a transform such as a one-dimensional Fourier transform, a one-dimensional wavelet transform, or a one-dimensional Discrete Cosine Transform (DCT) transform.

The operation of the audio watermarking unit 100 will be described below. The input audio is converted into a frequency domain through the frequency converter 101, and the perceptual modeling unit 102 extracts a masking threshold value considering human hearing characteristics. Here, the masking threshold refers to a threshold that can detect the audio original sound in the frequency domain. Therefore, even if a signal component below the masking threshold is added to the original sound of the audio, the added sound cannot be detected. This masking threshold can be obtained using the psychoacoustic model used in MPEG audio coding, which is based on "ISO / IEC 11172-3 Information Technlogy-Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit / s, 1993 ".

Meanwhile, the error correction encoder 103 corrects an error that may occur when watermark is detected in the audio copyright information. The pseudo noise sequence generator 104 generates and outputs pseudo noise, and the error-corrected audio copyright information is multiplied by the pseudo noise in the multiplier and modulated. The frequency converter 105 converts this modulated audio copyright information into data in the frequency domain. The frequency converter 105 performs the same function as the frequency converter 101.

The pseudo noise sequence generator 104 outputs a pseudo noise sequence having a sampling frequency of 32 kHz, and the pseudo noise sequence passes through a low pass filter having a cutoff frequency of 16 kHz. When the audio copyright information is modulated using this low-pass filtered pseudonoise sequence, the watermark information included in the audio information does not deviate beyond 16 kHz, so even if an attack such as a compression attack or lowpass filtering is performed, The watermark data does not disappear. The pseudo noise sequence generator 104 may output a pseudo noise sequence having a sampling frequency of up to 48 kHz. At this time, the bandwidth is 24 kHz, which is half of that, and the cutoff frequency of the low pass filter is also 24 kHz.

The time-varying adaptive filter 106 filters the signal passing through the frequency converter 105 using the masking threshold received from the perceptual modeling unit 102. That is, the time-varying adaptive filter unit 106 changes the characteristics of the filter according to the characteristics of the audio that changes with time, and the signal output from the frequency converter 105 has the masking threshold value obtained from the perceptual modeling unit 102. Adjust it to have a size close to the size. In addition, the frequency band of 16 kHz or more is filtered out.

When processing an audio signal, the audio signal is sequentially processed in units of a frame having a predetermined size, and different masking thresholds are obtained for each frame. That is, since the masking threshold changes every frame, the signal output from the frequency converter 105 is adjusted to have a size close to the size of the masking threshold obtained from the perceptual modeling unit 102. do. The time-varying adaptive filter 106 creates an auto-regressive (AR) filter based on the masking threshold obtained, and then passes the AR filter to the signal output from the frequency converter 105 to have a size close to the masking threshold. do.

The signal passing through the time-varying adaptive filter unit 106 passes through the frequency inverse transform unit 107 to be converted into a signal in the time domain, and finally becomes a watermark signal to be inserted into the audio signal. The watermark signal generated through the above process is added to the original audio by the adder 108, and finally, the audio signal having the watermark embedded therein is produced.

3 is a detailed block diagram of the video watermarking unit 200 according to the present invention.

Referring to FIG. 3, the video watermarking unit 200 receives a frequency conversion unit 201 for converting an input video into a frequency domain, and receives a frequency-converted video from the frequency conversion unit 201 to obtain human visual characteristics. Perceptual modeling unit 202 for extracting the masking threshold value considered, error correction encoder 203 for correcting an error that may occur when watermark is detected by receiving video copyright information, pseudo noise to modulate error-corrected audio copyright information A pseudo-noise sequence generator 204 for generating a signal, a frequency converter 205 for converting the modulated audio copyright information into a frequency domain, and a frequency converter 205 using a masking threshold extracted from the perceptual modeling unit 202. The time-varying adaptive filter unit 206 and the signal output from the time-varying adaptive filter unit 206 and the frequency converter 201 are converted into a frequency domain. An adder 207 for adding the call to output the watermarked video signal in the frequency domain, and a frequency inverse converter 208 for converting the watermarked video signal output from the adder 207 into the time domain. .

Here, the frequency converter 201 transforms the video into the frequency domain through a transform such as a two-dimensional Fourier transform, a two-dimensional wavelet transform, or a two-dimensional DCT (Discrete Cosine Transform) transform.

In addition, the frequency converter 201, 205 and the frequency inverse converter 208 may be removed. In this case, the video watermarking process is entirely performed in the time domain, and the perceptual modeling unit 202 also has a visual weight in the time domain. Will be extracted.

The operation of the video watermarking unit 200 will be described below. The input video is converted into the frequency domain through the frequency converter 201, and the perceptual modeling unit 202 extracts the visual weights of the frequency domain in consideration of the human visual characteristics.

In the time domain, an image signal is an edge where the edge corresponding to the boundary of the object is visually sensitive. The visual weight can be obtained with a Laplacian filter or similar filtering method that can find edges. In the frequency domain, a Just Noticeable Difference (JND) is obtained from a DCT or a wavelet transformed frequency domain and used as a visual weight.

The error correction encoder 203 corrects an error that may occur when watermark is detected from video copyright information. The pseudo noise sequence generator 204 generates and outputs pseudo noise, and the error-corrected video copyright information is multiplied by the pseudo noise in the multiplier and modulated. The frequency converter 205 converts the modulated video copyright information into data in the frequency domain. The frequency converter 105 performs the same function as the frequency converter 101.

The time-varying adaptive filter 206 filters the video signal passed through the frequency converter 205 by using the visual weight received from the perceptual modeling unit 202. That is, the time-varying adaptive filter unit 206 changes the characteristics of the filter differently according to the characteristics of the video that changes with time, and the signal output from the frequency converter 205 may be used in the frequency domain obtained by the perceptual modeling unit 202. Filter using visual weights.

The video signal is also processed in units of frames, and since each frame has different image data, its contents change with time. Therefore, different visual weights are obtained every frame, and multiplying the obtained visual weights of each frame means filtering using visual weights.

The signal passing through the time-varying adaptive filter unit 206 and the signal passing through the frequency converter 201 are added by the adder 207 to output a watermark embedded video signal, and the watermark embedded video signal is output. Passed through the frequency inverse converter 208 is converted into a signal in the time domain is output.

As described above, the frequency converter 201, the frequency converter 205, and the frequency inverse converter 207 may be omitted to obtain a computational gain. In this case, since the frequency transform and inverse transform processes are omitted, the video watermarking process is entirely performed in the time domain, and the perceptual modeling unit 202 also extracts the visual weights in the time domain.

When the frequency conversion and inverse transformation processes are omitted, the time-varying adaptive filter unit 206 changes the characteristics of the filter according to the characteristics of the video signal that changes with time, and the signal modulated by the pseudo noise sequence Filtering is performed using the visual weight of the time domain obtained by the perceptual modeling unit 202.

Referring back to FIG. 1, the audio and video signals including the watermark inserted through the audio watermarker 100 of FIG. 2 and the video watermarker 200 of FIG. 3 are respectively encoded by the audio encoder 300 and the video. The multimedia stream is output as a compressed audio and video bit stream through the encoding 400, and multiplexed through the multiplexer 500 to insert a watermark.

While the invention has been described above based on the preferred embodiments thereof, these embodiments are intended to illustrate rather than limit the invention. It will be apparent to those skilled in the art that various changes, modifications, or adjustments to the above embodiments can be made without departing from the spirit of the invention. Therefore, the protection scope of the present invention will be limited only by the appended claims, and should be construed as including all such changes, modifications or adjustments.

As described above, according to the present invention, since a watermark can be inserted without degrading the quality of audio / video and multimedia, it can be applied as a copyright protection technology of digital audio / video and multimedia. In addition, it can be used for authentication to confirm whether or not to manipulate or modify the digital audio and video. It can also be used for Usage Control, which allows you to control the number of duplicates allowed by watermark when playing digital data on portable devices or other audio and video equipment. Further, additional data other than copyright information may be used as a watermark and transmitted.

Claims (17)

An audio perception modeling unit for extracting an auditory masking threshold considering the human auditory characteristics after converting the original audio into a frequency domain; An audio copyright information modulator for multiplying audio copyright information by the pseudo noise, An audio frequency converter for converting the modulated audio copyright information into a frequency domain; An audio time-varying adaptive filter that filters audio copyright information output from the audio frequency converter by using an acoustic masking threshold extracted by the audio perceptual modeling unit; An audio frequency inverse converter for converting a signal output from the audio time-varying adaptive filter into a time domain to generate an audio watermark signal; An audio adder for adding the audio watermark signal and the original audio to output a watermarked audio signal; An audio encoder which compresses the watermark-embedded audio signal; A video perceptive modeling unit which extracts a visual masking threshold considering a human visual characteristic by receiving an original video; A video copyright information modulator for multiplying video copyright information by pseudo noise, A video time-varying adaptive filter that filters video copyright information modulated by the video copyright information modulator by using the visual masking threshold value extracted by the video perceptual modeling unit; A video adder for adding a signal output from the video time-varying adaptive filter unit and the original video to output a watermarked video signal; A video encoder which compresses the watermark-embedded video signal, and And a multiplexer for multiplexing the watermark-embedded compressed audio signal and the compressed video signal outputted from the audio encoder and the video encoder, respectively, and outputting the watermark-embedded multimedia. delete The watermark embedding of digital multimedia as claimed in claim 1, further comprising an audio error correction encoding unit which receives the audio copyright information and corrects an error that may occur when a watermark is detected. Device. The method of claim 1, wherein the audio copyright information modulation means, The pseudo noise is lowpass filtered to a frequency band corresponding to the sampling frequency of the pseudo noise, and then multiplied by the audio copyright information so that the watermark information inserted into the audio signal does not deviate from the frequency band corresponding to the sampling frequency of the pseudo noise. Watermark insertion device of the digital multimedia, characterized in that. The method of claim 4, wherein And a sampling frequency of the pseudo noise is 32 kHz, and the low pass filtering cutoff frequency is 16 kHz. The method of claim 4, wherein the audio time-varying adaptive filter means, The signal output from the audio frequency converting means is adjusted to have a size close to the size of the masking threshold, and the audio copyright information output from the audio perceptual modeling means is low-passed in a frequency band corresponding to the sampling frequency of the pseudo noise. Watermark insertion device of the digital multimedia, characterized in that for filtering. delete The apparatus of claim 1, further comprising: a first video frequency converter for converting the original video into a frequency domain and providing the video to the video perceptual modeling unit; A second video frequency converter converting the video copyright information modulated by the video copyright information modulator into a frequency domain and providing the video copyright variable to the video time-varying adaptive filter; And a video frequency inverse converter for inversely converting the watermark-embedded video signal output from the video adder into a time domain and providing the video encoder to the video encoder. The video perceptual modeling unit receives the video converted into the frequency domain and extracts the visual masking threshold in the frequency domain. And the video time-varying adaptive filter unit filters the frequency-converted video copyright information by using the visual masking threshold. The method according to claim 1 or 8, And a video error correction encoding means for correcting an error that may occur when watermark is detected by receiving the video copyright information and providing the video perception modeling means to the video perceptual modeling means. An audio perception modeling unit for extracting an auditory masking threshold considering the human auditory characteristics after converting the original audio into a frequency domain; An audio copyright information modulator for multiplying audio copyright information by the pseudo noise, An audio frequency converter for converting the modulated audio copyright information into a frequency domain; An audio time-varying adaptive filter that filters audio copyright information output from the audio frequency converter by using an acoustic masking threshold extracted by the audio perceptual modeling unit; An audio frequency inverse converter for converting a signal output from the audio time-varying adaptive filter into a time domain to generate an audio watermark signal; And an audio adder configured to add the audio watermark signal and the original audio to output a watermark-embedded audio signal. 11. The watermark of claim 10, further comprising audio error correction encoding means for receiving the audio copyright information, correcting an error that may occur when a watermark is detected, and providing the audio perception modeling means. Insertion device. The audio copyright information modulation means according to claim 10, The pseudo noise is lowpass filtered to a frequency band corresponding to the sampling frequency of the pseudo noise, and then multiplied by the audio copyright information so that the watermark information inserted into the audio signal does not deviate from the frequency band corresponding to the sampling frequency of the pseudo noise. Watermark insertion device of the digital audio, characterized in that. 13. The apparatus of claim 12, wherein the sampling frequency of the pseudo noise is 32 kHz, and the low pass filtering cutoff frequency is 16 kHz. The method of claim 12, wherein the audio time-varying adaptive filter means, Adjust the signal output from the audio frequency converting means to have a size close to the size of the acoustic masking threshold, and the audio copyright information output from the audio perceptual modeling means into a frequency band corresponding to the sampling frequency of the pseudo noise. A watermark embedding apparatus for digital audio, characterized by lowpass filtering. A video perceptive modeling unit which extracts a visual masking threshold considering a human visual characteristic by receiving an original video; A video copyright information modulator for multiplying video copyright information by pseudo noise, A video time-varying adaptive filter that filters video copyright information modulated by the video copyright information modulator by using the visual masking threshold value extracted by the video perceptual modeling unit; And a video adder for adding a signal output from the video time-varying adaptive filter unit and the original video to output a watermark-embedded video signal. 16. The apparatus of claim 15, further comprising: a first video frequency converter converting the original video into a frequency domain and providing the video to the video perceptual modeling unit; A second video frequency converter converting the video copyright information modulated by the video copyright information modulator into a frequency domain and providing the video copyright variable to the video time-varying adaptive filter; And a video frequency inverse converter for inversely converting the watermark-embedded video signal output from the video adder into a time domain and providing the video encoder to the video encoder. The video perceptual modeling unit receives the video converted into the frequency domain and extracts the visual masking threshold in the frequency domain. And the video time-varying adaptive filter unit filters the frequency-converted video copyright information by using the visual masking threshold. The method according to claim 15 or 16, And a video error correction encoding means for correcting an error that may occur when watermark is detected by receiving the video copyright information and providing the video perception modeling means to the video perceptual modeling means.