KR20110047106A - Method and apparatus for embedding and detecting watermark into/from contents - Google Patents

Abstract

PURPOSE: An apparatus for inserting a watermark to a content and detecting the inserted watermark is provided to improve the reliability of watermark detection. CONSTITUTION: A method for inserting a watermark to a content and detecting the inserted watermark comprises steps of: phase shifting for a bandwidth spread denotation to a predetermined shift amount by a first bet group value which is included in a data corresponding to the watermark; and selectively changing conversion factor of a content by the bit value of each bit sequence including the shifted bandwidth spread denotation.

Description

Method and apparatus for embedding a watermark in content and detecting the inserted watermark {Method and apparatus for embedding and detecting watermark into / from contents}

The present invention relates to a method and apparatus for embedding a watermark in content, and to a method and apparatus for detecting an inserted watermark. In particular, the present invention relates to a forensic technique that enables watermarks to be detected even in content that is illegally copied through an analog interface. forensic).

IPTV and Pay Per View (PPV) services are business models that allow viewers to purchase paid content and watch it. These services generally adopt Digital Right Management (DRM), such as a Conditional Access System (CAS), so that only users who pay a reasonable price can use it. CAS is a technology used to prevent unauthorized users from accessing content, and scrambling data at the transmitter so that only qualified viewers can receive a specific program. The scrambled data is de-scrambled so that the program can be viewed. 1A shows a general system using CAS. As shown in FIG. 1A, a CAS is generally implemented by a Settop Box 11 provided by a service provider 10. Of course, the TV itself has a built-in set-top box.

However, such a CAS system does not prevent illegal copying while watching the user with permission. In general, the image from the three-top box 11 is connected to the input terminal of the TV, as shown in Figure 1b, the equipment such as a capture board (capture board) on the output terminal of the set-top box 11 for the connection (15) If the connection is duplicated as it is, a replica having the same image quality as the original screen may be generated, and a replica may be generated by capturing (16a) a screen output to a TV with a camcorder (camcorder) 16. As such illegally copied contents are provided to others through a DVD or an Internet file sharing service 19, there are cases in which content providers or service providers suffer enormous damages.

The illegal copying of content through the analog interface method is called analog-hole, and various forensic watermarking techniques have been proposed to cope with such illegal copying. Forensic watermarking technology invisibly inserts information about the producer or distribution path, viewers, etc. in the content so that if the content is illegally distributed, whether the content is illegal or not, and whether the content is leaked or leaked. It's a technique that allows you to get the information you need to track your back.

One object of the present invention is watermarking and a method of detecting the same, which ensures the reliability of watermark detection even in the case of a content whose quality is deteriorated by an analog hall method as well as content having the quality of an illegally copied original. It is to provide a device.

Another object of the present invention is to provide a more robust feature to attacks such as geometric attacks, frame rate conversions, and re-encoding of images, such as those that attempt to destroy watermarks. It is to provide a watermark having the same and a method and apparatus for detecting the same.

It is still another object of the present invention to provide a watermarking method and apparatus in which the implementation of the watermarking apparatus is simpler and easier.

The object of the present invention is not limited to the explicit purpose described above, but also naturally includes the purpose of achieving the effect that can be derived from the specific and exemplary description of the present invention.

According to an embodiment of the present invention, a method of inserting watermark data into content includes determining a phase shift amount of a spread spectrum code according to a value of a first bit group belonging to data corresponding to a watermark. Phase shifting the spread spectrum code by the determined shift amount and selectively changing a transform coefficient of the content according to a value of each bit of the bit sequence including at least the phase shifted spread spectrum code It is made to include.

In one embodiment according to the present invention, the method may further include selecting one spread spectrum code from among a plurality of spread spectrum codes according to the value of the second bit group belonging to the data. In this embodiment, the selected spread spectrum code is shifted in phase according to the value of the first bit group.

In one embodiment according to the present invention, the phase shift amount is selected from k * 2 ^p (k = 0,1,2, .., p = 1,2, ..) according to the value of the first bit group. The value of p is fixedly used.

In one embodiment according to the present invention, a sync spread spectrum code having a fixed phase always is further included in the bit sequence. The sync spread spectrum code may have the same period as the spread spectrum code or may have a different period.

According to an embodiment of the present invention, the transform coefficient of the content is selectively changed only for an intra picture to which only spatial compression is applied in the image frame sequence constituting the content.

In an embodiment according to the present invention, the step of selectively changing the conversion coefficient of the content comprises: dividing the video frame sequence constituting the content into frame groups and for each of the divided frame groups. Corresponding bits selectively change the conversion coefficient of the frame group. In one embodiment according to the invention, the distinguished frame group corresponds to a certain time, for example a time of 0.5 seconds.

In another embodiment of the present invention, the step of selectively changing the conversion coefficient of the content may include dividing the video frame sequence constituting the content into frame groups and skipping each of the divided frame groups one by one. Corresponding to selectively transform coefficients of the corresponding frame group. In one embodiment according to the invention, the distinguished frame group corresponds to a certain time, for example a time of 0.5 seconds.

In one embodiment according to the present invention, a transform coefficient for a partial region of each transform coefficient block of an image frame belonging to the content is selectively changed.

According to an aspect of the present invention, an apparatus for inserting watermark data into content includes a code supply unit configured to provide a spread spectrum code and a code supply unit according to a value of a first bit group belonging to data corresponding to the watermark. Determine a phase shift amount of a provided spread spectrum code and phase shift the spread spectrum code by the determined shift amount, and according to the value of each bit of the bit sequence including at least the phase shifted spread spectrum code, And a watermark encoder configured to selectively change the transform coefficient of.

In one embodiment of the present invention, a descrambler for descrambled scrambled digital compressed image data and outputting the decoded digital compressed image data by the watermark device including the code supply unit and the watermark encoder, It is provided between decoders for decoding digital compressed video data and outputting the video signal. In the present embodiment, the watermark device receives the descrambled digital compressed image data, selectively changes the conversion coefficient of the digital compressed image data, and applies it to the decoder. According to an embodiment of the present invention, the watermark encoder performs entropy decoding on the received digital compressed image data to extract only an I (intra) picture or an I slice and extract the extracted I picture or I. After selectively changing the transform coefficient for the slice, entropy encoding is performed and applied to the decoder. In another embodiment of the present invention, the watermark encoder extracts only an I picture or an I slice by performing entropy decoding on the received digital compressed image data, and extracts only an I picture or an I slice from the extracted I picture or I slice. A decoding according to intra prediction is performed to reconstruct a picture or slice of a pixel value, and then to a picture or slice having a transform coefficient representing a frequency component with respect to the picture or slice having the reconstructed pixel value, The transform coefficient obtained in the transform is selectively changed, and entropy encoding is performed on the picture or slice in which the transform coefficient is selectively changed and applied to the decoder.

In one embodiment according to the present invention, the apparatus further includes a controller for controlling the operation of the watermark encoder. In this embodiment, the control unit inserts a synchronous quantized spreading code used for quantized movement of the spreading code, the number of spreading codes used, the period of spreading codes, and the synchronization of spreading codes. Receive a setting value for at least one of a frequency, a continuous or discontinuous mode corresponding to one bit of a spread spectrum code, and a time interval of content corresponding to one bit of a spread spectrum code, and perform an operation according to the setting value. And control the watermark encoder.

According to the present invention, a method of detecting watermark data from content includes extracting a level of a frequency component of the content, generating a bit sequence by matching bit values based on a change in the extracted level; Confirming a correlation while shifting a phase of a spread spectrum code with respect to the generated bit sequence, and when the identified correlation corresponds to auto-correlation, the spread spectrum code Checking a phase shift amount of the signal and determining a group of bits having a value corresponding to the identified shift amount.

In one embodiment according to the present invention, the method further includes determining another group of bits having a pre-assigned value for the spread spectrum code when the checked correlation corresponds to autocorrelation.

In one embodiment according to the present invention, specifying the same bit interval as the sync spread spectrum code on the bit sequence by checking the correlation between the generated bit sequence and the sync spread spectrum code having a fixed phase. Is further included. In this embodiment, a step of confirming the correlation while shifting the phase of the spread spectrum code with respect to the bit string before or after the specified bit section is performed. Further, in the present embodiment, the step of checking the time interval between a plurality of sync bit sections specified by the same bit section as the sync spread spectrum code may be further performed on the bit sequence. In addition, for at least a part of the bit strings in the sync bit section having a time interval out of a predetermined time interval, a process of checking correlation while shifting the phase of the spread spectrum code may not be performed.

In one embodiment according to the present invention, the amount of phase shift in the checking of the correlation is selected from k * 2 ^p (k = 0,1,2, .., p = 1,2 ,.), The p-value is fixed to one preset value.

In another embodiment according to the present invention, the step of determining the bit group, the phase shift amount of the spread spectrum code when the identified correlation corresponds to autocorrelation 2 ^p (p = 1,2, .. ), or 0 or an integer times of a bit, to be considered as a movement amount of the ^P 2 0, or an integer multiple of the most closely to determine the value corresponding thereto.

In an embodiment of the present disclosure, the generating of the bit sequence may include converting the extracted level into a level change rate, and converting the extracted bit sequence into time periods based on a point at which the level change rate passes an upper / lower threshold. A second step of dividing, and a third step of determining an insertion period of bits based on a frequency of time intervals between the upstream peak value, the downstream peak value, or the upstream and downstream peak values of the level change rate in each of the divided time periods; And generating the bit sequence by associating a bit value with each determined bit insertion period. In the case of this embodiment, the step 4 is a bit value for each of the determined bit insertion periods as starting points of the upstream peak value or the downside peak value within the interval between the upper peak value or the lower peak value having a time interval coinciding with the insertion period of the determined bit. To match. When the bit values are mapped, the first bit value, e.g., for the corresponding insertion period when there is both an upward peak value larger than the upper limit threshold value and a downward peak value smaller than the lower limit threshold value in each of the determined insertion periods. 1 ', otherwise, the complementary value of the first bit value, e.g.,' 0 ', for the corresponding insertion period. Further, in the case of this embodiment, if a value corresponding to autocorrelation is not detected, the upper / lower threshold is adjusted to perform again from step 2, and the generated bit sequence is '1' of the spread spectrum code. If the aberration characteristic of and '0' is not satisfied, the process is performed again from step 2.

In an embodiment of the present disclosure, the extracting of the level of the frequency component may generate a signal reflecting the magnitude of the coefficient of the partial region of each frequency coefficient block of the image frame belonging to the content.

According to an aspect of the present invention, there is provided an apparatus for detecting watermark data in a content, the apparatus comprising: a code supply unit providing a spread spectrum code, a signal extractor configured to extract a level of a frequency component of the content, and a change in the extracted level; And an information extracting unit configured to generate a bit sequence by matching bit values, and verifying correlation with the generated bit sequence while shifting a phase of a spread spectrum code provided from the code supplying unit. And a watermark decoder configured to determine a bit group having a value corresponding to the amount of phase shift of the spread spectrum code when the correlation corresponds to autocorrelation.

In one embodiment according to the present invention, the control unit is further configured to control the interface to receive the watermark data including the determined bit group from the watermark decoder and to display on the screen.

At least one embodiment according to the present invention has a characteristic that is robust to an attack for watermark damage, thereby ensuring the preservation of the watermark contained in the content to ensure the detection of the watermark from the watermarked content. This protects the copyright of the content creator and the copyright of the content creator.

In addition, at least one embodiment according to the present invention reduces the cost of designing / manufacturing a device for embedding a watermark by simplifying a reprocessing process for content for embedding a watermark. Has an effect.

Figure 1a shows an example of the configuration of a general video receiving apparatus equipped with a CAS (Conditional Access System) with a receiver restriction,
1b schematically illustrates a process of illegally copying / distributing an image through an image receiving apparatus equipped with a CAS.
2 illustrates a configuration of an apparatus for embedding a watermark into an image according to an embodiment of the present invention.
3 is a flowchart schematically illustrating a process of embedding a watermark in a video signal according to an embodiment of the present invention.
FIG. 4 illustrates autocorrelation characteristics of a pn sequence used for encoding watermark data according to an embodiment of the present invention.
5 illustrates a process of encoding unit bit groups constituting watermark data into pn sequences according to an embodiment of the present invention.
6A and 6B illustrate a process of encoding unit bit groups constituting watermark data into pn sequences according to another embodiment of the present invention.
FIG. 7 illustrates a process of encoding watermark data into one or multiple pn sequences according to another embodiment of the present invention.
8 illustrates examples of regions for adjusting coefficients according to bits of a pn sequence in each block constituting an image picture or a slice, according to an embodiment of the present invention.
9 illustrates an example of encoding each bit of a watermark payload into image frames according to an embodiment of the present invention.
10 illustrates an example of encoding each bit of a watermark payload into image frames according to another embodiment of the present invention.
11 illustrates an example of encoding each bit of a watermark payload into image frames according to another embodiment of the present invention.
12A and 12B illustrate examples in which a watermarking function is implemented in another device according to embodiments of the present invention.
13 illustrates a configuration of means for inserting watermark data into digital compressed video data in MPEG-2 format according to an embodiment of the present invention.
14 illustrates a configuration of means for inserting watermark data into digital compressed image data of H.264 / AVC format according to another embodiment of the present invention.
15 illustrates a configuration of an apparatus for detecting watermark data according to an embodiment of the present invention.
16 illustrates a flow of a method of detecting watermark data according to an embodiment of the present invention.
17 illustrates a theoretical waveform of an energy level signal detected from watermarked content, in accordance with an embodiment of the present invention.
18A and 18B illustrate theoretical waveforms of an energy level signal detected from a watermarked content and an energy ratio signal derived therefrom, respectively, according to an embodiment of the present invention.
19 illustrates an example of an actual waveform of an energy ratio signal derived from an energy level signal obtained from watermarked content, according to an embodiment of the present invention.
20A illustrates an example of dividing the energy ratio signal into sections to recover bit values from an energy ratio signal obtained from watermarked content, according to an embodiment of the present invention.
20B illustrates boundary points of bit prediction intervals in an energy ratio signal obtained from watermarked content according to one embodiment of the present invention;
21 illustrates an example in which a starting point and a bit insertion period for restoring a bit sequence in an energy ratio signal obtained from a watermarked content are determined according to an embodiment of the present invention.
FIG. 22 illustrates an example of allocating bit values according to a determined bit insertion period with respect to an energy ratio signal obtained from watermarked content according to an embodiment of the present invention.
23 illustrates an aberration characteristic of '1' and '0' of a spread spectrum code used for watermark embedding according to an embodiment of the present invention.
24 illustrates an example of selecting a target section for detecting correlation of a spread spectrum code in a bit sequence generated by allocating bit values according to a determined bit insertion period according to an embodiment of the present invention.
FIG. 25 illustrates an example in which bit sections between sync band spread codes are used as sections for detecting watermark data according to detection intervals of a sync spread spectrum code (pilot pn sequence) according to an embodiment of the present invention. It is shown.

Hereinafter, various embodiments of a method and apparatus for embedding / detecting a watermark in content according to the present invention will be described in detail with reference to the accompanying drawings.

2 is a configuration diagram of an apparatus for embedding a watermark into an image signal according to an embodiment of the present invention, and a user interface unit 130 performing input / output with a user and data for performing a watermark embedding operation. And a memory unit 140 for storing the same, and a P / N (pseudonoise, or pseudo-random noise) sequence (hereinafter, referred to as a "pn sequence") which is a spread spectrum code (or "noise code"). The PN supply unit 120, the watermarking unit 110 for encoding information corresponding to the set watermark into the video signal, that is, encoding the watermark, and the controller 100 for controlling the operations of the components as a whole. It is configured to include. The user interface 130 may include key input means 131 such as a keyboard, a mouse, and a touch pad for receiving a user input, and a display 132 for outputting an appropriate input guide screen and / or processing status to the user. ).

The apparatus illustrated in FIG. 2 encodes information corresponding to a watermark set by a user into a video signal according to the flow of an embodiment of the watermarking method according to the present invention illustrated in FIG. 3. Operations performed in the apparatus of the present invention will be described in detail in parallel with the flowchart illustrated in FIG. 3.

First, a pn sequence that is a spread spectrum code or a noise code used in embodiments of the present invention will be described. The pn sequence has a characteristic of wedge-shaped auto-correlation as shown in FIG. 4. That is, it has a high correlation value 401 only when the phases coincide, and has a correlation value of '-1' when the phases do not coincide with each other. Using this autocorrelation characteristic, it is possible to know the shift, that is, the shifted phase, of the pn sequence inserted into an arbitrary signal. Therefore, in embodiments of the present invention, when watermarking a video signal, the movement amount of the pn sequence is used as one of encoding resources. By using the movement amount of the pn sequence as an encoding resource of the information, the encoded information becomes more robust to errors.

The user inputs data to be used as a watermark through the user interface 130. The data at this time is preferably able to represent a name, name, or abbreviated name, etc., which can identify the user himself or his organization. The watermark data input in this way is stored in the memory unit 140 (S31). The user performs a watermarking operation or sets an operation mode through the user interface 130. The operation mode set at this time may be, for example, an 'automatic' mode in which watermarking insertion is performed when an image input is detected.

In response to the request, the controller 100 instructs the watermarking unit 110 to start the watermarking operation while transferring the watermark data set in the memory unit 140 to the watermarking unit 110. . The watermark data is transmitted to the watermarking unit 110 in such a manner that the entire watermark data stored in the memory unit 140 is transferred at once, or the watermark data is transmitted to one video signal in a pn sequence. It is divided into bit unit units that can be processed by encoding (hereinafter, referred to as 'unit bit amount' for the bit amount divided in this manner) and is transmitted to the watermarking unit 110. In the latter case, when the completion signal of the pn sequence encoding to the video signal is received from the watermarking unit 110, the process of transferring the next unit bit amount is repeatedly performed.

Meanwhile, the control unit 100 receives the watermarking mode of the user through the user interface unit 130 before instructing the watermarking unit 110 to start the watermarking operation, and determines the variables determined accordingly. The watermarking unit 110 may be set. Variables set in this manner include the quantized shift of the pn sequence, the number of pn sequences to use, and the like. A detailed description of these variables will be given later. In the embodiments described below, the shift amount of the pn sequence is not quantized, that is, the pn sequence is shifted continuously by one bit, and the pn sequence to be used is also single.

The watermarking unit 110 checks the first unit bit amount in the applied watermark data and determines the movement amount of the pn sequence provided from the PN supply unit 120 according to the value indicated by the unit bit amount (S33). This will be described in more detail below.

A pn sequence having N periods may represent N unique shift values. However, since the period (N) of the pn sequence generated by the n-bit size is generally '2 ⁿ -1', the information that can be expressed even when all N shift cases are used for encoding is the number of cases by n bits ( One less than 2 ⁿ ). Therefore, any particular one value (eg all 0s or all 1s) of information that can be represented by n bits is not used. For example, using a pn sequence with period N of 127, a value that can be represented as 7 bits ("0000000", "0000001", ..., "1111111"), or 128 to "0000000" or The value of " 1111111 " is limited to 127 values except for values that can be encoded by the shift amount of the pn sequence. According to the period N of the pn sequence, as shown in FIG. 4, the maximum value 401 of the autocorrelation is determined, and the maximum value is directly related to the probability of false detection. In other words, the probability of false detection is inversely proportional to the value of N. However, when N is large, the time required for the insertion of watermark data becomes long. Therefore, in the embodiments of the present invention, the N value is selected by an appropriate trade-off between the robustness against errors (low false detection probability) and the time required for watermarking. In another embodiment according to the present invention, if watermarking time is an important factor, N is set small, otherwise N is set large. In one embodiment according to the present invention, the control unit 100 may also receive a value for N through the user interface unit 130, and set the value for the input N to the PN supply unit 120. The PN supply unit 120 may include pn code generators of various bit sizes to generate a pn sequence having a set period of N.

In another embodiment according to the present invention, the PN supply unit 120 is a table that stores one or more pn sequences to be used for watermarking. In the present exemplary embodiment, a plurality of pn sequence tables grouped by pn sequences having different periods are provided, and the controller 100 designates a pn sequence table to be used for the watermarking unit 110 according to a value for the set N. Will give a command.

When the movement amount of the pn sequence is determined (S33), the watermarking unit 110 shifts the pn sequence obtained from the PN supply unit 120 or read from the designated table by the determined movement amount. In this way, each bit value of the pn sequence having the corresponding bits shifted is encoded into the input video signal (S35). Such encoding is performed in units of a unit bit amount of the set watermark data. In total, as illustrated in FIG. 5, when the set watermark data (L bits) 501 is composed of k unit bit amounts (each m bits), k pn sequences (each pn sequences The phase is determined according to the value represented by each m bit, that is, the bit is shifted. A bit sequence 511 composed of pn sequences in which all data (L bits) corresponding to a watermark is encoded in a phase of each pn sequence is called a "watermark payload".

In another embodiment according to the present invention, as illustrated in FIG. 6A, the watermarking unit 110 may generate a sync pn sequence 601 in order to facilitate detection of watermark data. Insert in front of the encoded watermark payload. A synchronous pn sequence (hereinafter referred to as a "pilot pn sequence") is predetermined for synchronization of watermark detection and always has a fixed phase. When the pilot pn sequence for synchronization is used, the watermark detector first checks the correlation value for the pilot pn sequence, so that the timing of the watermark payload that is repeated continuously or at regular intervals can be known. The period of the pilot pn sequence is equal to or different from the period of the pn sequence in which the watermark data is encoded.

In another embodiment according to the present invention, as illustrated in FIG. 6B, a pilot pn sequence may be inserted in front of each pn sequence of the watermark payload. Frequent insertion of pilot pn sequences can aid synchronization (this has the benefit of faster detection of pn sequences whose watermark data is encoded as phases), but requires more watermarking due to the larger amount of bits that need to be inserted Since the length of time required is longer, the frequency of insertion of the pilot pn sequence is determined according to the importance of the time required for watermarking.

In one embodiment according to the present invention, the control unit 100 receives a command for the insertion frequency or insertion method of the pilot pn sequence from the user interface unit 130 and specifies the command to the watermarking unit 110 By applying the watermarking unit 110, the pilot pn sequence is added to the watermark payload according to the designated mode.

In the above embodiments, the shift amount of the pn sequence used to encode the watermark data is used as a continuous variable. That is, the unit bit amount is determined and encoded with N being the number of cases of the shift amount for the pn sequence of period N. In other embodiments according to the present invention, the shift amount for the pn sequence of period N is quantized and used. In one embodiment according to the present invention, the amount of movement of the pn sequence of period N is limited to a value of 2 ^p (p = 1,2,3,4, ...). For example, with p = 4, the values by which the pn sequence can be shifted are 0 * 16 (= 0), 1 * 16 (= 16), 2 * 16 (= 32), 3 * 16 (= 48) Limit to use. When the movement amount of the pn sequence is quantized in this manner, an error occurs on the time axis by stretching or cutting part of the watermark-embedded video signal and correcting even if a value other than a predetermined (ie, quantized) phase is produced. There is a possibility to do it. In the quantization of the pn sequence movement amount as described above, for example, the p value at 2 ^p is set in the watermarking unit 110 by the controller 100 according to a user input.

According to an embodiment of the present invention, when the amount of movement of the pn sequence is quantized and used, the number of cases that can be represented by the pn sequence of period N is reduced, so that the unit bit amount of the watermark data that can be encoded once in the pn sequence is also reduced. Decreases. If the continuous shift amount is used for the pn sequence of period N, the shift amount is 2 when the n bit is a unit bit amount (of course, the value that can be expressed as described above becomes '2 ⁿ -1' with a small value of 1). When quantized to ^p , the number is reduced to N / 2 ^p if it can be encoded in the pn sequence. Therefore, the unit bit amount becomes 'n-p' when negligible that the representable value becomes smaller. For example, if the period N is 127 (= 2 ⁷ -1), 7 bits becomes the unit bit amount when the continuous shift amount is used, but if the shift amount is quantized to 16 (= 2 ⁴ ), the shift amount case is 0, As 8 (= 2 ³ ), which are 16, 32, 48, ..., 112, 3 (= 7-4) bits are the unit bit amount. Therefore, in the present embodiment, when using the amount of movement of the pn sequence in period N (= 2 ⁿ -1) by 2 ^p , the watermark data is encoded in the pn sequence by 'n-p' bits. That is, with respect to the above example, at the time of determining the movement amount of the pn sequence of the watermarking unit 110 (S33), the watermarking unit 110 converts the watermark data into bit group units having a 3-bit size. If the value of each bit group is "000", the movement amount of the pn sequence is determined as 0, 16 for "001", or 32 for "010".

In another embodiment according to the present invention, the moving quantization of the pn sequence may be selected any number, for example, any prime number, without selecting one of multiples of two. For example, when the reference number R for quantization is 3, the quantized shift amount is as follows: 0, 3, 6, 9, 12, ..., and in the case of 5, 0, 5, 10, 15, Becomes like In this case, the maximum value of ^S that satisfies (N / R) = 2 ^S is the unit bit amount. In the present embodiment, the number of cases of the quantized movement amount may be larger than 2 ^S by the determined S. Therefore, in this case, the number of quantized shifts may not be used for encoding, or may be used for signal transmission for other special purposes.

In one embodiment according to the present invention, the watermark data is encoded not only in the amount of movement of the pn sequence but also in the type of the pn sequence to be used. As in the above-described embodiment, when the movement amount of the pn sequence is quantized and used, the amount of unit bits that can be encoded into the pn sequence is relatively reduced. To compensate for this, a pn sequence having a large period N may be adopted. However, this increases the size of the watermark payload, which is disadvantageous in watermarking, and this embodiment can compensate for this disadvantage. FIG. 7 illustrates an example in which a pn sequence is selected and used for encoding according to the present embodiment, wherein a pn sequence belonging to a pool 710 of short period N pn sequences is based on an input bit string of the controller 100. By generating or selecting output from the PN supply unit 120, information may be encoded even when one pn sequence belonging to the pn sequence pool 710 is selected as a watermark payload (711). . The number of pn sequences belonging to the pool of pn sequences 710 is 2 ^q (For the convenience of the description of the present invention, for example, the exponent of 2, but the number of pn sequences of non-exponential numbers of 2 constitute a pool of pn sequences) If), in selecting one pn sequence from the pn sequence pool 710, it is possible to encode information of a bit group of q bit size.

In the present embodiment, as in the example shown in FIG. 7, the watermarking unit 110 selects one movement amount from the quantized movement amounts as described above with respect to one pn sequence 720 selected as described above. By selecting 721, the bit group belonging to the watermark data is additionally encoded. As in the above-described example, when the amount of movement of the pn sequence having a period N (= 2 ⁿ -1) is quantized to 2 ^p , the number of cases becomes 2 ^np , so that an additional 'np' bit can be encoded in addition to the q bit. Will be. That is, the unit bit amount is 'q + (np)'. For example, using a pn sequence with period 127 with n = 7, the number of pn sequences in the pn sequence pool is 16 (= 2 ⁴ , q = 4), and 8 phases (0, 16, 32) with p = 4. If only, 48, .., 112) is used as the movement amount, a bit group of 7 {= 4 + (7-4)} bit sizes can be encoded through one pn sequence. Of course, if the watermark payload 730 is composed of k pn sequence sequences, as illustrated in FIG. 7, the size of information that can be encoded with the watermark payload is increased by k times, resulting in a total k *. {q + (np)} bits. Also for a watermark payload in which watermark data is encoded according to the present embodiment, a pilot pn sequence for synchronization may be added according to the method illustrated in FIG. 6A or 6B.

The watermark payload to which the watermark payload or pilot pn sequence configured according to one of the above-described embodiments is added is continuously or periodically encoded in the video signal according to the method described below (S35). ). In this case, the continuous mode or the encoding period is set by the controller 100, and the continuous mode or the period may be designated by the user through the user interface 130.

Next, a description will be given of a method of an embodiment according to the present invention, in which the watermarking unit 110 encodes each bit belonging to the watermark payload into an input video signal. For the purpose of describing the exemplary embodiment of the present invention, it is assumed that the image signal input to the watermarking unit 110 is image data which is digitally compressed by an arbitrary codec. Of course, when the input video signal is an analog video signal, the apparatus of FIG. 2 converts the analog video signal into digital compressed video data and applies the above-described preprocessing process to the watermarking unit 110. The additional may be further configured. In this case, the codec unit may encode the input video signal using one of MPEG-2, H.264 / AVC, and other known encoding methods.

According to an embodiment of the present invention, the digital compressed image data input to the watermarking unit 110 is a signal converted from a real-time domain to a frequency domain, and the converted image frame is a size of each frequency component instead of a pixel value. Transform coefficients (e.g., transform coefficients by the discrete cosine transform method) are included. In order to know the conversion coefficient for each frequency component, the watermarking unit 110 may involve an appropriate pre-processing process for the digital compressed image data input thereto. This pretreatment will be described later. The conversion coefficients are closely related to the characteristics of the screen. In the edge part, the conversion coefficients corresponding to the high frequency are large, and in the plat area, the conversion coefficients are often small or zero. In general, when there is no scene change in the video signal, the screen is continuously changed with time, so the energy of the frequency of a specific band (reflected as <= conversion coefficient) in the entire screen is also greatly changed with time. Does not occur. Therefore, in one embodiment according to the present invention, the watermarking unit 110 changes the conversion coefficient (ie, energy) of a specific frequency band in an image frame according to the bit value of the watermark payload to be currently encoded. For example, when the bit value to be encoded is '1', the energy of a specific band is increased, for example, the corresponding conversion coefficient is doubled, and when it is '0', it is not changed or the energy is reduced (for example, It is also possible to multiply the conversion factor by 1/2). FIG. 8 shows an 8x8 unit block constituting an image frame, and shows a specific region 801, 802, 803, or 804 in which the adjustment coefficient is to be adjusted according to the bit value in the block according to an embodiment of the present invention. (Each area 801, 802, 803, or 804 of the unit block of conversion coefficients corresponds to a specific frequency band, respectively). Of course, it is also possible to adjust the corresponding conversion coefficients by combining the regions illustrated in the block of FIG. As described above, the modification of the conversion coefficient according to the bit value may be performed for the entire image frame or may be performed for a predetermined portion of the image frame. In addition, in one embodiment according to the present invention, damage to the image quality may be minimized by changing the conversion coefficient only in a region where a specific condition is satisfied and the condition is satisfied. The above specific condition includes whether or not the conversion coefficients are smaller than a specified ratio in comparison with the conversion coefficients of other regions in one image frame. According to an embodiment of the present invention, when the energy is changed for the entire frame, the frame may be more resistant to attack such as cropping or rotation for removing the watermark. In addition, according to an embodiment of the present invention, even when changing the frequency coefficients, that is, the transform coefficients, to code the bit value, it is not necessary to make the coefficients all have the same amount of change, and the amount of change for each transform coefficient may be appropriately selected. have. For example, the amount of change or the rate of change may vary depending on the magnitude of each conversion coefficient. The larger the amount of change in energy for encoding the bit value, the higher the detectability but the lower the image quality. Therefore, in the embodiments according to the present invention, an optimal coefficient change amount that can be balanced between deterioration of image quality and detectability is used.

In the embodiments according to the present invention, when the watermarking unit 110 needs to change a conversion coefficient according to a bit value to be currently encoded, each frame or each frame for a predetermined number of temporally consecutive image frames. The transform coefficients (frequency coefficients) of the regions 801, 802, 803 or 804 of FIG. 8 or the combined region of the unit blocks belonging to some regions in the image are changed. 9 illustrates this. In FIG. 9, the frequency coefficient is changed or maintained by mapping ten frames to one bit value, but the ten frames are for illustration only, and the number of frames, that is, the time interval, is appropriately selected. The number of frames (time interval) may also be set within a range limited to the watermarking unit 110 through the control unit 100 according to a user input.

As shown in FIG. 9, when the coefficient values are increased or maintained for each consecutive frame group 901 _j , the watermark detection apparatus subsequently changes the energy (conversion coefficient) level of a specific frequency band of the corresponding video signal. 910 can be detected to know the time period 911 in which the energy is increased, and for this time period, the value is recognized as '1'('0' depending on the encoding method). The liver can be recognized as '0'.

In one embodiment according to the present invention, the energy change of the frame according to the bit value is performed for all pictures of I / P / B Intra / Inter / Bi-directional picture.

In another embodiment according to the present invention, an energy change according to a bit value is performed only on an I (Intra) picture to which only spatial compression is applied without temporal compression. In the present embodiment, normally, when a coefficient whose energy is changed is present in an I picture, the value obtained by the coefficient uses a characteristic that the codec propagates to the coefficient of the P / B picture by a motion estimation / compensation algorithm. Since the energy change is selectively performed only for the I picture, the complexity of the device for watermarking is relatively low. In this embodiment, as illustrated in FIG. 10, for each bit of the watermark payload, a frame group or picture group 1010 _j in which at least one I picture 1001 _j exists is encoded correspondingly. In other words, the coefficients of the I pictures in the picture group are changed (1001 ₁ , 1001 ₃ ) or held (1001 ₂ ). Since the effect of changing the energy appears after the I picture in the group, when detecting the watermark payload bit, a predetermined frame delay (time from the start of each group of pictures illustrated in FIG. 10 to the I picture in the group of pictures). Appears. According to an embodiment of the present invention, the number of frames belonging to a time limit or a longer time period during which at least one I picture is inserted in a broadcast signal is a frame group for 1-bit encoding. For example, for an image signal according to the ATSC standard that specifies that an I picture is transmitted at intervals of 0.5 seconds or less, the watermark payload may be set based on 0.5 seconds (15 frames, if 30 frames per second). Encode by 1 bit.

In another embodiment according to the present invention, as illustrated in FIG. 11, a reference frame group 1120 _j for reference energy is allocated between frame groups to which each bit of the watermark payload is allocated. The reference frame group 1120 _j is assigned the same number of frames as the frame group 1110 _{j to} which the watermark payload bit is allocated. The reference frame group makes it easier to know whether the energy is changed in the detection device. In other words, even when the energy level of the image changes with time due to the change of scene, the relative change in energy due to the watermark payload bit relative to the adjacent reference energy is well detected. In the drawing illustrated in FIG. 11, the frame belonging to the reference frame group and the frame value belonging to the encoded frame group 1110 ₂ are colored so as to be distinguished from each other, but this means that the bit value is encoded. It is only for distinguishing, and since the bit value of '0' has not changed the energy of the encoded frame group 1110 ₂ , the energy level of the reference frame group also varies greatly from the energy level of the adjacent reference frame group upon detection. (1130). Of course, as mentioned in the above embodiment, in the present embodiment, when encoding the bit value '0', the energy may be relatively reduced instead of maintaining the original energy of the frame.

As in the above-described embodiment, when the energy change is performed only on the I picture (or I slice) according to the bit value, the reference frame group must also include the I picture. Of course, according to the present embodiment, when the number of frames of the frame group in which the reference frame group and the bit value are encoded is kept the same, the frame period includes a video section in which one I picture belongs to the frame group including the bit value. If it is set to, the requirement that one I picture necessarily belongs to the reference frame group is naturally satisfied.

In one embodiment according to the present invention, whether to allocate the reference frame group in allocating the bits of the watermark payload, the condition of whether the control mode (100) is also controlled through the user interface 130. It can also be set to). The controller 100 sets the continuous or discontinuous mode (reference frame group allocation mode) set in this way to the watermarking unit 110 so that the encoding of the watermark payload bits into the image frame is performed accordingly. do.

After encoding each bit of the watermark payload as described above, the watermarking unit 110 performs preprocessing to obtain blocks of frequency coefficients for image frames having frequency coefficients whose values are selectively changed. The reverse process is performed to complete and output image data having the same format as the input digital compressed image data. Examples of the aforementioned pretreatment process are described in more detail in the description of the embodiments with reference to FIGS. 14 and 15. According to an embodiment of the present invention, when watermarking is performed at the front end of the AV codec for decoding and outputting digital compressed image data, the completed output image data is applied to the AV codec. The device illustrated in FIG. 2 may be implemented as an independent device, but may also be implemented at the component level 1210 in a device that performs other functions as shown in FIG. 12A, for example, a digital set-top box. As described above, when the above-described watermarking is implemented at the component level of another device, the function of the above-described control unit 100 is integrally implemented in the control unit of the device. In addition, the component level watermarking function 1222 may be implemented in a chip 1220 integrated with a component of another function, such as the CAS function 1221, as illustrated in FIG. 12B.

Meanwhile, the above-described preprocessing process and the reverse process of the preprocessing performed by the watermarking unit 110 (or the component level watermarking function units 1210 and 1222 in FIGS. 12A and 12B) before and after watermarking are input. It depends on the format of the digital compressed video data. The following describes the preprocessing and vice versa for the two formats. However, the present invention is not limited to the limited conditions and / or examples described below, and any embodiment may be employed as long as the present invention adopts the concept and principle of the above-described watermark payload and encoding into a video frame. It should be regarded as included in the scope of the present invention.

First, when the digital compressed image data applied to the watermarking unit 110 (or the component level watermarking units 1210 and 1222 in FIGS. 12A and 12B) is MPEG-2 format, the watermarking unit ( As illustrated in FIG. 13, an I picture entropy decoder 1310, a watermark inserter 1320, and an I picture entropy encoder 1330 may be configured. The I picture entropy decoder 1310 performs a preprocessing process of decoding the information compressed data according to the code generation probability and outputting the image data corresponding to the I picture from the input digital compressed image data. This image data are coefficients of frequency components, for example DCT coefficients. The watermark inserter 1320 may be configured in one of the above-described embodiments or some selected combination schemes for DCT coefficients of an I picture (or I slice) input from the I picture entropy decoder 1310. Accordingly, the watermarked I picture 1321 is output by changing or maintaining the DCT coefficients of the I picture. The I picture entropy encoder 1330 performs entropy encoding on an I picture output from the watermark inserting unit 1320 according to a predetermined method, and thus the original digital compressed data input to the I picture entropy decoder 1310. Convert to In the sequential process of entropy encoding, watermark embedding, and entropy encoding, digital compressed data belonging to a picture (P or B picture) that does not correspond to an I picture are bypassed by the I picture entropy decoder 1310. 1340, which is transmitted to the I picture entropy encoder 1330, and the I picture entropy encoder 1330 inserts compressed data of an I picture entropy-compressed by the bypassed digital compressed data into a corresponding position. The original digital compressed image data is output.

Next, a case in which the digital compressed image data applied to the watermarking unit 110 (or the component-level watermarking units 1210 and 1222 in FIGS. 12A and 12B) is in the H.264 / AVC format is described. An embodiment of the invention will be described. The H.264 / AVC format uses an intra prediction method in the compression of information. Intra prediction is a method of coding a DC value of a current block by predicting a DC value from a neighboring block when encoding an I slice. That is, within one slice, there may be a mode in which one block depends on neighboring blocks earlier in order. Thus, in the case of simply changing the transform coefficients of one block to insert a watermark bit, even for blocks later in the coding order within a slice, the coefficients change sequentially even though they are not intended. It may also be reflected. In consideration of the features of the encoding scheme, according to an embodiment of the present invention, a process of performing DCT conversion after decoding an I slice into an image signal is performed. Of course, during the DCT conversion process, the DCT coefficient is changed according to the watermark payload bit. 14 illustrates an example of a configuration of the watermarking unit 110 according to an exemplary embodiment of the present invention, which includes an I-slice entropy decoder 1410, an intra decoder 1420, and a pixel. The adder 1430, the watermark inserter 1440, the inverse transformer 1450, the intra predictor 1460, the intra encoder 1470, and the I-slice entropy encoder 1480 may be configured. .

The I-slice entropy decoder 1410 decodes the information-compressed data according to the code generation probability from the input digital compressed image data and outputs image data corresponding to the I slice. This image data are coefficients of frequency components, for example DCT coefficients. The intra decoder 1420 performs inverse quantization and inverse transformation on each macroblock of an input slice in a 4x4 subblock unit 1411 (1421) and has a subblock having a pixel residual value (1421). 1421a). Meanwhile, according to the type of the intra prediction mode of the subblock, a subblock 1422a having a predicted pixel value for the subblock is configured from the pixel values of the neighboring subblock reconstructed first. The pixel adder 1430 adds the pixel value of the predicted subblock 1422a to the pixel residual value of the subblock 1421a to reconstruct the subblock 1431 having the original pixel value. The subblock having the reconstructed pixel value may be used to reconstruct the pixel value of the next subblock (S1430). The watermark inserting unit 1440 completes the above-described preprocessing process by performing DCT transformation on the 8x8 block 1432 composed of the 4x4 subblocks reconstructed as described above. For an 8x8 block 1441 having the transformed DCT coefficients obtained by completing the preprocessing, the water may be changed or maintained by changing or maintaining the DCT coefficients in the block according to one of the various embodiments described above or some selected combination. The 8x8 block 1442 into which the bits of the mark payload are inserted is outputted, and the inverse transformer 1450 performs IDCT on the 8x8 block 1442 into which the bits of the watermark payload are inserted, thereby performing a block of pixel values ( 1451). The intra predictor 1460 divides the block 1451 of the pixel value into 4 × 4 subblocks, and uses the original intra prediction mode type for each divided subblock as it is (S1422). After the block 1461 of the predicted pixel value is generated, the pixel residual subblock 1463 is output by subtracting the pixel value of the predicted block from the pixel value of the corresponding subblock 1462. Such pixel residual subblocks 1463 are sequentially output from the intra predictor 1460, and each pixel residual subblocks 1463 are DCT residuals through DCT transformation and quantization in the intra encoder 1470. It is applied to the I-slice entropy encoder 1480 as a subblock 1471. The I-slice entropy encoder 1480 performs entropy encoding on the data of the input DCT residual subblock, and is not I-slice to digital compressed data S1410 bypassed from the I-slice entropy decoder 1410. The original digital compressed image data is output by inserting the compressed data of the I slice entropy compressed by itself into the corresponding position.

In the embodiments of the method of encoding watermark data according to the present invention described with reference to FIGS. 13 and 14, a transform coefficient for a frequency component is selectively changed only for an I picture or a slice in the input image data. However, the present invention can selectively change the conversion coefficient obtained in an intermediate process by applying an image re-encoding process to a picture or slice encoded in an inter mode. Accordingly, the present invention is not limited to the selective adjustment of the transform coefficients for I pictures or slices described as examples for ease of implementation.

By one method according to one of the various embodiments described above or a combination of several embodiments, the watermark payload is continuously or periodically repeated, and the encoded image data is transferred to another device to decode the device. The screen is output or recorded through the process.

Hereinafter, a method of detecting a watermark from image content obtained by capturing a screen outputted or reproduced after recording as described above or transmitted to another device will be described in detail. Watermark detection process described in detail below will be described in detail as a limited embodiment to aid the understanding of the present invention. Due to the presentation of such limited examples, the description of the detection method for the watermark encoded by one or a combination of the various embodiments of the above-described encoding method may be omitted. However, if the principle of the specific detection method described below and the reverse process of the above-described encoding method are considered together, the detection method for the watermark encoding method whose description is omitted can be naturally derived. Therefore, even if not specifically or explicitly described in the description of the watermark detection method according to the present invention, any embodiment of the watermark as long as watermark detection according to the reverse process of the various watermark encoding method described above, Naturally, it should be regarded as belonging to the right scope of the watermark detection method of the present invention, which is claimed on the basis of the following specific method for the detection of.

FIG. 15 illustrates a configuration of an apparatus for detecting watermark data according to an embodiment of the present invention, wherein an energy detector 1510 for detecting energy of a frequency component of an input video signal, and the energy detector 1510 may be used. A bit for detecting a bit sequence in which watermark data is encoded, from a signal converter 1520 for converting the outputted energy level signal according to a specified scheme and outputting the signal, and a signal converted and output from the signal converter 1520. An extractor 1530, a PN supply unit 1541 for generating a predetermined pn sequence or storing a table of a predetermined pn sequence, and the PN supply unit (1) for the bit sequence inputted from the bit extractor 1530; A watermark decoder 1540 that decodes the watermark data by applying the pn sequence provided by step 1541, and a user interface that performs input / output with a user. It is configured to include a seubu 1550 and a controller 1500 and a memory unit 1501 in which data, etc. are stored for the implementation of the watermark detection operation for controlling the operation of the components as a whole. The user interface unit 1550 includes key input means 1551 such as a keyboard, a mouse, and a touch pad for receiving a user input, and a display unit 1552 for outputting an appropriate input guide screen and / or processing status to a user. The PN supply unit 1541 may include the same pn sequence as one or a plurality of pn sequences used by the PN supply unit 120 in the apparatus of FIG. 2 for watermarking, to the watermark decoder 1540. to provide.

The apparatus of FIG. 15 detects watermark data from an input video signal according to the flow of the detection method of the embodiment illustrated in FIG. 16. Hereinafter, an operation performed by the apparatus of FIG. 15 will be described. It will be described in detail in parallel with the flow chart illustrated in.

The energy detector 1510 detects a watermark related signal in each frame of the input image signal (S1610). The signal detected at this time is a signal reflecting the frequency coefficients specified by the regions 801, 802, 803, 804 or a combination of regions promised to be used in connection with watermarking in each block in the frame illustrated in FIG. The sum or average of these coefficients is output from the energy detector 1510. The sum or average of these coefficients is a signal that reflects the intensity or energy carried in the frequency component (hereinafter, referred to as "energy"). Abbreviated). Since the energy is increased for the watermark bit '1' in the above watermarking method, a signal having a high energy level is output in the frame group. When the input image signal is digital compressed image data, the energy detector 1510 performs entropy decoding to check the frequency coefficient of each frame to perform the above-described energy detection, and the input image signal is an analog image signal. In this case, the frequency coefficient of each frame is checked by transforming the frequency domain, for example, DCT. Alternatively, the energy level of a specific frequency band (frequency band corresponding to each region or combined region of FIG. 8) of the analog signal may be directly detected.

In the above-described embodiment of FIG. 11, since a reference frame group in which the bits of the watermark payload are not inserted is provided, the insertion period of the watermark bits is two in the watermark encoded video signal according to the embodiment of FIG. It becomes a frame group. In the case of the embodiments (FIGS. 9 and 10) which do not have the reference frame group, the time period of one frame group will be the insertion period of the watermark bit. Hereinafter, a method of detecting a watermark according to an embodiment of the present invention will be described in detail on the premise that an image signal encoded with watermark data is received according to the embodiment illustrated in FIG. 11. Of course, in the case of other embodiments (FIGS. 9 and 10), the watermark data may be detected by applying the principle of the detection method described below according to the embodiment.

For convenience of description, frame groups belonging to the bit period of the watermark payload are referred to as 'GoWmP' (Group of Watermarked Pictures). The energy level signal output from the energy detector 1510 should theoretically be a signal waveform as shown in FIG. 17. However, when a watermark is detected due to a characteristic difference such as brightness of an image signal, a signal having a waveform close to FIG. It is difficult to obtain. Accordingly, the signal converter 1520 of the rear stage may determine the moving average of the energy from the output signal of the energy detector 1510 in order to more easily know the change in the energy level for each insertion period of the watermark data bit. A ratio signal is generated (S1620). 18A and 18B show examples of theoretical waveforms of the energy ratio signal converted and output from the signal converter 1520. In this embodiment, the energy ratio between four image frames, i.e. {E (n + 4) -E (n)} / E (n) or E (n + 4) / E (n), where n is a frame (The number in the order of)), but the ratio between these frames can be selected to any other ratio depending on the watermark encoding and / or detection environment. FIG. 18A illustrates a case where GoWmP is 15, and FIG. 18B illustrates a case where GoWmP is 30. Since the ratio between the four frames is selected, as shown in FIGS. 18A and 18B, the signals 1810 and 1820 output from the signal converter 1520 are applied to approximately four frames corresponding to transition periods of changing energy levels. It will have a peak value with a corresponding interval and will have a flat signal level in the remaining intervals. The actual energy ratio signal output from the signal converter 1520 is different from the waveform shown in Figs. 18A or 18B in a theoretical manner, and as shown in Fig. 19, the boundary or peak value becomes a little smooth. In addition, the bit extractor 1530 of the rear stage applies a lower limit threshold value to an upstream peak value and a lower peak value to a signal obtained as shown in FIG. 19 so that '1' or 0 'is applied to the signal. Whether it is inserted can be judged more easily than the energy level signal.

The bit extractor 1530 performs a process of extracting bits constituting watermark data, that is, bits of a pn sequence, from the energy ratio signal (S1630). To this end, first, the initial values of the upper and lower thresholds are respectively selected (S1632), the insertion period of each bit of the bit sequence composed of one or more pn sequences is determined (S1634), and the signal at which the watermark payload is started. The determination process of the point or the start frame (S1636) and the like are performed. First, a method of determining an initial value of upper and lower threshold values according to an embodiment of the present invention will be described in detail. In this embodiment, the cumulative distribution is used. For an embodiment in which the insertion period of the watermark payload bit (ie, pn sequence bit, hereinafter abbreviated as "pn bit") is two GoWmPs (one GoWmP is used as a reference frame group), GoWmP If = 15, as shown in FIG. 18A, the cumulative distribution of the top portion is 4/15 = 26%, and if GoWmP = 30, the top portion is 4/30 = 14%. Accordingly, the bit extractor 1530 samples the input energy ratio signal in a period much shorter than the period of the image frame (typically 1/30 second), lists the sampled values in order of magnitude, and then selects a small value from the large value. In operation S1632, the corresponding sampling value when the cumulative value becomes 26% or 14% by accumulating the number of values is determined as an initial upper limit threshold value. Of course, the lower limit threshold is determined in the same manner. The above method of determining 26% or 14% can be applied when the insertion period of the pn bit is known. In another embodiment according to the present invention, when the insertion period of the pn bit is not known, the reference of the cumulative distribution number is determined between 7% and 26%. Preferably, the basis of the cumulative distribution number is set to approximately 15%. The cumulative distribution number set in this way is based on the difference between the frames when the GoWmP does not exceed 60 frames and the difference between the frames (k _f ) and the energy ratio signal described above ({E (n + k _f ) -E ( n)} / E (n) or E (n + k _f ) / E (n) is assumed to be 4, so if GoWmP and k _f are different, calculate the value accordingly. An appropriate value obtained by applying the principle is determined as a reference value of the cumulative distribution, and the initial value of the upper and lower thresholds is determined by applying the reference value to the cumulative distribution number (S1632). The reason why it is possible to determine and use any value deemed appropriate as an initial value is that the correct threshold value can be adaptively reset after only the insertion period of the pn bit and the frame where it starts, i.e., the starting point. Because. This process will be described later.

Next, the bit extractor 1530 determines the insertion period of the pn bit on the basis of the initial upper and lower thresholds determined above. To this end, each interval is first divided based on a point at which the ratio signal of the energy level crosses the determined upper and lower thresholds. 20A illustrates each section in which the energy ratio signal is divided in this manner. In the illustrated figure, there are intervals 2020 _p and 2030 _{q that} are divided before and after the baseline 2010 _r for the point where the energy ratio signal crosses the upper and lower thresholds THmax and THmin. In each divided section, the point from which the upper limit threshold THmax passes upward to the point that passes the lower threshold THmin downward becomes the high level derivation section 2030 _p , and the upper limit from the point passing the lower threshold threshold downward. The section up to the point passing the threshold upwards becomes a low level derivation section 2020 _q . The bit extractor 1530 divides the energy ratio signal into each section as described above, and then extracts the lowest point in the low level derivation section 2020 _q , and extracts the highest point in the high level derivation section 2030 _p . 20B shows the lowest point 2031 and the highest points 2021 extracted in this manner. When the highest point and the lowest point are identified in this way, the bit extractor 1530 refers to the time interval between each extracted adjacent highest point or the time interval between adjacent lowest points (hereinafter, such a time interval is referred to as a "bit prediction interval"). After calculating and storing the stored time intervals, the most frequent time interval is determined as an insertion period BP of pn bits (S1634).

In one embodiment according to the present invention, the time interval between adjacent highest and lowest points may be extracted, and their statistics may be examined to determine the pn bit insertion period at the most frequent time interval. In this embodiment, since the bit insertion period expected to be determined is one frame group, the watermarking according to the embodiment of FIG. 11 in which the reference frame groups 1120 ₁ and 1120 ₂ are inserted at the time of inserting the watermark data is performed. Is divided into one bit section for the signal section corresponding to the reference frame group. Therefore, in the present embodiment, according to the embodiment of FIG. 11, the bit extractor 1530 is configured to obtain a bit value for each divided section in units of determined bit periods, starting from a later determined start point (start frame). Alternately discards bits one by one to generate the final bit sequence.

In another embodiment according to the present invention, an insertion period of a pn bit may be input through the user interface 1550. The pn bit insertion period set through the input unit 1551 of the user interface 1550 is set and used by the bit extractor 1530 through the control unit 1500. In one embodiment according to the present invention, even when the pn bit insertion period is set from the outside as described above, the bit extractor 1530 automatically performs the above-described determination process of the pn bit insertion period and is determined accordingly. By presenting the pn bit insertion period to the user through the user interface 1550, the user can know the accuracy of the input insertion period or the reliability of automatic detection.

After determining the insertion period of the pn bit, the bit extractor 1530 determines a point (or an image frame corresponding thereto) at which the pn sequence is expected to start (S1636). At this time, the start point (frame) determined is, as illustrated in FIG. 21, the start point 2101 of the bit prediction section 2110 having a time period that matches the insertion period BP of the pn bit determined previously is determined. do. In the example of FIG. 21, when the pn bit insertion period BP determined above corresponds to 25 frames, the interval start point (or frame corresponding thereto) of the bit prediction interval having a time interval corresponding to 25 frames is determined as a bit. Here is an example that determines the starting point for assigning values. As such, when the start point is determined, the bit extractor 1530 allocates a bit value for each insertion period BP previously determined from the start point. At this time, if there is a point passing the upper threshold value downward (from top to bottom) within the insertion period, '1' is assigned, and if it does not have such a point, '0' is assigned (S1638). Of course, according to the embodiment, since the watermark encoding may increase energy for '0', bit values may be allocated as opposed to the above. According to an embodiment of the present invention, when a reference frame group is inserted between pn bits, as shown in FIG. 22, when the corresponding bit value is determined for each bit insertion period, an upper limit threshold value in the insertion period is determined. A value of '1' is assigned only to a period 2210 _i having all of the points passing the lower limit threshold, and '0' is allocated otherwise. In the example of FIG. 22, the second period 2220 has a point 2221 which passes the upper limit downward, but does not have a point that passes the lower limit (2222), and thus a value of '0' is assigned. In this manner, the bit extractor 1530 generates a bit sequence expected to be a watermark payload (S1638).

In another embodiment according to the present invention, the point at which the watermark payload is expected to start is not determined by the method illustrated in FIG. The bit sequence is generated by assigning a bit value to each bit insertion period in the same manner as described above while moving to the interval or every bit prediction interval, and then, during the pn sequence detection process on the bit sequence described later. It is also possible to use the starting point of the pn sequence as the point at which the optimal result is obtained.

The bit extractor 1530 generates a bit sequence, i.e., a bit sequence, expected to be a watermark payload according to the above-described process, and then verifies the accuracy and / or reliability of all or part of the bit sequence. To this end, the bit value characteristic of the pn sequence (the number of '1' is one more than the number of '0') is used. For example, in a pn sequence having a period of 31, there are 16 '1's and 15'0's. Thus, as illustrated in FIG. 23, if the bit sequence corresponds to five of the 31-bit period pn sequence output from the 5-bit pn generator, '1' will be 80 and '0 will be 75. Therefore, the bit extractor 1530 checks whether all or part of the bit sequence generated by allocating the bit value to the pn bit insertion period in the manner described above satisfies the above characteristics (S1639). The satisfaction requirement at this time may have a range of errors. For example, the aberration condition of 0 and 1 satisfying the characteristic may be set to N _L (N _L = 2,3, ...) instead of one. If the above satisfaction requirement is not satisfied, the above-described processes (pn bit insertion period determination, start point determination, bit sequence generation, etc.) are performed again by adjusting the upper and lower threshold values previously set. The size of the bit string that the bit extractor 1530 verifies for accuracy and / or reliability of bit extraction may be a preset value, or may be set as the number of pn sequences through the user interface 1550. When the aberration characteristic of 0 and 1 satisfies a predetermined condition, the bit extractor 1530 applies the generated bit sequence to the watermark decoder 1540. Even after outputting the bit sequence, the bit extractor 1530 stores all the information (the input energy ratio signal, the upper and lower threshold values, etc.) obtained from the bit sequence. This is for the case where the pn sequence having the desired correlation is not determined in the watermark decoding unit 1540 at the next stage.

In one embodiment according to the present invention, since the pilot pn sequence is inserted at regular intervals as in FIG. 6a or 6b to compensate for the characteristics of the pn sequence vulnerable to the occurrence of a temporal error (loss, etc.), the watermark decoder ( In operation 1540, a portion of the pilot pn sequence and a watermark data section are searched for the input bit sequence. In the case of watermarking according to the embodiment of FIG. 5 in which the pilot pn sequence is not inserted into the watermarking, the process of finding the pilot pn sequence is omitted, and a pn sequence in which the watermark data is encoded (hereinafter referred to as a "data pn sequence"). In step S1650, the watermark data is decoded and the watermark data is decoded. In order to find a part of the pilot pn sequence on the input bit sequence, the watermark decoder 1540 applies the pilot pn sequence provided by the PN supply unit 1541 while moving one bit on the input bit sequence to autocorrelate. As shown in FIG. 4, the group of bits in which the value appears the highest (in theory, the value corresponding to the period of the pn sequence) is found (S1642). The bit group found at this time has, of course, a bit string estimated at the same bit values as the pilot pn sequence. In the case of watermarking according to the embodiment of FIG. 5, one or more data pn sequences promised for use in encoding watermark data are used instead of the pilot pn sequence. In the embodiment of shifting the phase by using the quantized shift amount at the time of encoding, the data pn from which the autocorrelation value is derived by detecting the correlation value on the input bit sequence while shifting each used data pn sequence in a predetermined phase unit The sequence and its phase are derived. Meanwhile, the pn sequence in which the watermark decoding unit 1540 is used is provided by the PN supply unit 1541.

Meanwhile, in order to detect a pilot pn sequence, the watermark decoder 1540 sets a target interval in which at least one pilot pn sequence exists on a bit sequence input from the bit extractor 1530 to set the correlation value. Perform the detection process. To this end, the watermark decoding unit 1540 has a bit size of the target section, as shown in FIG. 24, WN (= (number of bits in a pn sequence period) * (pN sequences used for encoding watermark data). Number)), and the bit extractor 1530 also extracts a bit sequence having a size of at least the WN bit and applies it to the watermark decoder 1540. In some cases, the pilot pn sequence may not be detected even within the WN size section. This may occur because the pilot pn sequence spans the boundary region of the target section of the set WN bit size. Accordingly, if the bit group having a value corresponding to autocorrelation is not detected in the target period 2401 of the initially set WN bit size, the watermark decoder 1540 starts with a half point 2410 of the target period. After setting the target section 2411, the pilot pn sequence is found. To this end, the bit extractor 1530 extracts a bit sequence of at least 1.5 * WN (preferably 2 * WN) bits from the input energy ratio signal and applies the bit sequence to the watermark decoder 1540. The mark decoder 1540 performs the above-described process on the input bit sequence, that is, the pn sequence bit string.

According to an embodiment of the present invention, if the bit group that becomes a correlation value of more than a predetermined value (pV _auto ) is not detected on the input bit sequence even though the above process is performed (S1644), the feedback process (S1645) is performed. . At this time, the watermark decoder 1540 notifies the bit extractor 1530 of the pn sequence detection failure, and accordingly, the bit extractor 1530 adjusts the upper and lower thresholds for the energy ratio signal previously stored. Then, the process of re-extracting the bit sequence is performed again by adjusting the signal interval for bit extraction.

In one embodiment according to the present invention, when a plurality of bit groups corresponding to the pilot pn sequence are detected through the above-described process, the watermark decoder 1540 determines the distance between the detected pilot pn sequences on the input bit sequence. In operation S1646, a pn sequence bit section corresponding to the data pn sequence to be used for watermark decoding is determined. The reason for verifying the distance between pilot pn sequences is to confirm whether a temporal error has occurred in the watermark data carried in the video signal. In the case of a temporal error, wrong watermark information may be detected from the erroneous pn sequence, and this error can be avoided through this verification. As in the example of FIG. 25, if a pilot pn sequence is inserted such that a time interval of ₀ seconds is maintained between pilot pn sequences, the watermark decoder 1540 determines that a time interval between detected pilot pn sequences is within an tolerance range. Make sure that T is ₀ seconds. If the time interval between the identified pilot pn sequences is T ₀ within the tolerance range, one or more data pn sequences 2510 _{k in the} corresponding time interval are identified as an error-free pn sequence, and a pn sequence for decoding watermark data. If not, one or more data pn sequences in the time interval 2520 are not used for decoding the watermark data. In one embodiment according to the present invention, a watermark payload corresponding to a pn sequence period immediately after and / or immediately before a detected pilot pn sequence, even for a bit string in the time interval 2520 that is determined to have occurred a temporal error. Each bit stage 2521 and 2522 corresponding to the length of may be used for decoding watermark data. In another embodiment according to the present invention, when the time interval until the next pilot pn sequence is longer than T _o , the time interval until the next pilot pn sequence is checked and the time interval is an appropriate time interval (for example, A time interval corresponding to an integer multiple of T _{o in} the tolerance range) is used for decoding the watermark data for the bit end of payload data in the corresponding interval having a time interval longer than T _o . The figure illustrated in FIG. 25 overlaps the same bit stages immediately after and immediately before different pilot pn sequences to illustrate performing correlation detection on one or more data pn sequences for the bit strings before and after the pilot pn sequence. It is merely illustrated (2530) and does not exemplify that the bit stages are used in duplicate for detection.

As described above, when a pn sequence interval (hereinafter, referred to as a "data target interval") in which there is a data pn sequence to be used for decoding watermark data is determined, the watermark decoder 1540 may determine the corresponding data pn sequence. The watermark data detection process (S1650) will be described below in detail for the data target section starting from the beginning. As mentioned above, the watermark decoder 1540 uses one pn sequence or one pn sequence sequentially selected from the pn sequences provided by the PN supply unit 1541. A correlation value is detected for the first bit set from the first bit of the data target section to the bit corresponding to the pn sequence period. This detection process changes the phase of the pn sequence while changing the pn sequence until a peak value within a predetermined tolerance corresponding to autocorrelation is obtained for any one pn sequence and / or any quantized phase shift. It proceeds by moving. As for the embodiment which does not quantize the phase shift amount, the correlation value is detected while changing the phase shift amount of the pn sequence continuously, that is, by 1 bit. When a peak value corresponding to autocorrelation is detected, an encoded value is determined from the phase shift amount of the pn sequence used at that time, and a watermark bit group having the value is determined (inverse process of 721 of FIG. 7). That is, a watermark bit group having a value corresponding to the phase shift amount is determined. Of course, when watermarking with a plurality of pn sequences as in the embodiment illustrated in FIG. 7, when a peak value corresponding to autocorrelation is detected, the pn sequence used is allocated to the pn sequence. A predetermined value is determined as a decoding value and a watermark bit group having the value is determined (inverse process of 711 of FIG. 7). If there is no pn sequence for which a value corresponding to autocorrelation is detected for all data pn sequences determined to be used, the watermark decoder 1540 ignores the quantized phase shift amount for each data pn sequence. Then, perform a phase shift by 1 bit and check whether a peak value corresponding to autocorrelation is detected. When the peak value is detected, the quantized phase shift amount closest to the phase shift amount at that time is found, and a value corresponding to the quantized phase shift amount is determined as the decoding value. In one embodiment according to the present invention, if the phase shifted pn sequence representing the peak value corresponding to autocorrelation is not detected, the phase shifted pn sequence representing the maximum value for all correlation detection values for the first bit set is obtained. Determined to correspond, the decoding values for the phase and / or pn sequences may be determined.

The above process is performed in the pn sequence period unit for the data target section to determine a plurality of watermark bit groups, thereby restoring watermark data consisting of these bit groups. Of course, when the pilot pn sequence is added to every data pn sequence as in the embodiment of FIG. 6b, the data constituting the watermark is finally performed by performing the above-described process for the next one or more data target sections. Will be restored.

In another embodiment according to the present invention, after detecting the pilot pn sequence as described above, the bit string for the interval between the detected pilot pn sequences is again extracted from the energy ratio signal to obtain the bit string of the data pn sequence. . To this end, the watermark decoder 1540 notifies the bit extractor 1530 of a section corresponding to the detected pilot pn sequence and a bit string corresponding to one or more pn sequences before or after the section. Ask to reextract. Accordingly, the bit extractor 1530 corresponds to the data pn sequence by applying the same process as the derivation process S1632, S2634, S1636, S1638, and S1639 to the previously stored energy ratio signal. Extract the bit string. At this time, the upper and lower threshold values are adjusted until the upper and lower threshold values for which the pilot pn sequence has been detected as an initial value are detected until a bit string that satisfies the bit characteristics is detected. Since the interval for extracting the bit string of the data pn sequence is shorter than the initial stage, it is possible to extract a bit string corresponding to the data pn sequence more accurately by applying a more precise condition. When the bit string corresponding to the data pn sequence is extracted and received from the bit extractor 1530, the watermark decoder 1540 may appropriately before and / or after a period of the pilot pn sequence previously detected. By inserting and determining the data target section and performing the above-described detection process of the watermark data (S1650) for the target section, the encoded watermark data is decoded.

The watermark data corresponding to the watermark payload restored through the above described watermark detection process is transferred to the controller 1500, and the controller 1500 transmits the watermark data to the user interface 1550. This allows the user or operator to know the watermark on the target content.

The encoding of the watermark data described above and the detection process thereof can be applied to both audio and multimedia contents in addition to the video signal described in the above embodiments. When applied to audio, there may be no element for a temporal reference unlike an image signal having an image frame to be used as a temporal reference. In such a case, when applying the principles of the present invention to an audio signal, watermark bits may be inserted at predetermined time intervals. In addition, when applying the principles of the present invention to an audio signal, the energy may be increased or inserted at a specific intensity according to the bit value of the pn sequence to be encoded, for a component of a specific frequency band that is difficult for humans to hear. .

As described above, the embodiments of the present invention are disclosed for the purpose of illustration, and those skilled in the art can further improve various other embodiments within the spirit and technical scope of the present invention disclosed in the appended claims. Modifications, substitutions or additions may be made.

100: control unit 110: watermarking unit
120: PN supply unit 130: user interface
140: memory unit 1310: I picture entropy decoder
1320: watermark insertion unit 1330: I picture entropy encoder
1410: I slice entropy decoder 1420: intra decoder
1430: pixel adder 1440: watermark insertion unit
1450: Inverse Transformer 1460: Intra Predictor
1470: Intra Encoder 1480: I-Slice Entropy Encoder
1500: control unit 1501: memory unit
1510: energy detector 1520: signal converter
1530: bit extracting unit 1540: watermark decoding unit
1541: PN supply 1550: user interface

Claims (22)

In the method of embedding a watermark in the content,
Determining a phase shift amount of a spread spectrum code according to a value of a first bit group belonging to the data corresponding to the watermark, and phase shifting the spread spectrum code by the determined shift amount;
Selectively changing a conversion coefficient of the content according to the value of each bit of the bit sequence including at least the phase shifted spreading code. The method of claim 1,
The method may further include selecting one spreading code from among a plurality of spreading codes according to a value of a second group of bits belonging to the data.
And the phase shifting step shifts the selected spread spectrum code according to the value of the first bit group. The method of claim 1,
The phase shift amount determined in the phase shifting step is selected from k * 2 ^p (k = 0,1,2, .., p = 1,2, ..) according to the value of the first bit group. The p value is a method in which one preset value is fixedly used. The method of claim 1,
And including in the bit sequence a sync spread spectrum code having always a fixed phase. The method of claim 1,
Selectively changing the transform coefficient of the content is performed only on an intra picture to which only spatial compression is applied in a sequence of video frames constituting the content. The method of claim 1,
In the step of selectively changing the conversion coefficient of the content, the video frame sequence constituting the content is divided into frame groups, and the conversion coefficient of the corresponding frame group is selectively selected by matching each bit with respect to each divided frame group. To change. The method of claim 1,
In the step of selectively changing a conversion coefficient of the content, the video frame sequence constituting the content is divided into frame groups, and each of the divided frame groups is skipped one by one to correspond to the respective bits to obtain a conversion coefficient of the corresponding frame group. Selectively changing. The method according to claim 6 or 7,
The divided frame group corresponds to a predetermined time. The method of claim 1,
And selectively changing a transform coefficient of the content, selectively changing a transform coefficient for a partial region of each transform coefficient block of an image frame belonging to the content. An apparatus for embedding a watermark in content, the apparatus comprising:
A code supply configured to provide a spread spectrum code,
Determining a phase shift amount of a spread spectrum code provided by the code supply unit according to a value of a first bit group belonging to the data corresponding to the watermark, and phase shifting the spread spectrum code by the determined shift amount, And a watermark encoder configured to selectively change a transform coefficient of the content according to the value of each bit of the bit sequence including at least the phase shifted spread spectrum code. The method of claim 10,
The watermark encoder is further configured to perform a process of selecting one spread spectrum code from among a plurality of spread spectrum codes according to a value of a second group of bits belonging to the data. And shift the phase of the selected spread spectrum code according to the value of the first group of bits. The method of claim 10,
The amount of shifting the phase shifted by the watermark encoder is selected from k * 2 ^p (k = 0,1,2, .., p = 1,2, ..) according to the value of the first bit group. The p value is a device in which one preset value is fixedly used. The method of claim 10,
And the watermark encoder is further configured to include, in the bit sequence, a synchronization spread spectrum code having a fixed phase always provided by the code supply unit. The method of claim 10,
And the watermark encoder is configured to perform only on an intra picture to which only spatial compression is applied in a video frame sequence constituting the content when the transform coefficient of the content is selectively changed. The method of claim 10,
When the watermark encoder selectively changes the conversion coefficient of the content, the watermark encoder divides the video frame sequence constituting the content into frame groups, and associates each bit with each of the divided frame groups to determine the corresponding frame group. And to selectively change the conversion coefficient. The method of claim 10,
When the watermark encoder selectively changes the conversion coefficient of the content, the watermark encoder divides the video frame sequence constituting the content into frame groups, and skips each of the divided frame groups one by one to correspond to the respective bits. And selectively change the coefficient of transformation of the group. The method according to claim 15 or 16,
The divided frame group corresponds to a predetermined time. The method of claim 10,
And the watermark encoder is configured to selectively change a transform coefficient for a partial region of each frequency coefficient block of an image frame belonging to the content when selectively changing the transform coefficient of the content. The method of claim 10,
A device including the code supply unit and the watermark encoder may include a descrambler that descrambles scrambled digital compressed image data and outputs the digital compressed image data, and decodes the digital compressed image data as an image signal. And between the decoders to receive the descrambled digital compressed image data, selectively change the conversion coefficient of the digital compressed image data, and apply the decoded digital compressed image data to the decoder. The method of claim 19,
The watermark encoder,
Entropy decoding is performed on the received digital compressed image data to extract only I pictures or I slices, and selectively transform transform coefficients of the extracted I pictures or I slices, and then apply entropy encoding to the decoder. Device. The method of claim 19,
The watermark encoder,
Entropy decoding is performed on the received digital compressed image data to extract only I pictures or I slices,
Decoding the extracted I picture or I slice, and performing decoding according to intra prediction to reconstruct a picture or slice of a pixel value,
Converting a picture or slice having a transform coefficient representing a frequency component with respect to the picture or slice having the reconstructed pixel value and then selectively changing the transform coefficient obtained by the conversion;
And performing entropy encoding on a picture or slice in which the transform coefficient is selectively changed and applying the entropy encoding to the decoder. The method of claim 10,
It further comprises a control unit for controlling the operation of the watermark encoder,
The control unit may include a quantized shift amount for a spread spectrum code, the number of spread spectrum codes used, a cycle of spread spectrum codes, an insertion frequency of a sync spread spectrum code used for synchronization of spread spectrum codes, and one bit of spread spectrum code. Is configured to control the watermark encoder to receive a setting value for at least one of a continuous mode corresponding to the content and a time interval of the content corresponding to 1 bit of the spread spectrum code, and to perform an operation according to the setting value. Device.

Images