Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

A new perceptual video fingerprinting system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Search, retrieval and storage of video content over the Internet and online repositories can be efficiently improved using compact summarizations of this content. Robust and perceptual fingerprinting codes, extracted from local video features, are astutely used for identification and authentication purposes. Unlike existing fingerprinting schemes, this paper proposes a robust and perceptual fingerprinting solution that serves both video content identification and authentication. While content identification is served by the robustness of the proposed fingerprinting codes to content alterations and geometric attacks, their sensitivity to malicious attacks makes them fit for forgery detection and authentication. This dual usage is facilitated by a new concept of sequence normalization based on the circular shift properties of the discrete cosine and sine transforms (DCT and DST). Sequences of local features are normalized by estimating the circular shift required to align each of these sequences to a reference sequence. The fingerprinting codes, consisting of normalizing shifts, are properly modeled using information-theoretic concepts. Security, robustness and sensitivity analysis of the proposed scheme is provided in terms of the security of the secret keys used during the proposed normalization stage. The computational efficiency of the proposed scheme makes it appropriate for large scale and online deployment. Finally, the robustness (identification-based) and sensitivity (authentication-based) of the proposed fingerprinting codes to content alterations and geometric attacks is evaluated over a large set of video sequences where they outperform existing DCT-based codes in terms of robustness, discriminability and sensitivity to moderate and large size intentional alterations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. Our proposed fingerprinting scheme is based on a modified version of the DLBM-based one as shown in Section 4.4.

  2. The best SVD numerical implementations have a complexity order of O(m i n(m n 2, m 2 n)) on m × n matrices [13].

  3. The feature point-based technique extracts the 64 most robust features from image corners and edges [38].

  4. Typical frame rates are usually 60 or 30 fps [22].

  5. The DLBM concept, attributed to Oostoven et al. [43] is extended in this paper to cover all spatial and temporal directions in the fingerprinting code extraction.

  6. The sizes of F h o r , F v e r and F t e m p are R × (C − 1), (R − 1) × C and R × C, respectively.

  7. Using the z-transform concepts, the proof of (18)-(19) is provided in Appendix.

  8. To enhance the security of our proposed video fingerprinting solution, the random generator is set using a secret key K 1 ∈ 𝕂.

  9. L represents the number of frames in the video sequence after the frame rate is reduced to 5 fps in the preprocessing stage.

  10. The first bit in \( bin\_h_{K_{1}} \) is set to 1.

  11. It is assumed that the number of frames, L, is larger than the height of the block in each frame R.

References

  1. Cannons J, Moulin P (2004) Design and statistical analysis of a hash-aided image watermarking system. IEEE Trans Image Process 13(10):1393–1408

    Article  Google Scholar 

  2. Chiu C-Y, Chen C-S, Chien L-F (2008) A framework for handling spatiotemporal variations in video copy detection. IEEE Trans Circuits Syst Video Technol 18(3):412–417

    Article  Google Scholar 

  3. Coskun B, Sankur B, Memon N (2006) Spatio–temporal transform based video hashing. IEEE Trans Multimedia 8(6):1190–1208

    Article  Google Scholar 

  4. Daugman JG (1993) High confidence visual recognition of persons by a test of statistical independence. IEEE Trans Pattern Anal Mach Intell 15(11):1148–1161

    Article  Google Scholar 

  5. De Roover C, De Vleeschouwer C, Lefèbvre F, Macq B (2005) Robust video hashing based on radial projections of key frames. IEEE Trans Signal Process 53 (10):4020–4037

    Article  MathSciNet  MATH  Google Scholar 

  6. Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106

    Article  Google Scholar 

  7. Douze M, Jégou H, Schmid C (2010) An image-based approach to video copy detection with spatio-temporal post-filtering. IEEE Trans Multimedia 12(4):257–266

    Article  Google Scholar 

  8. Esmaeili MM, Fatourechi M, Ward RK (2011) A robust and fast video copy detection system using content-based fingerprinting. IEEE Trans Inf Forensics Secur 6 (1):213–226

    Article  Google Scholar 

  9. Fei M, Ju Z, Zhen X, Li J (2016) Real-time visual tracking based on improved perceptual hashing. Multimedia Tools and Applications:1–18

  10. Fridrich J, Goljan M (2000) Robust hash functions for digital watermarking International conference on information technology: coding and computing, pp 178–183

    Google Scholar 

  11. Ghouti L Robust perceptual color image hashing using randomized hypercomplex matrix factorizations, April 2016, paper submitted to the Signal Processing Journal - Special Issue on Hypercomplex Signal Processing

  12. Goldreich O (2009) Foundations of cryptography: Basic Applications, vol 2. Cambridge university press

  13. Golub GH, Van Loan CF (2012) Matrix computations, vol 3. John Hopkins University

  14. Guéziec AP, Pennec X, Ayache N (1997) Medical image registration using geometric hashing. Comput Sci Eng 4:29–41

    Article  Google Scholar 

  15. Haitsma J (2006) Fingerprint extraction, February 2006, uS Patent App. 10/529,360

  16. Haitsma J, Kalker T (2003) A highly robust audio fingerprinting system with an efficient search strategy. Journal of New Music Research 32(2):211–221

    Article  Google Scholar 

  17. Holliman MJ, Yeo B-L, Liu RG, Yeung MM-Y (1999) Partial protection of content, March 1999, uS Patent App. 09/275,514

  18. Holm F, Hicken WT (2006) Audio fingerprinting system and method, March 2006, uS Patent No. 7013301

  19. Schneider M, Chang S-f (1996) A robust content based digital signature for image authentication Proceedings of the international conference on image processing, 1996, vol 3, pp 227– 230

  20. Kalker A, Haitsma J (2009) Fingerprint database updating method, client and server, April 2009, uS Patent 7523312

  21. Kalker AACM, Haitsma J (2009) Fingerprint database updating method, client and server, April 2009, uS Patent 7,523,312

  22. Karam LJ, Reisslein M (2012) Yuv video sequences, http://trace.eas.asu.edu/yuv/index.html, 2012, [Online; accessed 20-April-2016]

  23. Katzenbeisser S, Lemma A, Celik MU, van der Veen M, Maas M (2008) A buyer–seller watermarking protocol based on secure embedding. IEEE Trans Inf Forensics Secur 3(4):783–786

  24. Lee S, Suh YH (2009) Video fingerprinting based on orientation of luminance centroid IEEE international conference on multimedia and expo, 2009. ICME 2009, pp 1386–1389

    Chapter  Google Scholar 

  25. Lee S, Yoo CD (2008) Robust video fingerprinting for content-based video identification. IEEE Trans Circuits Syst Video Technol 18(7):983–988

    Article  Google Scholar 

  26. Lee S, Yoo CD, Kalker T (2009) Robust video fingerprinting based on symmetric pairwise boosting. IEEE Trans Circuits Syst Video Technol 19(9):1379–1388

    Article  Google Scholar 

  27. Li M, Monga V (2012) Robust video hashing via multilinear subspace projections. IEEE Trans Image Process 21(10):4397–4409

    Article  MathSciNet  MATH  Google Scholar 

  28. Li M, Monga V (2013) Compact video fingerprinting via structural graphical models. IEEE Trans Inf Forensics Secur 8(11):1709–1721

    Article  Google Scholar 

  29. Li M, Monga V (2015) Twofold video hashing with automatic synchronization. IEEE Trans Inf Forensics Secur 10(8):1727–1738

    Article  Google Scholar 

  30. Li M, Vishal M (2014) Twofold video hashing with automatic synchronization 2014 IEEE international conference on image processing (ICIP), pp 5362–5366

    Chapter  Google Scholar 

  31. Lin C-Y, Chang S-F (1997) Robust image authentication method surviving jpeg lossy compression Photonics west’98 electronic imaging, pp 296–307

    Google Scholar 

  32. Liu Y, Zou L, Li J, Yan J, Shi W, Deng D (2016) Segmentation by weighted aggregation and perceptual hash for pedestrian detection. J Vis Commun Image Represent 36:80–89

    Article  Google Scholar 

  33. Loong T-W (2003) Understanding sensitivity and specificity with the right side of the brain. Br Med J 327(7417):716

    Article  Google Scholar 

  34. Lu J (2009) Video fingerprinting for copy identification: from research to industry applications IS&T/SPIE electronic imaging, pp 725402–725402

    Google Scholar 

  35. Macy WW, Holliman MJ, Yeung MM-Y (2004) Method for robust watermarking of content, November 2004, uS Patent No. 6823455

  36. Meyer S, Wang O, Zimmer H, Grosse M, Sorkine-Hornung A (2015) Phase-based frame interpolation for video Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1418

    Google Scholar 

  37. Mihcak MK, Venkatesan R (2007) Hash value computer of content of digital signals, July 2007, uS Patent 7240210

  38. Monga V, Evans BL (2006) Perceptual image hashing via feature points: performance evaluation and tradeoffs. IEEE Trans Image Process 15(11):3452–3465

    Article  Google Scholar 

  39. Monga V, Mihçak MK (2007) Robust and secure image hashing via non-negative matrix factorizations. IEEE Trans Inf Forensics Secur 2(3-1):376–390

    Article  Google Scholar 

  40. Oded G (2001) Foundations of cryptography: basic tools, vol 1. Cambridge University Press

  41. Oppenheim AV, Schafer RW (2009) Discrete-time signal processing, 3rd ed. Prentice Hall

  42. Oostveen JC, Kalker T, Haitsma J (2001) Visual hashing of digital video: applications and techniques International symposium on optical science and technology, pp 121–131

    Google Scholar 

  43. Oostveen J, Kalker T, Haitsma J (2002) Feature extraction and a database strategy for video fingerprinting Recent advances in visual information systems, pp 117–128

    Chapter  Google Scholar 

  44. Ouyang J, Wen X, Liu J, Chen J (2016) Robust hashing based on quaternion zernike moments for image authentication. ACM Trans Multimed Comput Commun Appl 12(4s):63

    Article  Google Scholar 

  45. Pramateftakis A, Oelbaum T, Diepold K (2004) Authentication of mpeg-4-based surveillance video 2004 International conference on image processing, 2004. ICIP’04, vol 1, pp 33–37

  46. Rao KR, Yip P (2014) Discrete cosine transform: algorithms, advantages applications. Academic Press

  47. Recognizer of audio-content in digital signals, December 2005, uS Patent 6973574

  48. Recognizer of content of digital signals, November 2005, uS Patent 6971013

  49. RIAA-IFPI Request for information on audio fingerprinting technologies, http://www.ifpi.org/site-content/press/20010615.html, 2001, [PDF; Cached copy]

  50. Rhee S, Kang MG (1999) Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Opt Eng 38(8):1348–1356

    Article  Google Scholar 

  51. Robust recognizer of perceptually similar content, September 2007, uS Patent 7266244

  52. Wolfson HJ, Rigoutsos I (1997) Geometric hashing: an overview. Comput Sci Eng 4:10–21

    Article  Google Scholar 

  53. Sarkar A, Ghosh P, Moxley E, Manjunath B (2008) Video fingerprinting: features for duplicate and similar video detection and query-based video retrieval Electronic imaging 2008, pp 68200E–68200E

    Google Scholar 

  54. Su P-C, Chen C-C, Chang H-M (2009) Towards effective content authentication for digital videos by employing feature extraction and quantization. IEEE Trans Circuits Syst Video Technol 19(5):668– 677

    Article  Google Scholar 

  55. Sun J, Wang J, Zhang J, Nie X, Liu J (2012) Video hashing algorithm with weighted matching based on visual saliency. IEEE Signal Process Lett 19(6):328–331

    Article  Google Scholar 

  56. Sun R, Yan X, Gao J (2017) Robust video fingerprinting scheme based on contourlet hidden markov tree model. Optik-International Journal for Light and Electron Optics 128:139–147

    Article  Google Scholar 

  57. Sutcu Y, Sencar HT, Memon N (2005) A secure biometric authentication scheme based on robust hashing Proceedings of the 7th workshop on multimedia and security. ACM, pp 111–116

    Google Scholar 

  58. Wu L-N (1990) Comments on” on the shift property of dcts and dsts. IEEE Trans Acoust Speech Signal Process 38(1):186–188

    Article  Google Scholar 

  59. Xu Z, Ling H, Zou F, Lu Z, Li P, Wang T (2009) Fast and robust video copy detection scheme using full dct coefficients IEEE international conference on multimedia and expo, 2009. ICME 2009, pp 434–437

    Google Scholar 

  60. Yang G, Chen N, Jiang Q (2012) A robust hashing algorithm based on surf for video copy detection. Comput Secur 31(1):33–39

    Article  Google Scholar 

  61. Yang OU, Rhee KH (2010) A survey on image hashing for image authentication. IEICE Trans Inf Syst 93(5):1020–1030

    Google Scholar 

  62. Ye C, Xiong Z, Ding Y, Zhang X, Wang G, Xu F (2016) Joint fingerprinting and encryption in the dwt domain for secure m2m communication. International Journal of Security and Its Applications 10(1):125–138

    Article  Google Scholar 

  63. Yip P, Rao K (1987) On the shift property of dct’s and dst’s. IEEE Trans Acoust Speech Signal Process 35(3):404–406

    Article  Google Scholar 

  64. Yuan F, Po L-M, Liu M, Xu X, Jian W, Wong K, Cheung KW (2016) Shearlet based video fingerprint for content-based copy detection. Journal of Signal and Information Processing 7(2):84

    Article  Google Scholar 

  65. Zauner C (2010) Implementation and benchmarking of perceptual image hash functions Master’s thesis, Upper Austria University of Applied Sciences, Hagenberg Campus, Austria

  66. Zhang Z, Cao C, Zhang R, Zou J (2010) Video copy detection based on speeded up robust features and locality sensitive hashing IEEE international conference on automation and logistics (ICAL), 2010, pp 13–18

    Chapter  Google Scholar 

Download references

Acknowledgment

The author would like to thank King Fahd University of Petroleum and Minerals for supporting this work.

Author information

Authors and Affiliations

  1. Department of Information and Computer Science, King Fahd University of Petroleum Minerals, Dhahran, 31261, Saudi Arabia

    Lahouari Ghouti

Authors
  1. Lahouari Ghouti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lahouari Ghouti.

Additional information

This research is supported by King Fahd University of Petroleum and Minerals.

Appendix A: Proof of (18)–(19)

Appendix A: Proof of (18)–(19)

In this appendix, we will provide the proof of (18)–(19)

Let \( \{ x_{0}(n) \}_{n = -0}^{N - 1} \) and \( \{ x_{1}(n) \}_{n = -0}^{N - 1} \) be an N-length sequence and its one-sample circularly-shifted version such as:

$$ \{ x_{0}(n) \}_{n = 0}^{N - 1} = \{ x_{0}(0), x_{0}(1), \ldots, x_{0}(N - 1) \} $$
(49)
$$ \{ x_{1}(n) \}_{n = 0}^{N - 1} = \{ x_{0}(1), x_{0}(2), \ldots, x_{0}(N - 1), x_{0}(0) \} $$
(50)

The z-transforms of the sequences, defined in (49)–(50) are given by [41]:

$$ X_{0}(z) = \sum\limits_{n = 0}^{N - 1} x_{0}(n) z^{-n} $$
(51)
$$ X_{1}(z) = \sum\limits_{n = 0}^{N - 1} x_{1}(n) z^{-n} $$
(52)

where \( z = x + j \cdot y \in \mathbb {C} \) and j 2 = −1.

Using (50), (52) is expanded as follows:

$$ X_{1}(z) = x_{0}(1) + x_{0}(2) \cdot z^{-1} + {\cdots} + x_{0}(N_{1}) \cdot z^{-(N - 2)} + x_{0}(0) \cdot z^{-(N - 1)} $$
(53)

To relate X 1(z) to X 0(z), let us multiply both terms of (51) by z:

$$ z \cdot X_{0}(z) = x_{0}(0) \cdot z + x_{0}(1) + x_{0}(2) \cdot z^{-1} + {\cdots} + x_{0}(N - 1) \cdot z^{-(N - 2)} $$
(54)

Equation (54) can be written in terms of X 1(z) using the following equality:

$$ x_{0}(1) + x_{0}(2) \cdot z^{-1} + {\cdots} + x_{0}(N - 1) \cdot z^{-(N - 2)} = X_{1}(z) - x_{0} \cdot z^{-(N - 1)} $$
(55)
$$ z \cdot X_{0}(z) = x_{0}(0) \cdot z + X_{1}(z) - x_{0} \cdot z^{-(N - 1)} $$
(56)

Then, X 1(z) is expressed as:

$$ X_{1}(z) = z \cdot X_{0}(z) + x_{0}(0) \left[ z^{-(N - 1)} - z \right] $$
(57)

Equation (16), that defines the one-sample circular shift property, is reproduced below for ease of reference:

$$\begin{array}{@{}rcl@{}} \mathcal{X}_{k+1} = \mathcal{A} \cdot \mathcal{X}_{k} \end{array} $$
(58)

where

$$\begin{array}{@{}rcl@{}} \mathcal{X}_{k+1} = \left[\begin{array}{c} X_{k+1}^{C_{2}}(m) \\ X_{k+1}^{S_{2}}(m - 1) \end{array}\right], \; \; \; \; \mathcal{A} = \left[\begin{array}{cc} \cos(v) & \sin(v) \\ -\sin(v) & \cos(v) \end{array}\right], \; \; \; \; \mathcal{X}_{k} = \left[\begin{array}{c} X_{k}^{C_{2}}(m) \\ X_{k}^{S_{2}}(m - 1) \end{array}\right] \end{array} $$
(59)

where \( v = \frac {m \pi }{N} \).

Using the shift property of the z-transform (see (57), the matrix form of the z-transform of (58) gives:

$$\begin{array}{@{}rcl@{}} \left( z \cdot \mathbf{I}_{2 \times 2} - \mathcal{A} \right) \cdot \mathcal{X}_{k}(z) = \mathbf{I}_{2 \times 2} \left( z - z^{-(N - 1)} \right) \cdot \mathcal{X}_{0} \end{array} $$
(60)

where \( \mathcal {X}_{k}(z) \) and I 2×2 represent the matrix-form z-transform of \( \mathcal {X}_{k}(z) \) and the 2 × 2 identity matrix, respectively.

Equation (60) is further simplified into:

$$\begin{array}{@{}rcl@{}} \mathcal{X}_{k}(z) = \left( z \cdot \mathbf{I}_{2 \times 2} - \mathcal{A} \right)^{-1} \cdot \left( z - z^{-(N - 1)} \right) \cdot \mathcal{X}_{0} \end{array} $$
(61)

Therefore, \( \mathcal {X}_{k}(z) \) is given by:

$$\begin{array}{@{}rcl@{}} \mathcal{X}_{k}(z) = \left( \begin{array}{cc} \frac{z - \cos(v)}{1 - 2 \cos(v)z + z^{2}} & \frac{\sin(v)}{1 - 2 \cos(v)z + z^{2}} \\ -\frac{1}{1 - 2 \cos(v)z + z^{2}} & \frac{1}{1 - 2 \cos(v)z + z^{2}} \end{array} \right) \cdot \left( z - z^{-(N - 1)} \right) \cdot \mathcal{X}_{0} \end{array} $$
(62)

To get \( \mathcal {X}_{k} \), the inverse z-transform is applied on (62) as follows:

$$\begin{array}{@{}rcl@{}} \mathcal{X}_{k} = \left[ \begin{array}{c} X_{0}^{C_{2}}(m) \cos(v \cdot k) + X_{0}^{S_{2}}(m - 1) \cdot \sin(v \cdot k) \\ -X_{0}^{C_{2}}(m) \sin(v \cdot k) + X_{0}^{S_{2}}(m - 1) \cdot \cos(v \cdot k) \end{array} \right] \end{array} $$
(63)

Finally, (63) can be rewritten in its final form:

$$\begin{array}{@{}rcl@{}} \mathcal{X}_{k} = \left[ \begin{array}{c} \sqrt{\left[ X_{0}^{C_{2}}(m) \right]^{2} + \left[ X_{0}^{S_{2}}(m - 1) \right]^{2}} \cdot \cos \left( v \cdot k - \arctan \left[ \frac{X_{0}^{S_{2}}(m - 1)}{X_{0}^{C_{2}(m)}} \right] \right) \\ \sqrt{\left[ X_{0}^{C_{2}}(m) \right]^{2} + \left[ X_{0}^{S_{2}}(m - 1) \right]^{2}} \cdot \cos \left( v \cdot k - \arctan \left[ -\frac{X_{0}^{C_{2}(m)}}{X_{0}^{S_{2}}(m - 1)} \right] \right) \end{array} \right] \end{array} $$
(64)

The elements of (64) represent (18)–(19) which concludes our proof.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghouti, L. A new perceptual video fingerprinting system. Multimed Tools Appl 77, 6713–6751 (2018). https://doi.org/10.1007/s11042-017-4595-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4595-z

Keywords

Search

Navigation