Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Representing dynamic textures based on polarized gradient features

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Efficiently representing dynamic textures (DTs) is one of the significant challenges for video understanding in real implementations of computer vision applications. In this work, an efficient approach for DT description is introduced by addressing the following prominent concepts. Firstly, high-order 2D/3D Gaussian-gradient filtering kernels are used for filtering a given video to obtain its Gaussian-gradient-filtered images/volumes. Secondly, taking advantage of the polarizing property of these responses, we propose a competent model of decomposition to decompose them into corresponding robust collections of separately polarized outcomes, which are complementary to DT representation. Finally, a simple variant of local binary patterns (LBPs) is applied to extract local polarized Gaussian-gradient features from the complemented collections for constructing discriminative local-based descriptors. Experimental results for DT recognition on benchmark datasets have remarkably validated the efficacy of our proposal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Algorithm 2
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data availability

The datasets used in this paper are publicly available.

Notes

  1. CLBP [63] operator is utilized in this work for the purpose of unity in implementing and evaluating the efficiency of the polarized features for DT description. Definitely, it could address other robust local-based operators for further enhancement in practice, e.g., LDP-based [60, 79], CLBC [68], LRP [56], LVP-based [37, 80], MRELBP [81], etc.

References

  1. Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. IJCV 51(2), 91–109 (2003)

    Article  MATH  Google Scholar 

  2. Nguyen, X.S., Nguyen, T.P., Charpillet, F., Vu, N.S.: Local derivative pattern for action recognition in depth images. Multimedia Tools Appl. 77(7), 8531–8549 (2018)

    Article  Google Scholar 

  3. Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. PAMI 29(6), 915–928 (2007)

    Article  Google Scholar 

  4. Kellokumpu, V., Zhao, G., Pietikäinen, M.: Recognition of human actions using texture descriptors. Mach. Vis. Appl. 22(5), 767–780 (2011)

    Article  Google Scholar 

  5. Monfort, M., Pan, B., Ramakrishnan, K., Andonian, A., McNamara, B.A., Lascelles, A., et al.: Multi-moments in time: learning and interpreting models for multi-action video understanding. IEEE Trans. Pattern Anal. Mach. Intell. 44(12), 9434–9445 (2022)

    Article  Google Scholar 

  6. Nguyen, T.P., Manzanera, A., Garrigues, M., Vu, N.S.: Spatial motion patterns: action models from semi-dense trajectories. IJPRAI 28(7), 1460011 (2014)

    Google Scholar 

  7. Péteri, R.: Tracking dynamic textures using a particle filter driven by intrinsic motion information. Mach. Vis. Appl. 22(5), 781–789 (2011)

    Article  Google Scholar 

  8. Zhang, C., Wang, Q., Li, X.: V-LPDR: towards a unified framework for license plate detection, tracking, and recognition in real-world traffic videos. Neurocomputing 449, 189–206 (2021)

    Article  Google Scholar 

  9. Kim, W., Han, J.: Directional coherence-based spatiotemporal descriptor for object detection in static and dynamic scenes. Mach. Vis. Appl. 28(1–2), 49–59 (2017)

    Article  Google Scholar 

  10. Wu, X., Lu, X., Leung, H.: Video smoke separation and detection via sparse representation. Neurocomputing 360, 61–74 (2019)

    Article  Google Scholar 

  11. Lin, Y., Yu, Q., Medioni, G.G.: Efficient detection and tracking of moving objects in geo-coordinates. Mach. Vis. Appl. 22(3), 505–520 (2011)

    Google Scholar 

  12. Zhang, C., Lam, K., Wang, Q.: CoF-Net: a progressive coarse-to-fine framework for object detection in remote-sensing imagery. IEEE Trans. Geosci. Remote Sens. 61, 1–17 (2023)

    Article  Google Scholar 

  13. Hoang, V., Jo, K.: Joint components based pedestrian detection in crowded scenes using extended feature descriptors. Neurocomputing 188, 139–150 (2016)

    Article  Google Scholar 

  14. Favaretto, R.M., Knob, P., Musse, S.R., Vilanova, F., Costa, A.B.: Detecting personality and emotion traits in crowds from video sequences. Mach. Vis. Appl. 30(5), 999–1012 (2019)

    Article  Google Scholar 

  15. Cancela, B., Ortega, M., Penedo, M.G.: Multiple human tracking system for unpredictable trajectories. Mach. Vis. Appl. 25(2), 511–527 (2014)

    Article  Google Scholar 

  16. Chan, A.B., Mahadevan, V., Vasconcelos, N.: Generalized Stauffer-Grimson background subtraction for dynamic scenes. Mach. Vis. Appl. 22(5), 751–766 (2011)

    Article  Google Scholar 

  17. Chan, K.L.: Detection of foreground in dynamic scene via two-step background subtraction. Mach. Vis. Appl. 26(6), 723–740 (2015)

    Article  Google Scholar 

  18. Chetverikov, D., Fazekas, S., Haindl, M.: Dynamic texture as foreground and background. Mach. Vis. Appl. 22(5), 741–750 (2011)

    Article  Google Scholar 

  19. Narayana, M., Hanson, A.R., Learned-Miller, E.G.: Background subtraction: separating the modeling and the inference. Mach. Vis. Appl. 25(5), 1163–1174 (2014)

    Article  Google Scholar 

  20. Nguyen, T.T., Nguyen, T.P.: A comprehensive taxonomy of dynamic texture representation. ACM Comput. Surv. 55(2), 23:1-23:39 (2023)

    Google Scholar 

  21. Saisan, P., Doretto, G., Wu, Y.N., Soatto, S.: Dynamic Texture Recognition. In: CVPR; pp. 58–63 (2001)

  22. Chan, A.B., Vasconcelos, N.: Classifying video with kernel dynamic textures. In: CVPR; pp. 1–6 (2007)

  23. Mumtaz, A., Coviello, E., Lanckriet, G.R.G., Chan, A.B.: Clustering dynamic textures with the hierarchical EM algorithm for modeling video. IEEE Trans. PAMI 35(7), 1606–1621 (2013)

    Article  Google Scholar 

  24. Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. In: CVPR; pp. 1651–1657 (2009)

  25. Wang, L., Liu, H., Sun, F.: Dynamic texture video classification using extreme learning machine. Neurocomputing 174, 278–285 (2016)

    Article  Google Scholar 

  26. Wang, Y., Hu, S.: Chaotic features for dynamic textures recognition. Soft Comput. 20(5), 1977–1989 (2016)

    Article  Google Scholar 

  27. Wang, Y., Hu, S.: Exploiting high level feature for dynamic textures recognition. Neurocomputing 154, 217–224 (2015)

    Article  Google Scholar 

  28. Mumtaz, A., Coviello, E., Lanckriet, G.R.G., Chan, A.B.: A scalable and accurate descriptor for dynamic textures using bag of system trees. IEEE Trans. PAMI 37(4), 697–712 (2015)

    Article  Google Scholar 

  29. Qiao, Y., Weng, L.: Hidden Markov model based dynamic texture classification. IEEE Signal Process. Lett. 22(4), 509–512 (2015)

    Article  Google Scholar 

  30. Qiao, Y., Xing, Z.: Dynamic texture classification using multivariate hidden markov model. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. 101(A(1)), 302–305 (2018)

    Article  Google Scholar 

  31. Peh, C.H., Cheong, L.F.: Synergizing spatial and temporal texture. IEEE Trans. IP 11(10), 1179–1191 (2002)

    MathSciNet  Google Scholar 

  32. Péteri, R., Chetverikov, D.: Qualitative Characterization of Dynamic Textures for Video Retrieval. In: Wojciechowski KW, Smolka B, Palus H, Kozera R, Skarbek W, Noakes L (eds). ICCVG. vol. 32 of Computational Imaging and Vision, pp. 33–38 (2004)

  33. Péteri, R., Chetverikov, D.: Dynamic texture recognition using normal flow and texture regularity. In: Marques JS, de la Blanca NP, Pina P (eds). IbPRIA. vol. 3523 of LNCS, pp. 223–230 (2005)

  34. Lu, Z., Xie, W., Pei, J., Huang, J.: Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms. In: WACV/MOTION, pp. 241–246 (2005)

  35. Fazekas, S., Chetverikov, D.: Analysis and performance evaluation of optical flow features for dynamic texture recognition. Signal Proc. Image Commun. 22(7–8), 680–691 (2007)

    Article  Google Scholar 

  36. Nguyen, T.T., Nguyen, T.P., Bouchara, F., Nguyen, X.S.: Directional beams of dense trajectories for dynamic texture recognition. In: Blanc-Talon J, Helbert D, Philips W, Popescu D, Scheunders P, editors. ACIVS, pp. 74–86 (2018)

  37. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Directional dense-trajectory-based patterns for dynamic texture recognition. IET Comput. Vis. 14(4), 162–176 (2020)

    Article  Google Scholar 

  38. Rivera, A.R., Chae, O.: Spatiotemporal directional number transitional graph for dynamic texture recognition. IEEE Trans. PAMI 37(10), 2146–2152 (2015)

    Article  Google Scholar 

  39. Xu, Y., Quan, Y., Ling, H., Ji, H.: Dynamic texture classification using dynamic fractal analysis. In: ICCV, pp. 1219–1226 (2011)

  40. Xu, Y., Huang, S.B., Ji, H., Fermüller, C.: Scale-space texture description on SIFT-like textons. CVIU 116(9), 999–1013 (2012)

    Google Scholar 

  41. Ji, H., Yang, X., Ling, H., Xu, Y.: Wavelet domain multifractal analysis for static and dynamic texture classification. IEEE Trans. IP 22(1), 286–299 (2013)

    MathSciNet  MATH  Google Scholar 

  42. Quan, Y., Sun, Y., Xu, Y.: Spatiotemporal lacunarity spectrum for dynamic texture classification. CVIU 165, 85–96 (2017)

    Google Scholar 

  43. Baktashmotlagh, M., Harandi, M.T., Lovell, B.C., Salzmann, M.: Discriminative non-linear stationary subspace analysis for video classification. IEEE Trans. Pattern Anal. Mach. Intell. 36(12), 2353–2366 (2014)

    Article  Google Scholar 

  44. Péteri, R., Fazekas, S., Huiskes, M.J.: DynTex: a comprehensive database of dynamic textures. Pattern Recognit. Lett. 31(12), 1627–1632 (2010)

    Article  Google Scholar 

  45. Ghanem, B., Ahuja, N.: Maximum margin distance learning for dynamic texture recognition. In: Daniilidis K, Maragos P, Paragios N (eds). ECCV. vol. 6312 of LNCS, pp. 223–236 (2010)

  46. Andrearczyk, V., Whelan, P.F.: Convolutional neural network on three orthogonal planes for dynamic texture classification. Pattern Recognit. 76, 36–49 (2018)

    Article  Google Scholar 

  47. Arashloo, S.R., Amirani, M.C., Noroozi, A.: Dynamic texture representation using a deep multi-scale convolutional network. JVCIR 43, 89–97 (2017)

    Google Scholar 

  48. Qi, X., Li, C.G., Zhao, G., Hong, X., Pietikainen, M.: Dynamic texture and scene classification by transferring deep image features. Neurocomputing 171, 1230–1241 (2016)

    Article  Google Scholar 

  49. Hong, S., Ryu, J., Im, W., Yang, H.S.: D3: recognizing dynamic scenes with deep dual descriptor based on key frames and key segments. Neurocomputing 273, 611–621 (2018)

    Article  Google Scholar 

  50. Quan, Y., Huang, Y., Ji, H.: Dynamic texture recognition via orthogonal tensor dictionary learning. In: ICCV, pp. 73–81 (2015)

  51. Quan, Y., Bao, C., Ji, H.: Equiangular kernel dictionary learning with applications to dynamic texture analysis. In: CVPR; pp. 308–316 (2016)

  52. Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: ICCV, pp. 4489–4497 (2015)

  53. Hadji, I., Wildes, R.P.: A new large scale dynamic texture dataset with application to ConvNet understanding. In: ECCV, pp. 334–351 (2018)

  54. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. PAMI 24(7), 971–987 (2002)

    Article  MATH  Google Scholar 

  55. Ren, J., Jiang, X., Yuan, J.: Dynamic texture recognition using enhanced LBP features. In: ICASSP, pp. 2400–2404 (2013)

  56. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Rubik Gaussian-based patterns for dynamic texture classification. Pattern Recognit. Lett. 135, 180–187 (2020)

    Article  Google Scholar 

  57. Zhao, G., Ahonen, T., Matas, J., Pietikäinen, M.: Rotation-invariant image and video description with local binary pattern features. IEEE Trans. IP 21(4), 1465–1477 (2012)

    MathSciNet  MATH  Google Scholar 

  58. Tiwari, D., Tyagi, V.: Dynamic texture recognition based on completed volume local binary pattern. MSSP 27(2), 563–575 (2016)

    Google Scholar 

  59. Tiwari, D., Tyagi, V.: A novel scheme based on local binary pattern for dynamic texture recognition. CVIU 150, 58–65 (2016)

    Google Scholar 

  60. Nguyen, T.T., Nguyen, T.P., Bouchara, F., Nguyen, X.S.: Momental directional patterns for dynamic texture recognition. CVIU 194, 102882 (2020)

    Google Scholar 

  61. Arashloo, S.R., Kittler, J.: Dynamic texture recognition using multiscale binarized statistical image features. IEEE Trans. Multimedia 16(8), 2099–2109 (2014)

    Article  Google Scholar 

  62. Zhao, X., Lin, Y., Liu, L., Heikkilä, J., Zheng, W.: Dynamic texture classification using unsupervised 3D filter learning and local binary encoding. IEEE Trans. Multimedia 21(7), 1694–1708 (2019)

    Article  Google Scholar 

  63. Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. IP 19(6), 1657–1663 (2010)

    MathSciNet  MATH  Google Scholar 

  64. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Completed statistical adaptive patterns on three orthogonal planes for recognition of dynamic textures and scenes. J. Electron. Imaging 27(05), 053044 (2018)

    Article  Google Scholar 

  65. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Prominent local representation for dynamic textures based on high-order Gaussian-gradients. IEEE Trans. Multimedia 23, 1367–1382 (2021)

    Article  Google Scholar 

  66. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Dynamic texture description using adapted bipolar-invariant and blurred features. Multidimens. Syst. Signal Process. 33(3), 945–979 (2022)

    Article  Google Scholar 

  67. Nguyen, T.P., Manzanera, A., Kropatsch, W.G., N’Guyen, X.S.: Topological attribute patterns for texture recognition. Pattern Recognit. Lett. 80, 91–97 (2016)

    Article  Google Scholar 

  68. Zhao, Y., Huang, D.S., Jia, W.: Completed local binary count for rotation invariant texture classification. IEEE Trans. IP 21(10), 4492–4497 (2012)

    MathSciNet  MATH  Google Scholar 

  69. Zhao, X., Lin, Y., Heikkilä, J.: Dynamic texture recognition using volume local binary count patterns with an application to 2D face spoofing detection. IEEE Trans. Multimedia 20(3), 552–566 (2018)

    Article  Google Scholar 

  70. Nguyen, T.T., Nguyen, T.P., Thirion-Moreau, N.: Locating robust patterns based on invariant of LTP-based features. Pattern Recognit. Lett. 165, 9–16 (2023)

    Article  Google Scholar 

  71. Jain, A.K., Farrokhnia, F.: Unsupervised texture segmentation using Gabor filters. Pattern Recognit. 24(12), 1167–1186 (1991)

    Article  Google Scholar 

  72. Nguyen, T.P., Vu, N.S., Manzanera, A.: Statistical binary patterns for rotational invariant texture classification. Neurocomputing 173, 1565–1577 (2016)

    Article  Google Scholar 

  73. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Smooth-invariant Gaussian features for dynamic texture recognition. In: ICIP, pp. 4400–4404 (2019)

  74. Nguyen, T.T., Nguyen, T.P., Bouchara, F., Vu, N.: Volumes of blurred-invariant Gaussians for dynamic texture classification. In: Vento M, Percannella G (eds). CAIP, pp. 155–167 (2019)

  75. Van Wyk, M., Wässle, H., Taylor, W.R.: Receptive field properties of ON-and OFF-ganglion cells in the mouse retina. Vis. Neurosci. 26(3), 297–308 (2009)

    Article  Google Scholar 

  76. Vu, N., Nguyen, T.P., Garcia, C.: Improving texture categorization with biologically-inspired filtering. Image Vis. Comput. 32(6–7), 424–436 (2014)

    Article  Google Scholar 

  77. Dubois, S., Péteri, R., Ménard, M.: Characterization and recognition of dynamic textures based on the 2D+T curvelet transform. Signal Image Video Process. 9(4), 819–830 (2015)

    Article  Google Scholar 

  78. Tiwari, D., Tyagi, V.: Dynamic texture recognition using multiresolution edge-weighted local structure pattern. Comput. Electr. Eng. 62, 485–498 (2017)

  79. Zhang, B., Gao, Y., Zhao, S., Liu, J.: Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans. IP 19(2), 533–544 (2010)

    MathSciNet  MATH  Google Scholar 

  80. Fan, K., Hung, T.: A novel local pattern descriptor - local vector pattern in high-order derivative space for face recognition. IEEE Trans. IP 23(7), 2877–2891 (2014)

    MathSciNet  MATH  Google Scholar 

  81. Liu, L., Lao, S., Fieguth, P.W., Guo, Y., Wang, X., Pietikäinen, M.: Median robust extended local binary pattern for texture classification. IEEE Trans. IP 25(3), 1368–1381 (2016)

    MathSciNet  MATH  Google Scholar 

  82. Fan, R., Chang, K., Hsieh, C., Wang, X., Lin, C.: LIBLINEAR: a library for large linear classification. JMLR 9, 1871–1874 (2008)

    MATH  Google Scholar 

  83. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Completed local structure patterns on three orthogonal planes for dynamic texture recognition. In: IPTA, pp. 1–6 (2017)

  84. Tiwari, D., Tyagi, V.: Improved Weber’s law based local binary pattern for dynamic texture recognition. Multimedia Tools Appl. 76(5), 6623–6640 (2017)

    Article  Google Scholar 

  85. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Dynamic texture representation based on hierarchical local patterns. In: ACIVS, pp. 277–289 (2020)

  86. Xu, Y., Quan, Y., Zhang, Z., Ling, H., Ji, H.: Classifying dynamic textures via spatiotemporal fractal analysis. Pattern Recognit. 48(10), 3239–3248 (2015)

    Article  Google Scholar 

  87. Nguyen, T.T., Nguyen, T.P., Bouchara, F.: Dynamic texture representation based on oriented magnitudes of Gaussian gradients. J. Vis. Commun. Image Represent. 81, 103330 (2021)

    Article  Google Scholar 

  88. Ren, J., Jiang, X., Yuan, J., Wang, G.: Optimizing LBP structure for visual recognition using binary quadratic programming. SPL 21(11), 1346–1350 (2014)

    Google Scholar 

  89. Hadji, I., Wildes, R.P.: A spatiotemporal oriented energy network for dynamic texture recognition. In: ICCV, pp. 3085–3093 (2017)

  90. Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. In: NIPS, pp. 568–576 (2014)

  91. Derpanis, K.G., Wildes, R.P.: Spacetime texture representation and recognition based on a spatiotemporal orientation analysis. IEEE Trans. PAMI 34(6), 1193–1205 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to express our sincere appreciation for the valuable and insightful comments of the editors and reviewers, which allow us to clarify the presentation of this work. Besides, we would like to send many thanks to those in Faculty of IT, HCMC University of Technology and Education, Thu Duc City, Ho Chi Minh City, Vietnam, who gave us crucial supports in high-performing computers for the experiments on the large datasets.

Author information

Authors and Affiliations

  1. Université de Toulon, Aix Marseille Univ, CNRS, LIS, Marseille, France

    Thanh Tuan Nguyen, Thanh Phuong Nguyen & Frédéric Bouchara

  2. HCMC University of Technology and Education, Faculty of IT, Thu Duc City, Ho Chi Minh City, Vietnam

    Thanh Tuan Nguyen

Authors
  1. Thanh Tuan Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thanh Phuong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frédéric Bouchara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thanh Tuan Nguyen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, T.T., Nguyen, T.P. & Bouchara, F. Representing dynamic textures based on polarized gradient features. Machine Vision and Applications 34, 92 (2023). https://doi.org/10.1007/s00138-023-01438-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01438-7

Keywords

Search

Navigation