Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

MetaGait: Learning to Learn an Omni Sample Adaptive Representation for Gait Recognition

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13665))

Included in the following conference series:

  • 2480 Accesses

Abstract

Gait recognition, which aims at identifying individuals by their walking patterns, has recently drawn increasing research attention. However, gait recognition still suffers from the conflicts between the limited binary visual clues of the silhouette and numerous covariates with diverse scales, which brings challenges to the model’s adaptiveness. In this paper, we address this conflict by developing a novel MetaGait that learns to learn an omni sample adaptive representation. Towards this goal, MetaGait injects meta-knowledge, which could guide the model to perceive sample-specific properties, into the calibration network of the attention mechanism to improve the adaptiveness from the omni-scale, omni-dimension, and omni-process perspectives. Specifically, we leverage the meta-knowledge across the entire process, where Meta Triple Attention and Meta Temporal Pooling are presented respectively to adaptively capture omni-scale dependency from spatial/channel/temporal dimensions simultaneously and to adaptively aggregate temporal information through integrating the merits of three complementary temporal aggregation methods. Extensive experiments demonstrate the state-of-the-art performance of the proposed MetaGait. On CASIA-B, we achieve rank-1 accuracy of 98.7%, 96.0%, and 89.3% under three conditions, respectively. On OU-MVLP, we achieve rank-1 accuracy of 92.4%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Antoniou, A., Edwards, H., Storkey, A.: How to train your MAML. arXiv preprint arXiv:1810.09502 (2018)

  2. Ariyanto, G., Nixon, M.S.: Model-based 3D gait biometrics. In: International Joint Conference on Biometrics, pp. 1–7 (2011)

    Google Scholar 

  3. Balazia, M., Plataniotis, K.N.: Human gait recognition from motion capture data in signature poses. IET Biom. 6, 129–137 (2017)

    Article  Google Scholar 

  4. Bashir, K., Xiang, T., Gong, S.: Gait recognition using gait entropy image. In: IET International Conference on Imaging for Crime Detection and Prevention, pp. 1–6 (2009)

    Google Scholar 

  5. Bengio, E., Bacon, P.L., Pineau, J., Precup, D.: Conditional computation in neural networks for faster models. arXiv preprint arXiv:1511.06297 (2015)

  6. Bodor, R., Drenner, A., Fehr, D., Masoud, O., Papanikolopoulos, N.: View-independent human motion classification using image-based reconstruction. Image Vis. Comput. 27(8), 1194–1206 (2009)

    Article  Google Scholar 

  7. Bouchrika, I.: A survey of using biometrics for smart visual surveillance: gait recognition. In: Karampelas, P., Bourlai, T. (eds.) Surveillance in Action, pp. 3–23. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-68533-5_1

    Chapter  Google Scholar 

  8. Boulgouris, N.V., Chi, Z.X.: Gait recognition based on human body components. In: IEEE International Conference on Image Processing, pp. 353–356 (2007)

    Google Scholar 

  9. Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)

    Google Scholar 

  10. Chao, H., He, Y., Zhang, J., Feng, J.: GaitSet: regarding gait as a set for cross-view gait recognition. In: AAAI (2019)

    Google Scholar 

  11. Chattopadhyay, P., Sural, S., Mukherjee, J.: Frontal gait recognition from incomplete sequences using RGB-D camera. IEEE Trans. Inf. Forensics Secur. 9(11), 1843–1856 (2014)

    Article  Google Scholar 

  12. Chen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., Liu, Z.: Dynamic convolution: attention over convolution kernels. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11030–11039 (2020)

    Google Scholar 

  13. Cheng, H.P., et al.: Swiftnet: using graph propagation as meta-knowledge to search highly representative neural architectures. arXiv preprint arXiv:1906.08305 (2019)

  14. Dai, J., et al.: Deformable convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 764–773 (2017)

    Google Scholar 

  15. Denil, M., Shakibi, B., Dinh, L., Ranzato, M., De Freitas, N.: Predicting parameters in deep learning. In: Advances in Neural Information Processing Systems (2013)

    Google Scholar 

  16. Devos, A., Chatel, S., Grossglauser, M.: Reproducing meta-learning with differentiable closed-form solvers. In: International Conference on Learning Representations (2019)

    Google Scholar 

  17. Dou, H., et al.: Versatilegait: a large-scale synthetic gait dataset with fine-grainedattributes and complicated scenarios. arXiv preprint arXiv:2101.01394 (2021)

  18. Du, Y., Wang, W., Wang, L.: Hierarchical recurrent neural network for skeleton based action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1110–1118 (2015)

    Google Scholar 

  19. Eigen, D., Ranzato, M., Sutskever, I.: Learning factored representations in a deep mixture of experts. arXiv preprint arXiv:1312.4314 (2013)

  20. Fan, C., et al.: Gaitpart: temporal part-based model for gait recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  21. Finn, C., Rajeswaran, A., Kakade, S., Levine, S.: Online meta-learning. In: International Conference on Machine Learning, pp. 1920–1930 (2019)

    Google Scholar 

  22. Gao, H., Zhu, X., Lin, S., Dai, J.: Deformable kernels: adapting effective receptive fields for object deformation. In: International Conference on Learning Representations (2019)

    Google Scholar 

  23. Goffredo, M., Bouchrika, I., Carter, J.N., Nixon, M.S.: Self-calibrating view-invariant gait biometrics. IEEE Trans. Cybern. 40(4), 997–1008 (2009)

    Article  Google Scholar 

  24. Zhao, G., Liu, G., Li, H., Pietikainen, M.: 3D gait recognition using multiple cameras. In: International Conference on Automatic Face Gesture Recognition, pp. 529–534 (2006)

    Google Scholar 

  25. Han, J., Bhanu, B.: Individual recognition using gait energy image. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 316–322 (2006)

    Article  Google Scholar 

  26. Harley, A.W., Derpanis, K.G., Kokkinos, I.: Segmentation-aware convolutional networks using local attention masks. In: International Conference on Computer Vision, pp. 5038–5047 (2017)

    Google Scholar 

  27. He, B., Yang, X., Wu, Z., Chen, H., Lim, S.N., Shrivastava, A.: GTA: global temporal attention for video action understanding. arXiv preprint arXiv:2012.08510 (2020)

  28. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  29. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017)

  30. Hospedales, T., Antoniou, A., Micaelli, P., Storkey, A.: Meta-learning in neural networks: a survey. arXiv preprint arXiv:2004.05439 (2020)

  31. Hou, S., Cao, C., Liu, X., Huang, Y.: Gait lateral network: learning discriminative and compact representations for gait recognition. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12354, pp. 382–398. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_22

    Chapter  Google Scholar 

  32. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

    Google Scholar 

  33. Huang, X., et al.: Context-sensitive temporal feature learning for gait recognition. In: International Conference on Computer Vision, pp. 12909–12918, October 2021

    Google Scholar 

  34. Huang, Y., Zhang, J., Zhao, H., Zhang, L.: Attention-based network for cross-view gait recognition. In: Advances in Neural Information Processing Systems, pp. 489–498 (2018)

    Google Scholar 

  35. Huang, Y., et al.: Curricularface: adaptive curriculum learning loss for deep face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  36. Huang, Z., et al.: 3D local convolutional neural networks for gait recognition. In: International Conference on Computer Vision, pp. 14920–14929, October 2021

    Google Scholar 

  37. Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Comput. 3(1), 79–87 (1991)

    Article  Google Scholar 

  38. Kastaniotis, D., Theodorakopoulos, I., Fotopoulos, S.: Pose-based gait recognition with local gradient descriptors and hierarchically aggregated residuals. J. Electron. Imaging 25(6), 063019 (2016)

    Article  Google Scholar 

  39. Kusakunniran, W., Wu, Q., Zhang, J., Li, H., Wang, L.: Recognizing gaits across views through correlated motion co-clustering. IEEE Trans. Image Process. 23(2), 696–709 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  40. Kusakunniran, W., Wu, Q., Zhang, J., Ma, Y., Li, H.: A new view-invariant feature for cross-view gait recognition. IEEE Trans. Inf. Forensics Secur. 8(10), 1642–1653 (2013)

    Article  Google Scholar 

  41. Li, S., Liu, W., Ma, H.: Attentive spatial-temporal summary networks for feature learning in irregular gait recognition. IEEE Trans. Multimedia 21(9), 2361–2375 (2019)

    Article  Google Scholar 

  42. Liao, R., Yu, S., An, W., Huang, Y.: A model-based gait recognition method with body pose and human prior knowledge. Pattern Recog. 98, 107069 (2020)

    Article  Google Scholar 

  43. Lin, B., Zhang, S., Bao, F.: Gait recognition with multiple-temporal-scale 3D convolutional neural network. In: ACM International Conference on Multimedia, pp. 3054–3062 (2020)

    Google Scholar 

  44. Lin, B., Zhang, S., Yu, X.: Gait recognition via effective global-local feature representation and local temporal aggregation. In: International Conference on Computer Vision, pp. 14648–14656, October 2021

    Google Scholar 

  45. Lin, J., Rao, Y., Lu, J., Zhou, J.: Runtime neural pruning. In: Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  46. Lin, P., Sun, P., Cheng, G., Xie, S., Li, X., Shi, J.: Graph-guided architecture search for real-time semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, June 2020

    Google Scholar 

  47. Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., Chi, E.H.: Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: SIGKDD, pp. 1930–1939 (2018)

    Google Scholar 

  48. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)

    Google Scholar 

  49. Macoveciuc, I., Rando, C.J., Borrion, H.: Forensic gait analysis and recognition: standards of evidence admissibility. J. Forensic Sci. 64(5), 1294–1303 (2019)

    Article  Google Scholar 

  50. Makihara, Y., Sagawa, R., Mukaigawa, Y., Echigo, T., Yagi, Y.: Gait recognition using a view transformation model in the frequency domain. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 151–163. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_12

    Chapter  Google Scholar 

  51. Radenović, F., Tolias, G., Chum, O.: Fine-tuning CNN image retrieval with no human annotation. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1655–1668 (2018)

    Article  Google Scholar 

  52. Samangooei, S., Nixon, M.S.: Performing content-based retrieval of humans using gait biometrics. Multimed. Tools Appl. 49, 195–212 (2010)

    Google Scholar 

  53. Sepas-Moghaddam, A., Etemad, A.: Deep gait recognition: a survey. arXiv preprint arXiv:2102.09546 (2021)

  54. Shan, S., Li, Y., Oliva, J.B.: Meta-neighborhoods. In: Advances in Neural Information Processing Systems, vol. 33, pp. 5047–5057 (2020)

    Google Scholar 

  55. Su, H., Jampani, V., Sun, D., Gallo, O., Learned-Miller, E., Kautz, J.: Pixel-adaptive convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11166–11175 (2019)

    Google Scholar 

  56. Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  57. Takemura, N., Makihara, Y., Muramatsu, D., Echigo, T., Yagi, Y.: Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Trans. Comput. Vis. Appl. 10(1), 1–14 (2018). https://doi.org/10.1186/s41074-018-0039-6

    Article  Google Scholar 

  58. Veit, A., Belongie, S.: Convolutional networks with adaptive inference graphs. In: European Conference on Computer Vision, pp. 3–18 (2018)

    Google Scholar 

  59. Wang, D., Zhang, S.: Unsupervised person re-identification via multi-label classification. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  60. Wang, L., Ning, H., Tan, T., Hu, W.: Fusion of static and dynamic body biometrics for gait recognition. IEEE TCSVT 14(2), 149–158 (2004)

    Google Scholar 

  61. Wang, X., Yu, F., Dou, Z.Y., Darrell, T., Gonzalez, J.E.: Skipnet: learning dynamic routing in convolutional networks. In: European Conference on Computer Vision, pp. 409–424 (2018)

    Google Scholar 

  62. Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: CBAM: convolutional block attention module. In: European Conference on Computer Vision, pp. 3–19 (2018)

    Google Scholar 

  63. Yang, B., Bender, G., Le, Q.V., Ngiam, J.: Condconv: conditionally parameterized convolutions for efficient inference. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  64. Yu, S., Tan, D., Tan, T.: A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: International Conference on Pattern Recognition, pp. 441–444 (2006)

    Google Scholar 

  65. Zhang, F., Wah, B.W.: Supplementary meta-learning: towards a dynamic model for deep neural networks. In: International Conference on Computer Vision, pp. 4344–4353 (2017)

    Google Scholar 

  66. Zhang, Y., Huang, Y., Yu, S., Wang, L.: Cross-view gait recognition by discriminative feature learning. IEEE Trans. Image Process. 29, 1001–1015 (2019)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgement

This work is supported in part by the National Natural Science Foundation of China under Grant U20A20222, National Key Research and Development Program of China under Grant 2020AAA0107400, Zhejiang Provincial Natural Science Foundation of China under Grant LR19F020004, NSFC (62002320, U19B2043) and the Key R &D Program of Zhejiang Province, China (2021C01119).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University, Hangzhou, China

    Huanzhang Dou, Pengyi Zhang, Wei Su & Xi Li

  2. College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, China

    Yunlong Yu

  3. Shanghai Institute for Advanced Study, Zhejiang University, Hangzhou, China

    Xi Li

  4. Shanghai AI Laboratory, Shanghai, China

    Xi Li

Authors
  1. Huanzhang Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunlong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yunlong Yu or Xi Li .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 142 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dou, H., Zhang, P., Su, W., Yu, Y., Li, X. (2022). MetaGait: Learning to Learn an Omni Sample Adaptive Representation for Gait Recognition. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13665. Springer, Cham. https://doi.org/10.1007/978-3-031-20065-6_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20065-6_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20064-9

  • Online ISBN: 978-3-031-20065-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation