Article

Interpretable and Generalizable Person Re-identification with Query-Adaptive Convolution and Temporal Lifting

Authors:

Ling ShaoAuthors Info & Claims

Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI

Pages 456 - 474

https://doi.org/10.1007/978-3-030-58621-8_27

Published: 23 August 2020 Publication History

Abstract

For person re-identification, existing deep networks often focus on representation learning. However, without transfer learning, the learned model is fixed as is, which is not adaptable for handling various unseen scenarios. In this paper, beyond representation learning, we consider how to formulate person image matching directly in deep feature maps. We treat image matching as finding local correspondences in feature maps, and construct query-adaptive convolution kernels on the fly to achieve local matching. In this way, the matching process and results are interpretable, and this explicit matching is more generalizable than representation features to unseen scenarios, such as unknown misalignments, pose or viewpoint changes. To facilitate end-to-end training of this architecture, we further build a class memory module to cache feature maps of the most recent samples of each class, so as to compute image matching losses for metric learning. Through direct cross-dataset evaluation, the proposed Query-Adaptive Convolution (QAConv) method gains large improvements over popular learning methods (about 10%+ mAP), and achieves comparable results to many transfer learning methods. Besides, a model-free temporal cooccurrence based score weighting method called TLift is proposed, which improves the performance to a further extent, achieving state-of-the-art results in cross-dataset person re-identification. Code is available at https://github.com/ShengcaiLiao/QAConv.

References

[1]

Ahmed, E., Jones, M., Marks, T.K.: An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3908–3916 (2015)

[2]

Bay H, Tuytelaars T, and Van Gool L Speeded-up robust features (SURF) Comput. Vis. Image Underst. 2008 110 3 346-359

[3]

Chang X, Yang Y, Xiang T, and Hospedales TM Disjoint label space transfer learning with common factorised space Proc. AAAI Conf. Artif. Intell. 2019 33 3288-3295

[4]

Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4690–4699 (2019)

[5]

DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)

[6]

Ergys, R., Francesco, S., Roger, Z., Rita, C., Carlo, T.: Performance measures and a data set for multi-target, multi-camera tracking. In: ECCV workshop on Benchmarking Multi-Target Tracking (2016)

[7]

Fan H, Zheng L, Yan C, and Yang Y Unsupervised person re-identification: clustering and fine-tuning TOMM 2018 14 4 83

[8]

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

[9]

Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

[10]

Hu, Y., Yi, D., Liao, S., Lei, Z., Li, S.Z.: Cross dataset person Re-identification. In: ACCV Workshop on Human Identification for Surveillance (HIS), pp. 650–664 (2014)

[11]

Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)

[12]

Jia, J., Ruan, Q., Hospedales, T.M.: Frustratingly easy person re-identification: Generalizing person re-id in practice. In: British Machine Vision Conference (2019)

[13]

Jin, H., Wang, X., Liao, S., Li, S.Z.: Deep person re-identification with improved embedding and efficient training. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 261–267. IEEE (2017)

[14]

Kalayeh, M.M., Emrah, B., Gökmen, M., Kamasak, M.E., Shah, M.: Human semantic parsing for person re-identification. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1062–1071 (2018)

[15]

Li, M., Zhu, X., Gong, S.: Unsupervised person re-identification by deep learning tracklet association. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 737–753 (2018)

[16]

Li M, Zhu X, and Gong S Unsupervised tracklet person re-identification TPAMI 2019 42 7 1770-1782

[17]

Li, W., Zhao, R., Xiao, T., Wang, X.: DeepReID: deep filter pairing neural network for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)

[18]

Li, W., Zhu, X., Gong, S.: Harmonious attention network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2285–2294 (2018)

[19]

Liao, S., Shao, L.: Interpretable and generalizable deep image matching with adaptive convolutions. CoRR abs/1904.10424v1 (23, April 2019), http://arxiv.org/abs/1904.10424v1

[20]

Lin, S., Li, H., Li, C.T., Kot, A.C.: Multi-task mid-level feature alignment network for unsupervised cross-dataset person re-identification. In: The British Machine Vision Conference (BMVC) (2018)

[21]

Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

[22]

Lin, T.Y., Roychowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: IEEE International Conference on Computer Vision (2015)

[23]

Liu, C., Loy, C.C., Gong, S., Wang, G.: Pop: person re-identification post-rank optimisation. In: International Conference on Computer Vision (2013)

[24]

Liu H, Feng J, Qi M, Jiang J, and Yan S End-to-end comparative attention networks for person re-identification IEEE Trans. Image Process. 2017 26 7 3492-3506

[25]

Liu, X., et al.: Hydraplus-net: attentive deep features for pedestrian analysis. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 350–359 (2017)

[26]

Lowe DG Distinctive image features from scale-invariant keypoints Int. J. Comput. Vis. 2004 60 2 91-110

[27]

Lv, J., Chen, W., Li, Q., Yang, C.: Unsupervised cross-dataset person re-identification by transfer learning of spatial-temporal patterns. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 7948–7956 (2018)

[28]

Lv, J., Chen, W., Li, Q., Yang, C.: Unsupervised cross-dataset person re-identification by transfer learning of spatial-temporal patterns. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, pp. 7948–7956 (2018)

[29]

Pan, X., Luo, P., Shi, J., Tang, X.: Two at once: Enhancing learning and generalization capacities via ibn-net. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 464–479 (2018)

[30]

Peng, P., Xiang, T., Wang, Y., Pontil, M., Gong, S., Huang, T., Tian, Y.: Unsupervised cross-dataset transfer learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1306–1315 (2016)

[31]

Qian, X., Fu, Y., Jiang, Y.G., Xiang, T., Xue, X.: Multi-scale deep learning architectures for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5399–5408 (2017)

[32]

Qian, X., et al.: Pose-normalized image generation for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 650–667 (2018)

[33]

Saquib Sarfraz, M., Schumann, A., Eberle, A., Stiefelhagen, R.: A pose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 420–429 (2018)

[34]

Shen, Y., Xiao, T., Li, H., Yi, S., Wang, X.: End-to-end deep kronecker-product matching for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6886–6895 (2018)

[35]

Si, J., et al.: Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5363–5372 (2018)

[36]

Song, J., Yang, Y., Song, Y.Z., Xiang, T., Hospedales, T.M.: Generalizable person re-identification by domain-invariant mapping network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 719–728 (2019)

[37]

Suh, Y., Wang, J., Tang, S., Mei, T., Lee, K.M.: Part-aligned bilinear representations for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 402–419 (2018)

[38]

Suh, Y., Wang, J., Tang, S., Mei, T., Mu Lee, K.: Part-aligned bilinear representations for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 402–419 (2018)

[39]

Sun, Y., Zheng, L., Yang, Y., Tian, Q., Wang, S.: Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

[40]

Ustinova, E., Ganin, Y., Lempitsky, V.: Multi-Region bilinear convolutional neural networks for person re-identification. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (2017)

[41]

Wang, G., Lai, J., Huang, P., Xie, X.: Spatial-temporal person re-identification. In: AAAI Conference on Artificial Intelligence (2019)

[42]

Wang, G., Yuan, Y., Chen, X., Li, J., Zhou, X.: Learning discriminative features with multiple granularities for person re-identification. In: 2018 ACM Multimedia Conference on Multimedia Conference, pp. 274–282. ACM (2018)

[43]

Wang, J., Zhu, X., Gong, S., Li, W.: Transferable joint attribute-identity deep learning for unsupervised person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2275–2284 (2018)

[44]

Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

[45]

Wei, L., Zhang, S., Gao, W., Tian, Q.: Person transfer gan to bridge domain gap for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 79–88 (2018)

[46]

Wen Yandong, Zhang Kaipeng, Li Zhifeng, and Qiao Yu Leibe Bastian, Matas Jiri, Sebe Nicu, and Welling Max A discriminative feature learning approach for deep face recognition Computer Vision – ECCV 2016 2016 Cham Springer 499-515

[47]

Wu, J., Liao, S., Wang, X., Yang, Y., Li, S.Z., et al.: Clustering and dynamic sampling based unsupervised domain adaptation for person re-identification. In: 2019 IEEE International Conference on Multimedia and Expo (ICME), pp. 886–891. IEEE (2019)

[48]

Wu, J., Yang, Y., Liu, H., Liao, S., Lei, Z., Li, S.Z.: Unsupervised graph association for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8321–8330 (2019)

[49]

Xu, J., Zhao, R., Zhu, F., Wang, H., Ouyang, W.: Attention-aware compositional network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2119–2128 (2018)

[50]

Xu, S., Cheng, Y., Gu, K., Yang, Y., Chang, S., Zhou, P.: Jointly attentive spatial-temporal pooling networks for video-based person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4733–4742 (2017)

[51]

Yang, Q., Yu, H.X., Wu, A., Zheng, W.S.: Patch-based discriminative feature learning for unsupervised person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3633–3642 (2019)

[52]

Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., Hoi, S.C.H.: Deep Learning for Person Re-identification: A Survey and Outlook. arXiv preprint arXiv:2001.04193 (2020)

[53]

Yi, D., Lei, Z., Liao, S., Li, S.Z.: Deep metric learning for person re-identification. In: International Conference on Pattern Recognition, pp. 34–39 (December 2014)

[54]

Yu, H.X., Wu, A., Zheng, W.S.: Unsupervised person re-identification by deep asymmetric metric embedding. In: IEEE Transactions on Pattern Analysis and Machine intelligence (2019)

[55]

Yu, H.X., Zheng, W.S., Wu, A., Guo, X., Gong, S., Lai, J.H.: Unsupervised person re-identification by soft multilabel learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2148–2157 (2019)

[56]

Yu, R., Zhou, Z., Bai, S., Bai, X.: Divide and fuse: a re-ranking approach for person re-identification. In: The British Machine Vision Conference (BMVC) (2017)

[57]

Zhao, H., et al.: Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1077–1085 (2017)

[58]

Zhao, L., Li, X., Zhuang, Y., Wang, J.: Deeply-learned part-aligned representations for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3219–3228 (2017)

[59]

Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: Proceedings of IEEE International Conference on Computer Vision (2015)

[60]

Zheng, Z., Zheng, L., Yang, Y.: Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. In: International Conference on Computer Vision, pp. 3774–3782 (2017)

[61]

Zhong, Z., Zheng, L., Zheng, Z., Li, S., Yang, Y.: Camstyle: A novel data augmentation method for person re-identification. In: IEEE Transactions on Image Processing (2018)

[62]

Zhong, Z., Zheng, L., Cao, D., Li, S.: Re-ranking person re-identification with k-reciprocal encoding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1318–1327 (2017)

[63]

Zhong, Z., Zheng, L., Kang, G., Li, S., Yang, Y.: Random erasing data augmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2020)

[64]

Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Invariance matters: Exemplar memory for domain adaptive person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 598–607 (2019)

Cited By

Lv KChen HZhao CTu KChen JLi YLi BLin Y(2024)Style Variable and Irrelevant Learning for Generalizable Person Re-identificationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367100320:9(1-22)Online publication date: 6-Jun-2024
https://dl.acm.org/doi/10.1145/3671003
Zhao QYu WJi T(2024)Style Elimination and Information Restitution for generalizable person re-identificationJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.10404898:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.jvcir.2024.104048
Peng WChen HLi YSun J(2024)Multi-source domain generalization peron re-identification with knowledge accumulation and distribution enhancementApplied Intelligence10.1007/s10489-024-05266-854:2(1818-1830)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-024-05266-8
Show More Cited By

Index Terms

Interpretable and Generalizable Person Re-identification with Query-Adaptive Convolution and Temporal Lifting
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Machine learning
    1. Learning paradigms
    2. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Learning to synthesize programs as interpretable and generalizable policies
NIPS '21: Proceedings of the 35th International Conference on Neural Information Processing Systems

Recently, deep reinforcement learning (DRL) methods have achieved impressive performance on tasks in a variety of domains. However, neural network policies produced with DRL methods are not human-interpretable and often have difficulty generalizing to ...
Debiased Contrastive Curriculum Learning for Progressive Generalizable Person Re-Identification
Domain generalization (DG) in person re-identification (ReID) is an extremely challenging but essential task, which aims to learn a generalizable model over multiple labeled source domains that can perform well on unseen target domains. Most existing DG ...
Unsupervised Person Re-identification: Clustering and Fine-tuning
Special Section on Deep Learning for Intelligent Multimedia Analytics

The superiority of deeply learned pedestrian representations has been reported in very recent literature of person re-identification (re-ID). In this article, we consider the more pragmatic issue of learning a deep feature with no or only a few labels. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI

Aug 2020

857 pages

ISBN:978-3-030-58620-1

DOI:10.1007/978-3-030-58621-8

Editors:
Andrea Vedaldi
University of Oxford, Oxford, UK
,
Horst Bischof
Graz University of Technology, Graz, Austria
,
Thomas Brox
University of Freiburg, Freiburg im Breisgau, Germany
,
Jan-Michael Frahm
University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

© Springer Nature Switzerland AG 2020.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 23 August 2020

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lv KChen HZhao CTu KChen JLi YLi BLin Y(2024)Style Variable and Irrelevant Learning for Generalizable Person Re-identificationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367100320:9(1-22)Online publication date: 6-Jun-2024
https://dl.acm.org/doi/10.1145/3671003
Zhao QYu WJi T(2024)Style Elimination and Information Restitution for generalizable person re-identificationJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.10404898:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.jvcir.2024.104048
Peng WChen HLi YSun J(2024)Multi-source domain generalization peron re-identification with knowledge accumulation and distribution enhancementApplied Intelligence10.1007/s10489-024-05266-854:2(1818-1830)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s10489-024-05266-8
Chang TYang XLuo XJi WWang MEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Learning Style-Invariant Robust Representation for Generalizable Visual Instance RetrievalProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611949(6171-6180)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611949
Dong NZhang LYan STang HTang J(2023)Erasing, Transforming, and Noising Defense Network for Occluded Person Re-IdentificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.333916734:6(4458-4472)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1109/TCSVT.2023.3339167
Delussu RPutzu LFumera G(2023)Human-in-the-loop cross-domain person re-identificationExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120216226:COnline publication date: 15-Sep-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120216
Liu JYang M(2023)Prompt-Based Transformer for Generalizable Person Re-identification with Image MaskingBiometric Recognition10.1007/978-981-99-8565-4_25(259-268)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/978-981-99-8565-4_25
Song PPeng J(2023)Learning Frequency-Based Disentanglement and Filtering for Generalizable Person Re-identificationPattern Recognition and Computer Vision10.1007/978-981-99-8555-5_38(482-494)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-8555-5_38
Cheng LKuang ZZhang HDing XHuang Y(2023)Boosting Generalization Performance in Person Re-identificationPattern Recognition and Computer Vision10.1007/978-981-99-8549-4_15(174-185)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-8549-4_15
Almansoori MFiaz MCholakkal H(2023)Anchor-ReID: A Test Time Adaptation for Person Re-identificationImage Analysis10.1007/978-3-031-31438-4_39(599-612)Online publication date: 18-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-31438-4_39
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents