Abstract
In spite of Convolutional Neural Network (CNN) has dominated in the area of Person Re-Identification, Transformer-based methods have emerged with their advantages in computer vision for processing long sequences in recent two years. In this work, for the purpose of reinforcing complementary advantages of Transformer and CNN in computer vision, a concise method combining Convolution and Transformer is proposed to boost the performance. Firstly, a convolutional network with attention mechanism is employed to generate features with channel and inter-channel relationship information. Moreover, a feature enhancement module is designed to combine pose information and ViT information, and the heatmap generated by the pose estimator is applied to guide ViT features to become good discriminative features. Finally, a relationship reinforced transformer layer is proposed to effectively increase the relationship between features. Experimental results show that the proposed method achieves superior results than interrelated advanced methods on two large-scale person re-Identification benchmark datasets and one occlusion dataset. For Market-1501, our method called Fusion Pose Guidance and Transformer Feature Enhancement for Person Re-Identification gain 94.3% and 87.0% for Rank-1 and mAP respectively. For DukeMTMC-reID our method reaches 88.7% and 77.2% for Rank-1 and mAP respectively. Especially, for the dataset Occluded-Duke, compared with the state of art model HONet, our method, with up to 2.7% and 4.5% performance gains in Rank-1 and mAP respectively.
Similar content being viewed by others
References
Adil M, Mamoon S, Zakir A et al (2020) Multi scale-adaptive super-resolution person re-identification using gan. IEEE Access 8(177):351–177,362
Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3908–3916
Ajagbe SA, Amuda KA, Oladipupo MA et al (2021) Multi-classification of alzheimer disease on magnetic resonance images (mri) using deep convolutional neural network (dcnn) approaches. Int J Adv Comput Res 11(53):51
Ajagbe SA, Oki OA, Oladipupo MA et al (2022) Investigating the efficiency of deep learning models in bioinspired object detection. In: 2022 International Conference on Electrical, Computer and Energy Technologies (ICECET), IEEE, pp 1–6
Cambria E, White B (2014) Jumping nlp curves: a review of natural language processing research. IEEE Comput Intell Mag 9(2):48–57
Cao Z, Simon T, Wei SE et al (2017) Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7291–7299
Carion N, Massa F, Synnaeve G et al (2020) End-to-end object detection with transformers. In: European conference on computer vision, Springer, pp 213–229
Chen B, Deng W, Hu J (2019) Mixed high-order attention network for person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 371–381
Chen T, Ding S, Xie J et al (2019) Abd-net: Attentive but diverse person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 8351–8361
Chen H, Wang Y, Guo T et al (2021) Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 12,299–12,310
Cheng D, Gong Y, Zhou S et al (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the iEEE conference on computer vision and pattern recognition, pp 1335–1344
De Boer PT, Kroese DP, Mannor S et al (2005) A tutorial on the cross-entropy method. Ann Oper Res 134(1):19–67
Dosovitskiy A, Beyer L, Kolesnikov A et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:201011929
Gao H, Chen S, Zhang Z (2019) Parts semantic segmentation aware representation learning for person re-identification. Appl Sci 9(6):1239
Gao S, Wang J, Lu H et al (2020) Pose-guided visible part matching for occluded person reid. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 11,744–11,752
Ge Y, Li Z, Zhao H et al (2018) Fd-gan: Pose-guided feature distilling gan for robust person re-identification. In: Advances in neural information processing systems, pp 1229–1240
Han L, Wang P, Yin Z et al (2020) Exploiting better feature aggregation for video object detection. In: Proceedings of the 28th ACM International conference on multimedia, pp 1469–1477
He L, Liang J, Li H et al (2018) Deep spatial feature reconstruction for partial person re-identification: Alignment-free approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7073–7082
He S, Luo H, Wang P et al (2021) Transreid: Transformer-based object re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 15,013–15,022
He L, Sun Z, Zhu Y et al (2018) Recognizing partial biometric patterns. arXiv:181007399
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Hendrycks D, Gimpel K (2016) Gaussian error linear units (gelus). arXiv:160608415
Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv:170307737
Huang H, Li D, Zhang Z et al (2018) Adversarially occluded samples for person re-identification. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 5098–5107
Huang Y, Sheng H, Zheng Y et al (2017) Deepdiff: Learning deep difference features on human body parts for person re-identification. Neurocomputing 241:191–203
Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, PMLR, pp 448–456
Kumar T A, Rajmohan R, Pavithra M et al (2022) Automatic face mask detection system in public transportation in smart cities using iot and deep learning. Electronics 11(6):904
Li D, Chen X, Zhang Z et al (2017) Learning deep context-aware features over body and latent parts for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 384–393
Li W, Zhao R, Xiao T et al (2014) Deepreid: Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 152–159
Lin TY, Maire M, Belongie S et al (2014) Microsoft coco: Common objects in context. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, Springer, pp 740–755
Lin Y, Zheng L, Zheng Z et al (2019) Improving person re-identification by attribute and identity learning. Pattern Recogn 95:151–161
Luo C, Chen Y, Wang N et al (2019) Spectral feature transformation for person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4976–4985
Luo H, Gu Y, Liao X et al (2019) Bag of tricks and a strong baseline for deep person re-identification. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 1487–1495. https://doi.org/10.1109/CVPRW.2019.00190
Matsukawa T, Suzuki E (2016) Person re-identification using cnn features learned from combination of attributes. In: 2016 23Rd international conference on pattern recognition (ICPR), IEEE, pp 2428–2433
Miao J, Wu Y, Liu P et al (2019) Pose-guided feature alignment for occluded person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 542–551
Park H, Ham B (2020) Relation network for person re-identification. In: Proceedings of the AAAI conference on artificial intelligence, pp 11,839–11,847
Ristani E, Tomasi C (2018) Features for multi-target multi-camera tracking and re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 6036–6046. https://doi.org/10.1109/CVPR.2018.00632
Song C, Huang Y, Ouyang W et al (2018) Mask-guided contrastive attention model for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1179–1188
Su C, Li J, Zhang S et al (2017) Pose-driven deep convolutional model for person re-identification. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp 3980–3989. https://doi.org/10.1109/ICCV.2017.427
Suh Y, Wang J, Tang S et al (2018) Part-aligned bilinear representations for person re-identification. In: Proceedings of the European conference on computer vision (ECCV). 402–419
Sun P, Cao J, Jiang Y et al (2020) Transtrack: Multiple object tracking with transformer. arXiv:201215460
Sun H, Chen Z, Yan S et al (2019) Mvp matching: A maximum-value perfect matching for mining hard samples, with application to person re-identification. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 6737–6747
Sun Y, Cheng C, Zhang Y et al (2020) Circle loss: A unified perspective of pair similarity optimization. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 6398–6407
Sun K, Xiao B, Liu D et al (2019) Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5693–5703
Sun Y, Xu Q, Li Y et al (2019) Perceive where to focus: Learning visibility-aware part-level features for partial person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 393–402
Sun Y, Zheng L, Yang Y et al (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European conference on computer vision (ECCV), pp 480–496
Tay CP, Roy S, Yap KH (2019) Aanet: Attribute attention network for person re-identifications. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7134–7143
Touvron H, Cord M, Douze M et al (2021) Training data-efficient image transformers & distillation through attention. In: International conference on machine learning, PMLR, pp 10,347–10,357
Varior RR, Haloi M, Wang G (2016) Gated siamese convolutional neural network architecture for human re-identification. In: European conference on computer vision, Springer, pp 791–808
Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Wang G, Yang S, Liu H et al (2020) High-order information matters: Learning relation and topology for occluded person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6449–6458
Wei L, Zhang S, Yao H et al (2017) Glad: Global-local-alignment descriptor for pedestrian retrieval. In: Proceedings of the 25th ACM international conference on multimedia, pp 420–428
Yuan L, Chen Y, Wang T et al (2021) Tokens-to-token vit: Training vision transformers from scratch on imagenet. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 558–567
Zhang Z, Lan C, Zeng W et al (2020) Relation-aware global attention for person re-identification. In: Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pp 3186–3195
Zhang X, Luo H, Fan X et al (2017) Alignedreid: Surpassing human-level performance in person re-identification. arXiv:171108184
Zhao L, Li X, Zhuang Y et al (2017) Deeply-learned part-aligned representations for person re-identification. In: Proceedings of the IEEE international conference on computer vision, pp 3219–3228
Zheng L, Huang Y, Lu H et al (2019) Pose-invariant embedding for deep person re-identification. IEEE Trans Image Process 28(9):4500–4509
Zheng L, Shen L, Tian L et al (2015) Scalable person re-identification: A benchmark. pp 1116–1124. https://doi.org/10.1109/ICCV.2015.133
Zheng L, Yang Y, Hauptmann AG (2016) Person re-identification: Past, present and future. arXiv:161002984
Zheng Z, Zheng L, Yang Y (2017) A discriminatively learned cnn embedding for person reidentification. ACM Trans Multimed Comput Commun Appl (TOMM) 14(1):1–20
Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. pp 3774–3782. https://doi.org/10.1109/ICCV.2017.405
Zhong Z, Zheng L, Kang G et al (2020) Random erasing data augmentation. In: Proceedings of the AAAI conference on artificial intelligence, pp 13,001–13,008
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 61972056, in part by the Hunan Provincial Natural Science Foundation of China under Grant 2021JJ30743, in part by the Degree & Post-graduate Education Reform Project of Hunan Province of China under Grant 2020JGZD043.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors declare that they have no conflicts of interest to report regarding the present study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhou, S., Zou, W. Fusion pose guidance and transformer feature enhancement for person re-identification. Multimed Tools Appl 83, 21745–21763 (2024). https://doi.org/10.1007/s11042-023-15303-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-15303-2