Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

MBA-Net: multi-branch attention network for occluded person re-identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Occluded person re-identification (ReID) aims to retrieve the same pedestrian from partially occluded pedestrian images across non-overlapping cameras. Current state-of-the-art methods generally use auxiliary models to obtain non-occluded regions, which not only result in more complex models, but also cannot effectively handle the more generalized ReID task. To this end, a Multi-Branch Attention Network (MBA-Net) is proposed to achieve multi-level refinement of features through an end-to-end multi-branch framework with attention mechanisms. Specifically, we first achieve preliminary feature refinement through a backbone network with a non-local attention mechanism. Then, a two-level multi-branch architecture in MBA-Net is proposed with two-level features refinement to obtain aware local discriminative features from the self-attention branch, non-occluded local complementary features from the cross-attention branch, and global features from the global branch. Finally, we can obtain retrieval features that are robust to occlusion by concatenating all the above features. Experimental results show that our MBA-Net achieves state-of-the-art performance on an occluded person ReID dataset Occluded-Duke and simultaneously achieves competitive performance on two general person ReID datasets Market-1501 and DukeMTMC-ReID.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

Market-1501 dataset analysed during the current study are available in https://drive.google.com/file/d/0B8-rUzbwVRk0c054eEozWG9COHM/view, DukeMTMC-reID dataset analysed during the current study is available from the corresponding author on reasonable request https://arxiv.org/abs/1609.01775, Occluded-Duke dataset analysed during the current study is available in the [https://github.com/lightas/Occluded-DukeMTMC-Dataset] repository.

References

  1. Chen W, Chen X, Zhang J, Huang K (2017) Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 403–412

  2. Chen G, Lin C, Ren L, Lu J, Zhou J (2019) Self-critical attention learning for person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9637–9646

  3. Chen W, Lu Y, Ma H, Chen Q, Wu X, Wu P (2022) Self-attention mechanism in person re-identification models. Multimed Tools Appl 81 (4):4649–4667

    Article  Google Scholar 

  4. Chen Y, Yang Y, Liu W, Huang Y, Li J (2022) Pose-guided counterfactual inference for occluded person re-identification. Image Vis Comput 128:104587

    Article  Google Scholar 

  5. Cheng D, Gong Y, Shi W, Zhang S (2018) Person re-identification by the asymmetric triplet and identification loss function. Multimed Tools Appl 77(3):3533–3550

    Article  Google Scholar 

  6. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv:2010.11929

  7. Gao S, Wang J, Lu H, Liu Z (2020) Pose-guided visible part matching for occluded person reid. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11744–11752

  8. He L, Liao X, Liu W, Liu X, Cheng P, Mei T (2020) Fastreid: a pytorch toolbox for general instance re-identification. arXiv:2006.02631

  9. He S, Luo H, Wang P, Wang F, Li H, Jiang W (2021) Transreid: transformer-based object re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 15013–15022

  10. Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv:1703.07737

  11. Jin H, Lai S, Qian X (2021) Occlusion-sensitive person re-identification via attribute-based shift attention. IEEE Trans Circuits Syst Video Technol 32(4):2170–2185

    Article  Google Scholar 

  12. Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2285–2294

  13. Li Y, He J, Zhang T, Liu X, Zhang Y, Wu F (2021) Diverse part discovery: occluded person re-identification with part-aware transformer. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 2898–2907

  14. Liao S, Li S (2015) Efficient psd constrained asymmetric metric learning for person re-identification. In: Proceedings of the IEEE international conference on computer vision, pp 3685–3693

  15. Luo H, Gu Y, Liao X, Lai S, Jiang W (2019) Bag of tricks and a strong baseline for deep person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 0–0

  16. Ma B, Su Y, Jurie F (2014) Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image Vis Comput 32 (6-7):379–390

    Article  Google Scholar 

  17. Miao J, Wu Y, Liu P, Ding Y, Yang Y (2019) Pose-guided feature alignment for occluded person re-identification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 542–551

  18. Ren X, Zhang D, Bao X, Shi L (2022) Double granularity relation network with self-criticism for occluded person re-identification. In: MultiMedia Modeling: 28th International Conference, MMM 2022, Phu Quoc, Vietnam, June 6–10, 2022, Proceedings, Part I, pp 325–338. Springer

  19. Ristani E, Solera F, Zou R, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European conference on computer vision, pp 17–35. Springer

  20. Shi Y, Tian W, Ling H, Li Z, Li P (2022) Spatial-wise and channel-wise feature uncertainty for occluded person re-identification. Neurocomputing 486:237–249

    Article  Google Scholar 

  21. Si J, Zhang H, Li CG, Kuen J, Kong X, Kot AC, Wang G (2018) Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5363–5372

  22. Srinivas S, Fleuret F (2019) Full-gradient representation for neural network visualization. Advances in Neural Information Processing Systems, 32

  23. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European conference on computer vision (ECCV), pp 480–496

  24. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  25. Wang F, Zuo W, Lin L, Zhang D, Zhang L (2016) Joint learning of single-image and cross-image representations for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1288–1296

  26. Wang C, Zhang Q, Huang C, Liu W, Wang X (2018) Mancs: a multi-task attentional network with curriculum sampling for person re-identification. In: Proceedings of the European conference on computer vision (ECCV), pp 365–381

  27. Wang G, Yuan Y, Chen X, Li J, Zhou X (2018) Learning discriminative features with multiple granularities for person re-identification. In: Proceedings of the 26th ACM international conference on multimedia, pp 274–282

  28. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  29. Wang G, Yang S, Liu H, Wang Z, Yang Y, Wang S, Yu G, Zhou E, Sun J (2020) High-order information matters: learning relation and topology for occluded person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6449–6458

  30. Wang H, Chen X, Liu C (2021) Pose-guided part matching network via shrinking and reweighting for occluded person re-identification. Image Vis Comput 111:104186

    Article  Google Scholar 

  31. Wang L, Zhou Y, Sun Y, Li S (2022) Occluded person re-identification based on differential attention siamese network. Appl Intell 52(7):7407–7419

    Article  Google Scholar 

  32. Wu W, Tao D, Li H, Yang Z, Cheng J (2021) Deep features for person re-identification on metric learning. Pattern Recogn 110:107424

    Article  Google Scholar 

  33. Xiang S, Fu Y, Chen H, Ran W, Liu T (2020) Multi-level feature learning with attention for person re-identification. Multimed Tools Appl 79(43):32079–32093

    Article  Google Scholar 

  34. Xu Y, Zhao L, Qin F (2021) Dual attention-based method for occluded person re-identification. Knowl-Based Syst 212:106554

    Article  Google Scholar 

  35. Yang Y, Yang J, Yan J, Liao S, Yi D, Li S (2014) Salient color names for person re-identification. In: European conference on computer vision, pp 536–551. Springer

  36. Yang J, Zhang C, Tang Y, Li Z (2022) Pafm: pose-drive attention fusion mechanism for occluded person re-identification. Neural Comput Applic 34(10):8241–8252

    Article  Google Scholar 

  37. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SC (2021) Deep learning for person re-identification: a survey and outlook. IEEE Trans Pattern Anal Mach Intell 44(6):2872–2893

    Article  Google Scholar 

  38. Yu R, Dou Z, Bai S, Zhang Z, Xu Y, Bai X (2018) Hard-aware point-to-set deep metric for person re-identification. In: Proceedings of the European conference on computer vision (ECCV), pp 188–204

  39. Zhang X, Luo H, Fan X, Xiang W, Sun Y, Xiao Q, Jiang W, Zhang C, Sun J (2017) Alignedreid: surpassing human-level performance in person re-identification. arXiv:1711.08184

  40. Zhang X, Yan Y, Xue JH, Hua Y, Wang H (2020) Semantic-aware occlusion-robust network for occluded person re-identification. IEEE Trans Circuits Syst Video Technol 31(7):2764–2778

    Article  Google Scholar 

  41. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1077–1085

  42. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: Proceedings of the IEEE international conference on computer vision, pp 1116–1124

  43. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE international conference on computer vision, pp 3754–3762

  44. Zheng F, Deng C, Sun X, Jiang X, Guo X, Yu Z, Huang F, Ji R (2019) Pyramidal person re-identification via multi-loss dynamic training. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8514–8522

  45. Zheng Z, Yang X, Yu Z, Zheng L, Yang Y, Kautz J (2019) Joint discriminative and generative learning for person re-identification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2138–2147

  46. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2020) Random erasing data augmentation. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 13001–13008

  47. Zhou Q, Zhong B, Lan X, Sun G, Zhang Y, Zhang B, Ji R (2020) Fine-grained spatial alignment model for person re-identification with focal triplet loss. IEEE Trans Image Process 29:7578–7589

    Article  Google Scholar 

  48. Zhu K, Guo H, Liu Z, Tang M, Wang J (2020) Identity-guided human semantic parsing for person re-identification. In: European conference on computer vision, pp 346–363. Springer

  49. Zou G, Fu G, Peng X, Liu Y, Gao M, Liu Z (2021) Person re-identification based on metric learning: a survey. Multimed Tools Applic 80(17):26855–26888

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61803161; in part by the Natural Science Foundation of Guangdong Province under Grant 2023A1515030119, Grant 2022A1515011887 and Grant 2022A1515110119; in part by the Science and Technology Plan Project of Jiangmen under Grant 2020030103080008999.

Author information

Authors and Affiliations

  1. School of Automation Science and Engineering, South China University of Technology, Guangdong, China

    Xing Hong, Langwen Zhang, Wei Xie & Yumin Xie

  2. Key Laboratory of Autonomous Systems and Networked Control, South China University of Technology, Guangdong, China

    Langwen Zhang & Wei Xie

  3. Guangdong Provincial Key Laboratory of Technique and Equipment for Macromolecular Advanced Manufacturing, South China University of Technology, Guangdong, China

    Langwen Zhang & Wei Xie

  4. Unmanned Aerial Vehicle Systems Engineering Technology Research Center of Guangdong, South China University of Technology, Guangdong, China

    Langwen Zhang & Wei Xie

  5. School of Physics and Telecommunication Engineering, South China Normal University, Guangdong, China

    Xiaoyuan Yu

Authors
  1. Xing Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Langwen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyuan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yumin Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xing Hong.

Ethics declarations

Conflict of Interests

The authors declared that they have no conflicts of interest to this work. No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hong, X., Zhang, L., Yu, X. et al. MBA-Net: multi-branch attention network for occluded person re-identification. Multimed Tools Appl 83, 6393–6412 (2024). https://doi.org/10.1007/s11042-023-15312-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15312-1

Keywords

Search

Navigation