research-article

Semantic consistent feature construction and multi-granularity feature learning for visible-infrared person re-identification

Authors:

Guanqiu QiAuthors Info & Claims

The Visual Computer, Volume 40, Issue 4

Pages 2363 - 2379

https://doi.org/10.1007/s00371-023-02923-w

Published: 27 June 2023 Publication History

Abstract

In the real-world 24/7 surveillance systems, the images collected during the day and night are visible light images and infrared images, respectively. Infrared images lack color and texture information. In this case, it is more practical to use cross-modality person re-identification (re-ID) to process visible-infrared images. In fact, the cross-modality semantic alignment and specific discriminative feature extraction of different modalities are important for the improvement of modal performance. Therefore, a Semantic Consistent Feature Construction and Multi-granularity Feature learning (SCC–MGL) method is proposed for visible-infrared person re-ID in this paper. The SCC–MGL consists of a Semantic Consistent Feature Construction (SCC) module and a Multi-Granularity Information Enhancement (MGIE) module. In SCC, the features of different modalities are guided by analyzing the relation between feature maps channels and pedestrian’s body parts to form consistent semantic information on the corresponding channels, which reduces the impact caused by the misalignment of semantic information. In MGIE, a local modality difference elimination strategy is proposed to remove the modality difference. Meanwhile, the local feature discrimination is improved by reasonably constraining multi-granularity features. The effectiveness and superiority of proposed method are validated by experimental results from SYSU-MM01 and RegDB datasets.

References

[1]

Wang S, Liu R, Li H, Qi G, and Yu Z Occluded person re-identification via defending against attacks from obstacles IEEE Trans. Inf. Forensics Secur. 2023 18 147-161

[2]

Li H, Chen Y, Tao D, Yu Z, and Qi G Attribute-aligned domain-invariant feature learning for unsupervised domain adaptation person re-identification IEEE Trans. Inf. Forensics Secur. 2021 16 1480-1494

[3]

Li H, Dong N, Yu Z, Tao D, and Qi G Triple adversarial learning and multi-view imaginative reasoning for unsupervised domain adaptation person re-identification IEEE Trans. Circuits Syst. Video Technol. 2022 32 5 2814-2830

Digital Library

[4]

Wang S, Huang B, Li H, Qi G, Tao D, and Yu Z Key point-aware occlusion suppression and semantic alignment for occluded person re-identification Inf. Sci. 2022 606 669-687

Digital Library

[5]

Li S, Li F, Wang K, Qi G, and Li H Mutual prediction learning and mixed viewpoints for unsupervised-domain adaptation person re-identification on blockchain Simul. Model. Pract. Theory 2022 119

[6]

Li L, Xie M, Li F, Zhang Y, Li H, and Tan T Unsupervised domain adaptive person re-identification guided by low-rank priori(in chinese) J. Chongqing Univ. 2021 44 57-70

[7]

Zhang, Y., Wang, Y., Li, H., Li, S.: Cross-compatible embedding and semantic consistent feature construction for sketch re-identification. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 3347–3355 (2022)

[8]

He, S., Luo, H., Wang, P., Wang, F., Li, H., Jiang, W.: Transreid: transformer-based object re-identification. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 14993–15002 (2021)

[9]

Dai, Y., Liu, J., Sun, Y., Tong, Z., Zhang, C., Duan, L.-Y.: Idm: an intermediate domain module for domain adaptive person re-id. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11844–11854 (2021)

[10]

Li H, Xu K, Li J, and Yu Z Dual-stream reciprocal disentanglement learning for domain adaptation person re-identification Knowl. Based Syst. 2022 251

Digital Library

[11]

Wang Y, Qi G, Li S, Chai Y, and Li H Body part-level domain alignment for domain-adaptive person re-identification with transformer framework IEEE Trans. Inf. Forensics Secur. 2022 17 3321-3334

Digital Library

[12]

Li H, Kuang Z, Yu Z, and Luo J Structure alignment of attributes and visual features for cross-dataset person re-identification Pattern Recognit. 2020 106

[13]

Li H, Xu J, Yu Z, and Luo J Jointly learning commonality and specificity dictionaries for person re-identification IEEE Trans. Image Process. 2020 29 7345-7358

Digital Library

[14]

Li H, Yan S, Yu Z, and Tao D Attribute-identity embedding and self-supervised learning for scalable person re-identification IEEE Trans. Circuits Syst. Video Technol. 2020 30 10 3472-3485

[15]

Wang, G., Yuan, Y., Chen, X., Li, J., Zhou, X.: Learning discriminative features with multiple granularities for person re-identification. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 274–282 (2018)

[16]

Zhuo, J., Chen, Z., Lai, J., Wang, G.: Occluded person re-identification. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2018)

[17]

Wang, G., Wang, G., Zhang, X., Lai, J., Lin, L.: Weakly supervised person re-identification: cost-effective learning with a new benchmark. CoRR, vol. abs/1904.03845 (2019)

[18]

Wang, G., Lai, J., Huang, P., Xie, X.: Spatial-temporal person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8933–8940 (2019)

[19]

Zhu Z, Luo Y, Chen S, Qi G, Mazur N, Zhong C, and Li Q Camera style transformation with preserved self-similarity and domain-dissimilarity in unsupervised person re-identification J. Vis. Commun. Image Represent. 2021 80

Digital Library

[20]

Li Y, Chen S, Qi G, Zhu Z, Haner M, and Cai R A gan-based self-training framework for unsupervised domain adaptive person re-identification J. Imaging 2021 7 4 62

[21]

Xie, J., Ge, Y., Zhang, J., Huang, S., Chen, F., Wang, H.: Low-resolution assisted three-stream network for person re-identification. Vis. Comput. 2515–2525 (2022)

[22]

Jia, Z., Li, Y., Tan, Z., Wang, W., Wang, Z., Yin, G.: Domain-invariant feature extraction and fusion for cross-domain person re-identification. Vis. Comput. 1205–1216 (2023)

[23]

Zhong C, Jiang X, and Qi G Video-based person re-identification based on distributed cloud computing J. Artif. Intell. Technol. 2021 1 2 110-120

[24]

Zhong C, Qi G, Mazur N, Banerjee S, Malaviya D, and Hu G A domain adaptive person re-identification based on dual attention mechanism and camstyle transfer Algorithms 2021 14 12 361

[25]

Liang W, Wang G, Lai J, and Xie X Homogeneous-to-heterogeneous: unsupervised learning for rgb-infrared person re-identification IEEE Trans. Image Process. 2021 30 6392-6407

Digital Library

[26]

Wang, G., Lai, J.-H., Liang, W., Wang, G.: Smoothing adversarial domain attack and p-memory reconsolidation for cross-domain person re-identification. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10565–10574 (2020)

[27]

Wen, X., Feng, X., Li, P., Chen, W.: Cross-modality collaborative learning identified pedestrian. Vis. Comput. (2022)

[28]

Wang, G., Zhang, T., Cheng, J., Liu, S., Yang, Y., Hou, Z.: Rgb-infrared cross-modality person re-identification via joint pixel and feature alignment. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3623–3632 (2019)

[29]

Wang, Z., Wang, Z., Zheng, Y., Chuang, Y.-Y., Satoh, S.: Learning to reduce dual-level discrepancy for infrared-visible person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), pp. 618–626 (2019)

[30]

Wang, G.-A., Zhang, T., Yang, Y., Cheng, J., Chang, J., Liang, X., Hou, Z.-G.: Cross-modality paired-images generation for rgb-infrared person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12144–12151 (2020)

[31]

Zhang ZY, Jiang S, Huang CZT, Li Y, Da X, and Yi R Rgb-ir cross-modality person reid based on teacher-student gan model Pattern Recognit. Lett. 2021 150 155-161

Digital Library

[32]

Dai, P.Y., Ji, R.R., Wang, H.B., Wu, Q., Huang, Y.Y.: Cross-modality person re-identification with generative adversarial training. In: International Joint Conference on Artificial Intelligence(IJCAI), p. 6 (2018)

[33]

Fan, X., Jiang, W., Luo, H., Mao, W.J.: Modality-transfer generative adversarial network and dual-level unified latent representation for visible thermal person re-identification. Vis. Comput. 1–16 (2020)

[34]

Li KF, Wang XL, Liu Y, Zhang BJ, and Zhang MH Cross-modality disentanglement and shared feedback learning for infrared-visible person re-identification Knowl. Based Syst. 2022 252

Digital Library

[35]

Kansal K, Subramanyam AV, Wang Z, and Satoh S Sdl: spectrum-disentangled representation learning for visible-infrared person re-identification IEEE Trans. Circuits Syst. Video Technol. 2020 30 3422-3432

[36]

Choi, S., Lee, S., Kim, Y., Kim, T., Kim, C.: Hi-cmd: hierarchical cross-modality disentanglement for visible-infrared person re-identification. In: The IEEE/CVF Conference on Computer Vision and Pattern recognition(CVPR), pp. 10257–10266 (2020)

[37]

Pu, N., Chen, W., Liu, Y., Bakker, E.M., Lew, M.S.: Dual gaussian-based variational subspace disentanglement for visible-infrared person re-identification. In: the 28th ACM International Conference on Multimedia, pp. 2149–2158 (2020)

[38]

Zhu, X.K., Zheng, M.H., Chen, X.P., Zhang, X.Y., Yuan, C.H., Zhang, F.: Information disentanglement based cross-modal representation learning for visible-infrared person re-identification. Multimed. Tools Appl. 1–27 (2022)

[39]

Zhang, L.Y., Du, G.D., Liu, F., Tu, H.W., Shu, X.B.: Global-local multiple granularity learning for cross-modality visible-infrared person re-identification. IEEE Trans. Neural Netw. Learn. Syst. 1–11 (2021)

[40]

Liu HJ, Tan XH, and Zhou XC Parameter sharing exploration and hetero-center triplet loss for visible-thermal person re-identification IEEE Trans. Multimed. 2020 23 4414-4425

[41]

Ling, Y.G., Luo, Z.M., Lin, Y.J., Li, S.Z.: A multi-constraint similarity learning with adaptive weighting for visible-thermal person re-identification. In: International Joint Conference on Artificial Intelligence(IJCAI), pp. 845–851 (2021)

[42]

Wang HZ, Zhao JQ, Zhou Y, Yao R, Chen Y, and Chen SL Amc-net: attentive modality-consistent network for visible-infrared person re-identification Neurocomputing 2021 463 226-236

Digital Library

[43]

Ye, M., Shen, J.B., David, J.C., Shao, L., Luo, J.B.: Dynamic dual-attentive aggregation learning for visible-infrared person re-identification. In: European Conference on Computer Vision(ECCV), pp. 229–247 (2020)

[44]

Ye M, Shen JB, Lin GJ, Xiang T, Shao L, and Hoi S Deep learning for person re-identification: a survey and outlook IEEE Trans. Pattern Anal. Mach. Intell. 2021 44 2872-2893

[45]

Radenović F, Tolias G, and Chum O Fine-tuning cnn image retrieval with no human annotation IEEE Trans. Pattern Anal. Mach. Intell. 2018 41 7 1655-1668

[46]

Deep feature learning with relative distance comparison for person re-identification. Pattern Recognit. 48(10), 2993–3003 (2015)

[47]

Park, H., Lee, S., Lee, J., Ham, B.: Learning by aligning: visible-infrared person re-identification using cross-modal correspondences. In: 2021 IEEE International Conference on Computer Vision (ICCV), pp. 12026–12035 (2021)

[48]

Wei Z, Yang X, Wang N, and Gao X Flexible body partition-based adversarial learning for visible infrared person re-identification IEEE Trans. Neural Netw. Learn. Syst. 2021 33 9 4676-4687

[49]

Ding CX, Wang K, Wang PF, and Tao DC Multi-task learning with coarse priors for robust part-aware person re-identification IEEE Trans. Pattern Anal. Mach. Intell. 2022 44 1474-1488

[50]

Zhao C, Lv X, Dou S, Zhang S, Wu J, and Wang L Incremental generative occlusion adversarial suppression network for person reid IEEE Trans. Image Process. 2021 30 4212-4224

[51]

Lin, M., Chen, Q., Yan, S.: Network in network (2013). arXiv preprint arXiv:1312.4400

[52]

Liang, W., Wang, G., Lai, J., Zhu, J.-Y.: M2m-gan : many-to-many generative adversarial transfer learning for person re-identification (2018)

[53]

Luo, H., Gu, Y., Liao, X., Lai, S., Jiang, W.: Bag of tricks and a strong baseline for deep person re-identification. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1487–1495 (2019)

[54]

Sun, Y.F., Zheng, L., Yang, Y., Tian, Q., Wang, S.J.: Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: European Conference on Computer Vision (ECCV), pp. 480–496 (2018)

[55]

Wu, A., Zheng, W.-S., Yu, H.-X., Gong, S., Lai, J.: Rgb-infrared cross-modality person re-identification. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 5390–5399 (2017)

[56]

Wu A, Zheng W-S, Gong S, and Lai J Rgb-ir person re-identification by cross-modality similarity preservation Int. J. Comput. Vis. 2021 128 1765-1785

Digital Library

[57]

Nguyen DT, Hong HG, Kim KW, and Park KR Person recognition system based on a combination of body images from visible light and thermal cameras Sensors 2017 17 605

[58]

Kang JK, Lee MB, Yoon HS, and Park KR As-rig: adaptive selection of reconstructed input by generator or interpolation for person re-identification in cross-modality visible and thermal images IEEE Access 2021 9 12055-12066

[59]

Sun, Y.F., Zheng, L., Yang, Y., Tian, Q., Wang, S.J.: Visible thermal person re-identification via dual-constrained top-ranking. In: International Joint Conference on Artificial Intelligence(IJCAI), p. 2 (2018)

[60]

Ye M, Lan XY, Wang Z, and Yuen P Bi-directional center-constrained top-ranking for visible thermal person re-identification Ann. Math. Stat. 2019 15 407-419

[61]

Wang, X.G., Doretto, G., Sebastian, T., Rittscher, J., Tu, P.: Shape and appearance context modeling. In: 2007 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1–8 (2007)

[62]

Zheng, L., Shen, L.Y., Tian, L., Wang, S.J., Wang, J.D., Tian, Q.: Scalable person re-identification: a benchmark. In: 2015 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1116–1124 (2015)

[63]

Ye, M., Ruan, W.J., Du, B., Shou, M.Z.: Channel augmented joint learning for visible-infrared recognition. In: 2021 IEEE/CVF International Conference on Computer Vision, pp. 13567–13576 (2021)

[64]

Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 400–407 (1951)

[65]

Fan X, Jiang W, Luo H, and Fei M Spherereid: deep hypersphere manifold embedding for person re-identification J. Vis. Commun. Image Represent. 2019 60 51-58

Digital Library

[66]

Wei, Z., Yang, X., Wang, N., Gao, X.: Rbdf: reciprocal bidirectional framework for visible infrared person reidentification. IEEE Trans. Cybern. 52(10), 10988–10998

[67]

Wang XJ, Chen CQ, Zhu Y, and Chen SG Feature fusion and center aggregation for visible-infrared person re-identification IEEE Access 2022 10 30949-30958

[68]

Huang, Z.P., Liu, J.W., Li, L., Zheng, K.C., Zha, Z.J.: Modality-adaptive mixup and invariant decomposition for rgb-infrared person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1034–1042 (2022)

[69]

Gao, Y.J., Liang, T.F., Jin, Y., Gu, X.Y., Liu, W., Li, Y.D., Lang, C.Y.: Mso: Multi-feature space joint optimization network for rgb-infrared person re-identification. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 5257–5265 (2021)

[70]

Liu HJ, Chai YX, Tan XH, Li D, and Zhou XC Strong but simple baseline with dual-granularity triplet loss for visible-thermal person re-identification IEEE Signal Process. Lett. 2021 28 653-657

[71]

Wei, Z., Yang, X., Wang, N., Gao, X.: Syncretic modality collaborative learning for visible infrared person re-identification. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 225–234 (2021)

[72]

Chen C, Ye M, Qi M, Wu J, Jiang J, and Lin C-W Structure-aware positional transformer for visible-infrared person re-identification IEEE Trans. Image Process. 2022 31 2352-2364

Cited By

Dong NZhang LYan STang HTang J(2023)Erasing, Transforming, and Noising Defense Network for Occluded Person Re-IdentificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.333916734:6(4458-4472)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1109/TCSVT.2023.3339167
Zhu HWu HHe DLan RLiu ZPan X(2023)AcFusion: Infrared and Visible Image Fusion Based on Self-Attention and Convolution With Enhanced Information ExtractionIEEE Transactions on Consumer Electronics10.1109/TCE.2023.334185270:1(4155-4167)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1109/TCE.2023.3341852

Recommendations

Learning dual attention enhancement feature for visible–infrared person re-identification
Abstract
Most previous visible–infrared person re-identification methods emphasized learning modality-shared features to narrow the modality differences, while neglecting the benefits of modality-specific features for feature embedding and narrowing the ...
Unbiased feature enhancement framework for cross-modality person re-identification
Abstract
Cross-modality person re-identification aims at matching the RGB images of a specific person in variable appearances with his/her images in another modality like infrared modality, sketch modality, etc. It is challenging due to domain gap and ...
Serialized Local Feature Representation Learning for Infrared-Visible Person Re-identification
Intelligent Computing Theories and Application
Abstract
Infrared-Visible person re-identification is a kind of cross-modality person re-identification. The purpose of the task is that given a person image we need to find another image on the same person from gallery. The query images and gallery images ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image The Visual Computer: International Journal of Computer Graphics

The Visual Computer: International Journal of Computer Graphics Volume 40, Issue 4

Apr 2024

748 pages

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 27 June 2023

Accepted: 28 May 2023

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dong NZhang LYan STang HTang J(2023)Erasing, Transforming, and Noising Defense Network for Occluded Person Re-IdentificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.333916734:6(4458-4472)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1109/TCSVT.2023.3339167
Zhu HWu HHe DLan RLiu ZPan X(2023)AcFusion: Infrared and Visible Image Fusion Based on Self-Attention and Convolution With Enhanced Information ExtractionIEEE Transactions on Consumer Electronics10.1109/TCE.2023.334185270:1(4155-4167)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1109/TCE.2023.3341852

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents