research-article

A Unified Generative Adversarial Framework for Image Generation and Person Re-identification

Authors:

Changsheng XuAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 163 - 172

https://doi.org/10.1145/3240508.3240573

Published: 15 October 2018 Publication History

Abstract

Person re-identification (re-id) aims to match a certain person across multiple non-overlapping cameras. It is a challenging task because the same person's appearance can be very different across camera views due to the presence of large pose variations. To overcome this issue, in this paper, we propose a novel unified person re-id framework by exploiting person poses and identities jointly for simultaneous person image synthesis under arbitrary poses and pose-invariant person re-identification. The framework is composed of a GAN based network and two Feature Extraction Networks (FEN), and enjoys following merits. First, it is a unified generative adversarial model for person image generation and person re-identification. Second, a pose estimator is utilized into the generator as a supervisor in the training process, which can effectively help pose transfer and guide the image generation with any desired pose. As a result, the proposed model can automatically generate a person image under an arbitrary pose. Third, the identity-sensitive representation is explicitly disentangled from pose variations through the person identity and pose embedding. Fourth, the learned re-id model can have better generalizability on a new person re-id dataset by using the synthesized images as auxiliary samples. Extensive experimental results on four standard benchmarks including Market-1501 [69], DukeMTMC-reID [40], CUHK03 [23], and CUHK01 [22] demonstrate that the proposed model can perform favorably against state-of-the-art methods.

References

[1]

E. Ahmed, M. Jones, and T. K. Marks. 2015. An improved deep learning architecture for person re-identification CVPR.

[2]

Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Real-time multi-person 2d pose estimation using part affinity fields CVPR.

[3]

Dapeng Chen, Zejian Yuan, Badong Chen, and Nanning Zheng. 2016. Similarity learning with spatial constraints for person re-identification CVPR. 1268--1277.

[4]

Shi-Zhe Chen, Chun-Chao Guo, and Jian-Huang Lai. 2016. Deep ranking for person re-identification via joint representation learning. TIP Vol. 25, 5 (2016), 2353--2367.

[5]

Weihua Chen, Xiaotang Chen, Jianguo Zhang, and Kaiqi Huang. 2017. Beyond triplet loss: a deep quadruplet network for person re-identification CVPR, Vol. 2.

[6]

Y. Chen, X. Zhu, and S. Gong. 2017. Person re-identification by deep learning multi-scale representations ICCVW. 2590--2680.

[7]

De Cheng, Yihong Gong, Sanping Zhou, Jinjun Wang, and Nanning Zheng. 2016. Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In CVPR. 1335--1344.

[8]

Jason V. Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S. Dhillon. 2007. Information-theoretic metric learning. In ICML. ACM, 209--216.

Digital Library

[9]

Junyu Gao, Tianzhu Zhang, and Changsheng Xu. 2017. A unified personalized video recommendation via dynamic recurrent neural networks ACM MM. 127--135.

Digital Library

[10]

Junyu Gao, Tianzhu Zhang, Xiaoshan Yang, and Changsheng Xu. 2018. P2t: Part-to-target tracking via deep regression learning. IEEE Transactions on Image Processing Vol. 27, 6 (2018), 3074--3086.

[11]

Mengyue Geng, Yaowei Wang, Tao Xiang, and Yonghong Tian. 2016. Deep transfer learning for person re-identification. arXiv preprint arXiv:1611.05244.

[12]

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In NIPS. 2672--2680.

Digital Library

[13]

Swaminathan Gurumurthy, Ravi Kiran Sarvadevabhatla, and Venkatesh Babu Radhakrishnan. 2017. DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data CVPR.

[14]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778.

[15]

Martin Hirzer, Peter M Roth, Martin Köstinger, and Horst Bischof. 2012. Relaxed pairwise learned metric for person re-identification ECCV. 780--793.

Digital Library

[16]

Phillip Isola, Jun Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks CVPR. 5967--5976.

[17]

Takuhiro Kaneko, Kaoru Hiramatsu, and Kunio Kashino. 2017. Generative Attribute Controller with Conditional Filtered Generative Adversarial Networks. In CVPR. 6089--6098.

[18]

D. P. Kingma and M. Welling. 2014. Auto-Encoding Variational Bayes. In ICLR.

[19]

Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. 2015. Autoencoding beyond pixels using a learned similarity metric. arXiv preprint arXiv:1512.09300.

[20]

Christoph Lassner, Gerard Pons-Moll, and Peter V. Gehler. 2017. A Generative Model of People in Clothing. In CVPR. 853--862.

[21]

Dangwei Li, Xiaotang Chen, Zhang Zhang, and Kaiqi Huang. 2017. Learning deep context-aware features over body and latent parts for person re-identification. In CVPR. 384--393.

[22]

Wei Li, Rui Zhao, and Xiaogang Wang. 2012. Human Reidentification with Transferred Metric Learning ACCV.

Digital Library

[23]

Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. 2014. DeepReID: Deep Filter Pairing Neural Network for Person Re-identification CVPR.

Digital Library

[24]

W. Li, R. Zhao, T. Xiao, and X. Wang. 2014. Deepreid: Deep filter pairing neural network for person re-identification CVPR.

Digital Library

[25]

Wei Li, Xiatian Zhu, and Shaogang Gong. 2017. Person re-identification by deep joint learning of multi-loss classification. IJCAI (2017).

Digital Library

[26]

S. Liao, Y. Hu, X. Zhu, and S. Z. Li. 2015. Person re-identification by local maximal occurrence representation and metric learning. In CVPR.

[27]

Y. Lin, L. Zheng, Z. Zheng, Y. Wu, and Y. Yang. 2017. Improving person re-identification by attribute and identity learning arXiv preprint arXiv:1703.07220.

[28]

Hao Liu, Jiashi Feng, Meibin Qi, Jianguo Jiang, and Shuicheng Yan. 2016. End-to-end comparative attention networks for person re-identification. arXiv preprint arXiv:1606.04404 (2016).

[29]

Jiawei Liu, Zheng-Jun Zha, QI Tian, Dong Liu, Ting Yao, Qiang Ling, and Tao Mei. 2016. Multi-scale triplet cnn for person re-identification Proceedings of the 2016 ACM on Multimedia Conference. ACM, 192--196.

Digital Library

[30]

Ming Yu Liu and Oncel Tuzel. 2016. Coupled generative adversarial networks. In NIPS. 469--477.

Digital Library

[31]

X. Liu, H. Zhao, M. Tian, L. Sheng, J. Shao, S. Yi, J. Yan, and X. Wang. 2017. Hydraplus-net: Attentive deep features for pedestrian analysis arXiv preprint arXiv:1709.09930.

[32]

Liqian Ma, Xu Jia, Qianru Sun, Bernt Schiele, Tinne Tuytelaars, and Luc Van Gool. 2017. Pose guided person image generation. In NIPS. 405--415.

[33]

Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. 2016. Adversarial autoencoders. In ICLR.

[34]

Niki Martinel, Abir Das, Christian Micheloni, and Amit K. Roy-Chowdhury. 2016. Temporal model adaptation for person re-identification ECCV. 858--877.

[35]

Tetsu Matsukawa, Takahiro Okabe, Einoshin Suzuki, and Yoichi Sato. 2016. Hierarchical gaussian descriptor for person re-identification CVPR. 1363--1372.

[36]

Sakrapee Paisitkriangkrai, Chunhua Shen, and Anton van den Hengel. 2015. Learning to rank in person re-identification with metric ensembles CVPR. 1846--1855.

[37]

Sateesh Pedagadi, James Orwell, Sergio Velastin, and Boghos Boghossian. 2013. Local fisher discriminant analysis for pedestrian re-identification CVPR. 3318--3325.

Digital Library

[38]

Xuelin Qian, Yanwei Fu, Wenxuan Wang, Tao Xiang, Yang Wu, Yu-Gang Jiang, and Xiangyang Xue. 2017. Pose-Normalized Image Generation for Person Re-identification. arXiv preprint arXiv:1712.02225.

[39]

Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR.

[40]

Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. 2016. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking European Conference on Computer Vision workshop on Benchmarking Multi-Target Tracking.

[41]

Chen Shen, Zhongming Jin, Yiru Zhao, Zhihang Fu, Rongxin Jiang, Yaowu Chen, and Xian-Sheng Hua. 2017. Deep siamese network with multi-level similarity perception for person re-identification. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 1942--1950.

Digital Library

[42]

H. Shi, Y. Yang, X. Zhu, S. Liao, Z. Lei, W. Zheng, and S. Z. Li. 2016. Embedding deep metric for person re-identification: A study against large variations ECCV.

[43]

Chi Su, Jianing Li, Shiliang Zhang, Junliang Xing, Wen Gao, and Qi Tian. 2017. Pose-driven Deep Convolutional Model for Person Re-identification ICCV. 3980--3989.

[44]

Y. Sun, L. Zheng, D. Weijian, and W. Shengjin. 2017. SVDNet for pedestrian retrieval. In ICCV.

[45]

R. R. Varior, M. Haloi, and G. Wang. 2016. Gated siamese convolutional neural network architecture for human reidentification. In ECCV.

[46]

R. R. Varior, B. Shuai, J. Lu, D. Xu, and G. Wang. 2016. A siamese long short-term memory architecture for human reidentification ECCV.

[47]

F. Wang, W. Zuo, L. Lin, D. Zhang, and L. Zhang. 2016. Joint learning of single-image and cross-image representations for person re-identification. In CVPR.

[48]

Xiaogang Wang and Rui Zhao. 2014. Person re-identification: System design and evaluation overview. In Person Re-Identification. 351--370.

[49]

Longhui Wei, Shiliang Zhang, Hantao Yao, Wen Gao, and Qi Tian. 2017. Glad: Global-local-alignment descriptor for pedestrian retrieval ACM Multimedia. 420--428.

Digital Library

[50]

L. Wu, C. Shen, and A.v.d. Hengel. 2016. PersonNet: Person Re-identification with Deep Convolutional Neural Networks. arXiv preprint arXiv:1601.07255 (2016).

[51]

Shangxuan Wu, Ying-Cong Chen, Xiang Li, An-Cong Wu, Jin-Jie You, and Wei-Shi Zheng. 2016. An enhanced deep feature representation for person re-identification WACV.

[52]

Qiqi Xiao, Kelei Cao, Haonan Chen, Fangyue Peng, and Chi Zhang. 2016. Cross domain knowledge transfer for person re-identification arXiv preprint arXiv:1611.06026.

[53]

Tong Xiao, Hongsheng Li, Wanli Ouyang, and Xiaogang Wang. 2016. Learning deep feature representations with domain guided dropout for person re-identification. In CVPR. 1249--1258.

[54]

Fei Xiong, Mengran Gou, Octavia Camps, and Mario Sznaier. 2014. Person re-identification using kernel-based metric learning methods ECCV. 1--16.

[55]

Hantao Yao, Shiliang Zhang, Yongdong Zhang, Jintao Li, and Qi Tian. 2017. Deep representation learning with part loss for person re-identification arXiv preprint arXiv:1707.00798.

[56]

D. Yi, Z. Lei, and S.Z. Li. 2014. Deep metric learning for practical person re-identification ICPR.

Digital Library

[57]

Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaolei Huang, Xiaogang Wang, and Dimitris Metaxas. 2017. Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In ICCV. 5907--5915.

[58]

Li Zhang, Tao Xiang, and Shaogang Gong. 2016. Learning a discriminative null space for person re-identification CVPR. 1239--1248.

[59]

Tianzhu Zhang, Adel Bibi, and Bernard Ghanem. 2016. In Defense of Sparse Tracking: Circulant Sparse Tracker CVPR.

[60]

Tianzhu Zhang, Changsheng Xu, and Ming-Hsuan Yang. 2018. Learning Multi-task Correlation Particle Filters for Visual Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. PP, 99 (2018), 1--1.

[61]

Xuan Zhang, Hao Luo, Xing Fan, Weilai Xiang, Yixiao Sun, Qiqi Xiao, Wei Jiang, Chi Zhang, and Jian Sun. 2017. Alignedreid: Surpassing human-level performance in person re-identification. arXiv preprint arXiv:1711.08184.

[62]

Zhifei Zhang, Yang Song, and Hairong Qi. 2017. Age Progression/Regression by Conditional Adversarial Autoencoder CVPR.

[63]

Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, and Xiaoou Tang. 2017. Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In CVPR. 1077--1085.

[64]

Junbo Zhao, Michael Mathieu, and Yann LeCun. 2017. Energy-based generative adversarial network. In ICLR.

[65]

Liming Zhao, Xi Li, Yueting Zhuang, and Jingdong Wang. 2017. Deeply-Learned Part-Aligned Representations for Person Re-Identification CVPR. 3219--3228.

[66]

Rui Zhao, Wanli Ouyang, and Xiaogang Wang. 2013. Unsupervised salience learning for person re-identification CVPR. 3586--3593.

Digital Library

[67]

Rui Zhao, Wanli Ouyang, and Xiaogang Wang. 2014. Learning mid-level filters for person re-identification CVPR. 144--151.

Digital Library

[68]

L. Zheng, Y. Huang, H. Lu, and Y. Yang. 2017. Pose invariant embedding for deep person re-identification arXiv preprint arXiv:1701.07732.

[69]

Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. 2015. Scalable Person Re-identification: A Benchmark. In ICCV.

Digital Library

[70]

Liang Zheng, Shengjin Wang, Lu Tian, Fei He, Ziqiong Liu, and Qi Tian. 2015. Query-adaptive late fusion for image search and person re-identification CVPR. 1741--1750.

[71]

L. Zheng, H. Zhang, S. Sun, M. Chandraker, and Q. Tian. 2016. Person Re-identification in the Wild. arXiv preprint arXiv:1604.02531 (2016).

[72]

Yuhui Zheng, Le Sun, Shunfeng Wang, Jianwei Zhang, and Jifeng Ning. 2018. Spatially Regularized Structural Support Vector Machine for Robust Visual Tracking. IEEE Transactions on Neural Networks and Learning System (2018).

[73]

Zhedong Zheng, Liang Zheng, and Yi Yang. 2017. A Discriminatively Learned CNN Embedding for Person Reidentification. TOMM Vol. 14, 1 (2017).

Digital Library

[74]

Z. Zheng, L Zheng, and Y Yang. 2017. Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro. In ICCV.

[75]

Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks CVPR.

Cited By

Zhang ZHe DLiu SXiao BDurrani T(2024)Completed Part Transformer for Person Re-IdentificationIEEE Transactions on Multimedia10.1109/TMM.2023.329481626(2303-2313)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3294816
Yan TWang CYuan CHuang D(2024)A Broader Study of Spectral Missing in Multi-spectral Vehicle Re-identificationApplied Intelligence10.1007/978-981-97-0827-7_5(51-63)Online publication date: 1-Mar-2024
https://doi.org/10.1007/978-981-97-0827-7_5
Liu XWen JChen ZLi DChen JLiu YWang HJin X(2023)FaaSLight: General Application-level Cold-start Latency Optimization for Function-as-a-Service in Serverless ComputingACM Transactions on Software Engineering and Methodology10.1145/358500732:5(1-29)Online publication date: 22-Feb-2023
https://dl.acm.org/doi/10.1145/3585007
Show More Cited By

Index Terms

A Unified Generative Adversarial Framework for Image Generation and Person Re-identification
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

JoT-GAN: A Framework for Jointly Training GAN and Person Re-Identification Model
To cope with the problem caused by inadequate training data, many person re-identification (re-id) methods exploit generative adversarial networks (GAN) for data augmentation, where the training of GAN is typically independent of that of the re-id model. ...
Pose-Normalized Image Generation for Person Re-identification
Computer Vision – ECCV 2018
Abstract
Person Re-identification (re-id) faces two major challenges: the lack of cross-view paired training data and learning discriminative identity-sensitive and view-invariant features in the presence of large pose variations. In this work, we address ...
Deep multi-instance learning for end-to-end person re-identification

In this paper, we introduce a deep multi-instance learning framework to boost the instance-level person re-identification performance. Motivated by the observation of considerably dramatic and complex varieties of visual appearances in many current ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Beijing Natural Science Foundation
PKU-NTU Joint Research Institute (JRI) sponsored by a donation from the Ng Teng Fong Charitable Foundation
National Nature Science Foundation of China
Key Research Program of Frontier Sciences CAS
National Key Research and Development Program of China

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
465
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)3

Reflects downloads up to 18 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang ZHe DLiu SXiao BDurrani T(2024)Completed Part Transformer for Person Re-IdentificationIEEE Transactions on Multimedia10.1109/TMM.2023.329481626(2303-2313)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3294816
Yan TWang CYuan CHuang D(2024)A Broader Study of Spectral Missing in Multi-spectral Vehicle Re-identificationApplied Intelligence10.1007/978-981-97-0827-7_5(51-63)Online publication date: 1-Mar-2024
https://doi.org/10.1007/978-981-97-0827-7_5
Liu XWen JChen ZLi DChen JLiu YWang HJin X(2023)FaaSLight: General Application-level Cold-start Latency Optimization for Function-as-a-Service in Serverless ComputingACM Transactions on Software Engineering and Methodology10.1145/358500732:5(1-29)Online publication date: 22-Feb-2023
https://dl.acm.org/doi/10.1145/3585007
Rinberg AKeidar I(2023)Intermediate Value Linearizability: A Quantitative Correctness CriterionJournal of the ACM10.1145/358469970:2(1-21)Online publication date: 18-Apr-2023
https://dl.acm.org/doi/10.1145/3584699
Chan PHu XSong HPeng PChen K(2023)Learning Disentangled Features for Person Re-identification under Clothes ChangingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/358435919:6(1-21)Online publication date: 31-May-2023
https://dl.acm.org/doi/10.1145/3584359
Antoniadis ACoester CEliáš MPolak ASimon B(2023)Online Metric Algorithms with Untrusted PredictionsACM Transactions on Algorithms10.1145/358268919:2(1-34)Online publication date: 15-Apr-2023
https://dl.acm.org/doi/10.1145/3582689
Liu DWu LZheng FLiu LWang M(2023)Verbal-Person Nets: Pose-Guided Multi-Granularity Language-to-Person GenerationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.315163134:11(8589-8601)Online publication date: Nov-2023
https://doi.org/10.1109/TNNLS.2022.3151631
Eom CLee WLee GHam B(2022)Disentangled Representations for Short-Term and Long-Term Person Re-IdentificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2021.312244444:12(8975-8991)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TPAMI.2021.3122444
Li YYao HXu C(2022)Intra-Domain Consistency Enhancement for Unsupervised Person Re-IdentificationIEEE Transactions on Multimedia10.1109/TMM.2021.305235424(415-425)Online publication date: 2022
https://doi.org/10.1109/TMM.2021.3052354
Li YYao HXu C(2021)TEST: Triplet Ensemble Student-Teacher Model for Unsupervised Person Re-IdentificationIEEE Transactions on Image Processing10.1109/TIP.2021.311203930(7952-7963)Online publication date: 2021
https://doi.org/10.1109/TIP.2021.3112039
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents