research-article

Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation

Authors:

David ZhangAuthors Info & Claims

IEEE Transactions on Image Processing, Volume 25, Issue 2

Pages 850 - 863

https://doi.org/10.1109/TIP.2015.2510498

Published: 01 February 2016 Publication History

Abstract

In this paper, we address the problem of unsupervised domain transfer learning in which no labels are available in the target domain. We use a transformation matrix to transfer both the source and target data to a common subspace, where each target sample can be represented by a combination of source samples such that the samples from different domains can be well interlaced. In this way, the discrepancy of the source and target domains is reduced. By imposing joint low-rank and sparse constraints on the reconstruction coefficient matrix, the global and local structures of data can be preserved. To enlarge the margins between different classes as much as possible and provide more freedom to diminish the discrepancy, a flexible linear classifier (projection) is obtained by learning a non-negative label relaxation matrix that allows the strict binary label matrix to relax into a slack variable matrix. Our method can avoid a potentially negative transfer by using a sparse matrix to model the noise and, thus, is more robust to different types of noise. We formulate our problem as a constrained low-rankness and sparsity minimization problem and solve it by the inexact augmented Lagrange multiplier method. Extensive experiments on various visual domain adaptation tasks show the superiority of the proposed method over the state-of-the art methods. The MATLAB code of our method will be publicly available at <uri xlink:href="http://www.yongxu.org/lunwen.html" xlink:type="simple">http://www.yongxu.org/lunwen.html</uri>.

References

[1]

M. Kan, J. Wu, S. Shan, and X. Chen, “Domain adaptation for face recognition: Targetize source domain bridged by common subspace,” Int. J. Comput. Vis., vol. 109, no. 1, pp. 94–109, 2014.

Digital Library

[2]

M. Shao, D. Kit, and Y. Fu, “Generalized transfer subspace learning through low-rank constraint,” Int. J. Comput. Vis., vol. 109, no. 1, pp. 74–93, 2014.

Digital Library

[3]

I.-H. Jhuo, D. Liu, D. T. Lee, and S.-F. Chang, “Robust visual domain adaptation with low-rank reconstruction,” in Proc. IEEE CVPR, Jun. 2012, pp. 2168–2175.

[4]

F. Zhu and L. Shao, “Weakly-supervised cross-domain dictionary learning for visual recognition,” Int. J. Comput. Vis., vol. 109, nos. 1–2, pp. 42–59, Aug. 2014.

Digital Library

[5]

Z. Deng, Y. Jiang, K.-S. Choi, F.-L. Chung, and S. Wang, “Knowledge-leverage-based TSK fuzzy system modeling,” IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 8, pp. 1200–1212, Aug. 2013.

[6]

Z. Deng, Y. Jiang, F.-L. Chung, H. Ishibuchi, and S. Wang, “Knowledge-leverage-based fuzzy system and its modeling,” IEEE Trans. Fuzzy Syst., vol. 21, no. 4, pp. 597–609, Aug. 2013.

Digital Library

[7]

Y. Yao and G. Doretto, “Boosting for transfer learning with multiple sources,” in Proc. IEEE CVPR, Jun. 2010, pp. 1855–1862.

[8]

S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345–1359, Oct. 2010.

Digital Library

[9]

L. Shao, F. Zhu, and X. Li, “Transfer learning for visual categorization: A survey,” IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 5, pp. 1019–1034, May 2015.

[10]

B. Gong, Y. Shi, F. Sha, and K. Grauman, “Geodesic flow kernel for unsupervised domain adaptation,” in Proc. IEEE CVPR, Jun. 2012, pp. 2066–2073.

[11]

R. Gopalan, R. Li, and R. Chellappa, “Domain adaptation for object recognition: An unsupervised approach,” in Proc. IEEE ICCV, Nov. 2011, pp. 999–1006.

[12]

K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual category models to new domains,” in Proc. ECCV, 2010, pp. 213–226.

[13]

S. Si, D. Tao, and B. Geng, “Bregman divergence-based regularization for transfer subspace learning,” IEEE Trans. Knowl. Data Eng., vol. 22, no. 7, pp. 929–942, Jul. 2010.

Digital Library

[14]

L. Duan, D. Xu, and S.-F. Chang, “Exploiting Web images for event recognition in consumer videos: A multiple source domain adaptation approach,” in Proc. IEEE CVPR, Jun. 2012, pp. 1338–1345.

[15]

L. Duan, D. Xu, and I. W. Tsang, “Domain adaptation from multiple sources: A domain-dependent regularization approach,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 3, pp. 504–518, Mar. 2012.

[16]

L. Duan, D. Xu, I. W.-H. Tsang, and J. Luo, “Visual event recognition in videos by learning from Web data,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 9, pp. 1667–1680, Sep. 2012.

Digital Library

[17]

J. Yang, R. Yan, and A. G. Hauptmann, “Cross-domain video concept detection using adaptive SVMs,” in Proc. Int. Conf. Multimedia, 2007, pp. 188–197.

[18]

Y. Xu, D. Zhang, J. Yang, and J.-Y. Yang, “A two-phase test sample sparse representation method for use with face recognition,” IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 9, pp. 1255–1262, Sep. 2011.

[19]

Y. Xu et al., “Data uncertainty in face recognition,” IEEE Trans. Cybern., vol. 44, no. 10, pp. 1950–1961, Oct. 2014.

[20]

J. Liu, Y. Chen, J. Zhang, and Z. Xu, “Enhancing low-rank subspace clustering by manifold regularization,” IEEE Trans. Image Process., vol. 23, no. 9, pp. 4022–4030, Sep. 2014.

[21]

L. Bruzzone and M. Marconcini, “Domain adaptation problems: A DASVM classification technique and a circular validation strategy,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 5, pp. 770–787, May 2010.

Digital Library

[22]

Y. Chen, G. Wang, and S. Dong, “Learning with progressive transductive support vector machine,” Pattern Recognit. Lett., vol. 24, no. 12, pp. 1845–1855, 2003.

Digital Library

[23]

Y. Xue, X. Liao, L. Carin, and B. Krishnapuram, “Multi-task learning for classification with Dirichlet process priors,” J. Mach. Learn. Res., vol. 8, pp. 35–63, May 2007.

Digital Library

[24]

X. He, S. Yan, Y. Hu, P. Niyogi, and H.-J. Zhang, “Face recognition using Laplacianfaces,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 3, pp. 328–340, Mar. 2005.

Digital Library

[25]

X. He, D. Cai, S. Yan, and H.-J. Zhang, “Neighborhood preserving embedding,” in Proc. IEEE CVPR, Oct. 2005, pp. 1208–1213.

[26]

D. Cai, X. He, and J. Han, “Isometric projection,” in Proc. AAAI, 2007, pp. 528–533.

[27]

S. Yan, D. Xu, B. Zhang, H.-J. Zhang, Q. Yang, and S. Lin, “Graph embedding and extensions: A general framework for dimensionality reduction,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 40–51, Jan. 2007.

[28]

M. Sugiyama, “Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis,” J. Mach. Learn. Res., vol. 8, no. 27, pp. 1027–1061, May 2007.

Digital Library

[29]

H.-T. Chen, H.-W. Chang, and T.-L. Liu, “Local discriminant embedding and its variants,” in Proc. IEEE CVPR, Jun. 2005, pp. 846–853.

[30]

J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, “Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization,” in Proc. Adv. NIPS, 2009, pp. 2080–2088.

[31]

G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma, “Robust recovery of subspace structures by low-rank representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 171–184, Jan. 2013.

Digital Library

[32]

G. Liu, Z. Lin, and Y. Yu, “Robust subspace segmentation by low-rank representation,” in Proc. ICML, 2010, pp. 663–670.

[33]

Z. Lin, M. Chen, and Y. Ma, “The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices”. arXiv preprint arXiv:1009.5055, 2010.

[34]

S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, “Domain adaptation via transfer component analysis,” IEEE Trans. Neural Netw., vol. 22, no. 2, pp. 199–210, Feb. 2011.

Digital Library

[35]

M. Shao, C. Castillo, Z. Gu, and Y. Fu, “Low-rank transfer subspace learning,” in Proc. IEEE ICDM, Dec. 2012, pp. 1104–1109.

[36]

Z. Ma, Y. Yang, N. Sebe, and A. G. Hauptmann, “Knowledge adaptation with partially shared features for event detection using few exemplars,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 9, pp. 1789–1802, Sep. 2014.

[37]

S. Xiang, F. Nie, G. Meng, C. Pan, and C. Zhang, “Discriminative least squares regression for multiclass classification and feature selection,” IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 11, pp. 1738–1754, Nov. 2012.

[38]

F. Nie, H. Wang, H. Huang, and C. Ding, “Adaptive loss minimization for semi-supervised elastic embedding,” in Proc. IJCAI, 2013, pp. 1565–1571.

[39]

M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transfer feature learning with joint distribution adaptation,” in Proc. IEEE ICCV, Dec. 2013, pp. 2200–2207.

[40]

M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transfer joint matching for unsupervised domain adaptation,” in Proc. IEEE CVPR, Jun. 2014, pp. 1410–1417.

[41]

B. Geng, D. Tao, and C. Xu, “DAML: Domain adaptation metric learning,” IEEE Trans. Image Process., vol. 20, no. 10, pp. 2980–2989, Oct. 2011.

Digital Library

[42]

I. Naseem, R. Togneri, and M. Bennamoun, “Linear regression for face recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 2106–2112, Nov. 2010.

Digital Library

Cited By

Feng WWang ZCao XCai BGuo WDing W(2024)Discriminative sparse subspace learning with manifold regularizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123831249:PCOnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123831
Gao MHuang W(2024)Latent space search approach for domain adaptationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123770249:PCOnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123770
Liu ZWang TZhu FChen XPelusi DVasilakos A(2024)Domain adaptive learning based on equilibrium distribution and dynamic subspace approximationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123673249:PBOnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123673
Show More Cited By

Index Terms

Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation
1. Mathematics of computing

Index terms have been assigned to the content through auto-classification.

Recommendations

Robust subspace segmentation via nonconvex low rank representation

A nonconvex formulation to determine the low rank representation from contaminated data is proposed.We provide a proximal iteratively reweighed algorithm for solving the nonconvex model.The proposed nonconvex model can recover the underlying low rank ...
Simultaneously Sparse and Low-Rank Matrix Reconstruction via Nonconvex and Nonseparable Regularization

Many real-world problems involve the recovery of a matrix from linear measurements, where the matrix lies close to some low-dimensional structure. This paper considers the problem of reconstructing a matrix with a simultaneously sparse and low-rank ...
Subspace clustering based on latent low rank representation with Frobenius norm minimization

The problem of subspace clustering which refers to segmenting a collection of data samples approximately drawn from a union of linear subspaces is considered in this paper. Among existing subspace clustering algorithms, low rank representation (LRR) ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Image Processing

IEEE Transactions on Image Processing Volume 25, Issue 2

Feb. 2016

501 pages

ISSN:1057-7149

Issue’s Table of Contents

Copyright © 2015.

Publisher

IEEE Press

Publication History

Published: 01 February 2016

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

81
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 19 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Feng WWang ZCao XCai BGuo WDing W(2024)Discriminative sparse subspace learning with manifold regularizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123831249:PCOnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123831
Gao MHuang W(2024)Latent space search approach for domain adaptationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123770249:PCOnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123770
Liu ZWang TZhu FChen XPelusi DVasilakos A(2024)Domain adaptive learning based on equilibrium distribution and dynamic subspace approximationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123673249:PBOnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123673
Liu ZOu WZhang KXiong H(2024)Robust manifold discriminative distribution adaptation for transfer subspace learningExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122117238:PDOnline publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122117
Liu ZOu WLiu JZhang KLai ZXiong H(2024)Discriminative transfer regression for low-rank and sparse subspace learningEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108445133:PCOnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.108445
Luo T(2024)Class-specific regularized joint distribution alignment for unsupervised domain adaptationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107877131:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.107877
Li KLi ZJia XLiu LChen M(2024)A domain adversarial graph convolutional network for intelligent monitoring of tool wear in machine toolsComputers and Industrial Engineering10.1016/j.cie.2023.109795187:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.cie.2023.109795
Luo LHu SChen L(2024)Discriminative Noise Robust Sparse Orthogonal Label Regression-Based Domain AdaptationInternational Journal of Computer Vision10.1007/s11263-023-01865-z132:1(161-184)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11263-023-01865-z
Karn RSanodiya R(2024)PSO-based unified framework for unsupervised domain adaptation in image classificationApplied Intelligence10.1007/s10489-024-05706-554:20(10106-10132)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10489-024-05706-5
Liu YLi LTan JRao YTan XLi Y(2024)Cross-domain Fisher Discrimination Criterion: A Domain Adaptive Method Based on the Nature of ClassifierApplied Intelligence10.1007/s10489-024-05376-354:7(5389-5405)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s10489-024-05376-3
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents