Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Domain adaptive collaborative representation based classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Conventional representation based classification methods, such as sparse representation based classification (SRC) and collaborative representation based classification (CRC) have been developed and shown great potential due to its effectiveness in various recognition tasks. However, when the test data and training data come from different distribution, the performance of SRC and CRC will be degraded significantly. Recently, several sparse representation based domain adaptation learning (DAL) methods have been proposed and achieve impressive performance. However, these sparse representation based DAL methods need to solve the ℓ1-norm optimization problem, which is extremely time-consuming. To address this problem, in this paper, we propose a simple yet much more efficient domain adaptive collaborative representation-based classification method (DACRC). By replacing the ℓ2-norm regularization term using the ℓ2-norm, we exploit the collaborative representation rather than sparse representation to jointly learn projections of data in the two domains. In addition, a common dictionary is also learned such that in the projected space the learned dictionary can optimal represent both training and test data. Furthermore, the proposed method is effective to deal with multiple domains problem and is easy to kernelized. Compared with other sparse representation based DAL methods, DACRC is computationally efficient and its performance is better or comparable to many state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ben-David S, Blitzer J, Crammer K, Pereira F (2007) Analysis of representation for domain adaptation. In: Proceedings of the Advances in Neural Information Processing Systems (NIPS), pp. 137–144

  2. Bibi A, Ghanem B (2017) High order tensor formulation for convolutional sparse coding. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 1772–1780

  3. Brian K, Saenko K, Darrell T (2011) What you saw is not what you get: Domain adaptation using asymmetric kernel transforms. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 1785–1792

  4. Cai S, Zhang L, Zuo W, Feng X (2016) A probabilistic collaborative representation based approach for pattern classification, in: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 5839–5847

  5. Courty N, Flamary R, Tuia D, Rakotomamonjy A (2017) Optimal transport for domain adaptation. IEEE Trans Pattern Anal Mach Intell 39(9):1853–1865

    Article  Google Scholar 

  6. Daume H III (2007) Frustratingly easy domain adaptation. In: Proceedings of the of ACL, pp. 256–263

  7. Deng W, Hu J, Guo J (2017) Face recognition via collaborative representation: its discriminant nature and superposed representation. IEEE Trans Pattern Anal Mach Intell

  8. Gao Y, Ma J, Yuille AL (2017) Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. IEEE Trans Image Processing 26(5):2545–2560

    Article  MathSciNet  Google Scholar 

  9. Gebru T, Hoffman J, Fei-Fei L (2017) Fine-grained recognition in the wild: a multi-task domain adaptation approach. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (ICCV), pp. 1349–1358

  10. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (ICCV), 2066–2073

  11. Gopalan R, Li R, Chellappa R (2011) Domain adaptation for object recognition: An unsupervised approach. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 999–1006

  12. Griffin G, Holub A, Perona P (2007) Caltech-256 object category data set, CIT Technical Report 7694

  13. Huang W, Harandi M, Zhang T, Fan L (2017) Efficient optimization for linear dynamical systems with applications to clustering and sparse coding, in: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 3447–3457

  14. Huang W, Sun F, Cao L, Zhao D, Liu H, Harandi M (2016) Sparse coding and dictionary learning with linear dynamical system. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 3938–3947

  15. Jiang X, Lai J (2015) Sparse and dense hybrid representation via dictionary decomposition for face recognition. IEEE Trans Pattern Anal Mach Intell 37(5):1067–1079

    Article  Google Scholar 

  16. Lu B, Chellapp A, Nasrabadi N (2015) Incremental dictionary learning for unsupervised domain adaptation. In: Proceedings of the British Machine Vision Conference (BMVC). pp. 108.1–108.2

  17. Lu J, Wang G, Zhou J (2017) Simultaneous feature and dictionary learning for image set based face recognition. IEEE Trans Image Processing 26(8):4042–4054

    Article  MathSciNet  Google Scholar 

  18. Monga V, Damera-Venkata N, Rehman H, Evans BL (2005) Halftoning toolbox for MATLAB. Available: http://users.ece.utexas.edu/bevans/projects/halftoning/toolbox/

  19. Ni J, Qiu Q, Chellappa R (2017) Subspace interpolation via dictionary learning for unsupervised domain adaptation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (ICCV), pp. 692–699

  20. Pan S, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl & Data Eng 22(10):1345–1359

    Article  Google Scholar 

  21. Patel VM, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: A survey of recent advances. IEEE Signal Process Mag 32(3):53–69

    Article  Google Scholar 

  22. Qi Y, Qin L, Zhang J, Zhang S, Huang Q, Yang MH (2018) Structure-aware local sparse coding for visual tracking. IEEE Trans Image Processing 27(8):3857–3869

    Article  MathSciNet  Google Scholar 

  23. Qiu Q, Patel VM, Turaga P, Chellappa R (2012) Domain adaptive dictionary learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 631–645

    Chapter  Google Scholar 

  24. Quan Y, Xu Y, Sun Y, Huang Y, Ji H (2016) Sparse coding for classification via discriminative ensemble. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 5839–5847

  25. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 213–226

    Chapter  Google Scholar 

  26. Shekhar S, Patel VM, Nguyen HV, Chellappa R (2013) Generalized domain-adaptive dictionaries. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 361–368

  27. Shekhar S, Patel VM, Nguyen HV, Chellappa R (2015) Couple projections for adaptation of dictionaries. IEEE Trans Image Process 24(10):2941–2954

    Article  MathSciNet  Google Scholar 

  28. Shi Y, Sha F (2012) Information-theoretical learning of discriminative adaption. In: Proceedings of the International Conference on Machine Learning, (ICML) pp. 1079–1086

  29. Shrivastava A, Shekhar S, Patel VM (2014) Unsupervised domain adaptation using parallel transport on Grassmann manifold. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 277–284

  30. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR), pp. 7167–7176

  31. Wen Z, Yin W (2013) A feasible method for optimization with orthogonality constraints. Math Program 142(1–2):397–434

    Article  MathSciNet  Google Scholar 

  32. Wright J, Yang A, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  33. Xia Z, Ma X, Shen Z, Sun X, Xiong N, Jeon B (2018) Secure image LBP feature extraction in cloud-based smart campus. IEEE Access 6(1):30392–30401

    Article  Google Scholar 

  34. Xiong N, Vasilakos AV, Yang LT, Song L, Pan Y, Kannan R, Li Y (2009) Comparative analysis of quality of service and memory usage for adaptive failure detectors in healthcare systems. IEEE Journal on Selected Areas in Communications 27(4):495–509

    Article  Google Scholar 

  35. Yang M, Chen L (2017) Discriminative semi-supervised dictionary learning with entropy regularization for pattern classification. In: Proceedings of AAAI conf. Aritificial Intell, pp. 1626–1632

  36. Yang J, Wright J, Ma Y, Huang T (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  37. Yang M, Zhang L, Yang J, Zhang D (2010) Metaface learning for sparse representation based face recognition. In: Proceedings of the International Conference on Image Processing (ICIP), pp. 1601–1604

  38. Zhang H, Patel VM, Shekhar S, Chellappa R (2015) Domain-adaptive sparse representation-based classification, in: Proceedings of the Automatic Face and Gesture Recognition Workshops, pp. 1–8

  39. Zhang G, Sun H, Ji Z, Sun Q (2016) Label propagation based on collaborative representation for face recognition. Neurocomputing 171:1193–1204

    Article  Google Scholar 

  40. Zhang G, Sun H, Ji Z, Yuan Y, Sun Q (2016) Cost-sensitive dictionary learning for face recognition. Pattern Recogn 60:613–629

    Article  Google Scholar 

  41. Zhang G, Sun H, Porikli F, Liu Y, Sun Q (2017) Optimal couple projections for domain adaptive sparse representation-based classification. IEEE Trans Image Process 26(12):5922–5935

    Article  MathSciNet  Google Scholar 

  42. Zhang G, Sun H, Xia G, Feng L, Sun Q (2016) Kernel dictionary learning based discriminant analysis. J Vis Commun Image Represent 40:470–484

    Article  Google Scholar 

  43. Zhang G, Sun H, Xia G, Sun Q (2016) Multiple kernel sparse representation based orthogonal discriminative projection and its cost-sensitive extension. IEEE Trans Image Processing 25(9):4271–4285

    MathSciNet  MATH  Google Scholar 

  44. Zhang G, Sun H, Xia G, Sun Q (2016) Kernel collaborative representation based dictionary learning and discriminative projection. Neurocomputing 207:300–309

    Article  Google Scholar 

  45. Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: Which helps face recognition? In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 471–478

  46. Zhang L, Zhou W, Chang P, Liu J, Yan Z, Wang T, Li F (2012) Kernel sparse representation-based classifier. IEEE Trans Signal Process 60(4):1684–1695

    Article  MathSciNet  Google Scholar 

  47. Zheng H, Guo W, Xiong N (2017) A kernel-based compressive sensing approach for mobile data gathering in wireless sensor network systems. IEEE Trans Systems, Man and Cybernetics: System 8(9):1–13

    Google Scholar 

  48. Zhu X, Li X, Zhang S, Ju C, Wu X (2017) Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Trans Neural Netw Learn Syst 28(6):1263–1275

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, Jiangsu, China

    Guoqing Zhang & Yuhui Zheng

  2. School of Information and Control, Nanjing University of Information Science and Technology, Nanjing, 210044, Jiangsu, China

    Guiyu Xia

Authors
  1. Guoqing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuhui Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guiyu Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoqing Zhang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

In the appendix, the optimization procedure of KDA-CRC is provided.

Updating\( \tilde{A} \): For fixed B and \( \tilde{\mathbf{X}} \), the optimization problem can be written as:

$$ {\displaystyle \begin{array}{c}{\tilde{A}}^{\ast }=\underset{\tilde{A}}{\mathrm{argmin}}{\left\Vert {\tilde{A}}^TK\left(I-B\tilde{\mathbf{X}}\right)\right\Vert}_F^2+\sum \limits_{i=1}^c{\left\Vert {\tilde{A}}^TK\Big(\cdot, {\tilde{\mathbf{Y}}}_i\Big)-{\tilde{A}}^TK{B}_i{\tilde{\mathbf{X}}}_i^i\right\Vert}_F^2\\ {}+\sum \limits_{i=1}^c\sum \limits_{j=1,j\ne i}^c{\left\Vert {\tilde{A}}^TK{B}_j{\tilde{\mathbf{X}}}_i^j\right\Vert}_F^2-{\lambda}_1 trace\left(\left({\tilde{A}}^TK\right){\left({\tilde{A}}^TK\right)}^T\right)\end{array}} $$
(30)

In order to efficiently solve \( \tilde{A} \), we have the following proposition.

Proposition

The optimal solution of Eq. (29) when B and \( \tilde{\mathbf{X}} \) are fixed is

$$ {\tilde{A}}^{\ast }=V{S}^{-\frac{1}{2}}{G}^{\ast } $$
(31)

whereV and S come from the eigen decomposition of K = VSVT, and \( {G}^{\ast }=\left[{G}_1^{\ast T},{G}_2^{\ast T},\cdots, {G}_M^{\ast T}\right]\in {R}^{\sum {N}_i\times d} \) is the optimal solution of the following problem:

$$ {\displaystyle \begin{array}{l}{G}^{\ast }=\underset{G}{\mathrm{argmin}} tr\left({G}^T HG\right)\\ {}s.t.\kern0.3em {G}_i^T{G}_i=I,i=1,2,\cdots, M.\end{array}} $$
(32)

where \( {\displaystyle \begin{array}{c}H={S}^{\frac{1}{2}}{V}^T\Big[\left(I-B\tilde{\mathbf{X}}\right){\left(I-B\tilde{\mathbf{X}}\right)}^T+\sum \limits_{i=1}^cV{S}^{-1}{V}^T\left(K\left(\cdot, {\tilde{\mathbf{Y}}}_i\right)-K{B}_i{\tilde{\mathbf{X}}}_i^i\right){\left(K\left(\cdot, {\tilde{\mathbf{Y}}}_i\right)-K{B}_i{\tilde{\mathbf{X}}}_i^i\right)}^T\\ {}+\sum \limits_{i=1}^c\sum \limits_{j=1,j\ne i}^c\left({B}_j{\tilde{\mathbf{X}}}_i^j\right){\left({B}_j{\tilde{\mathbf{X}}}_i^j\right)}^T-{\lambda}_1I\Big]V{S}^{\frac{1}{2}}\end{array}} \).

Proof

Let \( G={S}^{\frac{1}{2}}{V}^T\tilde{A} \). Substituting H and G into Eq. (32), we get the required form of the optimization in Eq. (30).

Equation (32) is a non-convex problem due to non-linear equality constraints. We solve this problem using the optimization technique described in [31].

UpdatingB: For fixed \( \tilde{A} \) and \( \tilde{\mathbf{X}} \), Eq. (29) can be rewritten as

$$ {\displaystyle \begin{array}{c}{\tilde{\mathbf{X}}}^{\ast }=\underset{\tilde{\mathbf{X}}}{\mathrm{argmin}}\sum \limits_{i=1}^c\Big({\left\Vert {\tilde{A}}^TK\left(\cdot, {\tilde{\mathbf{Y}}}_i\right)-{\tilde{A}}^T KB{\tilde{\mathbf{X}}}_i\right\Vert}_F^2+{\left\Vert {\tilde{A}}^TK\Big(\cdot, {\tilde{\mathbf{Y}}}_i\Big)-{\tilde{A}}^TK{B}_i{\tilde{\mathbf{X}}}_i^i\right\Vert}_F^2\\ {}+\sum \limits_{j=1,j\ne i}^c{\left\Vert {\tilde{A}}^TK{B}_j{\tilde{\mathbf{X}}}_i^j\right\Vert}_F^2\Big)+{\lambda}_2{\left\Vert \tilde{\mathbf{X}}\right\Vert}_F^2\end{array}} $$
(33)

Similar to the optimization procedure proposed in section 3.3, the solution of \( {\tilde{x}}_{i,s} \) can be easily and analytically derived as

$$ {\displaystyle \begin{array}{c}{\tilde{x}}_{i,s}={\left({B}^TK\tilde{A}{\tilde{A}}^T KB+{\widehat{B}}_i^TK\tilde{A}{\tilde{A}}^TK{\widehat{B}}_i+\sum \limits_{j=1,j\ne i}^c{\widehat{B}}_j^TK\tilde{A}{\tilde{A}}^TK{\widehat{B}}_j\right)}^{-1}\\ {}\kern0.1em \left({B}^T+{\widehat{B}}_i^T\right)K\tilde{A}{\tilde{A}}^TK\left(\cdot, {\tilde{y}}_{i,s}\right)\end{array}} $$
(34)

where \( {\widehat{B}}_i=\left[0,\cdots, 0,{B}_i,0,\cdots, 0\right] \) is a matrix, which has the same size of B, 0 is a zero matrix that corresponding to each sub-dictionary of Bi. Similarly, \( {\widehat{B}}_j=\left[0,\cdots, 0,{B}_j,0,\cdots, 0\right] \).

UpdatingB: For fixed \( \tilde{A} \) and \( \tilde{\mathbf{X}} \), Eq. (29) can be rewritten as

$$ {\displaystyle \begin{array}{c}{B}^{\ast }=\underset{B}{\mathrm{argmin}}\sum \limits_{i=1}^c\Big({\left\Vert {\tilde{A}}^TK\left(\cdot, {\tilde{\mathbf{Y}}}_i\right)-{\tilde{A}}^T KB{\tilde{\mathbf{X}}}_i\right\Vert}_F^2+{\left\Vert {\tilde{A}}^TK\Big(\cdot, {\tilde{\mathbf{Y}}}_i\Big)-{\tilde{A}}^TK{B}_i{\tilde{\mathbf{X}}}_i^i\right\Vert}_F^2\\ {}+\sum \limits_{j=1,j\ne i}^c{\left\Vert {\tilde{A}}^TK{B}_j{\tilde{\mathbf{X}}}_i^j\right\Vert}_F^2\Big)\end{array}} $$
(35)

which can be solved by using the algorithm in [37]. We repeat the above steps until the algorithm converges.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, G., Zheng, Y. & Xia, G. Domain adaptive collaborative representation based classification. Multimed Tools Appl 78, 30175–30196 (2019). https://doi.org/10.1007/s11042-018-7007-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-7007-0

Keywords

Search

Navigation