Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Simultaneous Learning of Nonlinear Manifolds Based on the Bottleneck Neural Network

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Manifold learning methods are important techniques for nonlinear extraction of high-dimensional data structures. These methods usually extract a global manifold for data. However, in many real-world problems, there is not only one global manifold, but also additional information about the objects is shared by a large number of manifolds. These manifolds can share information for data reconstruction. To simultaneously extract these data manifolds, this paper proposes a nonlinear method based on the deep neural network (NN) named nonlinear manifold separator NN (NMSNN). Unlike unsupervised learning of bottleneck NN, data labels were used for simultaneous manifold learning. This paper makes use of NMSNN for extracting both expression and identity manifolds for facial images of the CK+ database. These manifolds have been evaluated by different metrics. The identity manifold is used for changing image identity. The result of identity recognition by K-nearest neighbor classifier shows that virtual identities are exactly sanitized. The virtual images for different expressions of test subjects are generated by expression manifold. The facial expression recognition rate of 92.86 % is achieved for virtual expressions of test persons. It is shown that NMSNN can be used to enrich datasets by sanitizing virtual images. As a result, 8 and 19 % improvements are gained in the face recognition task by a single image of each person on CK+ and Bosphorus databases, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Fukunaga K (1990) Introduction to statistical pattern recognition. Academic Press Professional, San Diego

    MATH  Google Scholar 

  2. Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7:179–188

    Article  Google Scholar 

  3. Jolliffe I (2005) Principal component analysis. Wiley Online Library

  4. Comon P (1994) Independent component analysis: a new concept? Signal Process 36(3):287–314

    Article  MATH  Google Scholar 

  5. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  6. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 14:585–591

    Google Scholar 

  7. Zhang Z, Zha H (2004) Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. J Shanghai Univ 8(4):406–424

    Article  MathSciNet  Google Scholar 

  8. Cox T, Cox M (1994) Multidimensional scaling. Chapman & Hall, London

    MATH  Google Scholar 

  9. Tenenbaum JB (1998) Mapping a manifold of perceptual observations. Adv Neural Inf Process Syst 10:682–688

    Google Scholar 

  10. Faust M, Altenburger R, Backhaus T, Blanck H, Boedeker W, Gramatica P, Hamer V, Scholze M, Vighi M, Grimme LH (2003) Joint algal toxicity of 16 dissimilarly acting chemicals is predictable by the concept of independent action. Aquat Toxicol 63(1):43–63

    Article  Google Scholar 

  11. Kramer MA (1991) Nonlinear principal component analysis using autoassociative neural networks. AIChE J 37(2):233–243

    Article  Google Scholar 

  12. Daszykowski M, Walczak B, Massart D (2003) A journey into low-dimensional spaces with autoassociative neural networks. Talanta 59(6):1095–1105

    Article  Google Scholar 

  13. Van der Maaten L, Postma E, Van Den Herik H (2009) Dimensionality reduction: a comparative review. J Mach Learn Res 10:1–41

    Google Scholar 

  14. Seow MJ (2006) Learning as a nonlinear line of attraction for pattern association, classification and recognition. M.S. Thesis, Old Dominion University

  15. Dadashi N, Abdolali F, Seyyedsalehi SA (2011) Improving face recognition from a single image per person via virtual images produced by imagination using neural networks. Biannual J Signal Data Process 15(1):33–43

    Google Scholar 

  16. Erhan D, Bengio Y, Courville A, Manzagol P-A, Vincent P, Bengio S (2010) Why does unsupervised pre-training help deep learning? J Mach Learn Res 11:625–660

    MATH  MathSciNet  Google Scholar 

  17. Erhan D, Manzagol P-A, Bengio Y, Bengio S, Vincent P (2009) The difficulty of training deep architectures and the effect of unsupervised pre-training. In: Proceedings of the twelfth international conference on artificial intelligence and, statistics (AISTATS’09), pp 153–160

  18. Plath N (2008) Extracting low-dimensional features by means of Deep Network Architectures. PhD. Thesis, Technische Universität Berlin

  19. Bengio Y (2012) Evolving culture vs local minima. arXiv, preprint arXiv:12032990.

  20. Martens J (2010) Deep learning via Hessian-free optimization. In: Proceedings of the 27th international conference on machine learning (ICML)

  21. Nejadgholi I, Seyyedsalehi SA (2004) experiments towards bidirectional neural networks. Technical report, Research Center of Intelligent Signal Processing (In persian)

  22. Ghasemi M (2006) Nonlinear independent component analysis of Speech signal. M.S. Thesis, Amirkabir University of Technology (In persian)

  23. Nejadgholi I (2012) A brain-inspired model of feature extraction and binding considering their interactions. Ph.D. Thesis, Amirkabir University of Technology (In persian)

  24. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MATH  MathSciNet  Google Scholar 

  25. Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst 19:153

    Google Scholar 

  26. Seyyedsalehi SZ, Seyyedsalehi SA (2012) New fast pre training method for deep neural network learning. In: 19th Iranian conference on biomedical (ICBME 2012)

  27. Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended Cohn–Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW), pp 94–101

  28. Savran A, Alyüz N, Dibeklioğlu H, Çeliktutan O, Gökberk B, Sankur B, Akarun L (2008) Bosphorus database for 3D face analysis. In: Biometrics and identity management (BIOID 2008), pp 47–56

  29. Seyyedsalehi SA, Seyyedsalehi SZ (in press) A new fast pre training method for training of deep neural network. Biannual J Signal Data Process 19(1) (In persian)

  30. Tan X, Chen S, Zhou ZH, Zhang F (2006) Face recognition from a single image per person: a survey. Pattern Recognit 39(9):1725–1745

    Article  MATH  Google Scholar 

  31. Ahn W-K, Brewer WF (1992) Psychological studies of explanation: based learning. In: DeJong G (ed) Investigating explanation-based learning. Kluwer Academic Publishers, Boston, pp 295–316

    Google Scholar 

  32. Li S, Liu X, Chai X, Zhang H, Lao S, Shan S (2012) Morphable displacement field based image matching for face recognition across pose. Comput Vis (ECCV) 2012:102–115

    Google Scholar 

  33. Mohammadzade H, Hatzinakos D (2013) Projection into expression subspaces for face recognition from single sample per person. IEEE Trans Affect Comput 4(1):69–82

    Article  Google Scholar 

  34. Nourabadi NS, Dizaj KG, Seyyedsalehi SA (2013) Face pose normalization for identity recognition using 3D information by means of neural networks. The 5th conference on information and knowledge technology (IKT2013)

Download references

Author information

Authors and Affiliations

  1. Faculty of Biomedical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Hafez Street, Tehran, Iran

    Seyyede Zohreh Seyyedsalehi & Seyyed Ali Seyyedsalehi

Authors
  1. Seyyede Zohreh Seyyedsalehi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seyyed Ali Seyyedsalehi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyyed Ali Seyyedsalehi.

Appendix A

Appendix A

$$\begin{aligned}&\displaystyle \mathbf{M}_\mathrm{la}^\mathrm{A} (t)=\gamma \mathbf{M}_\mathrm{la}^\mathrm{A}(t-1)+(1-\gamma )\mathbf{Y}_{\mathrm{c}(1:\mathrm{n}_\mathrm{A})} (t)&\\&\displaystyle \mathbf{M}_\mathrm{la}^\mathrm{A}(t-1)=\gamma \mathbf{M}_\mathrm{la}^\mathrm{A}( {t-2})+(1-\gamma )\mathbf{Y}_{\mathrm{c}(1:\mathrm{n}_\mathrm{A})}(t-1)&\\&\displaystyle \mathbf{M}_\mathrm{la}^\mathrm{A}(t)=\gamma ^{2}\mathbf{M}_\mathrm{la}^\mathrm{A} (t-2)+\gamma (1-\gamma )\mathbf{Y}_{\mathrm{c}(1:\mathrm{n}_\mathrm{A})}(t-1)+ (1-\gamma )\mathbf{Y}_{\mathrm{c}(1:\mathrm{n}_\mathrm{A})}(t)&\\&\displaystyle \mathbf{M}_{\mathrm{la}}^\mathrm{A}(t)=\gamma ^{t}\mathbf{M}_{\mathrm{la}}^\mathrm{A}(0) +\sum \limits _{r=1}^t {\gamma ^{t-r}(1-\gamma )\mathbf{Y}_{\mathrm{c}(1:\mathrm{n}_\mathrm{A})} (r)\cong \sum \limits _{r=1}^t {\gamma ^{t-r}\left( {1-\gamma } \right) \mathbf{Y}_{\mathrm{c} (1:\mathrm{n}_\mathrm{A})}(r)}}&\end{aligned}$$

Rights and permissions

Reprints and permissions

About this article

Cite this article

Seyyedsalehi, S.Z., Seyyedsalehi, S.A. Simultaneous Learning of Nonlinear Manifolds Based on the Bottleneck Neural Network. Neural Process Lett 40, 191–209 (2014). https://doi.org/10.1007/s11063-013-9322-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-013-9322-9

Keywords

Search

Navigation