Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Two-stage local details restoration framework for face hallucination

  • Original paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Face hallucination is of great importance in many applications. In this paper, a novel two-stage framework is proposed for hallucinating high-resolution (HR) face image from the given low-resolution (LR) one. In contrast to the existing methods, where the finer details are ignored, our framework pays more attention to the further local details enhancement. In the first stage, the local position-patch-based method with locality constraint is introduced to obtain the initial estimate image. In order to generate more reasonable face image and reduce noise, our method only represents the input LR patches over the similar training patches in the same position. In the second stage, the initial estimate image rather than residual image is directly used as the input to obtain the final HR image via local position-patch-based method. Besides, contextual information of position-patch is taken into consideration to generate more precise details in the second stage. Extensive experiments on the open face database illustrate that the proposed method achieves superior performance in comparison with state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Baker, S., Kanade, T.: Hallucinating faces. In: Proceedings 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, pp. 83–88 (2000)

  2. Wang, Z., Miao, Z., Wu, Q.M.J., Wan, Y., Tang, Z.: Low-resolution face recognition: a review. Vis. Comput. 30(4), 359–386 (2014)

    Article  Google Scholar 

  3. Liu, C., Shum, H., Freeman, W.: Face hallucination: theory and practice. Int. J. Comput. Vis. 7(1), 115–134 (2007)

    Article  Google Scholar 

  4. Liu, C., Shum, H.Y., Zhang, C.S.: A two-step approach to hallucinating faces: global parametric model and local non parametric model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 192–198 (2001)

  5. Zhuang, Y.T., Zhang, J., Wu, F.: Hallucinating faces: LPH super-resolution and neighbor reconstruction for residue compensation. Pattern Recognit. 40(40), 3178–3194 (2007)

    Article  MATH  Google Scholar 

  6. Hui, Z., Liu, W., Lam, K.M.: A novel correspondence-based face-hallucination method. Image Vis. Comput. 60, 171–184 (2017)

    Article  Google Scholar 

  7. Wang, X., Tang, X.: Hallucinating face by eigentransformation. IEEE Trans. Syst. Man Cybern. Part C 35(3), 425–434 (2005)

    Article  Google Scholar 

  8. Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 1, 275–282 (2004)

    Google Scholar 

  9. Wei, F., Yeung, D.: Image hallucination using neighbor embedding over visual primitive manifolds. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2007)

  10. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: Proceedings Computer Vision and Pattern Recognition (2008)

  11. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  12. Ma, X., Zhang, J., Qi, C.: Position-based face hallucination method. In: Proc. IEEE Int. Conf. Multimedia and Expo. (ICME), pp. 290–293 (2009)

  13. Ma, X., Zhang, J., Qi, C.: Hallucinating face by position-patch. Pattern Recognit. 43(6), 3178–3194 (2010)

    Article  Google Scholar 

  14. Jung, C., Jiao, L., Liu, B., Gong, M.: Position-patch based face hallucination using convex optimization. IEEE Signal Process. Lett. 18(6), 367–370 (2011)

    Article  Google Scholar 

  15. Jiang, J., Hu, R., Han, Z., Lu, T., Huang, K.: Position-patch based face hallucination via locality-constrained representation. In: Proc. IEEE Int. Conf. Multimedia and Expo (ICME), pp. 212–217 (2012)

  16. Jiang, J., Hu, R., Wang, Z., Han, Z.: Noise robust face hallucination via locality-constrained representation. IEEE Trans. Multimed. 16(5), 1268–1281 (2014)

    Article  Google Scholar 

  17. Chen, H-Y., Chien, S-Y.: Eigen-patch: position-patch based face hallucination using eigen transformation. In: IEEE Int. Conf. on Multimedia and Expo, pp. 1–6 (2014)

  18. Gao, G., Yang, J., Lai, Z., Huang, P.: Nuclear norm regularized coding with local position-patch and nonlocal similarity for face hallucination. Digital Signal Process. 64, 107–120 (2017)

    Article  MathSciNet  Google Scholar 

  19. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  20. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  21. Yu, K., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Proc. Advances in Neural Information Processing Systems (NIPS), pp. 2223–2231 (2009)

  22. Wang, J., Lu, C.Y., Wang, M., Li, P.P., Yan, S.C., Hu, X.G.: Robust face recognition via adaptive sparse representation. IEEE Trans. Cybern. 44(12), 2368–2378 (2014)

    Article  Google Scholar 

  23. Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)

    Article  Google Scholar 

  24. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  25. Jiang, J., Hu, R., Wang, Z., Han, Z.: Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE TIP 23(10), 4220–4231 (2014)

    MathSciNet  MATH  Google Scholar 

  26. Jiang, J., Ma, J., Chen, C., Jiang, X., Wang, Z.: Noise robust face image super-resolution through smooth sparse representation. In: CYB, vol. PP, no. 99, pp. 1–12 (2016)

  27. Jiang, J., Chen, C., Huang, K., Cai, Z., Hu, R.: Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation. Inf. Sci. 367–368, 354–372 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

This work has been supported by National Natural Science Foundation of China (61203261; 61876099), China Postdoctoral Science Foundation funded project (2012M521335), Research Fund of Guangxi Key Lab of Multi-source Information Mining & Security (MIMS16-02), Shenzhen science and technology research and development funds (JCYJ20170307093018753) and The Fundamental Research Funds of Shandong University (2017JC043).

Author information

Authors and Affiliations

  1. School of Control Science and Engineering, Shandong University, Jinan, 250061, People’s Republic of China

    Di Zhao, Zhenxue Chen & Chengyun Liu

  2. Shenzhen Research Institute of Shandong University, Shandong University, Shenzhen, 518057, People’s Republic of China

    Zhenxue Chen

  3. Department of Electrical and Computer Engineering, University of Windsor, Windsor, N9B 3P4, Canada

    Q. M. Jonathan Wu

Authors
  1. Di Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenxue Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Q. M. Jonathan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chengyun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenxue Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, D., Chen, Z., Wu, Q.M.J. et al. Two-stage local details restoration framework for face hallucination. Machine Vision and Applications 30, 153–162 (2019). https://doi.org/10.1007/s00138-018-0983-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-018-0983-2

Keywords

Search

Navigation