research-article

Face hallucination via multiple feature learning with hierarchical structure

Authors:

C. L. Philip ChenAuthors Info & Claims

Volume 512, Issue C

Pages 416 - 430

https://doi.org/10.1016/j.ins.2019.06.017

Published: 01 February 2020 Publication History

Abstract

In the past few years, neighbor-embedding (NE) based methods have been widely exploited for face hallucination. However, the existing NE based methods in spatial domain just employ single type of features for data representation, ignoring the compensatory information among multiple image features, resulting in bias in high resolution (HR) face image reconstruction. To tackle such problem, this paper presents a novel Multiple feature Learning model with Hierarchical Structure (MLHS) for face hallucination. Compared with conventional NE based methods, the proposed MLHS makes full use of multi-level information of face images, which can effectively remedy the flaw caused by just using single type of spatial pixel features, and adopts hierarchical structure to better maintain the manifold consistency hypothesis between the HR and low resolution (LR) patch spaces. The multiple learning strategy and hierarchical structure admit the proposed MLHS to well reconstruct the face details such as eyes, nostrils and mouth. The validity of the proposed MLHS method is confirmed by the comparison experiments in some public face databases.

References

[1]

J. Jiang, R. Hu, Z. Wang, Z. Han, Noise robust face hallucination via locality-constrained representation, IEEE Trans. Multimed. 16 (5) (2014) 1268–1281,.

[2]

S.P. Mudunuri, S. Biswas, Low resolution face recognition across variations in pose and illumination, IEEE Trans. Pattern Anal. Mach. Intell. 38 (5) (2016) 1034–1040,.

Digital Library

[3]

M. Jian, L. Kin-Man, J. Dong, Facial-feature detection and localization based on a hierarchical scheme, Inf. Sci. 262 (2014) 1–14,.

Digital Library

[4]

K. Li, Y. Zhu, J. Yang, J. Jiang, Video super-resolution using an adaptive superpixel-guided auto-regressive model, Pattern Recognit. 51 (2016) 59–71,.

Digital Library

[5]

Y. Zhu, K. Li, J. Jiang, Video super-resolution based on automatic key-frame selection and feature-guided variational optical flow, Signal Process. Image Commun. 29 (8) (2014) 875–886,.

[6]

Z. Wang, Z. Han, R. Hu., J. Jiang, Noise robust face hallucination employing Gaussianc–Laplacian mixture model, Neurocomputing 133 (2014) 153–160,.

Digital Library

[7]

G. Gao, J. Yang, A novel sparse representation based framework for face image super-resolution, Neurocomputing 134 (2014) 92–99,.

Digital Library

[8]

J.S. Park, S.W. Lee, An example-based face hallucination method for single-frame, low-resolution facial images, IEEE Trans. Image Process. 17 (10) (2008) 1806–1816,.

Digital Library

[9]

A. Chakrabarti, A.N. Rajagopalan, R. Chellappa, Super-resolution of face images using kernel PCA-based prior, IEEE Trans. Multimed. 9 (4) (2007) 888–892,.

Digital Library

[10]

H. Huang, H. He, X. Fan, J. Zhang, Super-resolution of human face image using canonical correlation analysis, Pattern Recognit. 43 (7) (2010) 2532–2543,.

Digital Library

[11]

X. Zhang, S. Peng, J. Jiang, An adaptive learning method for face hallucination using locality preserving projections, 8th IEEE International Conference on Automatic Face & Gesture Recognition (ICAFGR), 2008, pp. 1–8,.

[12]

M. Jian, L. Kin-Man, J. Dong, A novel face-hallucination scheme based on singular value decomposition, Pattern Recognit. 46 (11) (2013) 3091–3102,.

[13]

M. Jian, K.M. Lam, Simultaneous hallucination and recognition of low-resolution faces based on singular value decomposition, IEEE Trans. Circuits Syst. Video Technol. 25 (11) (2015) 1761–1772,.

Digital Library

[14]

H. Chang, D.-Y. Yeung, Y. Xiong, Super-resolution through neighbor embedding, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, 2004, pp. I275–I282.,.

[15]

R.S. T, S.L. K, Nonlinear dimensionality reduction by locally linear embedding, Science 290 (5500) (2000) 2323–2326.

[16]

J. Jiang, C. Chen, K. Huang, Z. Cai, R. Hu., Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation, Inf. Sci. 367–368 (2016) 354–372,.

Digital Library

[17]

Q. Liu, C. Liu, A novel locally linear KNN method with applications to visual recognition, IEEE Trans. Neural Netw. Learn. Syst. 28 (9) (2017) 2010–2021,.

[18]

W. Zhang, W.K. Cham, Hallucinating face in the DCT domain, IEEE Trans. Image Process. 20 (10) (2011) 2769–2779,.

Digital Library

[19]

X. Ma, J. Zhang, C. Qi, Hallucinating face by position-patch, Pattern Recognit. 43 (6) (2010) 2224–2236,.

Digital Library

[20]

C. Jung, L. Jiao, B. Liu, M. Gong, Position-patch based face hallucination using convex optimization, IEEE Signal Process. Lett. 18 (6) (2011) 367–370,.

[21]

Z. Wang, R. Hu, S. Wang, J. Jiang, Face hallucination via weighted adaptive sparse regularization, IEEE Trans. Circuits Syst. Video Technol. 24 (5) (2014) 802–813.

[22]

J. Jiang, J. Ma, C. Chen, X. Jiang, Z. Wang, Noise robust face image super-resolution through smooth sparse representation, IEEE Trans. Cybern. 47 (11) (2017) 3991–4002,.

[23]

J. Jiang, R. Hu, Z. Wang, Z. Han, J. Ma, Facial image hallucination through coupled-layer neighbor embedding, IEEE Trans. Circuits Syst. Video Technol. 26 (9) (2016) 1674–1684,.

Digital Library

[24]

K. Yu, T. Zhang, Y. Gong, Nonlinear learning using local coordinate coding, Advances in Neural Information Processing Systems (NIPS), 2009, pp. 2223–2231.

[25]

R.A. Farrugia, C. Guillemot, Face hallucination using linear models of coupled sparse support, IEEE Trans. Image Process. 26 (9) (2017) 4562–4577,.

Digital Library

[26]

J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, Y. Gong, Locality-constrained linear coding for image classification, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 3360–3367,.

[27]

X. Peng, L. Zhang, Z. Yi, K.K. Tan, Learning locality-constrained collaborative representation for robust face recognition, Pattern Recognit. 47 (9) (2014) 2794–2806,.

[28]

L. Liu, S. Li, C.L.P. Chen, Quaternion locality-constrained coding for color face hallucination, IEEE Trans. Cybern. 48 (5) (2018) 1474–1485,.

[29]

J. Jiang, R. Hu, Z. Wang, Z. Han, Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning, IEEE Trans. Image Process. 23 (10) (2014) 4220–4231,.

[30]

J. Jiang, C. Chen, J. Ma, Z. Wang, Z. Wang, R. Hu, SRLSP: a face image super-resolution algorithm using smooth regression with local structure prior, IEEE Trans. Multimed. 19 (1) (2017) 27–40,.

Digital Library

[31]

L. Liu, C.L.P. Chen, S. Li, Y.Y. Tang, L. Chen, Robust face hallucination via locality-constrained bi-layer representation, IEEE Trans. Cybern. 48 (4) (2018) 1189–1201,.

[32]

L. Liu, S. Li, C.L.P. Chen, Iterative relaxed collaborative representation with adaptive weights learning for noise robust face hallucination, IEEE Trans. Circuits Syst. Video Technol. (2018).

[33]

R.A. Farrugia, C. Guillemot, Face hallucination using linear models of coupled sparse support, IEEE Trans. Image Process. 26 (9) (2017) 4562–4577.

Digital Library

[34]

J. Shi, X. Liu, Y. Zong, C. Qi, G. Zhao, Hallucinating face image by regularization models in high-resolution feature space, IEEE Trans. Image Process. 27 (6) (2018) 2980–2995.

[35]

C. Dong, C.C. Loy, K. He, X. Tang, Image super-resolution using deep convolutional networks, IEEE Trans. Pattern Anal. Mach. Intell. 38 (2) (2016) 295–307,.

Digital Library

[36]

J. Kim, J.K. Lee, K.M. Lee, Accurate image super-resolution using very deep convolutional networks, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1646–1654,.

[37]

T. Lu, H. Wang, Z. Xiong, J. Jiang, Y. Zhang, H. Zhou, Z. Wang, Face hallucination using region-based deep convolutional networks, IEEE International Conference on Image Processing (ICIP), 2017, pp. 1657–1661,.

Digital Library

[38]

J. Jiang, Y. Yu, J. Hu, S. Tang, J. Ma, Deep CNN denoiser and multi-layer neighbor component embedding for face hallucination, arXiv:1806.10726(2018).

[39]

C. Ledig, L. Theis, F. Huszcr, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, W. Shi, Photo-realistic single image super-resolution using a generative adversarial network, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 105–114,.

[40]

D. Jin, X. Bai, Patch-sparsity-based image inpainting through facet deduced directional derivative, IEEE Trans. Circuits Syst. Video Technol. (2018),.

Digital Library

[41]

K. Makantasis, A.D. Doulamis, N.D. Doulamis, A. Nikitakis, Tensor-based classification models for hyperspectral data analysis, IEEE Trans. Geosci. Remote Sens. 56 (12) (2018) 6884–6898,.

[42]

J. Jiang, R. Hu, Z. Han, T. Lu, K. Huang, Position-patch based face hallucination via locality-constrained representation, IEEE International Conference on Multimedia and Expo (ICME), 2012, pp. 212–217,.

Digital Library

[43]

W. Zhou, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process. 13 (4) (2004) 600–612,.

Digital Library

[44]

P.P. Gajjar, M.V. Joshi, New learning based super-resolution: use of DWT and IGMRF prior., IEEE Trans. Image Process. 19 (5) (2010) 1201.

[45]

H.A. Rowley, S. Baluja, T. Kanade, Neural network-based face detection, IEEE Trans. Pattern Anal. Mach. Intell. 20 (1) (1998) 23–38,.

Digital Library

Cited By

Savitha SPounambal M(2024)Locality-constraint Representation with Minkowski distance metric for an effective Face HallucinationJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.104142100:COnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.jvcir.2024.104142
Nagar SJain ASingh PKumar A(2023)Structural similarity-based Bi-representation through true noise level for noise-robust face super-resolutionMultimedia Tools and Applications10.1007/s11042-022-14325-682:17(26255-26288)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1007/s11042-022-14325-6
Tang SShu Z(2023)Mixed noise face hallucination via adaptive weighted residual and nuclear-norm regularizationApplied Intelligence10.1007/s10489-022-04018-w53:10(11979-11996)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1007/s10489-022-04018-w

Index Terms

Face hallucination via multiple feature learning with hierarchical structure
1. Computing methodologies

Index terms have been assigned to the content through auto-classification.

Recommendations

Low-resolution face recognition based on feature-mapping face hallucination
Abstract
The image recognition approaches based on Convolutional Neural Network (CNN) have already achieved tremendous performance on super-resolution face images. However, there exist several challenges in face recognition field. For example, ...
Cooperative face hallucination using multiple references
ICME'09: Proceedings of the 2009 IEEE international conference on Multimedia and Expo

This paper proposes a cooperative example-based face hallucination method using multiple references. The proposed method first uses clustering and residual prototype faces construction to improve the performance of hallucinating a single low-resolution (...
Face Hallucination: Theory and Practice

In this paper, we study face hallucination, or synthesizing a high-resolution face image from an input low-resolution image, with the help of a large collection of other high-resolution face images. Our theoretical contribution is a two-step statistical ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Information Sciences: an International Journal

Information Sciences: an International Journal Volume 512, Issue C

Feb 2020

1606 pages

ISSN:0020-0255

Issue’s Table of Contents

Copyright © 2019.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 February 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Savitha SPounambal M(2024)Locality-constraint Representation with Minkowski distance metric for an effective Face HallucinationJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.104142100:COnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.jvcir.2024.104142
Nagar SJain ASingh PKumar A(2023)Structural similarity-based Bi-representation through true noise level for noise-robust face super-resolutionMultimedia Tools and Applications10.1007/s11042-022-14325-682:17(26255-26288)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1007/s11042-022-14325-6
Tang SShu Z(2023)Mixed noise face hallucination via adaptive weighted residual and nuclear-norm regularizationApplied Intelligence10.1007/s10489-022-04018-w53:10(11979-11996)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1007/s10489-022-04018-w

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents