research-article

A robust kinship verification scheme using face age transformation

Authors:

Eenjun HwangAuthors Info & Claims

Volume 231, Issue C

https://doi.org/10.1016/j.cviu.2023.103662

Published: 01 June 2023 Publication History

Abstract

Kinship verification determines the existence of a kin relationship between people through facial image analysis, and can be used in various real-life applications, such as finding missing family members, analyzing social media, and genealogy research. Recently, many convolutional neural network (CNN)-based kinship verification methods have been proposed, owing to the good image-processing performance of the CNNs. Nevertheless, insufficient labeled data and age differences in the kinship images, make kinship verification quite difficult. To mitigate these limitations, herein, we first propose a face age transformation model to generate facial images of various age groups. Then, we construct a cross-age kinship verification model constructed using the generated images as a training dataset. To show the effectiveness of the proposed scheme, we conducted various comparative experiments with other models using popular kinship datasets and confirmed that our proposed method exhibited an improved verification accuracy.

Highlights

•

We designed a new face age transformation model that can preserve the identity of the input image during image transformation.

•

We built a classification network to verify kinship by considering common identity features across different age groups.

•

To validate the effectiveness of our proposed method, we conducted extensive experiments using two benchmark datasets.

References

[1]

Albawi S., Mohammed T.A., Al-Zawi S., Understanding of a convolutional neural network, in: 2017 International Conference on Engineering and Technology, ICET, IEEE, 2017, pp. 1–6.

[2]

Almuashi M., Hashim S.Z.M., Mohamad D., Alkawaz M.H., Ali A., Automated kinship verification and identification through human facial images: a survey, Multimedia Tools Appl. 76 (1) (2017) 265–307.

[3]

Antipov G., Baccouche M., Dugelay J.L., Face aging with conditional generative adversarial networks, in: 2017 IEEE International Conference on Image Processing, ICIP, IEEE, 2017, pp. 2089–2093.

[4]

Cao Q., Shen L., Xie W., Parkhi O.M., Zisserman A., VGGFace2: A dataset for recognising faces across pose and age, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2018, IEEE, 2018, pp. 67–74.

[5]

Chen B.C., Chen C.S., Hsu W.H., Cross-age reference coding for age-invariant face recognition and retrieval, in: European Conference on Computer Vision, Springer, Cham, 2014, pp. 768–783.

[6]

Creswell A., White T., Dumoulin V., Arulkumaran K., Sengupta B., Bharath A.A., Generative adversarial networks: An overview, IEEE Signal Process. Mag. 35 (1) (2018) 53–65.

[7]

Dahan E., Keller Y., A unified approach to kinship verification, IEEE Trans. Pattern Anal. Mach. Intell. 43 (8) (2020) 2851–2857.

[8]

Dalal N., Triggs B., Histograms of oriented gradients for human detection, in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 1, CVPR’05, IEEE, 2005, pp. 886–893.

[9]

DeBruine L.M., Smith F.G., Jones B.C., Roberts S.C., Petrie M., Spector T.D., Kin recognition signals in adult faces, Vis. Res. 49 (1) (2009) 38–43.

[10]

Deng, J., Guo, J., Xue, N., Zafeiriou, S., 2019. Arcface: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4690–4699.

[11]

Fang, H., Deng, W., Zhong, Y., Hu, J., 2020. Triple-GAN: Progressive face aging with triple translation loss. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 804–805.

[12]

Fang R., Tang K.D., Snavely N., Chen T., Towards computational models of kinship verification, in: 2010 IEEE International Conference on Image Processing, IEEE, 2010, pp. 1577–1580.

[13]

Fang Y., Yan Y., Chen S., Wang H., Shu C., Sparse similarity metric learning for kinship verification, in: 2016 Visual Communications and Image Processing, VCIP, IEEE, 2016, pp. 1–4.

[14]

He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778.

[15]

Hu J., Lu J., Tan Y.P., Sharable and individual multi-view metric learning, IEEE Trans. Pattern Anal. Mach. Intell. 40 (9) (2017) 2281–2288.

Digital Library

[16]

Huang D., Shan C., Ardabilian M., Wang Y., Chen L., Local binary patterns and its application to facial image analysis: a survey, IEEE Trans. Syst. Man Cybern. C (Appl. Rev.) 41 (6) (2011) 765–781.

[17]

Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A., 2017. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1125–1134.

[18]

Jain A., Bhagat N., Srivastava V., Tyagi P., Jain P., A Feature-Based kinship verification technique using convolutional neural network, in: Advances in Data Sciences, Security and Applications, Springer, Singapore, 2020, pp. 353–362.

[19]

Kaminski G., Dridi S., Graff C., Gentaz E., Human ability to detect kinship in strangers’ faces: effects of the degree of relatedness, Proc. R. Soc. B Biol. Sci. 276 (1670) (2009) 3193–3200.

[20]

Kim H., Kim H., Ko B., Shim J., Hwang E., Two-stage person re-identification scheme using cross-input neighborhood differences, J. Supercomput. 78 (3) (2022) 3356–3373.

[21]

Kim H.W., Kim H.J., Rho S., Hwang E., Augmented EMTCNN: A fast and accurate facial landmark detection network, Appl. Sci. 7 (2253) (2020).

[22]

Li L., Feng X., Wu X., Xia Z., Hadid A., Kinship verification from faces via similarity metric based convolutional neural network, in: International Conference on Image Analysis and Recognition, Springer, Cham, 2016, pp. 539–548.

[23]

Li, Q., Liu, Y., Sun, Z., 2020a. Age Progression and Regression with Spatial Attention Modules. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 07. pp. 11378–11385.

[24]

Li W., Zhang Y., Lv K., Lu J., Feng J., Zhou J., Graph-based kinship reasoning network, in: 2020 IEEE International Conference on Multimedia and Expo, ICME, IEEE, 2020, pp. 1–6.

[25]

Liang J., Guo J., Lao S., Li J., Using deep relational features to verify kinship, in: CCF Chinese Conference on Computer Vision, Springer, Singapore, 2017, pp. 563–573.

[26]

Liang J., Hu Q., Dang C., Zuo W., Weighted graph embedding-based metric learning for kinship verification, IEEE Trans. Image Process. 28 (3) (2018) 1149–1162.

Digital Library

[27]

Lowe D.G., Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vis. 60 (2) (2004) 91–110.

Digital Library

[28]

Lu J., Zhou X., Tan Y.P., Shang Y., Zhou J., Neighborhood repulsed metric learning for kinship verification, IEEE Trans. Pattern Anal. Mach. Intell. 36 (2) (2013) 331–345.

Digital Library

[29]

Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Smolley, S.Paul., 2017. Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2794–2802.

[30]

Mehendale N., Facial emotion recognition using convolutional neural networks (FERC), SN Appl. Sci. 2 (3) (2020) 1–8.

[31]

Merget, D., Rock, M., Rigoll, G., 2018. Robust facial landmark detection via a fully-convolutional local–global context network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 781–790.

[32]

Mirza M., Osindero S., Conditional generative adversarial nets, 2014, arXiv preprint arXiv:1411.1784.

[33]

Or-El R., Sengupta S., Fried O., Shechtman E., Kemelmacher-Shlizerman I., Lifespan age transformation synthesis, in: European Conference on Computer Vision, Springer, Cham, 2020, pp. 739–755.

[34]

Qin X., Liu D., Wang D., A literature survey on kinship verification through facial images, Neurocomputing 377 (2020) 213–224.

[35]

Ramanathan N., Chellappa R., Modeling age progression in young faces, in: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 1, CVPR’06, IEEE, 2006, pp. 387–394.

[36]

Ramchoun H., Idrissi M.A.J., Ghanou Y., Ettaouil M., Multilayer perceptron: Architecture optimization and training, Int. J. Interact. Multim. Artif. Intell. 4 (1) (2016) 26–30.

[37]

Robinson J.P., Shao M., Fu Y., Survey on the analysis and modeling of visual kinship: A decade in the making, 2020, arXiv preprint arXiv:2006.16033.

[38]

Robinson, J.P., Shao, M., Wu, Y., Fu, Y., 2016. Families in the Wild (FIW) Large-Scale Kinship Image Database and Benchmarks. In: Proceedings of the 24th ACM International Conference on Multimedia. pp. 242–246.

[39]

Robinson J.P., Shao M., Wu Y., Liu H., Gillis T., Fu Y., Visual kinship recognition of families in the wild, IEEE Trans. Pattern Anal. Mach. Intell. 40 (11) (2018) 2624–2637.

[40]

Rothe, R., Timofte, R., Gool, L.Van., 2015. Dex: Deep expectation of apparent age from a single image. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. pp. 10–15.

[41]

Schroff, F., Kalenichenko, D., Philbin, J., 2015. Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 815–823.

[42]

Somanath G., Kambhamettu C., Can faces verify blood-relations?, in: 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems, BTAS, IEEE, 2012, pp. 105–112.

[43]

Song J., Zhang J., Gao L., Liu X., Shen H.T., Dual conditional GANs for face aging and rejuvenation, in: IJCAI, 2018, pp. 899–905.

[44]

Sun Y., Tang J., Shu X., Sun Z., Tistarelli M., Facial age synthesis with label distribution-guided generative adversarial network, IEEE Trans. Inf. Forensics Secur. 15 (2020) 2679–2691.

Digital Library

[45]

Suo J., Zhu S.C., Shan S., Chen X., A compositional and dynamic model for face aging, IEEE Trans. Pattern Anal. Mach. Intell. 32 (3) (2009) 385–401.

Digital Library

[46]

Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A., 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence.

[47]

Tazoe Y., Gohara H., Maejima A., Morishima S., Facial aging simulator considering geometry and patch-tiled texture, in: ACM SIGGRAPH 2012 Posters, 2012, p. 1.

[48]

Tiddeman B., Burt M., Perrett D., Prototyping and transforming facial textures for perception research, IEEE Comput. Graph. Appl. 21 (5) (2001) 42–50.

Digital Library

[49]

Wang, Z., Tang, X., Luo, W., Gao, S., 2018. Face aging with identity-preserved conditional generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7939–7947.

[50]

Wu X., Feng X., Cao X., Xu X., Hu D., López M.B., Liu L., Facial kinship verification: A comprehensive review and outlook, Int. J. Comput. Vis. (2022) 1–32.

[51]

Wu Y., Ji Q., Facial landmark detection: A literature survey, Int. J. Comput. Vis. 127 (2) (2019) 115–142.

Digital Library

[52]

Xia S., Shao M., Fu Y., Toward kinship verification using visual attributes, in: Proceedings of the 21st International Conference on Pattern Recognition, ICPR2012, IEEE, 2012, pp. 549–552.

[53]

Xu Z., Zhang Y., Cao L., Social image analysis from a non-IID perspective, IEEE Trans. Multimed. 16 (7) (2014) 1986–1998.

[54]

Yan H., Lu J., Zhou X., Prototype-based discriminative feature learning for kinship verification, IEEE Trans. Cybern. 45 (11) (2014) 2535–2545.

[55]

Yan H., Wang S., Learning part-aware attention networks for kinship verification, Pattern Recognit. Lett. 128 (2019) 169–175.

[56]

Yao X., Puy G., Newson A., Gousseau Y., Hellier P., High resolution face age editing, in: 2020 25th International Conference on Pattern Recognition, ICPR, IEEE, 2021, pp. 8624–8631.

[57]

Zhang L., Duan Q., Zhang D., Jia W., Wang X., AdvKin: Adversarial convolutional network for kinship verification, IEEE Trans. Cybern. (2020).

[58]

Zhang, Z., Song, Y., Qi, H., 2017. Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 5810–5818.

[59]

Zhang K., Zhang Z., Li Z., Qiao Y., Joint face detection and alignment using multitask cascaded convolutional networks, IEEE Signal Process. Lett. 23 (10) (2016) 1499–1503.

[60]

Zhang1 2.K., Huang Y., Song C., Wu H., Wang L., Intelligence S.M., Kinship verification with deep convolutional neural networks, in: British machine vision conference, BMVA Press, 2015.

[61]

Zhou, X., Hu, J., Lu, J., Shang, Y., Guan, Y., 2011. Kinship Verification from Facial Images under Uncontrolled Conditions. In: Proceedings of the 19th ACM International Conference on Multimedia. pp. 953–956.

[62]

Zhou, X., Lu, J., Hu, J., Shang, Y., 2012. Gabor-Based Gradient Orientation Pyramid for Kinship Verification under Uncontrolled Environments. In: Proceedings of the 20th ACM International Conference on Multimedia. pp. 725–728.

[63]

Zhou X., Shang Y., Yan H., Guo G., Ensemble similarity learning for kinship verification from facial images in the wild, Inf. Fusion 32 (2016) 40–48.

Recommendations

Video-based kinship verification using distance metric learning

We investigate the problem of kinship verification from facial videos.We present a new video face dataset for the video-based kinship verification study.We develop a benchmark to evaluate state-of-the-art metric learning methods in video-based kinship ...
A survey on kinship verification
Abstract
In this survey, kinship verification is defined as the automatic process of verifying whether two or more persons are blood relatives (kin) by analyzing images of their faces. Kinship verification is an important research field in computer vision ...
Age-uniform Feature Learning for Image-based Kinship Verification
CSAI '21: Proceedings of the 2021 5th International Conference on Computer Science and Artificial Intelligence

Kinship verification based on face image is an important topic in computer vision and has many applications in practice, such as family pedigree organization, missing person search, etc. Although parent and children share certain similarities in facial ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computer Vision and Image Understanding

Computer Vision and Image Understanding Volume 231, Issue C

Jun 2023

80 pages

ISSN:1077-3142

Issue’s Table of Contents

Elsevier Inc.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 June 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents