Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Threshold-based outer lip segmentation using support vector regression

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Automated lip reading from videos requires lip segmentation. Threshold-based segmentation is straightforward, but it is rarely used. This study proposes a histogram threshold based on the feedback of shape information. Both good and bad lip segmentation examples were used to train an \(\epsilon \)-support vector regression model to infer the segmentation accuracy from the region shape. The histogram threshold was optimised to minimise the segmentation error. The proposed method was tested on 895 images from 112 subjects using the AR Face Database. The proposed method, implemented in simple segmentation algorithms, reduced segmentation errors by 23.1%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Zhang, X., Cheng, F., Shilin, W.: Spatio-temporal fusion based convolutional sequence learning for lip reading. In: Proceedings IEEE/CVF International Conference on Computer Vision (ICCV), pp. 713–722 (2019)

  2. Guan, C., Wang, S., Liu, G., Liew, A.W.: Lip image segmentation in mobile devices based on alternative knowledge distillation. In: Proceedings 2019 IEEE International Conference on Image Processing (ICIP), pp. 1540–1544 (2019)

  3. Guan, C., Wang, S., Liew, A.W.: Lip image segmentation based on a fuzzy convolutional neural network. IEEE Trans. Fuzzy Syst. 28(7), 1242–1251 (2020)

    Google Scholar 

  4. Nascimento, J.C., Carneiro, G.: One shot segmentation: unifying rigid detection and non-rigid segmentation using elastic regularization. IEEE Trans. Pattern Anal. Mach. Intell. 42, 3054–3070 (2020)

    Article  Google Scholar 

  5. Spyridonos, P., Saint, A.F., Likas, A., Gaitanis, G., Bassukas, I.: Multi-threshold lip contour detection. In: Proceedings 25th IEEE International Conference on Image Processing (ICIP), pp. 1912–1916 (2018)

  6. Cheung, Y.M., Li, M., Peng, Q., Chen, C.P.: A cooperative learning-based clustering approach to lip segmentation without knowing segment number. IEEE Trans. Neural Netw. Learn. Syst. 28(1), 80–93 (2017)

    Article  MathSciNet  Google Scholar 

  7. Zheng, Z., Jiong, J., Chunjiang, D., Liu, X., Yang, J.: Facial feature localization based on an improved active shape model. Inf. Sci. 178(9), 2215–2223 (2008)

    Article  Google Scholar 

  8. Wang, S.L., Liew, A.W.C., Lau, W.H., Leung, S.H.: An automatic lipreading system for spoken digits with limited training data. IEEE Trans. Circuits Syst. Video Technol. 18(12), 1760–1765 (2008)

    Article  Google Scholar 

  9. Saeed, U., Dugelay, J.L.: Combining edge detection and region segmentation for lip contour extraction. In: International Conference on Articulated Motion and Deformable Objects, pp. 11–20 (2010)

  10. Gritzman, A.D., Aharonson, V., Rubin, D.M., Pantanowitz, A.: Automatic computation of histogram threshold for lip segmentation using feedback of shape information. SIViP 10(5), 869–876 (2016)

    Article  Google Scholar 

  11. Smola, A.J., Schölkopf, B.: A tutorial on support vector regression. Stat. Comput. 14(3), 199–222 (2004)

    Article  MathSciNet  Google Scholar 

  12. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 1–27 (2011)

    Article  Google Scholar 

  13. Hsu, C.W., Chang, C.C., Lin, C.J.: A practical guide to support vector classification. Technical report, Department of Computer Science, National Taiwan University (2003)

  14. Lewis, R.M., Torczon, V., Trosset, M.W.: Direct search methods: then and now. J. Comput. Appl. Math. 124(1–2), 191–207 (2000)

    Article  MathSciNet  Google Scholar 

  15. Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11(285–296), 23–27 (1975)

    Google Scholar 

  16. Martinez, A.M.: The AR face database. CVC Technical Report, vol. 24 (1998)

  17. Ding, L., Martinez, A.M.: Features versus context: an approach for precise and detailed detection and delineation of faces and facial features. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 2022–2038 (2010)

    Article  Google Scholar 

  18. Gritzman, A.D.: Adaptive threshold optimisation for colour-based lip segmentation in automatic lip-reading systems. University of the Witwatersrand, Johannesburg, Ph.D. thesis (2016)

Download references

Acknowledgements

This study has been supported by the National Research Foundation of South Africa Grant Numbers 97742 and 127102. It has been based on a Doctoral project [18].

Author information

Authors and Affiliations

  1. Department of Electrical and Information Engineering, University of the Witwatersrand, Johannesburg, South Africa

    Ashley D. Gritzman, Michiel Postema, David M. Rubin & Vered Aharonson

  2. IBM Research, Johannesburg, South Africa

    Ashley D. Gritzman

  3. BioMediTech, Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland

    Michiel Postema

  4. School of Sciences, University of Central Lancashire — Cyprus, Pyla, Cyprus

    Vered Aharonson

  5. Afeka Tel Aviv Academic College of Engineering, Tel Aviv, Israel

    Vered Aharonson

Authors
  1. Ashley D. Gritzman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michiel Postema
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David M. Rubin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vered Aharonson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vered Aharonson.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gritzman, A.D., Postema, M., Rubin, D.M. et al. Threshold-based outer lip segmentation using support vector regression. SIViP 15, 1197–1202 (2021). https://doi.org/10.1007/s11760-020-01849-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01849-3

Keywords

Search

Navigation