Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3378393.3402265acmconferencesArticle/Chapter ViewAbstractPublication PagescompassConference Proceedingsconference-collections
short-paper

An Approach for Assessing Quality of Labeled Data for a Machine Learning Task in Malaria Detection

Published: 01 July 2020 Publication History

Abstract

While microscopy diagnosis through supervised learning for image analysis notably contributes to malaria detection, it has limitations. Among its principle challenges is the manual and tiresome process of data annotation for the classification task. The manual annotation of data is prone to inaccuracy defects due to bias, subjectivity and unclear images resulting into many false positives. This is normally due to personal independent judgements that vary from individual microscopists hence summatively affecting the accuracy of the model. In this paper, we seek to investigate the possibility of classifying the negative far examples and the positive near examples from the positives in thick blood smear images for malaria detection. Assessing the classification performance could potentially inform us of the quality of training dataset and guide n selecting the best training dataset for a malaria parasite detection task. We employ the Mean Squared Error (MSE) to distinguish between positive and negative images. We later investigate the performance of the VGG-16 classification model based on how close or far negative examples are from positives. Experimental results showed that negative examples far from the positives produce better results than those near and that the proposed method could potentially be used to reduce false positives and bias in the training data.

References

[1]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http://tensorflow.org/ Software available from tensorflow.org.
[2]
Carla E. Brodley and Mark A. Friedl. 2011. Identifying Mislabeled Training Data. CoRR abs/1106.0219 (2011). http://arxiv.org/abs/1106.0219
[3]
Robert Bryan, Christos Davatzikos, Edward Herskovits, Suyash Mohan, Jeffrey Rudie, and Andreas Rauschecker. [n.d.]. Medical Image Analysis: Human and Machine. ([n. d.]). https://doi.org/10.1016/j.acra.2019.09.011
[4]
FranÃğois Chollet. 2015. keras. https://github.com/fchollet/keras.
[5]
Hossein Talebi Esfandarani and Peyman Milanfar. 2017. NIMA: Neural Image Assessment. CoRR abs/1709.05424 (2017). arXiv:1709.05424 http://arxiv.org/abs/1709.05424
[6]
BenoÃőt FrÃl'nay and Verleysen Michel. 2014. Classification in the Presence of Label Noise: A Survey. Neural Networks and Learning Systems, IEEE Transactions on 25 (05 2014), 845--869. https://doi.org/10.1109/TNNLS.2013.2292894
[7]
Menzies G. 2020. MEAN SQUARE ERRORS OF POINT DETERMINATIONS. Empire Survey Review 7 (02 2020), 279--294. https://doi.org/10.1179/sre.1944.7.53.279
[8]
Battaglia G.J. 1996. Mean square error. AMP Journal of Technology 5 (01 1996), 31--36.
[9]
Qassim Hussam, Verma Abhishek, and Feinzimer David. 2018. Compressed residual-VGG16 CNN model for big data places image recognition. 169--175. https://doi.org/10.1109/CCWC.2018.8301729
[10]
Bradford P. Jeffrey, Kunz Clayton, Kohavi Ron, Brunk Clifford, and Brodley E. Carla. 1998. Pruning Decision Trees with Misclassification Costs. In Proceedings of the 10th European Conference on Machine Learning (ECML âÃŹ98). Springer-Verlag, Berlin, Heidelberg, 131âĂŞ136.
[11]
Quinn John, Nakasi Rose, Mugagga Pius, Byanyima Patrick, Lubega William, and Andama Alfred. 2016. Deep Convolutional Neural Networks for Microscopy-Based Point of Care Diagnostics. In preceedings of International Conference on Machine Learning for Health Care, Volume 50. (08 2016).
[12]
TÃŭnnies Klaus. 2012. Guide to Medical Image Analysis: The Analysis of Medical Images. 1--19. https://doi.org/10.1007/978-1-4471-2751-2_1
[13]
Martire Kristy, Bali Agnes, Ballantyne Kaye, Edmond Gary, Kemp Richard, and Found Bryan. 2020. Forensic science evidence: Naive estimates of false positive error rates and reliability. https://doi.org/10.31234/osf.io/anxur
[14]
Bornmann Lutz and Marx Werner. 2014. Distributions Instead of Single Numbers: Percentiles and Beam Plots for the Assessment of Single Researchers. Journal of the Association for Information Science and Technology 65 (01 2014). https://doi.org/10.1002/asi.22996
[15]
Alemu Megbaru, Tadesse Desalegn, Hailu Tesfaye, Mulu Wondemagegn, Derbie Awoke, Hailu Tadesse, and Abera Bayeh. 2017. Performance of Laboratory Professionals Working on Malaria Microscopy in Tigray, North Ethiopia. Journal of Parasitology Research (12 2017). https://doi.org/10.1155/2017/9064917
[16]
Zhao Qi and Nishida T. [n.d.]. Using qualitative hypotheses to identify inaccurate data. Journal of Articial Intelligence Research, 3, 119145., year = 1995 ([n. d.]).
[17]
Kahla Rabeb, Barkaoui Abdelwahed, and Merzouki Tarek. 2019. Medical Image Analysis and Processing. 49--77. https://doi.org/10.1002/9781119681625.ch2
[18]
Mazahreh Raied, Tarn HaiJo, and Rao Raghavendar.2013. Minimum mean square error processing. (12 2013).
[19]
Sukumar Sreenivas Rangan, Natarajan Ramachandran, and Ferrell Regina. 2015. Quality of Big Data in health care. International journal of health care quality assurance 28 (07 2015), 621--34. https://doi.org/10.1108/IJHCQA-07-2014-0080
[20]
Dennis Reidsma, op den Akker, and Hendrikus J.A. 2008. In Coling 2008: Proceedings of the workshop on Human Judgements in Computational Linguistics. Coling 2008 Organizing Committee, 8--16.
[21]
WHO. 2018. World Malaria report. WHO Press, World Health Organization. Geneva, Switzerland (2018).

Index Terms

  1. An Approach for Assessing Quality of Labeled Data for a Machine Learning Task in Malaria Detection

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    COMPASS '20: Proceedings of the 3rd ACM SIGCAS Conference on Computing and Sustainable Societies
    June 2020
    359 pages
    ISBN:9781450371292
    DOI:10.1145/3378393
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 July 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. MSE
    2. Malaria
    3. VGG-16
    4. data quality assessment
    5. thick blood smear images

    Qualifiers

    • Short-paper
    • Research
    • Refereed limited

    Funding Sources

    • Swedish International Development Cooperation Agency

    Conference

    COMPASS '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 25 of 50 submissions, 50%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 69
      Total Downloads
    • Downloads (Last 12 months)11
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 26 Sep 2024

    Other Metrics

    Citations

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media