Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Leaf segmentation in plant phenotyping: a collation study

  • Survey
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Image-based plant phenotyping is a growing application area of computer vision in agriculture. A key task is the segmentation of all individual leaves in images. Here we focus on the most common rosette model plants, Arabidopsis and young tobacco. Although leaves do share appearance and shape characteristics, the presence of occlusions and variability in leaf shape and pose, as well as imaging conditions, render this problem challenging. The aim of this paper is to compare several leaf segmentation solutions on a unique and first-of-its-kind dataset containing images from typical phenotyping experiments. In particular, we report and discuss methods and findings of a collection of submissions for the first Leaf Segmentation Challenge of the Computer Vision Problems in Plant Phenotyping workshop in 2014. Four methods are presented: three segment leaves by processing the distance transform in an unsupervised fashion, and the other via optimal template selection and Chamfer matching. Overall, we find that although separating plant from background can be accomplished with satisfactory accuracy (\(>\)90 % Dice score), individual leaf segmentation and counting remain challenging when leaves overlap. Additionally, accuracy is lower for younger leaves. We find also that variability in datasets does affect outcomes. Our findings motivate further investigations and development of specialized algorithms for this particular application, and that challenges of this form are ideally suited for advancing the state of the art. Data are publicly available (online at http://www.plant-phenotyping.org/datasets) to support future challenges beyond segmentation within this application domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://www.plant-phenotyping.org/CVPPP2014.

  2. http://www.phenotiki.com.

  3. http://www.garnics.eu/.

  4. http://www.dppn.de/.

  5. http://www.plant-phenotyping.org/.

  6. http://www.plant-phenotyping-network.eu/.

  7. http://www.iplantcollaborative.org/.

  8. http://www.plant-phenotyping.org/CVPPP2014-dataset.

  9. To measure Dice per leaf, we first find matches between a leaf in ground truth and an algorithm’s result that maximally overlap, and then report the Dice (Eq. 1) of matched leaves; for non-matched leaves a zero is reported.

  10. This indicates that additional (possibly tailored) evaluation metrics may be necessary, although our testing with some common in the literature did not yield any improvement.

  11. See the new Leaf Counting Challenge of CVPPP 2015 at BMVC (http://www.plant-phenotyping.org/CVPPP2015).

  12. https://www.codalab.org/.

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Adams, R., Bischof, L.: Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 641–647 (1994)

    Article  Google Scholar 

  3. Aksoy, E., Abramov, A., Wörgötter, F., Scharr, H., Fischbach, A., Dellen, B.: Modeling leaf growth of rosette plants using infrared stereo image sequences. Comput. Electron. Agric. 110, 78–90 (2015)

    Article  Google Scholar 

  4. Alenyà, G., Dellen, B., Torras, C.: 3D modelling of leaves from color and ToF data for robotized plant measuring. In: IEEE International Conference on Robotics and Automation, pp. 3408–3414 (2011)

  5. Arvidsson, S., Pérez-Rodríguez, P., Mueller-Roeber, B.: A growth phenotyping pipeline for Arabidopsis thaliana integrating image analysis and rosette area modeling for robust quantification of genotype effects. New Phytol 191(3), 895–907 (2011)

    Article  Google Scholar 

  6. Augustin, M., Haxhimusa, Y., Busch, W., Kropatsch, W.G.: Image-based phenotyping of the mature Arabidopsis shoot system. In: Computer Vision—ECCV 2014 Workshops, vol. 8928, pp. 231–246. Springer (2015)

  7. Bansal, S., Aggarwal, D.: Color image segmentation using CIELab color space using ant colony optimization. Int. J. Comput. Appl. 29(9), 28–34 (2011)

    Google Scholar 

  8. Barrow, H., Tenenbaum, J., Bolles, R., Wolf, H.: Parametric correspondence and chamfer matching: two new techniques for image matching. Tech. rep, DTIC (1977)

  9. Beucher, S.: The watershed transformation applied to image segmentation. Scanning Microsc. Int. 6, 299–314 (1992)

    Google Scholar 

  10. Biskup, B., Scharr, H., Schurr, U., Rascher, U.: A stereo imaging system for measuring structural parameters of plant canopies. Plant Cell Environ. 30, 1299–1308 (2007)

    Article  Google Scholar 

  11. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)

    Article  Google Scholar 

  12. Casanova, D., Florindo, J.B., Gonçalves, W.N., Bruno, O.M.: IFSC/USP at ImageCLEF 2012: plant identification task. In: CLEF (Online Working Notes/Labs/Workshop) (2012)

  13. Cerutti, G., Antoine, V., Tougne, L., Mille, J., Valet, L., Coquin, D., Vacavant, A.: ReVeS participation: tree species classification using random forests and botanical features. In: Conference and Labs of the Evaluation Forum (2012)

  14. Cerutti, G., Tougne, L., Mille, J., Vacavant, A., Coquin, D.: Understanding leaves in natural images: a model-based approach for tree species identification. Comput. Vis. Image Underst. 10(117), 1482–1501 (2013)

    Article  Google Scholar 

  15. CORESTA, C.: A scale for coding growth stages in tobacco crops (2009). http://www.coresta.org/Guides/Guide-No07-Growth-Stages_Feb09.pdf

  16. De Vylder, J., Ochoa, D., Philips, W., Chaerle, L., Van Der Straeten, D.: Leaf segmentation and tracking using probabilistic parametric active contours. In: International Conference on Computer Vision/Computer Graphics Collaboration Techniques, pp. 75–85 (2011)

  17. De Vylder, J., Vandenbussche, F.J., Hu, Y., Philips, W., Van Der Straeten, D.: Rosette Tracker: an open source image analysis tool for automatic quantification of genotype effects. Plant Physiol. 160(3), 1149–1159 (2012)

    Article  Google Scholar 

  18. Dellen, B., Scharr, H., Torras, C.: Growth signatures of rosette plants from time-lapse video. IEEE/ACM Trans. Comput. Biol. Bioinform. PP(99), 1–11 (2015)

  19. Dubuisson, M.P., Jain, A.K.: A modified Hausdorff distance for object matching. In: Proceedings of the 12th IAPR International Conferenced on Pattern Recognition, vol. 1, pp. 566–568 (1994)

  20. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal Visual Object Classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  21. Giuffrida, M.V., Minervini, M., Tsaftaris, S.A.: Learning to count leaves in rosette plants. In: Proceedings of the Computer Vision Problems in Plant Phenotyping (CVPPP) Workshop, pp. 1.1–1.13. BMVA Press (2015)

  22. Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768–1783 (2006)

    Article  Google Scholar 

  23. Granier, C., Aguirrezabal, L., Chenu, K., Cookson, S.J., Dauzat, M., Hamard, P., Thioux, J.J., Rolland, G., Bouchier-Combaud, S., Lebaudy, A., Muller, B., Simonneau, T., Tardieu, F.: PHENOPSIS, an automated platform for reproducible phenotyping of plant responses to soil water deficit in Arabidopsis thaliana permitted the identification of an accession with low sensitivity to soil water deficit. New Phytol. 169(3), 623–635 (2006)

    Article  Google Scholar 

  24. Hartmann, A., Czauderna, T., Hoffmann, R., Stein, N., Schreiber, F.: HTPheno: an image analysis pipeline for high-throughput plant phenotyping. BMC Bioinform. 12(1), 148 (2011)

    Article  Google Scholar 

  25. He, X., Gould, S.: An exemplar-based CRF for multi-instance object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 296–303 (2014)

  26. Jansen, M., Gilmer, F., Biskup, B., Nagel, K., Rascher, U., Fischbach, A., Briem, S., Dreissen, G., Tittmann, S., Braun, S., Jaeger, I.D., Metzlaff, M., Schurr, U., Scharr, H., Walter, A.: Simultaneous phenotyping of leaf growth and chlorophyll fluorescence via GROWSCREEN FLUORO allows detection of stress tolerance in Arabidopsis thaliana and other rosette plants. Funct. Plant Biol. 36(10/11), 902–914 (2009)

    Article  Google Scholar 

  27. Jin, J., Tang, L.: Corn plant sensing using real-time stereo vision. J. Field Robot. 26(6–7), 591–608 (2009)

    Article  MathSciNet  Google Scholar 

  28. Kalyoncu, C., Toygar, Ö.: Geometric leaf classification. Comput. Vis. Image Underst. 133, 102–109 (2015)

    Article  Google Scholar 

  29. Klukas, C., Chen, D., Pape, J.M.: Integrated analysis platform: an open-source information system for high-throughput plant phenotyping. Plant Physiol. 165(2), 506–518 (2014)

    Article  Google Scholar 

  30. Kurugollu, F., Sankur, B., Harmanci, A.E.: Color image segmentation using histogram multithresholding and fusion. Image Vis. Comput. 19(13), 915–928 (2001)

    Article  Google Scholar 

  31. Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 530–549 (2004)

    Article  Google Scholar 

  32. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Int. Conf. Comput. Vis. (ICCV) 2, 416–423 (2001)

    Google Scholar 

  33. Mezaris, V., Kompatsiaris, I., Strintzis, M.: Still image objective segmentation evaluation using ground truth. In: 5th COST 276 Workshop, pp. 9–14 (2003)

  34. Minervini, M., Abdelsamea, M.M., Tsaftaris, S.A.: Image-based plant phenotyping with incremental learning and active contours. Ecol. Inform. 23, 35–48 (2014). (Special Issue on Multimedia in Ecology and Environment)

    Article  Google Scholar 

  35. Minervini, M., Fschbach, A., Scharr, H., Tsaftaris, S.: Finely-grained annotated datasets for image-based plant phenotyping. Pattern Recogn. Lett. (2015) (In press)

  36. Minervini, M., Giuffrida, M.V., Tsaftaris, S.A.: An interactive tool for semi-automated leaf annotation. In: Proceedings of the Computer Vision Problems in Plant Phenotyping (CVPPP) Workshop, pp. 6.1-6.13. BMVA Press (2015)

  37. Minervini, M., Scharr, H., Tsaftaris, S.A.: Image analysis: the new bottleneck in plant phenotyping. IEEE Signal Process. Mag. 32(4), 126–131 (2015)

    Article  Google Scholar 

  38. Müller-Linow, M., Pinto-Espinosa, F., Scharr, H., Rascher, U.: The leaf angle distribution of natural plant populations: assessing the canopy with a novel software tool. Plant Methods 11(1), 11 (2015)

    Article  Google Scholar 

  39. Nagel, K., Putz, A., Gilmer, F., Heinz, K., Fischbach, A., Pfeifer, J., Faget, M., Blossfeld, S., Ernst, M., Dimaki, C., Kastenholz, B., Kleinert, A.K., Galinski, A., Scharr, H., Fiorani, F., Schurr, U.: GROWSCREEN-Rhizo is a novel phenotyping robot enabling simultaneous measurements of root and shoot growth for plants grown in soil-filled rhizotrons. Funct. Plant Biol. 39, 891–904 (2012)

    Article  Google Scholar 

  40. Nieuwenhuis, C., Cremers, D.: Spatially varying color distributions for interactive multilabel segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1234–1247 (2013)

    Article  Google Scholar 

  41. Pape, J.M., Klukas, C.: 3-D histogram-based segmentation and leaf detection for rosette plants. In: Computer Vision—ECCV 2014 Workshops, vol. 8928, pp. 61–74. Springer (2015)

  42. Polak, M., Zhang, H., Pi, M.: An evaluation metric for image segmentation of multiple objects. Image Vis. Comput. 27(8), 1223–1227 (2009)

    Article  Google Scholar 

  43. Pratt, W.K.: Digital Image Processing. Wiley-Interscience, New York, NY (1978)

    MATH  Google Scholar 

  44. Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., Kang, S.: Image-based plant modeling. ACM Trans. Graph. 25(3), 599–604 (2006)

    Article  Google Scholar 

  45. Riemenschneider, H., Sternig, S., Donoser, M., Roth, P.M., Bischof, H.: Hough regions for joining instance localization and segmentation. In: Computer Vision—ECCV 2012, vol. 7574, pp. 258–271. Springer (2012)

  46. Scharr, H., Minervini, M., Fischbach, A., Tsaftaris, S.A.: Annotated image datasets of rosette plants. Tech. Rep. FZJ-2014-03837, Forschungszentrum Jülich GmbH, (2014). http://hdl.handle.net/2128/5848

  47. Silva, L., Koga, M., Cugnasca, C., Costa, A.: Comparative assessment of feature selection and classification techniques for visual inspection of pot plant seedlings. Comput. Electron. Agric. 97, 47–55 (2013)

    Article  Google Scholar 

  48. Soares, J.V.B., Jacobs, D.W.: Efficient segmentation of leaves in semi-controlled conditions. Mach. Vis. Appl. 24(8), 1623–1643 (2013)

    Article  Google Scholar 

  49. Song, Y., Wilson, R., Edmondson, R., Parsons, N.: Surface modelling of plants from stereo images. In: Proceedings of the 6th International Conference on 3-D Digital Imaging and Modeling (3DIM ’07), pp. 312–319 (2007)

  50. Teng, C.H., Kuo, Y.T., Chen, Y.S.: Leaf segmentation, classification, and three-dimensional recovery from a few images with close viewpoints. Opt. Eng. 50(3), 1–13 (2011)

  51. Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13(6), 583–598 (1991)

    Article  Google Scholar 

  52. van der Heijden, G., Song, Y., Horgan, G., Polder, G., Dieleman, A., Bink, M., Palloix, A., van Eeuwijk, F., Glasbey, C.: SPICY: towards automated phenotyping of large pepper plants in the greenhouse. Funct. Plant Biol. 39(11), 870–877 (2012)

    Article  Google Scholar 

  53. W3C: Portable network graphics (PNG) specification (2003)

  54. Walter, A., Scharr, H., Gilmer, F., Zierer, R., Nagel, K.A., Ernst, M., Wiese, A., Virnich, O., Christ, M.M., Uhlig, B., Jünger, S., Schurr, U.: Dynamics of seedling growth acclimation towards altered light conditions can be quantified via GROWSCREEN: a setup and procedure designed for rapid optical phenotyping of different plant species. New Phytol. 174(2), 447–455 (2007)

    Article  Google Scholar 

  55. Walter, A., Schurr, U.: The modular character of growth in Nicotiana tabacum plants under steady-state nutrition. J. Exp. Bot. 50(336), 1169–1177 (1999)

    Article  Google Scholar 

  56. Wang, J., He, J., Han, Y., Ouyang, C., Li, D.: An adaptive thresholding algorithm of field leaf image. Comput. Electron. Agric. 96, 23–39 (2013)

    Article  Google Scholar 

  57. Wu, B., Nevatia, R.: Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses. Int. J. Comput. Vis. 82(2), 185–204 (2009)

    Article  Google Scholar 

  58. Yanikoglu, B., Aptoula, E., Tirkaz, C.: Automatic plant identification from photographs. Mach. Vis. Appl. 6(25), 1369–1383 (2014)

    Article  Google Scholar 

  59. Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-leaf alignment from fluorescence plant images. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 437–444 (2014)

  60. Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-leaf tracking from fluorescence plant videos. In: IEEE International Conference on Image Processing (ICIP), pp. 408–412 (2014)

  61. Yin, X., Liu, X., Chen, J., Kramer, D.M.: Multi-Leaf Segmentation, Alignment and Tracking from Fluorescence Plant Videos. arXiv:1505.00353 (2015)

  62. Ziou, D., Tabbone, S.: Edge detection techniques: an overview. Int. J. Pattern Recogn. Image Anal. 8(4), 537–559 (1998)

    Google Scholar 

Download references

Acknowledgments

We would like to thank participants of the 2014 CVPPP workshop for comments and annotators that have contributed to this work.

Author contributions SAT coordinated this collation study. SAT and HS organized the original LSC challenge. SAT, HS, and MM, wrote the paper and performed analysis. All other authors have contributed methods, text, and results. All authors have approved the manuscript.

Author information

Authors and Affiliations

  1. Institute of Bio- and Geosciences: Plant Sciences (IBG-2) Forschungszentrum Jülich GmbH, Jülich, Germany

    Hanno Scharr

  2. IMT Institute for Advanced Studies, Lucca, Italy

    Massimo Minervini & Sotirios A. Tsaftaris

  3. Schools of Biosciences and Computer Science, Centre for Plant Integrative Biology, University of Nottingham, Nottingham, UK

    Andrew P. French & Imanol Luengo

  4. Department of Molecular Genetics, Leibniz Institute of Plant Genetics and Crop Plant Research (IPK), Gatersleben, Germany

    Jean-Michel Pape

  5. Departments of Energy Plant Research Lab, and Biochemistry and Molecular Biology, Michigan State University, East Lansing, MI, USA

    David M. Kramer

  6. Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, USA

    Xiaoming Liu & Xi Yin

  7. Greenhouse Horticulture, Wageningen University and Research Centre, Wageningen, Netherlands

    Gerrit Polder & Danijela Vukadinovic

  8. School of Engineering, University of Edinburgh, Edinburgh, UK

    Sotirios A. Tsaftaris

  9. LemnaTec GmbH, Aachen, Germany

    Christian Klukas

Authors
  1. Hanno Scharr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Massimo Minervini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrew P. French
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christian Klukas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David M. Kramer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaoming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Imanol Luengo
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jean-Michel Pape
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Gerrit Polder
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Danijela Vukadinovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Xi Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Sotirios A. Tsaftaris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Massimo Minervini or Sotirios A. Tsaftaris.

Additional information

MM and SAT acknowledge a Marie Curie Action: “Reintegration Grant” (Grant #256534) of the EU’s Seventh Framework Programme (FP7/2007-2013). HS acknowledges funding from EU-FP7 no. 247947 (GARNICS). HS, JMP, and CK acknowledge the support of the German-Plant-Phenotyping Network, which is funded by the German Federal Ministry of Education and Research (Project Identification Number: 031A053). XY, XL, and DK acknowledge the support of US Department of Energy, Office of Science, Basic Energy Sciences Program (DE-FG02-91ER20021) and the MSU centre for Advanced Algal and Plant Phenotyping.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Scharr, H., Minervini, M., French, A.P. et al. Leaf segmentation in plant phenotyping: a collation study. Machine Vision and Applications 27, 585–606 (2016). https://doi.org/10.1007/s00138-015-0737-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0737-3

Keywords

Search

Navigation