Abstract
Research on marine species recognition is an important part of the actions for the protection of the ocean environment. It is also an under-exploited application area in the computer vision community. However, with the developments of deep learning, there has been an increasing interest about this topic. In this chapter, we present a comprehensive review of the computer vision techniques for marine species recognition, mainly from the perspectives of both classification and detection. In particular, we focus on capturing the evolution of various deep learning techniques in this area. We further compare the contemporary deep learning techniques with traditional machine learning techniques, and discuss the complementary issues between these two approaches. This chapter examines the attributes and challenges of a number of popular marine species datasets (which involve coral, kelp, plankton and fish) on recognition tasks. In the end, we highlight a few potential future application areas of deep learning in marine image analysis such as segmentation and enhancement of image quality.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
T. Wernberg, S. Bennett, R.C. Babcock et al., Climate-driven regime shift of a temperate marine ecosystem. Science 353(6295), 169–172 (2016)
T.C. Bridge, R. Ferrari, M. Bryson et al., Variable responses of benthic communities to anomalously warm sea temperatures on a high-latitude coral reef. PLoS ONE 9(11), e113079 (2014)
H. Singh, R. Armstrong, G. Gilbes et al., Imaging coral I: imaging coral habitats with the SeaBED AUV. Subsurf. Sens. Technol. Appl. 5(1), 25–42 (2004)
J.W. Nicholson, A.J. Healey, The present state of autonomous underwater vehicle (AUV) applications and technologies. Mar. Technol. Soc. J. 42(1), 44–51 (2008)
O. Beijbom, P.J. Edmunds, C. Roelfsema et al., Towards automated annotation of benthic survey images: variability of human experts and operational modes of automation. PLoS ONE 10(7), e0130312 (2015)
A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems (2012), pp. 1097–1105
K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)
K. He, X. Zhang, S. Ren et al., Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778
J. Donahue, Y. Jia, O. Vinyals et al., DeCAF: a deep convolutional activation feature for generic visual recognition, in Proceedings of the 31st International Conference on Machine Learning (ICML), Beijing, China, vol. 32, June 2014, pp. 647–655
S. Razavian, H. Azizpour, J. Sullivan et al., CNN features off-the-shelf: an astounding baseline for recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2014), pp. 806–813
J. Jaeger, M. Simon, J. Denzler et al., Croatian Fish dataset: fine-grained classification of fish species in their natural habitat (2015), pp. 1–7. http://dx.doi.org/10.5244/C.29.MVAB.6
A. Mahmood, M. Bennamoun, S. An et al., Automatic annotation of coral reefs using deep learning, in Proceedings of OCEANS 16, Monterey, California, USA, Sept 2016, pp. 17–23
M. Bewley, A. Friedman, R. Ferrari et al., Australian seafloor survey data, with images and expert annotations. Sci. Data 2 (2015)
A. Mahmood, M. Bennamoun, S. An et al., ResFeats: residual network based features for image classification. arXiv:1611.06656 (2016)
O. Beijbom, P.J. Edmunds, D.I. Kline et al., Automated annotation of coral reef survey images, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, June 2012, pp. 16–21
A. Mahmood, M. Bennamoun, S. An et al., Coral classification with hybrid feature representations, in Proceedings of IEEE International Conference on Image Processing (ICIP), Phoenix, Arizona, USA, Sept 2016, pp. 25–28
N. Dalal, B. Triggs. Histograms of oriented gradients for human detection, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1 (IEEE, 2005), pp. 886–893
D.G. Lowe, Object recognition from local scale-invariant features, in The Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, vol. 2 (IEEE, 1999), pp. 1150–1157
M. Marcos, S. Angeli, L. David et al., Automated Benthic counting of living and non-living components in Ngedarrak Reef, Palau via Subsurface Underwater video. Environ. Monit. Assess. 125(1), 177–184 (2008)
A. Pizarro, P. Rigby, M. Johnson-Roberson et al., Towards image-based marine habitat classification, in Proceedings of OCEANS 08, Quebec City, QC, Canada, Sept 2008, pp. 15–18
M.D. Stokes, G.B. Deane, Automated processing of coral reef benthic images. Limnol. Oceanogr.: Methods 7(2), 157–168 (2009)
M. Bewley, B. Douillard, N. Nourani-Vatani et al., Automated species detection: an experimental approach to kelp detection from sea-floor AUV images, in Proceedings of Australasian Conference on Robotics and Automation (2012)
M. Bewley, N. Nourani-Vatani, D. Rao et al., Hierarchical Classification in AUV imagery, in Springer Tracts in Advanced Robotics, vol. 105, Jan 2015, pp. 3–16
IMOS: integrated marine observing system, Sept 2013. http://www.imos.org.au
R.M. Haralick, K. Shanmugam, I. Dinstein, Textural features for image Classification. IEEE Trans. Syst. Man Cybern. SMC-3(6), 610–621 (1973)
L. Soh, C. Tsatsoulis, Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices. IEEE Trans. Geosci. Remote Sens. 37(2), 780–795 (1999)
D.A. Clausi, An analysis of co-occurrence texture statistics as a function of grey level quantization. Can. J. Remote Sens. 28(1), 45–62 (2002)
G.M. Haley, B.S. Manjunath, Rotation-invariant texture classification using a complete space-frequency model. IEEE Trans. Image Process. 8(2), 255–269 (1999)
Z. Guo, L. Zhang, A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19(6), 1657–1663 (2010)
J. Van de Weijer, C. Schmid, Coloring local feature extraction, in Proceedings of the 9th European Conference on Computer Vision (ECCV 06), Graz, Austria, May 2006, pp. 334–438
Z. Chao, J.C. Principe, B. Ouyang, Marine animal classification using combined CNN and hand-designed image features, in OCEANS’15 MTS/IEEE, Washington (IEEE, 2015), pp. 1–6
H. Peng, F. Long, C. Ding, Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)
C. Zheng, Jose C. Principe, B. Ouyang, Group feature selection in image classification with multiple kernel learning, in 2015 International Joint Conference on Neural Networks (IJCNN) (IEEE, 2015), pp. 1–5
H. Qin, X. Li, J. Liang et al., DeepFish: accurate underwater live fish recognition with a deep architecture. Neurocomputing 187, 49–58 (2016)
J.N. Blanchet et al., Automated annotation of corals in natural scene images using multiple texture representations. PeerJ Preprints 4, e2026v2 (2016)
A.S.M. Shihavuddin, N. Gracias, R. Garcia et al., Image-based coral reef classification and thematic mapping. Remote Sens. 5(4), 1809–1841 (2013)
O. Beijbom, T. Treibitz, D.I. Kline et al., Improving automated annotation of benthic survey images using wide-band fluorescence. Sci. Rep. 6 (2016)
Y. LeCun, Y. Bengio, Convolutional networks for images, speech, and time series, in The Handbook of Brain Theory and Neural Networks, vol. 3361, no. 10 (1995)
X. Li, Z. Cui, Deep residual networks for plankton classification, in OCEANS 2016 MTS/IEEE, Monterey, Sept 2016, pp. 1–4
S.H. Khan, M. Hayat, M. Bennamoun et al., Cost-sensitive learning of deep feature representations from imbalanced data. IEEE Trans. Neural Netw. Learn. Syst. (2017) (in press)
P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1 (2001), pp. I–I
S. Choi, Fish identification in underwater video with deep convolutional neural network: SNUMedinfo at LifeCLEF fish task 2015, in CLEF (Working Notes) (2015)
R. Girshick, J. Donahue, T. Darrell et al., Rich feature hierarchies for accurate object detection and semantic segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587, 2014
J.R. Uijlings, K.E. Van De Sande, T. Gevers et al., Selective search for object recognition. Int. J. Comput. Vis. 104(2), 154–171 (2013)
C. Zitnick, P. Dollr, Edge boxes: locating object proposals from edges, in European Conference on Computer Vision (Springer International Publishing, 2014), pp. 391–405
M. Cheng, Z. Zhang, W. Lin et al., BING: binarized normed gradients for objectness estimation at 300 fps, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014), pp. 3286–3293
K. He, X. Zhang, S. Ren et al., Spatial pyramid pooling in deep convolutional networks for visual recognition, in European Conference on Computer Vision (Springer International Publishing, 2014), pp. 346–361
R. Girshick, Fast R-CNN, in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 1440–1448
X. Li, M. Shang, H. Qin et al., Fast accurate fish detection and recognition of underwater images with Fast R-CNN, in OCEANS’15 MTS/IEEE, Washington, Oct 2015, pp. 1–5
S. Ren, K. He, R. Girshick et al., Faster R-CNN: towards real-time object detection with region proposal networks, in Advances in Neural Information Processing Systems (2015), pp. 91–99
X. Li, M. Shang, J. Hao et al., Accelerating fish detection and recognition by sharing CNNs with objectness learning, in OCEANS 2016-Shanghai, 10 Apr 2016 (IEEE, 2016), pp. 1–5
J. Redmon, S. Divvala, R. Girshick et al., You only look once: unified, real-time object detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 779–788
W. Liu, D. Anguelov, D. Erhan et al., SSD: single shot multibox detector, in European Conference on Computer vision (Springer, Cham, 2016), pp. 21–37
C. Dong, C.L. Chen, K. He et al., Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)
J. Sun, W. Cao, Z. Xu et al., Learning a convolutional neural network for non-uniform motion blur removal, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 769–777
C.J. Schuler, M. Hirsch, S. Harmeling et al., Learning to deblur. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1439–1451 (2016)
W. Shen, X. Wang, Y. Wang et al., Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 3982–3991
J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 3431–3440
Acknowledgements
This research was partially supported by China Scholarship Council funds (CSC, 201607565016) and Australian Research Council Grants (DP150104251 and DE120102960).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Xu, L., Bennamoun, M., An, S., Sohel, F., Boussaid, F. (2019). Deep Learning for Marine Species Recognition. In: Balas, V., Roy, S., Sharma, D., Samui, P. (eds) Handbook of Deep Learning Applications. Smart Innovation, Systems and Technologies, vol 136. Springer, Cham. https://doi.org/10.1007/978-3-030-11479-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-11479-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11478-7
Online ISBN: 978-3-030-11479-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)