Abstract
Fisheye cameras have recently became very popular in computer vision applications due to their wide field of view. In addition to a better overview of the surrounding area, they enable to capture objects at extremely close ranges. These advantages come at a cost of strong image distortion, which cannot be removed completely maintaining image continuity. This complicates the use of traditional computer vision algorithms, which expect a single image as an input. This paper presents a performance evaluation of neural network algorithms for object detection and segmentation on fisheye camera images. Three approaches are evaluated: semantic image segmentation with Fully Convolutional Network (FCN) [13], a fully convolutional approach to instance segmentation with U-Net [18] and a region-based approach to instance segmentation with Mask R-CNN [10]. All of these networks successfully solved the task. However, as they were designed to different purposes, each of them has its own strengths and shortcomings. These three approaches are used to perform euro container image segmentation task. An image dataset was created in order to train and evaluate these algorithms. Huge part of this dataset was generated artificially, which simplified the task of ground truth labeling. The power of neural networks enable for fast and reliable image segmentation. As to our knowledge, this is the first neural networks application for euro container fisheye image detection and segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. CoRR abs/1511.00561 (2015). http://arxiv.org/abs/1511.00561
Chen, L., Hermans, A., Papandreou, G., Schroff, F., Wang, P., Adam, H.: Masklab: instance segmentation by refining object detection with semantic and direction features. CoRR abs/1712.04837 (2017). http://arxiv.org/abs/1712.04837
Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS. CoRR abs/1606.00915 (2016). http://arxiv.org/abs/1606.00915
Chen, L., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. CoRR abs/1706.05587 (2017). http://arxiv.org/abs/1706.05587
Coors, B., Condurache, A.P., Geiger, A.: Spherenet: learning spherical representations for detection and classification in omnidirectional images. In: European Conference on Computer Vision (ECCV), September 2018. http://www.cvlibs.net/publications/Coors2018ECCV.pdf
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR 2009 (2009)
Deng, L., Yang, M., Li, H., Li, T., Hu, B., Wang, C.: Restricted deformable convolution based road scene semantic segmentation using surround view cameras. CoRR abs/1801.00708 (2018). http://arxiv.org/abs/1801.00708
Deng, L., Yang, M., Qian, Y., Wang, C., Wang, B.: CNN-based semantic segmentation for urban traffic scenes using fisheye camera, pp. 231–236, June 2017. https://doi.org/10.1109/IVS.2017.7995725, https://www.researchgate.net/publication/318805934_CNN_based_semantic_segmentation_for_urban_traffic_scenes_using_fisheye_camera
Hayder, Z., He, X., Salzmann, M.: Shape-aware instance segmentation. CoRR abs/1612.03129 (2016). http://arxiv.org/abs/1612.03129
He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. CoRR abs/1703.06870 (2017). http://arxiv.org/abs/1703.06870
Iglovikov, V.I., Seferbekov, S.S., Buslaev, A.V., Shvets, A.: Ternausnetv2: fully convolutional network for instance segmentation. CoRR abs/1806.00844 (2018). http://arxiv.org/abs/1806.00844
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates Inc., New York (2012). http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. CoRR abs/1411.4038 (2014). http://arxiv.org/abs/1411.4038
Mohamed, I.S., Capitanelli, A., Mastrogiovanni, F., Rovetta, S., Zaccaria, R.: Detection, localisation and tracking of pallets using machine learning techniques and 2D range data. CoRR abs/1803.11254 (2018). http://arxiv.org/abs/1803.11254
Pinheiro, P.H.O., Lin, T., Collobert, R., Dollár, P.: Learning to refine object segments. CoRR abs/1603.08695 (2016). http://arxiv.org/abs/1603.08695
Ran, L., Zhang, Y., Zhang, Q., Yang, T.: Convolutional neural network-based robot navigation using uncalibrated spherical images. Sensors 17, 1341 (2017)
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015). http://arxiv.org/abs/1506.01497
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015). http://arxiv.org/abs/1505.04597
Saez, A., Bergasa, L., Romeral, E., Guillén, M., Barea, R., Sanz, R.: CNN-based fisheye image real-time semantic segmentation, pp. 1039–1044, June 2018. https://doi.org/10.1109/IVS.2018.8500456
Salvador, A., Bellver, M., Baradad, M., Marqués, F., Torres, J., Giró i Nieto, X.: Recurrent neural networks for semantic instance segmentation. CoRR abs/1712.00617 (2017). http://arxiv.org/abs/1712.00617
Shipitko, O.: 3D pose estimation algorithm for intelligent box picking of warehouse automation robot (2018). https://www.researchgate.net/publication/325780223_3D_pose_estimation_algorithm_for_intelligent_box_picking_of_warehouse_automation_robot
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556
Su, Y., Grauman, K.: Flat2sphere: learning spherical convolution for fast features from 360\(^\circ \) imagery. CoRR abs/1708.00919 (2017). https://arxiv.org/pdf/1708.00919.pdf
Szegedy, C., et al.: Going deeper with convolutions. CoRR abs/1409.4842 (2014). http://arxiv.org/abs/1409.4842
Ter-Sarkisov, A., Ross, R.J., Kelleher, J.D., Earley, B., Keane, M.: Beef cattle instance segmentation using fully convolutional neural network. CoRR abs/1807.01972 (2018). http://arxiv.org/abs/1807.01972
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. CoRR abs/1511.07122 (2015). http://arxiv.org/abs/1511.07122
Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: ICNet for real-time semantic segmentation on high-resolution images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 418–434. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_25
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. CoRR abs/1612.01105 (2016). http://arxiv.org/abs/1612.01105
Acknowledgments
This work is supported by the German-Russian Interdisciplinary Science Center (G-RISC) funded by the German Federal Foreign Office via the German Academic Exchange Service (DAAD).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Beloshapko, A., Korkhov, V., Knoll, C., Iben, U. (2019). Industrial Fisheye Image Segmentation Using Neural Networks. In: Misra, S., et al. Computational Science and Its Applications – ICCSA 2019. ICCSA 2019. Lecture Notes in Computer Science(), vol 11622. Springer, Cham. https://doi.org/10.1007/978-3-030-24305-0_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-24305-0_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-24304-3
Online ISBN: 978-3-030-24305-0
eBook Packages: Computer ScienceComputer Science (R0)