EDO-SANet: Shape-Aware Network with Edge Detection Operator for Polyp Segmentation
Article No.: 57, Pages 1 - 8
Abstract
Automatic and accurate segmentation of colonic polyps can effectively help physicians quickly identify the size and location of polyps during endoscopy. However, due to the variable intestinal environment, polyps are highly similar to the surrounding tissue leading to over-segmentation or under-segmentation. To tackle these concerns, we propose a novel network with shape-awareness. Specifically, the network employs a dual-stream encoder, including the mainstream Transoformer-based encoder and an auxiliary encoder to extract global and local representations. The auxiliary encoder is built on deformable convolutional v3 for sensing the shape of polyps. In order to better identify the edges of polyps, the Edge Detection Guided Module (EDGM) is proposed to utilize edge detection operators to highlight the importance of edge information in low-level features. Furthermore, we introduce a Semantic Interaction Module (SIM) to better localize and calibrate targets by integrating global and local semantics in high-level features. Compared to other state-of-the-art methods, our model achieves better performance both in learning and generalization ability on five polyp segmentation datasets. The potential applicability of our model to other biomedical fields is demonstrated through its outstanding performance. The code is available at https://github.com/xff12138/EDO-SANet
References
[1]
Mojtaba Akbari, Majid Mohrekesh, Ebrahim Nasr-Esfahani, S. M. Reza Soroushmehr, Nader Karimi, Shadrokh Samavi, and Kayvan Najarian. 2018. Polyp Segmentation in Colonoscopy Images Using Fully Convolutional Network. 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2018), 69–72. https://doi.org/10.1109/EMBC.2018.8512197
[2]
Jorge Bernal, Francisco Javier Sánchez, Glòria Fernández-Esparrach, Debora Gil, Cristina Rodríguez de Miguel, and Fernando Vilariño. 2015. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society 43 (2015), 99–111. https://doi.org/10.1016/j.compmedimag.2015.02.007
[3]
Juan C. Caicedo, Allen Goodman, Kyle W. Karhohs, Beth A. Cimini, Jeanelle Ackerman, Marzieh Haghighi, Cherkeng Heng, Tim Becker, Minh Doan, Claire McQuin, Mohammad Hossein Rohban, Shantanu Singh, and Anne E Carpenter. 2019. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nature Methods 16 (2019), 1247 – 1253. https://doi.org/10.1038/s41592-019-0612-7
[4]
John F. Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8 (1986), 679–698. https://doi.org/10.1109/TPAMI.1986.4767851
[5]
Qi Chang, Danish Ahmad, J.W. Toth, Rebecca Bascom, and W.E. Higgins. 2022. ESFPNet: efficient deep learning architecture for real-time lesion segmentation in autofluorescence bronchoscopic video. In Medical Imaging. https://doi.org/10.1117/12.2647897
[6]
Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan Loddon Yuille, and Yuyin Zhou. 2021. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. ArXiv abs/2102.04306 (2021). https://doi.org/10.48550/arXiv.2102.04306
[7]
Xinru Chen, Chengbo Dong, Jiaqi Ji, Juan Cao, and Xirong Li. 2021. Image Manipulation Detection by Multi-View Multi-Scale Supervision. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021), 14165–14173. https://doi.org/10.1109/ICCV48922.2021.01392
[8]
Noel C. F. Codella, Veronica M Rotemberg, Philipp Tschandl, M. E. Celebi, Stephen W. Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Armando Marchetti, Harald Kittler, and Allan C. Halpern. 2019. Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC). ArXiv abs/1902.03368 (2019). https://doi.org/10.48550/arXiv.1902.03368
[9]
Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. 2017. Deformable Convolutional Networks. 2017 IEEE International Conference on Computer Vision (ICCV) (2017), 764–773. https://doi.org/10.1109/ICCV.2017.89
[10]
Jin Ding, Jie Zhao, Yongyang Sun, Ping Tan, Ji en Ma, and You tong Fang. 2023. Improving the Robustness of Deep Convolutional Neural Networks Through Feature Learning. ArXiv abs/2303.06425 (2023). https://doi.org/10.48550/arXiv.2303.06425
[11]
B. Dong, Wenhai Wang, Deng-Ping Fan, Jinpeng Li, H. Fu, and Ling Shao. 2021. Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers. ArXiv abs/2108.06932 (2021). https://doi.org/10.26599/air.2023.9150015
[12]
Stefan Elfwing, Eiji Uchibe, and Kenji Doya. 2017. Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning. Neural networks : the official journal of the International Neural Network Society 107 (2017), 3–11. https://doi.org/10.1016/j.neunet.2017.12.012
[13]
Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, H. Fu, Jianbing Shen, and Ling Shao. 2020. PraNet: Parallel Reverse Attention Network for Polyp Segmentation. ArXiv abs/2006.11392 (2020). https://doi.org/10.1007/978-3-030-59725-2_26
[14]
Pasqualino Favoriti, Gabriele Carbone, Marco Greco, Felice Pirozzi, Raffaele Pirozzi, and Francesco Corcione. 2016. Worldwide burden of colorectal cancer: a review. Updates in Surgery 68 (2016), 7–11. https://doi.org/10.1007/s13304-016-0359-y
[15]
Jacques Ferlay, Murielle Colombet, Isabelle Soerjomataram, Donald Maxwell Parkin, Marion Piñeros, Ariana Znaor, and Freddie Bray. 2021. Cancer statistics for the year 2020: An overview. International Journal of Cancer 149 (2021), 778 – 789. https://doi.org/10.1002/ijc.33588
[16]
Kerr Fitzgerald and Bogdan J. Matuszewski. 2023. FCB-SwinV2 Transformer for Polyp Segmentation. ArXiv abs/2302.01027 (2023). https://doi.org/10.48550/arXiv.2302.01027
[17]
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 770–778. https://doi.org/10.1109/cvpr.2016.90
[18]
Debesh Jha, M. Riegler, Dag Johansen, Paal Halvorsen, and Haavard D. Johansen. 2020. DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation. 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS) (2020), 558–564. https://doi.org/10.1109/CBMS49503.2020.00111
[19]
Debesh Jha, Pia Helen Smedsrud, M. Riegler, Paal Halvorsen, Thomas de Lange, Dag Johansen, and Haavard D. Johansen. 2019. Kvasir-SEG: A Segmented Polyp Dataset. In Conference on Multimedia Modeling. https://doi.org/10.1007/978-3-030-37734-2_37
[20]
Debesh Jha, Pia Helen Smedsrud, M. Riegler, Dag Johansen, Thomas de Lange, P. Halvorsen, and Håvard Dagenborg Johansen. 2019. ResUNet++: An Advanced Architecture for Medical Image Segmentation. 2019 IEEE International Symposium on Multimedia (ISM) (2019), 225–2255. https://doi.org/10.1109/ISM46123.2019.00049
[21]
Nick Kanopoulos, Nagesh Vasanthavada, and Robert L. Baker. 1988. Design of an image edge detection filter using the Sobel operator. IEEE Journal of Solid-state Circuits 23 (1988), 358–367. https://doi.org/10.1109/4.996
[22]
Taehun Kim, Hyemin Lee, and Daijin Kim. 2021. UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation. Proceedings of the 29th ACM International Conference on Multimedia (2021). https://doi.org/10.1145/3474085.3475375
[23]
Ange Lou, Shuyue Guan, and Murray H. Loew. 2021. CaraNet: context axial reverse attention network for segmentation of small medical objects. In Medical Imaging. https://doi.org/10.1117/12.2611802
[24]
Yuichi Mori and Shin ei Kudo. 2018. Detecting colorectal polyps via machine learning. Nature Biomedical Engineering 2 (2018), 713 – 714. https://doi.org/10.1038/s41551-018-0308-9
[25]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. ArXiv abs/1505.04597 (2015). https://doi.org/10.1007/978-3-319-24574-4_28
[26]
Edward Sanderson and Bogdan J. Matuszewski. 2022. FCN-Transformer Feature Fusion for Polyp Segmentation. In Annual Conference on Medical Image Understanding and Analysis. https://doi.org/10.1007/978-3-031-12053-4_65
[27]
Juan Silva, Aymeric Histace, Olivier Romain, Xavier Dray, and B. Granado. 2014. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. International Journal of Computer Assisted Radiology and Surgery 9 (2014), 283–293. https://doi.org/10.1007/s11548-013-0926-3
[28]
Abhishek Srivastava, Debesh Jha, Sukalpa Chanda, Umapada Pal, Haavard D. Johansen, Dag Johansen, M. Riegler, Sharib Ali, and Paal Halvorsen. 2021. MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation. IEEE Journal of Biomedical and Health Informatics 26 (2021), 2252–2263. https://doi.org/10.1109/JBHI.2021.3138024
[29]
Nima Tajbakhsh, Suryakanth R. Gurudu, and Jianming Liang. 2016. Automated Polyp Detection in Colonoscopy Videos Using Shape and Context Information. IEEE Transactions on Medical Imaging 35 (2016), 630–644. https://doi.org/10.1109/TMI.2015.2487997
[30]
Feilong Tang, Qi Hong Huang, Jinfeng Wang, Xianxu Hou, Jionglong Su, and Jingxin Liu. 2022. DuAT: Dual-Aggregation Transformer Network for Medical Image Segmentation. ArXiv abs/2212.11677 (2022). https://doi.org/10.48550/arXiv.2212.11677
[31]
Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Neural Information Processing Systems. https://doi.org/10.48550/arXiv.1706.03762
[32]
David Vázquez, Jorge Bernal, Francisco Javier Sánchez, Glòria Fernández-Esparrach, Antonio M. López, Adriana Romero, Michal Drozdzal, and Aaron C. Courville. 2016. A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images. Journal of Healthcare Engineering 2017 (2016). https://doi.org/10.1155/2017/4037190
[33]
Jiacheng Wang, Fei Chen, Yuxi Ma, Liansheng Wang, Zhaodong Fei, Jia Shuai, Xiangdong Tang, Qichao Zhou, and Jing Qin. 2023. XBound-Former: Toward Cross-Scale Boundary Modeling in Transformers. IEEE Transactions on Medical Imaging 42 (2023), 1735–1745. https://doi.org/10.1109/TMI.2023.3236037
[34]
Jinfeng Wang, Qiming Huang, Feilong Tang, Jia Meng, Jionglong Su, and Sifan Song. 2022. Stepwise Feature Fusion: Local Guides Global. In International Conference on Medical Image Computing and Computer-Assisted Intervention. https://doi.org/10.48550/arXiv.2203.03635
[35]
Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiao hua Hu, Tong Lu, Lewei Lu, Hongsheng Li, Xiaogang Wang, and Y. Qiao. 2022. InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 14408–14419. https://doi.org/10.1109/CVPR52729.2023.01385
[36]
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2021. PVT v2: Improved baselines with Pyramid Vision Transformer. Computational Visual Media 8 (2021), 415 – 424. https://doi.org/10.1007/s41095-022-0274-8
[37]
Yuxin Wu and Kaiming He. 2018. Group Normalization. International Journal of Computer Vision 128 (2018), 742 – 755. https://doi.org/10.1007/s11263-019-01198-w
[38]
Wenqiang Zhang, Zilong Huang, Guozhong Luo, Tao Chen, Xinggang Wang, Wenyu Liu, Gang Yu, and Chunhua Shen. 2022. TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 12073–12083. https://doi.org/10.1109/CVPR52688.2022.01177
[39]
Yundong Zhang, Huiye Liu, and Qiang Hu. 2021. TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation. ArXiv abs/2102.08005 (2021). https://doi.org/10.1007/978-3-030-87193-2_2
[40]
Xiaoqi Zhao, Hongpeng Jia, Youwei Pang, Long Lv, Feng Tian, Lihe Zhang, Weiwei Sun, and Huchuan Lu. 2023. M2SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation. ArXiv abs/2303.10894 (2023). https://doi.org/10.48550/arXiv.2303.10894
[41]
Gaodian Zhou, Jiahui Xu, Weitao Chen, Xianju Li, Jun Li, and Lizhe Wang. 2023. Deep Feature Enhancement Method for Land Cover With Irregular and Sparse Spatial Distribution Features: A Case Study on Open-Pit Mining. IEEE Transactions on Geoscience and Remote Sensing 61 (2023), 1–20. https://doi.org/10.1109/TGRS.2023.3241331
[42]
Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. 2018. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support : 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, held in conjunction with MICCAI 2018, Granada, Spain, S... 11045 (2018), 3–11. https://doi.org/10.1007/978-3-030-00889-5_1
[43]
Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. 2018. Deformable ConvNets V2: More Deformable, Better Results. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018), 9300–9308. https://doi.org/10.1109/CVPR.2019.00953
Index Terms
- EDO-SANet: Shape-Aware Network with Edge Detection Operator for Polyp Segmentation
Recommendations
FCN-Transformer Feature Fusion for Polyp Segmentation
Medical Image Understanding and AnalysisAbstractColonoscopy is widely recognised as the gold standard procedure for the early detection of colorectal cancer (CRC). Segmentation is valuable for two significant clinical applications, namely lesion detection and classification, providing means to ...
Polyp-SES: Automatic Polyp Segmentation with Self-enriched Semantic Model
Computer Vision – ACCV 2024AbstractAutomatic polyp segmentation is crucial for effective diagnosis and treatment in colonoscopy images. Traditional methods encounter significant challenges in accurately delineating polyps due to limitations in feature representation and the ...
Colorectal polyp segmentation based on geodesic active contours with a shape-prior model
MICCAI'10: Proceedings of the Second international conference on Virtual Colonoscopy and Abdominal Imaging: computational challenges and clinical opportunitiesAutomated polyp segmentation is important both in measuring polyp size and in improving polyp detection performance in CTC. We present a polyp segmentation method that is based on the combination of geodesic active contours and a shape-prior model of ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
373 pages
ISBN:9798400716607
DOI:10.1145/3663976
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Beijing Natural Science Foundation
- Key R&D Program of the Scientific Research Department
- National Natural Science Foundation of China
- National Natural Science Foundation of China
- Key R&D Program of the Scientific Research Department
Conference
CVIPPR 2024
CVIPPR 2024: 2024 2nd Asia Conference on Computer Vision, Image Processing and Pattern Recognition
April 26 - 28, 2024
Xiamen, China
Acceptance Rates
Overall Acceptance Rate 14 of 38 submissions, 37%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 21Total Downloads
- Downloads (Last 12 months)21
- Downloads (Last 6 weeks)2
Reflects downloads up to 27 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format