research-article

Open access

Augmented Synthetic Dataset with Structured Light to Develop Ai-Based Methods for Breast Depth Estimation

Authors:

Bruno Oliveira,

Helena R. Torres,

Jaime C. Fonseca,

Joao L. VilacaAuthors Info & Claims

ICBRA '22: Proceedings of the 9th International Conference on Bioinformatics Research and Applications

Pages 82 - 88

https://doi.org/10.1145/3569192.3569206

Published: 27 January 2023 Publication History

All formats PDF

Abstract

Breast interventions are common healthcare procedures that normally require experienced professionals, expensive setups, and high execution times. With the evolution of robot-assisted technologies and image analysis algorithms, new methodologies can be implemented to facilitate the interventions in this area. To enable the introduction of robot-assisted approaches for breast procedures, strategies with real-time capacity and high precision for 3D breast shape estimation are required. In this paper, it is proposed to fuse the structured light (SL) and deep learning (DL) techniques to perform the depth estimation of the breast shape with high precision. First, multiple synthetic datasets of breasts with different printed patterns, resembling the SL technique, are created. Thus, it is possible to take advantage of the pattern's deformation induced by the breast surface in order to improve the quality of the depth information and to study the most suitable design. Then, distinct DL architectures, taken from the literature, were implemented to estimate the breast shape from the created datasets and study the DL architectures’ influence on depth estimation. The results obtained with the introduction of a yellow grid pattern, composed of thin stripes, fused with the DenseNet-161 architecture achieved the best results. Overall, the current study demonstrated the potential of the proposed practice for breast depth estimation or other human body parts in the future when we rely exclusively on 2D images.

References

[1]

E. Dhamija, R. Singh, S. Mishra, and S. Hari, “Image-Guided Breast Interventions: Biopsy and beyond,” Indian Journal of Radiology and Imaging, vol. 31, no. 2, pp. 391–399, Apr. 2021.

[2]

K. L. Maughan, M. A. Lutterbie, and P. S. Ham, “Treatment of Breast Cancer,” Am Fam Physician, vol. 81, no. 11, pp. 1339–1346, 2010, Accessed: May 20, 2022. [Online]. Available: www.aafp.org/afpAmericanFamilyPhysician1339

[3]

Z. Xinran, D. Haiyan, L. Mingyue, and Z. Yongde, “Breast intervention surgery robot under image navigation: A review:,” https://doi.org/10.1177/16878140211028113, vol. 13, no. 6, pp. 1–17, Jun. 2021.

[4]

R. J. Valkenburg and A. M. McIvor, “Accurate 3D measurement using a structured light system,” Image and Vision Computing, vol. 16, no. 2, pp. 99–110, Feb. 1998.

[5]

J. Geng, “Structured-light 3D surface imaging: a tutorial,” Advances in Optics and Photonics, Vol. 3, Issue 2, pp. 128-160, vol. 3, no. 2, pp. 128–160, Jun. 2011.

[6]

N. Jamwal, N. Jindal, and K. Singh, “A survey on depth map estimation strategies,” pp. 14 (5 .)-14 (5 .), Mar. 2018.

[7]

Y. Liu, J. Jiang, J. Sun, L. Bai, and Q. Wang, “A survey of depth estimation based on computer vision,” Proceedings - 2020 IEEE 5th International Conference on Data Science in Cyberspace, DSC 2020, pp. 135–141, Jul. 2020.

[8]

A. Shrestha and A. Mahmood, “Review of Deep Learning Algorithms and Architectures”.

[9]

F. Liu, C. Shen, and G. Lin, “Deep Convolutional Neural Fields for Depth Estimation from a Single Image,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 07-12-June-2015, pp. 5162–5170, Nov. 2014.

[10]

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-December, pp. 770–778, Dec. 2015.

[11]

I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab, “Deeper Depth Prediction with Fully Convolutional Residual Networks,” Proceedings - 2016 4th International Conference on 3D Vision, 3DV 2016, pp. 239–248, Jun. 2016.

[12]

S. Xie, R. Girshick, P. Dollár, Z. Tu, K. He, and U. San Diego, “Aggregated Residual Transformations for Deep Neural Networks”, Accessed: Apr. 28, 2022. [Online]. Available: https://github.com/facebookresearch/ResNeXt

[13]

G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, vol. 2017-January, pp. 2261–2269, Aug. 2016.

[14]

I. A. Kaust and P. Wonka, “High Quality Monocular Depth Estimation via Transfer Learning,” Dec. 2018, Accessed: Jan. 31, 2022. [Online]. Available: https://arxiv.org/abs/1812.11941v2

[15]

C. Sui, K. He, C. Lyu, Z. Wang, and Y. H. Liu, “3D surface reconstruction using a two-step stereo matching method assisted with five projected patterns,” Proceedings - IEEE International Conference on Robotics and Automation, vol. 2019-May, pp. 6080–6086, May 2019.

Digital Library

[16]

C. Rocchini, P. Cignoni, C. Montani, P. Pingi, and R. Scopigno, “A low cost 3D scanner based on structured light,” Computer Graphics Forum, vol. 20, no. 3, pp. 299–308, Sep. 2001.

[17]

M. Alzuhiri, K. Farrag, E. Lever, and Y. Deng, “An Electronically Stabilized Multi-Color Multi-Ring Structured Light Sensor for Gas Pipelines Internal Surface Inspection,” IEEE Sensors Journal, vol. 21, no. 17, pp. 19416–19426, Sep. 2021.

[18]

L. Song, S. Tang, and Z. Song, “A robust structured light pattern decoding method for single-shot 3D reconstruction,” 2017 IEEE International Conference on Real-Time Computing and Robotics, RCAR 2017, vol. 2017-July, pp. 668–672, Mar. 2018.

[19]

M. T. Chen and N. J. Durr, “Rapid tissue oxygenation mapping from snapshot structured-light images with adversarial deep learning,” https://doi.org/10.1117/1.JBO.25.11.112907, vol. 25, no. 11, p. 112907, Nov. 2020.

[20]

F. Li, Q. Li, T. Zhang, Y. Niu, and G. Shi, “Depth acquisition with the combination of structured light and deep learning stereo matching,” Signal Processing: Image Communication, vol. 75, pp. 111–117, Jul. 2019.

Digital Library

[21]

“www.makehumancommunity.org.” http://www.makehumancommunity.org/ (accessed May 02, 2022).

[22]

“blender.org - Home of the Blender project - Free and Open 3D Creation Software.” https://www.blender.org/ (accessed May 02, 2022).

[23]

“DenseNet | Papers With Code.” https://paperswithcode.com/model/densenet?variant=densenet-161 (accessed May 02, 2022).

Cited By

Duarte BOliveira BTorres HMorais PFonseca JVilaça J(2023)Robust 3D breast reconstruction based on monocular images and artificial intelligence for robotic guided oncological interventions2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)10.1109/EMBC40787.2023.10341168(1-4)Online publication date: 24-Jul-2023
https://doi.org/10.1109/EMBC40787.2023.10341168

Index Terms

Augmented Synthetic Dataset with Structured Light to Develop Ai-Based Methods for Breast Depth Estimation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Reconstruction

Recommendations

CNN-Based CAD for Breast Cancer Classification in Digital Breast Tomosynthesis
ICGSP '18: Proceedings of the 2nd International Conference on Graphics and Signal Processing

Digital breast tomosynthesis (DBT) is a promising new technique for breast cancer diagnosis. DBT has the potential to overcome the tissue superimposition problems that occur on traditional mammograms for tumor detection. However, DBT generates numerous ...
Combination of Spatially-Modulated ToF and Structured Light for MPI-Free Depth Estimation
Computer Vision – ECCV 2018 Workshops
Abstract
Multi-path Interference (MPI) is one of the major sources of error in Time-of-Flight (ToF) camera depth measurements. A possible solution for its removal is based on the separation of direct and global light through the projection of multiple ...
The ETS2 Dataset, Synthetic Data from Video Games for Monocular Depth Estimation
Pattern Recognition and Image Analysis
Abstract
In this work, we present a new dataset for monocular depth estimation created by extracting images, dense depth maps, and odometer data from a realistic video game simulation, Euro Truck Simulator 2 $^{TM}$ . The dataset is used to train state-of-the-art ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICBRA '22: Proceedings of the 9th International Conference on Bioinformatics Research and Applications

September 2022

165 pages

ISBN:9781450396868

DOI:10.1145/3569192

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICBRA 2022

ICBRA 2022: 2022 9th International Conference on Bioinformatics Research and Applications

September 18 - 20, 2022

Berlin, Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
254
Total Downloads

Downloads (Last 12 months)140
Downloads (Last 6 weeks)41

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Duarte BOliveira BTorres HMorais PFonseca JVilaça J(2023)Robust 3D breast reconstruction based on monocular images and artificial intelligence for robotic guided oncological interventions2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)10.1109/EMBC40787.2023.10341168(1-4)Online publication date: 24-Jul-2023
https://doi.org/10.1109/EMBC40787.2023.10341168

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents