Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3150165.3150166acmotherconferencesArticle/Chapter ViewAbstractPublication PagescvmpConference Proceedingsconference-collections
research-article

Cloud-based collaborative 3D reconstruction using smartphones

Published: 11 December 2017 Publication History

Abstract

This article presents a pipeline that enables multiple users to collaboratively acquire images with monocular smartphones and derive a 3D point cloud using a remote reconstruction server. A set of key images are automatically selected from each smartphone's camera video feed as multiple users record different viewpoints of an object, concurrently or at different time instants. Selected images are automatically processed and registered with an incremental Structure from Motion (SfM) algorithm in order to create a 3D model. Our incremental SfM approach enables on-the-fly feedback to the user to be generated about current reconstruction progress. Feedback is provided in the form of a preview window showing the current 3D point cloud, enabling users to see if parts of a surveyed scene need further attention/coverage whilst they are still in situ. We evaluate our 3D reconstruction pipeline by performing experiments in uncontrolled and unconstrained real-world scenarios. Datasets are publicly available.

References

[1]
R. Arandjelovic and A. Zisserman. 2012. Three things everyone should know to improve object retrieval. In Proc. of Computer Vision and Pattern Recognition. Providence, US.
[2]
S.-Y. Bao and S. Savarese. 2011. Semantic structure from motion. In Proc. of Computer Vision and Pattern Recognition. Colorado Springs, US.
[3]
P.E. Carbonneau and J.T. Dietrich. 2017. Cost-effective non-metric photogrammetry from consumer-grade sUAS: implications for direct georeferencing of structure from motion photogrammetry. Earth Surface Processes and Landforms 42, 3 (Mar. 2017), 473--486.
[4]
J. Engel, T. Schops, and D. Cremers. 2014. LSD-SLAM: Large-Scale Direct Monocular SLAM. In Proc. of European Conference on Computer Vision. Zurich, CH.
[5]
C. Forster, S. Lynen, L. Kneip, and D. Scaramuzza. 2013. Collaborative monocular SLAM with multiple Micro Aerial Vehicles. In Proc. of Intelligent Robots and Systems. Tokyo, JP.
[6]
R. Gherardi, M. Farenzena, and A. Fusiello. 2010. Improving the efficiency of hierarchical Structure-and-Motion. In Proc. of Computer Vision and Pattern Recognition. Colorado Springs, US.
[7]
R.I. Hartley and A. Zisserman. 2004. Multiple View Geometry in Computer Vision. Cambridge University Press.
[8]
M. Havlena, A. Torii, J. Knopp, and T. Pajdla. 2009. Turning mobile phones into 3D scanners. In Workshop in Computer Vision and Pattern Recognition. Miami, US.
[9]
A. Irschara, C. Zach, and H. Bischof. 2007. Towards wiki-based dense city modelling. In Proc. of International Conference on Computer Vision. Rio de Janeiro, BR.
[10]
ItSeez3D. 2017. (Aug. 2017). http://www.itseez3d.com
[11]
K. Kolev, P. Tanskanen, P. Speciale, and M. Pollefeys. 2014. Turning mobile phones into 3D scanners. In Proc. of Computer Vision and Pattern Recognition. Columbus, US.
[12]
A. Locher, M. Perdoch, H. Riemenschneider, and L. Van Gool. 2016. Mobile Phone and Cloud - a Dream Team for 3D Reconstruction. In Proc. of Winter Conference on Applications of Computer Vision. Lake Placid, US.
[13]
D.G. Lowe. 2004. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 2, 60 (Nov. 2004), 91--110.
[14]
J.G. Morrison, D. Galvez-López, and G. Sibley. 2016. MOARSLAM: Multiple Operator Augmented RSLAM. In Proc. of Distributed Autonomous Robotic Systems. London, UK.
[15]
O. Muratov, Y. Slynko, V. Chernov, M. Lyubimtseva, A. Shamsuarov, and Victor Bucha. 2016. 3DCapture: 3D Reconstruction for a Smartphone. In Computer Vision and Pattern Recognition Workshops. Las Vegas, US.
[16]
D. Nister. 2004. An efficient solution to the five-point relative pose problem. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 6 (Apr. 2004), 756--770.
[17]
D. Nister and H. Stewenius. 2006. Scalable Recognition with a Vocabulary Tree. In Proc. of Computer Vision and Pattern Recognition. New York, US.
[18]
E. Nocerino, F. Lago, D. Morabito, and F. Remondino et al. 2017. A Smartphone-based pipeline for the creative industry - The REPLICATE project. In International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Nafplio, GR.
[19]
P. Ondruska, P. Kohli, and S. Izadi. 2015. MobileFusion: Real-time volumetric surface reconstruction and dense tracking on mobile phones. IEEE Trans. on Visualization and Computer Graphics 21, 11 (Nov. 2015), 1251--1258.
[20]
J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. 2007. Object retrieval with large vocabularies and fast spatial matching. In Proc. of Computer Vision and Pattern Recognition. Minneapolis, US.
[21]
V.A. Prisacariu, O. Kahler, D.W. Murray, and I.D. Reid. 2015. IEEE Trans. on Visualization and Computer Graphics. Real-time 3D tracking and reconstruction on mobile phones 5, 21 (May 2015), 557--570.
[22]
E. Rublee, V. Rabaud, K. Konolige, and G. Bradski. 2011. ORB: An efficient alternative to SIFT and SURF. In Proc. of International Conference on Computer Vision. Sydney, AU.
[23]
P. Schmuck and M. Chli. 2017. Multi-UAV collaborative monocular SLAM. In Proc. of International Conference on Robotics and Automation. Singapore.
[24]
J.L. Schonberger and J.-M. Frahm. 2016. Structure-from-Motion Revisited. In Proc. of Computer Vision and Pattern Recognition. Las Vegas, US.
[25]
T. Sieberth, R. Wackrow, and J.H. Chandler. 2016. Automatic detection of blurred images in UAV image sets. Journal of Archaeological Science 122, 12 (Dec. 2016), 1--16.
[26]
N. Snavely, S.M. Seitz, and R. Szeliski. 2008. Modeling the world from Internet photo collections. International Journal on Computer Vision 80, 2 (Dec. 2008), 189--210.
[27]
C. Sweeney, T. Sattler, T. Hollerer, M. Turk, and M. Pollefeys. 2015. Optimizing the viewing graph for Structure-from-Motion. In Proc. of Computer vision and Pattern Recognition. Boston, US.
[28]
P. Tanskanen, K. Kolev, L. Meier, F. Camposeco, O. Saurer, and M. Pollefeys. 2013. Live metric 3D reconstruction on mobile phones. In Proc. of International Conference on Computer Vision. Sydney, AU.
[29]
B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon. 2000. Bundle adjustment - a modern synthesis. Vision Algorithms: Theory and Practice, Springer-Verlag, Berlin, GE.
[30]
TRNIO. 2017. (Aug. 2017). http://www.trnio.com
[31]
O. Untzelmann, T. Sattler, S. Middelberg, and L. Kobbelt. 2013. A scalable collaborative online system for city reconstruction. In Workshop in International Conference on Computer Vision. Sydney, AU.
[32]
C. Wu. 2013. Turning mobile phones into 3D scanners. In Proc. of 3D Vision. Tokyo, JP.
[33]
S. Zhang, J. Shan, Z. Zhang, J. Yan, and Y. Hou. 2016. Integrating smartphone images and airborne LIDAR data for complete urban building modelling. In Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. Prague, CZ.
[34]
J. Sivic; A. Zisserman. 2003. Video Google: a text retrieval approach to object matching in videos. In Proc. of International Conference on Computer Vision. Nice, FR.

Cited By

View all
  • (2023)Implementation of Digital Geotwin-Based Mobile Crowdsensing to Support Monitoring System in Smart CitySustainability10.3390/su1505394215:5(3942)Online publication date: 21-Feb-2023
  • (2022)Detailed Three-Dimensional Building Façade Reconstruction: A Review on Applications, Data and TechnologiesRemote Sensing10.3390/rs1411257914:11(2579)Online publication date: 27-May-2022
  • (2022)Towards Open-Source Web-Based 3D Reconstruction for Non-ProfessionalsFrontiers in Virtual Reality10.3389/frvir.2021.7865582Online publication date: 3-Feb-2022
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
CVMP '17: Proceedings of the 14th European Conference on Visual Media Production (CVMP 2017)
December 2017
93 pages
ISBN:9781450353298
DOI:10.1145/3150165
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • The Foundry: The Foundry Visionmongers Ltd.
  • University of Bath: University of Bath

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 December 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Collaborative 3D Reconstruction
  2. Mobile Device
  3. Structure from Motion

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

CVMP 2017
CVMP 2017: 14th European Conference on Visual Media Production
December 11 - 13, 2017
London, United Kingdom

Acceptance Rates

CVMP '17 Paper Acceptance Rate 10 of 16 submissions, 63%;
Overall Acceptance Rate 40 of 67 submissions, 60%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)20
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2023)Implementation of Digital Geotwin-Based Mobile Crowdsensing to Support Monitoring System in Smart CitySustainability10.3390/su1505394215:5(3942)Online publication date: 21-Feb-2023
  • (2022)Detailed Three-Dimensional Building Façade Reconstruction: A Review on Applications, Data and TechnologiesRemote Sensing10.3390/rs1411257914:11(2579)Online publication date: 27-May-2022
  • (2022)Towards Open-Source Web-Based 3D Reconstruction for Non-ProfessionalsFrontiers in Virtual Reality10.3389/frvir.2021.7865582Online publication date: 3-Feb-2022
  • (2022)AR Cloud: Towards Collaborative Augmented Reality at a Large-Scale2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00155(733-738)Online publication date: Oct-2022
  • (2022)Development of AR-based scanning support system for 3D model reconstruction of work sitesJournal of Nuclear Science and Technology10.1080/00223131.2021.201836959:7(934-948)Online publication date: 14-Jan-2022
  • (2022)3D Dense & Scaled Reconstruction Pipeline with Smartphone AcquisitionIntelligent Systems and Pattern Recognition10.1007/978-3-031-08277-1_1(3-18)Online publication date: 17-Jun-2022
  • (2021)A Photogrammetry-Based Workflow for the Accurate 3D Construction and Visualization of Museums AssetsRemote Sensing10.3390/rs1303048613:3(486)Online publication date: 30-Jan-2021
  • (2021)Characterizing real-time dense point cloud capture and streaming on mobile devicesProceedings of the 3rd ACM Workshop on Hot Topics in Video Analytics and Intelligent Edges10.1145/3477083.3480155(1-6)Online publication date: 25-Oct-2021
  • (2021)Multi-view data capture for dynamic object reconstruction using handheld augmented reality mobilesJournal of Real-Time Image Processing10.1007/s11554-021-01095-x18:2(345-355)Online publication date: 1-Apr-2021
  • (2020)Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile PhoneIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.302363426:12(3446-3456)Online publication date: Dec-2020
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media