Abstract
We present a novel structure from motion pipeline, which estimates motion and wiry 3D structure from imaged line segments across multiple views. Although the position and orientation of line segments can be determined more accurately than point features, the instability of their endpoints and the fact that lines are not constrained by epipolar geometry diverted most research focus away to point-based methods. In our approach, we tackle the problem of instable endpoints by utilizing relaxed constraints on their positions, both during matching and as well in the following bundle adjustment stage. Furthermore, we gain efficiency in estimating trifocal image relations by decoupling rotation and translation. To this end, a novel linear solver for relative translation estimation given rotations from five line correspondences in three views is introduced. Extensive experiments on long image sequences show that our line-based structure from motion pipeline advantageously complements point-based methods, giving more meaningful 3D representation for indoor scenarios.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Recall that fisheye images possessing field-of-view larger than 180\(^\circ \) cannot be undistorted and warped into a perspective image with straight lines.
References
Agarwal, S., Mierle, K. et al. (2010). Ceres Solver. http://www.ceres-solver.org.
Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, M. S., et al. (2011). Building Rome in a day. Commununications of ACM, 54(10), 105–112.
Ansar, A., & Daniilidis, K. (2002). Linear pose estimation from points or lines. In Proceedings of the European Conference on Computer Vision (ECCV).
Antone, M. E., & Teller, S. J. (2000). Automatic recovery of relative camera rotations for urban scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2282–2289).
Barrow, H. G., Tenenbaum, J. M., Bolles, R. C., & Wolf, H. C. (1977). Parametric correspondence and chamfer matching: Two new techniques for image matching. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (pp. 659–663).
Bartoli, A., & Sturm, P. (2005). Structure-from-motion using lines: Representation, triangulation, and bundle adjustment. Computer Vision and Image Understanding (CVIU), 100(3), 416–441.
Bay, H., Ferrari, V., & Van Gool, L. (2005). Wide-baseline stereo matching with line segments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 329–336).
Bazin, J. C., & Pollefeys, M. (2012). 3-line ransac for orthogonal vanishing point detection. In Proceedings of the International Conference on Intelligent Robots and Systems (IROS) (pp. 4282–4287).
Bazin, J. C., Demonceaux, C., Vasseur, P., & Kweon, I. (2010). Motion estimation by decoupling rotation and translation in catadioptric vision. Computer Vision and Image Understanding (CVIU), 114(2), 254–273.
Burns, J. B., Hanson, A. R., & Riseman, E. M. (1986). Extracting straight lines. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 8(4), 425–455.
Chen, H. (1991). Pose determination from line-to-plane correspondences: Existence condition and closed-form solutions. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 13(6), 530–541.
Davison, A. J., Reid, I. D., Molton, N., & Stasse, O. (2007). Monoslam: Real-time single camera SLAM. IEEE Trans Pattern Anal Mach Intell, 29(6), 1052–1067.
Deriche, R., & Faugeras, O. (1990). Tracking line segments. In Proceedings of the European Conference on Computer Vision (ECCV).
Elqursh, A., & Elgammal, A. M. (2011). Line-based relative pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Fan, B., Wu, F., & Hu, Z. (2010). Line matching leveraged by point correspondences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 390–397).
Faugeras, O., Lustman, F., & Toscani, G. (1987). Motion and structure from motion from point and line matches. In Proceedings of the International Conference on Computer Vision (ICCV).
Geyer, C., & Daniilidis, K. (2001). Catadioptric projective geometry. International Journal of Computer Vision (IJCV), 45(3), 223–243.
Hartley, R. (1997). Lines and points in three views and the trifocal tensor. International Journal of Computer Vision (IJCV), 22(2), 125–140.
Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.
Hartley, R. I., Trumpf, J., Dai, Y., & Li, H. (2013). Rotation averaging. International Journal of Computer Vision (IJCV), 103(3), 267–305.
Hirose, K., & Saito, H. (2012). Fast line description for line-based SLAM. In Proceedings of the Britisch Machine Vision Conference (BMVC).
Hofer, M., Wendel, A., & Bischof, H. (2013). Incremental line-based 3D reconstruction using geometric constraints. In Proceedings of the Britisch Machine Vision Conference (BMVC).
Huynh, D. Q. (2009). Metrics for 3d rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35(2), 155–164.
Jain, A., Kurz, C., Thormählen, T., & Seidel, H.P. (2010). Exploiting global connectivity constraints for reconstruction of 3d line segments from images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Jiang, N., Cui, Z., & Tan, P. (2013). A global linear method for camera pose registration. In Proceedings of the International Conference on Computer Vision (ICCV).
Liu, Y., & Huang, T. S. (1988a). Estimation of rigid body motion using straight line correspondences. Computer Vision, Graphics, and Image Processing (CVGIP), 43(1), 35–52.
Liu, Y., & Huang, T. S. (1988b). A linear algorithm for motion estimation using straight line correspondences. Computer Vision, Graphics, and Image Processing (CVGIP), 44(1), 35–57.
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision (IJCV), 60(2), 91–110.
Martinec, D., & Pajdla, T. (2003). Line reconstruction from many perspective images by factorization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Micusik, B., & Pajdla, T. (2006). Structure from motion with wide circular field of view cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 28(7), 1135–1149.
Micusik, B., & Wildenauer, H. (2015). Descriptor free visual indoor localization with line segments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Morris, D. D., & Kanade, T. (1998). A unified factorization algorithm for points, line segments and planes with uncertainty models. In Proceedings of the International Conference on Computer Vision (ICCV) (pp. 696–702).
Rother, C. (2003). Multi-view reconstruction and camera recovery using a real or virtual reference plane. PhD thesis, KTH.
Schindler, G., Krishnamurthy, P., & Dellaert, F. (2006). Line-based structure from motion for urban environments. In Proceedings of the International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT).
Sinha, S. N., Steedly, D., & Szeliski, R. (2009). Piecewise planar stereo for image-based rendering. In Proceedings of the International Conference on Computer Vision (ICCV) (pp .1881–1888).
Sinha, S. N., Steedly, D., & Szeliski, R. (2010). A multi-stage linear approach to structure from motion. In: Proceedings of the ECCV Workshop—Trends and Topics in Computer Vision (pp. 267–281).
Smith, P., Reid, I., & Davison, A. (2006). Real-time monocular SLAM with straight lines. In Proceedings of the Britisch Machine Vision Conference (BMVC).
Spetsakis, M. E. (1992). A linear algorithm for point and line-based structure from motion. Computer Vision, Graphics, and Image Processing: Image Understanding (CVGIPIU), 56(2), 230–241.
Strecha, C., von Hansen, W., Van Gool, L., Fua, P., & Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Subbarao, R., & Meer, P. (2009). Nonlinear mean shift over Riemannian manifolds. International Journal of Computer Vision (IJCV), 84(1), 1–20.
Taylor, J. C., & Kriegman, J. D. (1995). Structure and motion from line segments in multiple images. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 17(11), 1021–1032.
Verhagen, B., Timofte, R., & Gool, L. J. V. (2014). Scale-invariant line descriptors for wide baseline matching. In: IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, CO, USA, March 24–26, (pp. 493–500).
VISCODA (2012). Voodoo camera tracker.
Wang, L., Neumann, U., & You, S. (2009). Wide-baseline image matching using line signatures. In Proceedings of the International Conference on Computer Vision (ICCV).
Weng, J., Huang, T., & Ahuja, N. (1992). Motion and structure from line correspondences; closed-form solution, uniqueness, and optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 14(3), 318–336.
Weng, J., Huang, T., & Ahuja, N. (1993). Motion and structure from image sequences. Springer series in information sciences. Berlin: Springer.
Werner, T., & Zisserman, A. (2002). New techniques for automated architectural reconstruction from photographs. In Proceedings of the European Conference on Computer Vision (ECCV).
Wildenauer, H., & Hanbury, A. (2012). Robust camera self-calibration from monocular images of manhattan worlds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2831–2838).
Wu, C. (2011). VisualSFM: A visual structure from motion system.
Zhang, L., & Koch, R. (2014). Structure and motion from line correspondences: Representation, projection, initialization and sparse bundle adjustment. Elsevier Journal of Visual Communication and Image Representation (JVCIR), 25(5), 904–915.
Zhang, Z. (1995). Estimating motion and structure from correspondences of line segments between two perspective images. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 17(12), 1129–1139.
Zhang, Z. (1997). Parameter estimation techniques: A tutorial with application to conic fitting. Image and Vision Computing (IVC), 15(1), 59–76.
Acknowledgements
This research received funding from the Austrian Research Promotion Agencys (FFG) Projects LOLOG 840168, LARAH 4586620 and PAMON 835916.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by K. Ikeuchi.
Rights and permissions
About this article
Cite this article
Micusik, B., Wildenauer, H. Structure from Motion with Line Segments Under Relaxed Endpoint Constraints. Int J Comput Vis 124, 65–79 (2017). https://doi.org/10.1007/s11263-016-0971-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-016-0971-9