Abstract
This paper describes a new approach to motion estimation in video. We represent video motion using a set of particles. Each particle is an image point sample with a long-duration trajectory and other properties. To optimize particle trajectories we measure appearance consistency along the particle trajectories and distortion between the particles. The resulting motion representation is useful for a variety of applications and cannot be directly obtained using existing methods such as optical flow or feature tracking. We demonstrate the algorithm on challenging real-world videos that include complex scene geometry, multiple types of occlusion, regions with low texture, and non-rigid deformations.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alvarez, L., Deriche, R., Papadopoulo, T., & Sanchez, J. (2002). Symmetrical dense optical flow estimation with occlusion detection. In ECCV (pp. 721–735).
Amiaz, T., & Kiryati, N. (2005). Dense discontinuous optical flow via contour-based segmentation. In ICIP (pp. 1264–1267).
Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56(3), 221–255.
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M., & Szeliski, R. (2007). A database and evaluation methodology for optical flow. In ICCV.
Barrett, R., Berry, M., Chan, T. F., Demmel, J., Donato, J. M., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., & der Vorst, H. V. (1994). Templates for the solution of linear systems: building blocks for iterative methods. Philadelphia: SIAM.
Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994). Performance of optical flow techniques. International Journal of Computer Vision, 12(1), 43–77.
Beauchemin, S. S., & Barron, J. L. (1995). The computation of optical flow. ACM Computing Surveys, 27(3), 433–467.
Bergen, J. R., Anandan, P., Hanna, K. J., & Hingorani, R. (1992). Hierarchical model-based motion estimation. In ECCV (pp. 237–252).
Black, M., & Anandan, P. (1991). Robust dynamic motion estimation over time. In CVPR (pp. 296–302).
Black, M. J. (1994). Recursive non-linear estimation of discontinuous flow fields. In ECCV (pp. 138–144).
Black, M. J., & Anandan, P. (1996). The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104.
Brand, M. (2001). Morphable 3D models from video. In CVPR (pp. 456–463).
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In ECCV (pp. 25–36).
Chin, T. M., Karl, W. C., & Willsky, A. S. (1994). Probabilistic and sequential computation of optical flow using temporal coherence. IEEE Transactions on Image Processing, 3(6), 773–788.
Elad, M., & Feuer, A. (1998). Recursive optical flow estimation–adaptive filtering approach. Visual Communication and Image Representation, 9(2), 119–138.
Fusiello, A., Trucco, E., Tommasini, T., & Roberto, V. (1999). Improving feature tracking with robust statistics. Pattern Analysis and Applications, 2(4), 312–320.
Goldman, D. B., Curless, B., Salesin, D., & Seitz, S. M. (2007). Interactive video object annotation (Technical Report UW-CSE-2007-04-01). University of Washington.
Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.
Haussecker, H. W., & Fleet, D. J. (2001). Computing optical flow with physical models of brightness variation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 661–673.
Irani, M. (1999). Multi-frame optical flow estimation using subspace constraints. In ICCV (pp. 626–633).
Lischinski, D. (1994). Incremental Delaunay triangulation. In Graphics gems IV (pp. 47–59). San Diego: Academic Press.
Liu, C., Torralba, A., Freeman, W. T., Durand, F., & Adelson, E. H. (2005). Motion magnification. ACM Transactions on Graphics, 24(3), 519–526.
Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In International joint conference on artificial intelligence (pp. 674–679).
Murray, D. W., & Buxton, B. F. (1987). Scene segmentation from visual motion using global optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(2), 220–228.
Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.
Sand, P. (2006). Long-range video motion estimation using point trajectories. PhD thesis, MIT.
Sand, P., & Teller, S. (2006). Particle video: Long-range motion estimation using point trajectories. In CVPR (pp. 2195–2202).
Sawhney, H. S., Guo, Y., Hanna, K., Kumar, R., Adkins, S., & Zhou, S. (2001). Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences. In SIGGRAPH (pp. 451–460).
Shi, J., & Malik, J. (1998). Motion segmentation and tracking using normalized cuts. In ICCV (pp. 1154–1160).
Shi, J., & Tomasi, C. (1994). Good features to track. In CVPR (pp. 593–600).
Silva, C., & Santos-Victor, J. (2001). Motion from occlusions. Robotics and Autonomous Systems, 35(3–4), 153–162.
Strecha, C., Fransens, R., & Gool, L. V. (2004). A probabilistic approach to large displacement optical flow and occlusion detection. In Statistical methods in video processing (pp. 71–82).
Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 2319–2322.
Thompson, W. (1998). Exploiting discontinuities in optical flow. International Journal of Computer Vision, 30(3), 163–174.
Weickert, J., Bruhn, A., Papenberg, N., & Brox, T. (2004). Variational optic flow computation: From continuous models to algorithms. In International workshop on computer vision and image analysis (pp. 1–6).
Xiao, J., Cheng, H., Sawhney, H., Rao, C., & Isnardi, M. (2006). Bilateral filtering-based optical flow estimation with occlusion detection. In ECCV (pp. 211–224).
Zitnick, C. L., Jojic, N., & Kang, S. B. (2005). Consistent segmentation for optical flow estimation. In ICCV (pp. 1308–1315).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sand, P., Teller, S. Particle Video: Long-Range Motion Estimation Using Point Trajectories. Int J Comput Vis 80, 72–91 (2008). https://doi.org/10.1007/s11263-008-0136-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-008-0136-6