research-article

Extraction and Analysis of Multiple Periodic Motions in Video Sequences

Authors:

Alexia Briassouli,

Narendra AhujaAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 29, Issue 7

Pages 1244 - 1261

https://doi.org/10.1109/TPAMI.2007.1042

Published: 01 July 2007 Publication History

Abstract

The analysis of periodic or repetitive motions is useful in many applications, such as the recognition and classification of human and animal activities. Existing methods for the analysis of periodic motions first extract motion trajectories using spatial information and then determine if they are periodic. These approaches are mostly based on feature matching or spatial correlation, which are often infeasible, unreliable, or computationally demanding. In this paper, we present a new approach, based on the time-frequency analysis of the video sequence as a whole. Multiple periodic trajectories are extracted and their periods are estimated simultaneously. The objects that are moving in a periodic manner are extracted using the spatial domain information. Experiments with synthetic and real sequences display the capabilities of this approach.

References

[1]

Y.A. Ivanov and A.F. Bobick, “Recognition of Visual Activities and Interactions by Stochastic Parsing,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 852-872, Aug. 2000.

Digital Library

[2]

M. Brand and V. Kettnaker, “Discovery and Segmentation of Activities in Video,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 844-851, Aug. 2000.

Digital Library

[3]

C. Lu and N. Ferrier, “Repetitive Motion Analysis: Segmentation and Event Classification,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 258-263, Feb. 2004.

Digital Library

[4]

R. Cutler and L.S. Davis, “Robust Real-Time Periodic Motion Detection, Analysis, and Applications,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 781-796, Aug. 2000.

Digital Library

[5]

D. Cremers, S. Osher, and S. Soatto, “Kernel Density Estimation and Intrinsic Alignment for Knowledge-Driven Segmentation: Teaching Level Sets to Walk,” Pattern Recognition, vol. 3175, pp.36-44, Feb. 2004.

[6]

D. Cremers and S. Soatto, “Probabilistic and Sequential Computation of Optical Flow Using Temporal Coherence,” Int'l J. Computer Vision, vol. 62, no. 3, pp. 249-265, May 2005.

Digital Library

[7]

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High Accuracy Optical Flow Estimation Based on a Theory for Warping,” Proc. Eighth European Conf. Computer Vision, vol. 4, pp. 25-36, May 2004.

[8]

S. Seitz and C.R. Dyer, “View-Invariant Analysis of Cyclic Motion,” Int'l J. Computer Vision, vol. 25, no. 3, pp. 231-251, 1997.

Digital Library

[9]

P. Tsai, M. Shah, K. Keiter, and T. Kasparis, “Cyclic Motion Detection for Motion Based Recognition,” Pattern Recognition, vol. 27, no. 12, pp. 1591-1603, 1994.

[10]

F. Liu and R.W. Picard, “Periodicity, Directionality, and Randomness: Wold Features for Image Modeling and Retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 7, pp.722-733, July 1996.

Digital Library

[11]

R. Polana and R. Nelson, “Detection and Recognition of Periodic, Nonrigid Motion,” Int'l J. Computer Vision, vol. 23, no. 3, pp. 261-282, 1997.

Digital Library

[12]

L. Wang, T. Tan, W. Hu, and H. Ning, “Automatic Gait Recognition Based on Statistical Shape Analysis,” IEEE Trans. Image Processing, vol. 12, no. 9, pp. 1120-1131, Sept. 2003.

Digital Library

[13]

A. Briassouli and N. Ahuja, “Fusion of Frequency and Spatial Domain Information for Motion Analysis,” Proc. 17th Int'l Conf. Pattern Recognition, vol. 2, pp. 175-178, Aug. 2004.

Digital Library

[14]

J.L. Barron, D.J. Fleet, S.S. Beauchemin, and T.A. Burkitt, “Performance of Optical Flow Techniques,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 236-242, June 1992.

[15]

J. Domingo, G. Ayala, and E. Dias, “A Method for Multiple Rigid-Object Motion Segmentation Based on Detection and Consistent Matching of Relevant Points in Image Sequences,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 4, pp. 3021-3024, Apr. 1997.

Digital Library

[16]

P. Milanfar, “Two-Dimensional Matched Filtering for Motion Estimation,” IEEE Trans. Image Processing, vol. 8, no. 3, pp. 438-444, Mar. 1999.

Digital Library

[17]

W. Hoge, D. Mitsouras, F. Rybicki, R. Mulkern, and C.-F. Westin, “Registration of Multi-Dimensional Image Data via Sub-Pixel Resolution Phase Correlation,” Proc. IEEE Int'l Conf. Image Processing, pp. 707-710, Sept. 2003.

[18]

R.E. Blahut, Fast Algorithms for Digital Signal Processing. Addison-Wesley, 1984.

Digital Library

[19]

P. Duhamel and M. Vetterli, “Fast Fourier Transforms: A Tutorial Review,” Signal Processing, vol. 19, pp. 259-299, 1990.

Digital Library

[20]

J.S. Walker, Fast Fourier Transform, second ed. CRC Press, 1996.

[21]

R.W. Young and N.G. Kingsbury, “Frequency-Domain Motion Estimation Using a Complex Lapped Transform,” IEEE Trans. Image Processing, vol. 2, no. 1, pp. 2-17, 1993.

Digital Library

[22]

W. Yu, G. Sommer, and K. Daniilidis, “Multiple Motion Analysis: In Spatial or in Spectral Domain?” Computer Vision and Image Understanding, vol. 90, pp. 129-152, 2003.

Digital Library

[23]

W. Chen, G.B. Giannakis, and N. Nandhakumar, “A Harmonic Retrieval Framework for Discontinuous Motion Estimation,” IEEE Trans. Image Processing, vol. 7, no. 9, pp. 1242-1257, Sept. 1998.

Digital Library

[24]

M. Piccardi, “Background Subtraction Techniques: A Review,” Proc. IEEE Conf. Systems, Man, and Cybernetics, pp. 3099-3104, 2004.

[25]

R. Pless, J. Larson, S. Siebers, and B. Westover, “Evaluation of Local Models of Dynamic Backgrounds,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 1063-1069, June 2003.

[26]

C. Stauffer and W. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. IEEE Conf. Computer Vision Pattern Recognition, pp. 246-252, June 1999.

[27]

D.H.A. Elgammal and L. Davis, “Nonparametric Model for Background Subtraction,” Proc. European Conf. Computer Vision, pp. 751-767, June 2000.

Digital Library

[28]

L. Cohen, “Time-Frequency Distributions—A Review,” Proc. IEEE, vol. 77, no. 7, pp. 941-981, July 1989.

[29]

A.M. Sayeed and D.L. Jones, “Analysis and Synthesis of Multicomponent Signals Using Positive Time-Frequency Distributions,” IEEE Trans. Signal Processing, vol. 47, no. 2, pp. 493-504, Feb. 1999.

Digital Library

[30]

R. Czerwinski and D. Jones, “Adaptive Short-Time Fourier Analysis,” IEEE Signal Processing Letters, vol. 4, no. 2, pp. 42-45, Feb. 1997.

[31]

A.M. Sayeed and D.L. Jones, “Optimal Quadratic Detection and Estimation Using Generalized Joint Signal Representations,” IEEE Trans. Signal Processing, vol. 44, no. 12, pp. 3031-3043, Dec. 1996.

Digital Library

[32]

A.M. Sayeed and D.L. Jones, “Optimal Kernels for Nonstationary Spectral Estimation,” IEEE Trans. Signal Processing, vol. 43, no. 2, pp. 478-490, Feb. 1995.

Digital Library

[33]

F.J. Harris, “On the Use of Windows for Harmonic Analysis with the Discrete Fourier Transform,” Proc. IEEE, vol. 66, pp. 51-83, 1978.

[34]

P. Kootsookos, B. Lovell, and B. Boashash, “A Unified Approach to the STFT, TFDs and Instantaneous Frequency,” IEEE Trans. Signal Processing, vol. 40, no. 8, pp. 1971-1982, 1992.

[35]

I. Djurovic and S. Stankovic, “Estimation of Time-Varying Velocities of Moving Objects by Time-Frequency Representations,” IEEE Trans. Signal Processing, vol. 47, no. 2, pp. 493-504, Feb. 1999.

[36]

S. Stankovic and I. Djurovic, “Motion Parameter Estimation by Using Time-Frequency Representations,” Electronics Letters, vol. 37, no. 24, pp. 1446-1448, Nov. 2001.

[37]

P. Kornprobst, R. Deriche, and G. Aubert, “Nonlinear Operators in Image Restoration,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 325-331, June 1997.

Digital Library

[38]

A. Hirani and T. Totsuka, “Combining Frequency and Spatial Domain Information for Fast Interactive Image Noise Removal,” Proc. ACM SIGGRAPH, pp. 269-276, 1996.

Digital Library

[39]

B. Barkat and K. Abed-Meraim, “A Blind Components Separation Procedure for FM Signal Analysis,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 2, pp. 1425-1428, June 2002.

[40]

S.M. Kay, Modern Spectral Estimation, Theory and Applications. Prentice-Hall, 1988.

[41]

A. Papoulis, Probability, Random Variables, and Stochastic Processes, second ed. McGraw-Hill, 1987.

[42]

D. Jones and T. Parks, “A Resolution Comparison of Several Time-Frequency Representations,” IEEE Trans. Signal Processing, vol. 40, no. 2, pp. 413-420, Feb. 1992.

Digital Library

Cited By

Li CShao MYang QXia S(2023)High‐precision skeleton‐based human repetitive action countingIET Computer Vision10.1049/cvi2.1219317:6(700-709)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1049/cvi2.12193
Li JChen BWang ZLiu H(2023)Full Resolution Repetition CountingIntelligent Robotics and Applications10.1007/978-981-99-6483-3_48(563-574)Online publication date: 5-Jul-2023
https://dl.acm.org/doi/10.1007/978-981-99-6483-3_48
Kamel NKajo IRuichek Y(2019)On Visual Periodicity Estimation Using Singular Value DecompositionJournal of Mathematical Imaging and Vision10.1007/s10851-019-00894-z61:8(1135-1153)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s10851-019-00894-z
Show More Cited By

Recommendations

Estimation of multiple periodic motions from video
ECCV'06: Proceedings of the 9th European conference on Computer Vision - Volume Part I

The analysis of periodic or repetitive motions is useful in many applications, both in the natural and the man-made world. An important example is the recognition of human and animal activities. Existing methods for the analysis of periodic motions ...
Video-guided motion synthesis using example motions

Video taken from a single monocular camera is the most common means of recording human motion. In this article, we present a practical, semiautomatic method for synthesizing a human motion that is guided by such video. After preprocessing an input video,...
Multiscale motion saliency for keyframe extraction from motion capture sequences

Motion capture is an increasingly popular animation technique; however data acquired by motion capture can become substantial. This makes it difficult to use motion capture data in a number of applications, such as motion editing, motion understanding, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 29, Issue 7

July 2007

193 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © 2007.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 July 2007

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li CShao MYang QXia S(2023)High‐precision skeleton‐based human repetitive action countingIET Computer Vision10.1049/cvi2.1219317:6(700-709)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1049/cvi2.12193
Li JChen BWang ZLiu H(2023)Full Resolution Repetition CountingIntelligent Robotics and Applications10.1007/978-981-99-6483-3_48(563-574)Online publication date: 5-Jul-2023
https://dl.acm.org/doi/10.1007/978-981-99-6483-3_48
Kamel NKajo IRuichek Y(2019)On Visual Periodicity Estimation Using Singular Value DecompositionJournal of Mathematical Imaging and Vision10.1007/s10851-019-00894-z61:8(1135-1153)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s10851-019-00894-z
Cuzzocrea AMumolo EMoro AOssowski S(2016)Real-time counting of moving objects in complex environmentsProceedings of the 31st Annual ACM Symposium on Applied Computing10.1145/2851613.2851714(588-595)Online publication date: 4-Apr-2016
https://dl.acm.org/doi/10.1145/2851613.2851714
Rajagopalan SEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Computational behaviour modelling for autism diagnosisProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2532191(361-364)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2532191
Song DXu Y(2010)A low false negative filter for detecting rare bird species from short video segments using a probable observation data set-based EKF methodIEEE Transactions on Image Processing10.1109/TIP.2010.204815119:9(2321-2331)Online publication date: 1-Sep-2010
https://dl.acm.org/doi/10.1109/TIP.2010.2048151
Ribnick EPapanikolopoulos N(2010)3D Reconstruction of Periodic Motion from a Single ViewInternational Journal of Computer Vision10.1007/s11263-010-0334-x90:1(28-44)Online publication date: 1-Oct-2010
https://dl.acm.org/doi/10.1007/s11263-010-0334-x
Ren WSingh SSingh MZhu Y(2009)State-of-the-art on spatio-temporal information-based video retrievalPattern Recognition10.1016/j.patcog.2008.08.03342:2(267-282)Online publication date: 1-Feb-2009
https://dl.acm.org/doi/10.1016/j.patcog.2008.08.033

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents