Abstract
Digital video applications exploit the intrinsic structure of video sequences. In order to obtain and represent this structure for video annotation and indexing tasks, the main initial step is automatic shot partitioning. This paper analyzes the problem of automatic TV commercials recognition, and a new algorithm for scene break detection is then introduced. The structure of each commercial is represented by the set of its key-frames, which are automatically extracted from the video stream. The particular characteristics of commercials make commonly used shot boundary detection techniques obtain worse results than with other video content domains. These techniques are based on individual image features or visual cues, which show significant performance lacks when they are applied to complex video content domains like commercials. We present a new scene break detection algorithm based on the combined analysis of edge and color features. Local motion estimation is applied to each edge in a frame, and the continuity of the color around them is then checked in the following frame. By separately considering both sides of each edge, we rely on the continuous presence of the objects and/or the background of the scene during each shot. Experimental results show that this approach outperforms single feature algorithms in terms of precision and recall.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
J.S. Boreczky and L.A. Rowe, “Comparison of video shot boundary detection techniques,” Journal of Electronic Imaging, Vol. 5, No. 2, pp. 122-128, 1996.
C. Colombo, A. Del Bimbo, and P. Pala, “Retrieval of commercials by video semantics,” in Proc. Computer Vision and Pattern Recognition, 1998, pp. 572-577.
G. Finlayson, M. Drew, and B. Funt, “Colour constancy: Generalized diagonal transforms suffice,” Journal of the Optical Society of America A, Vol. 11, No. 11, pp. 3011-3020, 1994.
U. Gargi, R. Kasturi, and S. Antani, “Performance characterization and comparison of video indexing algorithms,” in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, 1998, pp. 559-565.
J. Huang, S.R. Kumar, M. Mitra, W.-J. Zhu, and R. Zabih, “Image indexing using color correlograms,” in Proc. IEEE Computer Vision and Pattern Recognition Conference, CVPR'97, San Juan, Puerto Rico, 1997.
M. Irani and P. Anandan, “Video indexing based on mosaic representations,” Proceedings of IEEE, 1998.
R. Lienhart, C. Kuhmünch, and W. Effelsberg, “On the detection and recognition of television commercials,” in Proc. IEEE Conf. on Multimedia Computing and Systems, Ottawa, Canada, 1997, pp. 509-516.
A. Nagasaka and Y. Tanaka, “Automatic video indexing and full-video search for object appearances,” in Visual Database Systems II, E. Knuth and L. Wegner (Eds.), Elsevier Science Publishers, 1992, pp. 113-127.
G. Pass and R. Zabih, “Histogram refinement for content-based image retrieval,” in Proc. of the 3rdWorkshop on Applications of Computer Vision, Sarasota, Florida, 1996.
A. Pentland, R.W. Picard, and S. Sclaroff, “Photobook: Content-based manipulation of image databases,” in SPIE Storage and Retrieval for Image and Video Databases II, vol. 2185, San Jose, CA, 1994.
J.M. Sánchez and X. Binefa, “Color normalization for appearance based recognition of video key-frames,” in Proc. International Conference on Pattern Recognition, Barcelona, Spain, 2000, Vol. 1, pp. 815-818.
M.J. Swain and D.H. Ballard, “Color indexing,” International Journal of Computer Vision, Vol. 7, No. 1, pp. 11-32, 1991.
R. Zabih, J. Miller, and K. Mai, “A feature-based algorithm for detecting and classifying scene breaks,” in ACM Conference on Multimedia, San Francisco, California, 1995.
H.J. Zhang, A. Kankanhalli, and S. Smoliar, “Automatic partitioning of video,” Multimedia Systems, Vol. 1, No. 1, pp. 10-28, 1993.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Sánchez, J.M., Binefa, X. & Vitrià, J. Shot Partitioning Based Recognition of TV Commercials. Multimedia Tools and Applications 18, 233–247 (2002). https://doi.org/10.1023/A:1019996817159
Issue Date:
DOI: https://doi.org/10.1023/A:1019996817159