Abstract
A semantic video database system, based on an interactive approach that maps low-level features to high-level concepts, is proposed. A database of ontological semantic object models allows the user to get information about specific semantic objects, such as certain actors or other features of a TV drama. The system searches the database for key frames within the video to detect similarities in detailed or “low-level” features, such as the color, area, and position of a specific part of the frame. Since image recognition techniques are limited in their ability to fully identify and compare images, we propose an additional function in which a coarse model is used to recover a greater number of similar key frames, thus providing more relevant results. From these results, the content provider can select relevant key frames interactively; the matched objects in them are then automatically annotated according to descriptions that are added into the model by content provider. Therefore, more complex content can be generated with greater accuracy by using a combination of application-oriented operations. The system has high potential for use in object-based interactive multimedia applications. We also present an object-based video content generation application called the Drama Characters’ Popularity Voting System.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adali S, Candan KS, Chen S-S, Erol K, Subrahmanian VS (1996) Advanced video information system: data structures and query processing. ACM-Springer Multimedia System 4:172–186
Aguierre-Smith TG, Davenport G (1992, November) The stratification system: a design environment for random access video. Proceeding of the Third International Workshop Network and Operating System Support for Digital Audio and Video
Ardizzone E, Cascia ML (1997) Automatic video database indexing and retrieval. Multimed Tools Appl 4(1)
Bargeron D, Gupta A, Grudin J, Sanocki E (1999, May 17) Annotation for streaming video on the web: system design and usage studies. Computer Networks (Netherlands), Elsevier Science, 31(11–16), pp 1139–1153
Chang S-F, Chen W, Meng H J, Sundaram H, Zhong D (1998, September) A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Transactions on Circuits and Systems for Video Technology 8(5):602–615
Comaniciu D, Meer P (1997) Robust analysis of feature spaces: color image segmentation. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp 750–755
Deng Y, Kenney C, Moore MS, Manjunath BS (1999) Peer group filtering and perceptual color image quantization. Proc of IEEE ISCAS 4:21–24
Deng Y, Manjunath BS, Shin H (1999) Color image segmentation. Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, pp 446–451
Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995, September) Query by image and video content: the QBIC system. IEEE Comput Mag 28:23–32
Hacid M-S, Decleir C, Kouloumdjian J (2000, September/October) A database approach for modeling and query video data. IEEE Trans on Knowledge and Data Engineering 12(5):729–750
Hauptmann A, Yan R, Qi Y, Jin R, Christel M, Derthick M, Chen M-Y, Baron R, Lin W-H, Ng TD (2002, November) Video classification and retrieval with the informedia digital video library system. Text retrieval conference (TREC’02), Gaithersburg, Maryland
Hsieh JW, et al (2000) Region-based image retrieval. Proc. ICIP
Kamijo S, Matsushita Y, Ikeuchi K, Sakauchi M (2000, June) Traffic monitoring and accident detection at intersections. IEEE Transaction on ITS 1(2):108–118
Liu Yan, Li Fei (2002) Semantic extration and semanties-based annotation and retrieval for video databases. Multimed Tools Appl 17:5–20
Mehtre BM, Kankanhalli MS, Lee WF (1998) Content-based image retrieval using a composite color-shape approach. Inf Process Manag 34(1):109–120
MPEG-7 Application Document V.3, ISO/IECJTC1/SC29/WG11 N2084, February 1998
Nishida Tsunetoshi, Matsushita Takeshi, Kamijo Shunsuke, Sakauchi Masao (2002) Interactive system of analyzing traffic event statistics based on occlusion robust vehicle tracking method. Proc. of 9th world congress on ITS (CD-ROM)
Oomoto E, Tanaka K (1993) OVID: design and implementation of a video-object database system. IEEE Trans Knowl Data Eng 5(4):62–643
Osawa Y, Sakauchi M (1990,Feb) A new type data structure with homogeneous nodes suitable for a very large spatial database. Proc. of 6th International Conference on Data Engineering, pp 296–303
Rui Y, Huang TS (1998, September) Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans CSVT 8(5):644–655
Smith J R, Chang S-F (1996, November) VisualSEEk: a fully automated content-based image query system. ACM Multimedia 96, Boston, Massachusetts
Tran D A, Hua K A, Vu K (2000, September 4–8) Semantic reasoning based video database systems. In: Proc. of the 11th Int’l Conf. on Database and Expert Systems Applications, pp 41–50
Yoon J, Jayant N (2001) Relevant feedback for semantics for semantics based image retrieval. Proc. ICIP
Zhang W, Wu X, Kamijo S, Yaginuma Y, Sakauchi M (2002, December) Movie content retrieval and semi-automatic annotation based on low-level descriptions. The proceeding of third IEEE pacific rim conference on multimedia, pp 261–269, Springer LNCS 2532
Author information
Authors and Affiliations
Corresponding author
Additional information
W. Zhang graduated in 2004 from Sakauchi Lab, The 3rd Department, Institute of Industrial Science, University of Tokyo, Tokyo, Japan.
Rights and permissions
About this article
Cite this article
Zhang, W., Wu, X., Kamijo, S. et al. Semantic video database system with semi-automatic secondary-content generation capability. Multimed Tools Appl 30, 27–54 (2006). https://doi.org/10.1007/s11042-006-0007-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-006-0007-5