Abstract
In automatic video content analysis and film preservation, Shot Boundary Detection (SBD) is a fundamental pre-processing step. While previous research focuses on detecting Abrupt Transitions (AT) as well as Gradual Transitions (GT) in different video genres such as sports movies or news clips only few studies investigate in the detection of shot transitions in historical footage. The main aim of this paper is to create an SBD mechanism inspired by state-of-the-art algorithms which is applied and evaluated on a self-generated historical dataset as well as a publicly available dataset called Clipshots. Therefore, a three-stage pipeline is introduced consisting of a Candidate Frame Range Selection based on the network DeepSBD, Extraction of Convolutional Neural Network (CNN) Features and Similarity Calculation. A combination of pre-trained backbone CNNs such as ResNet, VGG19 and SqueezeNet with different similarity metrics like Cosine Similarity and Euclidean Distance are used and evaluated. The outcome of this paper displays that the proposed algorithm reaches promising results on detecting ATs in historical videos without the need of complex optimization and re-training processes. Furthermore, it points out the main challenges concerning historical footage such as damaged film reels, scratches or splices. The results of this paper contribute a significant base for future research on automatic video analysis of historical videos.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
https://www.vhh-project.eu/en/summary/ - last visit: 2019/05/29.
- 2.
http://efilms.ushmm.org/ - last visited: 2019/05/30.
- 3.
https://github.com/Tangshitao/ClipShots_basline - last visit: 2019/06/09.
- 4.
https://github.com/owenzlz/Shot_Boundary_Detection_Using_CNN_features.git-last visit: 2019/06/11.
References
Adjeroh, D., Lee, M.C., Banda, N., Kandaswamy, U.: Adaptive edge-oriented shot boundary detection. EURASIP J. Image Video Process. 2009(1), 859371 (2009). https://doi.org/10.1155/2009/859371
Baraldi, L., Grana, C., Cucchiara, R.: Shot and scene detection via hierarchical clustering for re-using broadcast video. In: Azzopardi, G., Petkov, N. (eds.) CAIP 2015. LNCS, vol. 9256, pp. 801–811. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23192-1_67. https://www.ebook.de/de/product/25073344/computer_analysis_of_images_and_patterns.html
Bolei, Z., Agata, L., Aditya, K., Aude, O., Antonio, T.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018). https://doi.org/10.1109/TPAMI.2017.2723009
Bouthemy, P., Gelgon, M., Ganansia, F.: A unified approach to shot change detection and camera motion characterization. IEEE Trans. Circ. Syst. Video Technol. 9(7), 1030–1044 (1999). https://doi.org/10.1109/76.795057
Jiang, D., Kim, J.: Video searching and fingerprint detection by using the image query and PlaceNet-based shot boundary detection method. Appl. Sci. 8(10), 1735 (2018). https://doi.org/10.3390/app8101735
Gygli, M.: Ridiculously fast shot boundary detection with fully convolutional neural networks. Biochimica et Biophysica Acta 89, 95–108 (2018). https://doi.org/10.1109/RoEduNet.2013.6511763
Hassanien, A., Elgharib, M.A., Selim, A., Hefeeda, M., Matusik, W.: Large-scale, fast and accurate shot boundary detection through spatio-temporal convolutional neural networks. CoRR abs/1705.03281 (2017). http://arxiv.org/abs/1705.03281
Xu, J., Song, L., Xie, R.: Shot boundary detection using convolutional neural networks. In: VCIP 2016–30th Anniversary of Visual Communication and Image Processing, pp. 1–4 (2017)
Küçüktunç, O., Gudukbay, U., Ulusoy, Ö.: Fuzzy color histogram-based video segmentation. Comput. Vis. Image Underst. 114, 125–134 (2010). https://doi.org/10.1016/j.cviu.2009.09.008
Porter, S., Mirmehdi, M., Thomas, B.: Temporal video segmentation and classification of edit effects. Image Vis. Comput. 21(13–14), 1097–1106 (2003). https://doi.org/10.1016/j.imavis.2003.08.014
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
Seidl, M., Zeppelzauer, M., Mitrović, D., Breiteneder, C.: Gradual transition detection in historic film material - a systematic study. J. Comput. Cult. Herit. 4(3), 10:1–10:18 (2011). https://doi.org/10.1145/2069276.2069279
Smeaton, A.F., Over, P., Doherty, A.R.: Video shot boundary detection: seven years of TRECVid activity. Comput. Vis. Image Underst. 114(4), 411–418 (2010). https://doi.org/10.1016/j.cviu.2009.03.011
Tang, S., Feng, L., Kuang, Z., Chen, Y., Zhang, W.: Fast video shot transition localization with deep structured models. CoRR abs/1808.04234 (2018). http://arxiv.org/abs/1808.04234
Tong, W., Song, L., Yang, X., Qu, H., Xie, R.: CNN-based shot boundary detection and video annotation. In: 2015 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp. 1–5 (2015). https://doi.org/10.1109/BMSB.2015.7177222
Zeppelzauer, M., Mitrović, D., Breiteneder, C.: Archive film material - a novel challenge for automated film analysis. Frames Cin. J. 1(1) (2012). http://publik.tuwien.ac.at/files/PubDat_216640.pdf
Li, Z., Liu, X., Zhang, S.: Shot boundary detection based on multilevel difference of colour histograms. In: Proceedings - 2016 1st International Conference on Multimedia and Image Processing, ICMIP 2016, pp. 15–22 (2016). https://doi.org/10.1109/ICMIP.2016.24
Acknowledgement
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Grant Agreement No. 822670.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Helm, D., Kampel, M. (2019). Shot Boundary Detection for Automatic Video Analysis of Historical Films. In: Cristani, M., Prati, A., Lanz, O., Messelodi, S., Sebe, N. (eds) New Trends in Image Analysis and Processing – ICIAP 2019. ICIAP 2019. Lecture Notes in Computer Science(), vol 11808. Springer, Cham. https://doi.org/10.1007/978-3-030-30754-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-30754-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30753-0
Online ISBN: 978-3-030-30754-7
eBook Packages: Computer ScienceComputer Science (R0)