Abstract
Deep learning-based approaches have become an important method for media content analysis, and are useful tools for multimedia analytics, as they enable organising and visualising multimedia content items. However, the use of deep neural networks also raises issues of traceability, reproducability and understanding analysis results. The issues are caused by the dependency on training data sets and their possible bias, the change of training data sets over time and the lack of transparent and interoperable representations of models. In this paper we analyse these problems in detail and provide examples. We propose six recommendations to address these issues, which include having interoperable representations of trained models, the identification of training data and models (including versions) and the description of provenance of data sets, models and results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
It is worth noting that this topic was addressed long before the term “fake news” came into everyday use.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
References
Cui, Y., Widom, J.: Lineage tracing for general data warehouse transformations. VLDB J.—Int. J. Very Large Data Bases 12(1), 41–58 (2003)
Esteves, D., Moussallem, D., Neto, C.B., Soru, T., Usbeck, R., Ackermann, M., Lehmann, J.: Mex vocabulary: a lightweight interchange format for machine learning experiments. In: Proceedings of the 11th International Conference on Semantic Systems, pp. 169–176. ACM (2015)
Groth, P., Moreau, L. (eds.) PROV-overview: an overview of the PROV family of documents. Technical report, W3C Working Group Note (2013)
Jónsson, B.Þ., Worring, M., Zahálka, J., Rudinac, S., Amsaleg, L.: Ten research questions for scalable multimedia analytics. In: Tian, Q., Sebe, N., Qi, G.-J., Huet, B., Hong, R., Liu, X. (eds.) MMM 2016. LNCS, vol. 9517, pp. 290–302. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-27674-8_26
Klare, B.F., Burge, M.J., Klontz, J.C., Bruegge, R.W.V., Jain, A.K.: Face recognition performance: role of demographic information. IEEE Trans. Inf. Forensics Secur. 7(6), 1789–1801 (2012)
L’Heureux, A., Grolinger, K., Elyamany, H.F., Capretz, M.A.M.: Machine learning with big data: challenges and approaches. IEEE Access 5, 7776–7797 (2017)
Li, H., Trocan, M.: Deep neural network based single pixel prediction for unified video coding. Neurocomputing 272, 558–570 (2018)
Maali, F., Erickson, J., Archer, P. (eds.) Data Catalog Vocabulary (DCAT). Technical report, W3C Recommendation (2014)
Moreau, L., Groth, P., Cheney, J., Lebo, T., Miles, S.: The rationale of prov. Web Semant.: Sci. Serv. Agents World Wide Web 35, 235–257 (2015)
O’Donoghue, J., Roantree, M.: A toolkit for analysis of deep learning experiments. In: Boström, H., Knobbe, A., Soares, C., Papapetrou, P. (eds.) IDA 2016. LNCS, vol. 9897, pp. 134–145. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46349-0_12
Parker, C.: Unexpected challenges in large scale machine learning. In: Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications, BigMine 2012, pp. 1–6. ACM, New York (2012)
Phillips, P.J., Jiang, F., Narvekar, A., Ayyad, J., O’Toole, A.J.: An other-race effect for face recognition algorithms. ACM Trans. Appl. Percept. 8(2), 14:1–14:11 (2011)
Rubtsov, D., Butakov, S.: A unified format for trained neural network description. In: Proceedings of International Joint Conference on Neural Networks, IJCNN 2001, vol. 4, pp. 2367–2372 (2001)
Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)
Simmhan, Y.L., Plale, B., Gannon, D.: A survey of data provenance techniques. Technical report, Computer Science Department, Indiana University, Bloomington, IN (2005)
Zahálka, J., Rudinac, S., Worring, M.: Analytic quality: evaluation of performance and insight in multimedia collection analysis. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 231–240. ACM (2015)
Acknowledgments
The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 732461, ReCAP (“Real-time Content Analysis and Processing”, http://recap-project.com). The author thanks Sophia Hebenstreit for collecting the data from AWS Rekognition.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Bailer, W. (2018). On the Traceability of Results from Deep Learning-Based Cloud Services. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_50
Download citation
DOI: https://doi.org/10.1007/978-3-319-73603-7_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73602-0
Online ISBN: 978-3-319-73603-7
eBook Packages: Computer ScienceComputer Science (R0)