Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Theoretical Foundations: Formalized Temporal Models for Hyperlinked Multimedia Documents

  • Chapter
  • First Online:
MediaSync

  • 706 Accesses

Abstract

Consistent linking and accurate synchronization of multimedia elements in hypervideos or multimedia documents are essential to provide a good quality of experience to viewers. Temporal models are needed to define relationships and constraints between multimedia elements and create an appealing presentation. However, no commonly used description language for temporal models exists. This makes existing temporal models harder to understand, compare, and transform from one to another temporal model. Using a formal description is more accurate than commonly used textual descriptions or figures of temporal models. This abstract representation makes it is easier to precisely define algorithms and constraints for delivery and buffering, as well as behavior of user and/or multimedia document. The use of a common formalism for all temporal models makes it possible to define synchronization constraints and media management. The same variables and terminology can then be used for describing algorithms that are applied to the documents, for example, to implement pre-fetching or download and cache management in order to increase the quality of experience for users. In this chapter, we give an overview of different existing temporal models for linked and temporally synchronized multimedia documents, like point-based, event-based, or interval-based temporal models. We analyze their common features and formally define their elementary components. We then give formal definitions for each temporal model covering essential features. These can then be used to computationally solve existing problems. We show this by defining basic functions that can be used in algorithms. We also show how user interaction and resulting video behavior can be precisely defined.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 59.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. AFNOR Expert Group: Multimedia synchronization: Definitions and model, input contribution on time variant aspects and synchronization in oda-extensions. ISO IE JTC 1/SC 18/WG3 (1989)

    Google Scholar 

  2. Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983)

    Google Scholar 

  3. Appelt, W., Scheller, A.: HyperODA–going beyond traditional document structures. Comput. Stand. Interfaces 17(1), 13–21 (1995)

    Article  Google Scholar 

  4. Benbernou, S., Makhoul, A., Hacid, M.S., Mostefaoui, A.: A spatio-temporal adaptation model for multimedia presentations. In 7th IEEE International Symposium on Multimedia (ISM’05), pp. 8–pp. Dec (2005)

    Google Scholar 

  5. Blakowski, G., Steinmetz, R.: A media synchronization survey: reference model, specification, and case studies. IEEE J. Sel. Areas Commun. 14(1), 5–35 (1996)

    Article  Google Scholar 

  6. Blakowski, G., Hübel, J., Langrehr, U., Mühlhäuser, M.: Tool support for the synchronization and presentation of distributed multimedia. Comput. Commun. 15(10), 611–618 (1992)

    Article  Google Scholar 

  7. Boll, S., Klas, W., Westermann, U.: A comparison of multimedia document models concerning advanced requirements. Technical Report UIB-1999–01, DBIS (1999)

    Google Scholar 

  8. Boll, S., Klas, W.: ZYX—A Semantic Model for Multimedia Documents and Presentations, pp. 189–209. Springer, US, Boston, MA (1999)

    Google Scholar 

  9. Buchanan, M.C., Zellweger, P.T.: Automatic temporal layout mechanisms. In: Proceedings of the First ACM International Conference on Multimedia, MULTIMEDIA ’93, pp. 341–350. ACM, New York, NY, USA (1993)

    Google Scholar 

  10. Buchanan, M.C., Zellweger, P.T.: Automatic temporal layout mechanisms revisited. ACM Trans. Multimedia Comput. Commun. Appl. 1(1), 60–88 (2005)

    Article  Google Scholar 

  11. Campbell, and A. N. Habermann. The specification of process synchronization by path expressions. In: Operating Systems, Proceedings of an International Symposium, pp. 89–102. Springer, London, UK (1974)

    Google Scholar 

  12. Chen, J.-J., Hang, H.-M.: Source model for transform video coder and its application ii. variable frame rate coding. IEEE Trans. Circuits Syst. Video Technol. 7(2), 299–311 (1997)

    Article  Google Scholar 

  13. Courtiat, J.P., De Oliveira, R.C.: Proving temporal consistency in a new multimedia synchronization model. In: Proceedings of the 4th ACM International Conference on Multimedia, MULTIMEDIA ’93, pp. 141–152. ACM, New York, NY, USA (1996)

    Google Scholar 

  14. Drapeau, G.D.: Synchronization in the maestro multimedia authoring environment. In: Proceedings of the First ACM International Conference on Multimedia, MULTIMEDIA ’93, pp. 331–339. ACM, New York, NY, USA (1993)

    Google Scholar 

  15. Duda, A., Keramane, C.: Structured temporal composition of multimedia data. In: 1995 Proceedings International Workshop on Multi-Media Database Management Systems, pp. 136. Aug 1995

    Google Scholar 

  16. Eijk, P.V., Diaz, M.: Formal Description Technique Lotos: Results of the Esprit Sedos Project. Elsevier Science Inc., New York, NY, USA (1989)

    MATH  Google Scholar 

  17. Euzenat, J., Layaïda, N., Dias, V.: A semantic framework for multimedia document adaptation. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence IJCAI’2003, pp. 31–36. Morgan Kauffman, Acapulco, Mexico, Aug 2003

    Google Scholar 

  18. Fiume, E., Tsichritzis, D., Dami, L.: A temporal scripting language for object-oriented animation. In: EG 1987-Technical Papers Eurographics Association (1987)

    Google Scholar 

  19. Fujikawa, K., Shimojo, S., Matsuura, T., Nishio, S., Miyahara, H.: Multimedia presentation system “harmony” with temporal and activemedia. In: Multimedia for Now and the Future, USENIX (1991)

    Google Scholar 

  20. Gibbs, S.J., Breiteneder, C., Tsichritzis, D.: Audio/video databases: an object-oriented approach. In: Proceedings of the 9th International Conference on Data Engineering, pp. 381–390. IEEE Computer Society, Washington, DC, USA (1993)

    Google Scholar 

  21. Gotthardt, K.: Grundlagen der Informationstechnik. LIT Verlag Münster, Einführungen–Informatik (2001)

    Google Scholar 

  22. Hirzalla, N., Falchuk, B., Karmouch, A.: A temporal model for interactive multimedia scenarios. IEEE Multimedia 2(3), 24–31 (1995)

    Article  Google Scholar 

  23. Hoepner, P.: Synchronisation der Präsentation von Multimedia-Objekten, pp. 455–464. Springer, Berlin Heidelberg (1991)

    Google Scholar 

  24. Illik, J.: Formale Methoden der Informatik: Von der Automatentheorie zu Algorithmen und Datenstrukturen. Expert, Reihe Technik (2009)

    Google Scholar 

  25. ISO/EIC: Iso/iec 13522-6:1998(en) information technology—coding of multimedia and hypermedia information. Website (1998)

    Google Scholar 

  26. ISO/IEC: Information technology—hypermedia/time-based structuring language (hytime) (iso/iec jtc 1/sc 34) (1997)

    Google Scholar 

  27. Kim, W., Kenchammana-hosekote, D., Lim, E.P., Srivastava, J.: Synchronization relation tree: a model for temporal synchronization in multimedia presentations. Technical Report (1992)

    Google Scholar 

  28. Kim, J.W., Kim, Y.-G., Song, H., Kuo, T.-Y., Chung, Y.J., Kuo, C.-C.J.: Tcp-friendly internet video streaming employing variable frame-rate encoding and interpolation. IEEE Trans. Circuits Syst. Video Technol. 10(7), 1164–1177 (2000)

    Article  Google Scholar 

  29. Kretz, F., Colaitis, F.: Standardizing hypermedia information objects. Comm. Mag. 30(5), 60–70 (1992)

    Google Scholar 

  30. Little, T.D.C., Ghafoor, A.: Interval-based conceptual models for time-dependent multimedia data. IEEE Trans. Knowl. Data Eng. 5(4), 551–563 (1993)

    Google Scholar 

  31. Little, T.D.C., Ghafoor, A.: Scheduling of bandwidth-constrained multimedia traffic. In: Proceedings of the 2nd International Workshop on Network and Operating System Support for Digital Audio and Video, pp. 120–131. Springer, London, UK (1992)

    Google Scholar 

  32. Little, T.D.C., Ghafoor, A.: Synchronization and storage models for multimedia objects. IEEE J. Sel. Areas Commun. 8(3), 413–427 (1990)

    Article  Google Scholar 

  33. Little, T.D.C., Ghafoor, A.: Spatio-temporal composition of distributed multimedia objects for value-added networks. Computer 24(10), 42–50 (1991)

    Article  Google Scholar 

  34. Macromind Director: Version 3.0: Overview Manual. MacroMind (1991)

    Google Scholar 

  35. Meixner, B., Einsiedler, C.: Download and cache management for HTML5 hypervideo players. In: Proceedings of the 27th ACM Conference on Hypertext and Social Media HT ’16, pp. 125–136. ACM, New York, NY, USA (2016)

    Google Scholar 

  36. Meixner, B., John, S., Handschigl, C.: Siva suite: framework for hypervideo creation, playback and management. In: Proceedings of the 23rd ACM International Conference on Multimedia, MM ’15, pp. 713–716. ACM, New York, NY, USA (2015)

    Google Scholar 

  37. Meixner, B., Kosch, H.: Interactive non-linear video: definition and xml structure. In: Proceedings of the 2012 ACM Symposium on Document Engineering, DocEng ’12, pp. 49–58. ACM, New York, NY, USA (2012)

    Google Scholar 

  38. Meixner, B.: Annotated interactive non-linear video–software suite, download and cache management. Ph.D. thesis, Universität Passau (2014)

    Google Scholar 

  39. Meixner, B.: Hypervideos and interactive multimedia presentations. ACM Comput. Surv. 50(1), 9:1–9:34 (2017)

    Google Scholar 

  40. Ogawa, R., Harada, H., Kaneko, A.: Scenario-based hypermedia: a model and a system. In: Rizk, A., Streitz, N., Andre, J. (eds.) Hypertext: Concepts, Systems and Applications–Proceeding the First European Conference on Hypertext, pp. 38–51. Cambridge University Press, Cambridge (1990)

    Google Scholar 

  41. Oxford University Press: British & world english. Website https://en.oxforddictionaries.com/ (2017). Accessed 03 May 2017

  42. Pan, F., Lin, X., Rahardja, S., Lim, K.P., Li, Z.G., Wu, D.J., Wu, S.: Proactive frame-skipping decision scheme for variable frame rate video coding. In: 2004 IEEE International Conference on Multimedia and Expo 2004. ICME ’04, Vol. 3, pp. 1903–1906. IEEE (2004)

    Google Scholar 

  43. Poole, L.: Quicktime in motion. MacWorld. 8(9), 154–159 (1991)

    Google Scholar 

  44. Rousseau, F., Duda, A.: An execution architecture for synchronized multimedia presentations, pp. 42–55. Springer, Berlin, Heidelberg (1998)

    Google Scholar 

  45. Shepherd, D., Salmony, M.: Extending osi to support synchronization required by multimedia applications. Comput. Commun. 13(7), 399–406 (1990)

    Article  Google Scholar 

  46. Shue, J.-S., Hsieh, C.-H., Tsai, H.-S., Wang, C.-C.: Variable-rate video codec using frame adaptive finite-state vector quantization. In: 1993 IEEE International Symposium on Circuits and Systems, 1993 ISCAS ’93, vol. 1, pp. 28–31. IEEE (1993)

    Google Scholar 

  47. Steinmetz, R.: Synchronization properties in multimedia systems. IEEE J. Sel. A. Commun. 8(3), 401–412 (1990)

    Article  Google Scholar 

  48. Vazirgiannis, M., Kostalas, I., Sellis, T.: Specifying and authoring multimedia scenarios. IEEE Multimedia 6(3), 24–37 (1999)

    Article  Google Scholar 

  49. Wahl, T., Rothermel, K.: Representing time in multimedia systems. In: Proceedings of the International Conference on Multimedia Computing and Systems, pp. 538–543. May 1994

    Google Scholar 

  50. Wahl, T., Wirag, S., Rothermel, K.: Tiempo: temporal modeling and authoring of interactive multimedia. In: Proceedings of the International Conference on Multimedia Computing and Systems, pp. 274–277. May 1995

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CWI, Science Park 123, 1098 XG, Amsterdam, Netherlands

    Britta Meixner

Authors
  1. Britta Meixner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Britta Meixner .

Editor information

Editors and Affiliations

  1. Departamento de Comunicaciones, Universitat Politècnica de València (UPV) – Campus de Gandia, Valencia, Grao de Gandia, Spain

    Mario Montagud

  2. Centrum Wiskunde & Informatica (CWI), Amsterdam, The Netherlands

    Pablo Cesar

  3. Centrum Wiskunde & Informatica (CWI), Amsterdam, The Netherlands

    Fernando Boronat

  4. Departamento de Comunicaciones, Universitat Politècnica de València (UPV) – Campus de Gandia, Valencia, Grao de Gandia, Spain

    Jack Jansen

Appendices

Appendix

Table 3.4 List of symbols
Full size table

Definitions

 

Multimedia element :

A multimedia element is an image, a video, an audio, a text or any other type of audiovisual medium. It is the atomic object of any multimedia document.

Annotation :

An annotation is additional information displayed with a main medium. It consists of an anchor attaching it to the main medium and a body. The body of an annotation is a multimedia element that can be shown in a player [39].

Static multimedia element :

Static multimedia elements are time independent and always show the same content, like images and/or text.

Continuous multimedia element :

Continuous multimedia elements are time dependent showing/playing different contents over time, like videos or audios.

Hyperlinked media :

Hyperlinked media are multimedia elements which are linked with each other by hyperlinks (as known from hypertext). Static media may be clickable or have clickable areas. Continuous media may provide links depending on the media time.

Media synchronization :

Synchronization of multimedia elements requires mechanisms to prepare the media for display (i.e. pre-fetch, buffering, rendering) and to ensure that timing constraints are met.

Multimedia document :

A multimedia document is a self-contained presentation of linked and synchronized multimedia elements which allows user interaction and navigation. Usually it is about a certain topic.

Point :

A (time) point is a precise moment in time [41]. It is synchronized with a clock.

Event :

An event is something that happens or takes place [41]. It may be triggered by a clock or by a user interaction.

Interval :

A (time) interval is the time between start and end of a time span.

Hypermedia :

Hypermedia is an extension to hypertext providing multimedia facilities, such as those handling sound and video [41]. Keeping the hyperlink structure from hypertext, multimedia elements of different types are added.

Multimedia :

Multimedia uses a variety of artistic or communicative media that are presented in one presentation [41].

Passive multimedia :

Passive multimedia presentations are started and then watched with little to no interaction. Available forms of interaction are starting, pausing, and stopping the presentation.

Active multimedia :

Active multimedia presentations allow more interaction compared to passive multimedia presentations. They may have hyperlinks or other interactive control elements.

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Meixner, B. (2018). Theoretical Foundations: Formalized Temporal Models for Hyperlinked Multimedia Documents. In: Montagud, M., Cesar, P., Boronat, F., Jansen, J. (eds) MediaSync. Springer, Cham. https://doi.org/10.1007/978-3-319-65840-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-65840-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-65839-1

  • Online ISBN: 978-3-319-65840-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation