Article

Computational models for experiences in the arts, and multimedia

Authors:

Harini Sridharan,

Thanassis RikakisAuthors Info & Claims

ETP '03: Proceedings of the 2003 ACM SIGMM workshop on Experiential telepresence

Pages 31 - 44

https://doi.org/10.1145/982484.982490

Published: 07 November 2003 Publication History

Abstract

In this paper, we develop formal computational models for three aspects of experiential systems for browsing media -- (a) context (b) interactivity through hyper-mediation and (c) context evolution using a memory model. Experiential systems deal with the problem of developing context adaptive mechanisms for knowledge acquisition and insight. Context is modeled as a union of graphs whose nodes represent concepts and where the edges represent the semantic relationships. The system context is the union of the contexts of the user, the environment and the media being accessed. We also develop a novel concept dissimilarity. We then develop algorithms to determine the optimal hyperlink for each media element by determining the relationship between the user context and the media. As the user navigates through the hyper-linked sources, the memory model captures the interaction of the user with the hyper-linked sources and updates the user context. Finally, this results in new hyper-links for the media. Our pilot user studies show excellent results, validating our framework.

References

[1]

Merriam Webster Dictionary http://www.m-w.com.

[2]

Google http://www.google.com.

[3]

house{raw} http://isa.asu.edu/projects_house.html.

[4]

OpenCyc http://www.opencyc.org.

[5]

L. AGNIHOTRI, K. DEVARA, T. MCGEE, et al. (2001). Summarization of Video Programs Based on Closed Captioning, SPIE Conf. on Storage and Retrieval in Media Databases, San Jose, CA, 599--607, January 2001.

[6]

B. ARONS (1994). Pitch-Based Emphasis Detection For Segmenting Speech Recordings, Proc. ICSLP, Yokohama Japan, 1931--1934, Sep. 1994.

[7]

R. ASCOT (1966). Behaviourist Art and Cybernetic Vision. Cybernetica 9: pp. 247--264.

[8]

R. BARTHES and S. HEATH (1988). Image, music, text. New York, Noonday Press.

[9]

A. B. BENITEZ, J. R. SMITH and S.-F. CHANG (2000). MediaNet: A Multimedia Information Network for Knowledge Representation, Proceedings of the 2000 SPIE Conference on Internet Multimedia Management Systems (IS&T/SPIE-2000), Boston MA, Nov 6--8, 2000.

[10]

A. B. BENITEZ and S.-F. CHANG (2002). Multimedia Knowledge Integration, Summarization and Evaluation, Proc. of the 2002 International Workshop On Multimedia Data Mining in conjuction with the International Conference on Knowledge Discovery and Data Mining (MDM/KDD-2002), Edmonton, Alberta, Canada, July 2002.

[11]

J. D. BOLTER and R. GRUSIN (1999). Remediation: understanding new media. Cambridge, Mass., MIT Press.

Digital Library

[12]

F. BRAUDEL, R. D. AYALA, P. BRAUDEL, et al. (1998). Les mémoires de la Méditerranée: préhistoire et antiquité. Paris, Editions de Fallois.

[13]

V. BUSH (1945). As We May Think. The Atlantic Monthly. 176: 101--108, http://www.theatlantic.com/unbound/flashbks/computer/bushf.htm.

[14]

J. CAGE (1966). Diary: Audience 1966. A year from Monday, Middletown CT, Weslyan University Press, 1966.

[15]

E. CHANG, K. GOH, G. SYCHAY, et al. (2003). CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines. IEEE Transactions on Circuits and Systems for Video Technology 13(1): 26--38.

Digital Library

[16]

M. CHRISTEL, S. STEVENS, T. KANADE, et al. (1996). Techniques for the Creation and Exploration of Digital Video Libraries. Multimedia Tools and Applications. B. FURHT. Boston MA, Kluwer Academic Publishers.

[17]

M. G. CHRISTEL, M. A. SMITH, C. R. TAYLOR, et al. (1998). Evolving video skims into useful multimedia abstractions, Proceedings of the SIGCHI conference on Human factors in computing systems, Los Angeles, California, United States, 171--178, 1998.

Digital Library

[18]

M. G. CHRISTEL (1999). Visual digests for news video libraries, Proceedings of the 7th ACM international conference on Multimedia, Orlando, Florida, USA, 303--311, 1999.

Digital Library

[19]

M. G. CHRISTEL, A. G. HAUPTMANN, H. D. WACTLAR, et al. (2002). Collages as dynamic summaries for news video, Proceedings of the 10th ACM international conference on Multimedia, Juan Les-Pins, France, 561--569, Dec. 2002.

Digital Library

[20]

B. CLARKSON, N. SAWHNEY and A. PENTLAND (1998). Auditory Context Awareness in Wearable Computing, Workshop on Perceptual User Interfaces, San Francisco, CA, November 5--6, 1998.

[21]

R. W. DEVAUL, S. J. SCHWARTZ and A. PENTLAND (2001) MIThril: context-aware computing for daily life http://www.media.mit.edu/wearables/mithril/MIThril.pdf.

[22]

A. K. DEY and G. D. ABOWD (1999). Towards a Better Understanding of Context and Context-Awareness, Proceedings of the 3rd International Symposium on Wearable Computers, San Francisco, CA, pp. 21--28, October 20--21, 1999.

Digital Library

[23]

A. K. DEY, M. FUTAKAWA, D. SALBER, et al. (1999). The Conference Assistant: Combining Context-Awareness with Wearable Computing, Proceedings of the 3rd International Symposium on Wearable Computers, San Francisco, CA, pp. 21--28, October 20--21, 1999.

Digital Library

[24]

A. K. DEY (2001). Understanding and Using Context. Personal and Ubiquitous Computing Journal 5(1): 4--7.

Digital Library

[25]

R. JAIN (2003). Experiential Computing. Communications of the ACM 46(7): 48--55.

Digital Library

[26]

R. JAIN (2003). Folk Computing. Communications of the ACM 46(4): 27--29.

Digital Library

[27]

M. I. JORDAN (1999). Learning in graphical models. Cambridge, Mass., MIT Press.

Digital Library

[28]

I. KANT (1920). Critique de la raison pure. Paris, Flammarion.

[29]

I. KANT, P. GUYER and A. W. WOOD (1998). Critique of pure reason. Cambridge; New York, Cambridge University Press.

[30]

M. KRUGER (1996). Responsive Environments. Theories and Documents of Contemporary Arts: A sourcebook for Artists' Writings, Berkeley CA, University of California Press, 1996.

[31]

L. MANOVICH (2001). The language of new media. Cambridge, Mass., MIT Press.

[32]

M. MCLUHAN (1994). Understanding media: the extensions of man. Cambridge, Mass., MIT Press.

[33]

G. A. MILLER, R. BECKWITH, C. FELLBAUM, et al. (1993). Introduction to WordNet: An On-line Lexical Database. International Journal of Lexicography 3(4): 235--244.

[34]

M. R. NAPHADE and T. S. HUANG (2001). Recognizing high-level audio-visual concepts using context, Proc. IEEE Conference on Image Processing, 2001, Thessaloniki, Greece, pp. 46--49, Oct. 2001.

[35]

M. R. NAPHADE and T. S. HUANG (2002). Discovering recurrent events in video using unsupervised methods, Proceedings IEEE International Conference on Image Processing. 2002., pp. 13--16, 2002.

[36]

M. R. NAPHADE, C.-Y. LIN and J. R. SMITH (2002). Learning semantic multimedia representations from a small set of examples, Proc. IEEE International Conference on Multimedia and Expo, 2002, Lausanne, Switzerland, pp. 513--516, Aug. 2002.

[37]

T. H. NELSON (1987). Computer lib/dream machines. Redmond, Wash., Tempus Books of Microsoft Press.

Digital Library

[38]

R. PACKER and K. JORDAN (2001). Multimedia: from Wagner to virtual reality. New York, Norton.

Digital Library

[39]

N. J. PAIK (1996). Cybernated Art. Theories and documents of Contemporary Art: A Sourcebook of Artists' Writings, Berkeley, CA, University of California Press, 1996.

[40]

S. PFEIFFER, R. LIENHART, S. FISCHER, et al. (1996). Abstracting Digital Movies Automatically. Journal of Visual Communication and Image Representation 7(4): 345--353.

[41]

J. C. PLATT (2000). AutoAlbum: clustering digital photographs using probabilistic model merging, IEEE Workshop on Content-based Access of Image and Video Libraries, SC. USA, 96--100, Jun. 2000.

Digital Library

[42]

J. RAMAMOORTHY, S. WONG and H. SUNDARAM (2003). Dynamic Adaptive Visualizations. Arts Media and Engineering Center, ASU, AME-TR-2003-03, 2003.

[43]

N. SAWHNEY and C. SCHMANDT. (1999). Nomadic Radio: Scaleable and Contextual Notification for Wearable Audio Messaging, Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Pittsburgh, PA, May 15--20, 1999.

Digital Library

[44]

B. N. T. SCHILIT, M. M. (1994). Disseminating active map information to mobile hosts. IEEE Network 8(5): 22--32.

Digital Library

[45]

B. SHEVADE and H. SUNDARAM (2003). Vidya: An Experiential Annotation System. Arts Media and Engineering Center, Arizona State University, AME-TR-2003-04, 2003.

[46]

H. SRIDHARAN, H. SUNDARAM and T. RIKAKIS (2003). Context, memory and Hyper-mediation in Experiential Systems. Arts Media and Engineering Center, ASU, AME-TR-2003-02, 2003.

[47]

H. SUNDARAM (2002). Segmentation, structure detection and summarization of multimedia sequences: xxviii, 331 leaves, bound., Thesis Ph D--Columbia University 2002.

Digital Library

[48]

H. SUNDARAM and S.-F. CHANG (2002). Video skims: taxonomies and an optimal generation framework, Proc. of the International Conference on Image Processing 2002, Rochester, NY, 21--24, Sep. 2002.

[49]

H. SUNDARAM, L. XIE and S.-F. CHANG (2002). A utility framework for the automatic generation of audio-visual skims, ACM Multimedia 2002, Juan-les-Pins, France, ACM Press, 189--198, Dec. 2002.

Digital Library

[50]

H. SUNDARAM and T. RIKAKIS (2003). An Introduction to Experiential Systems. Arts Media and Engineering Center, ASU, AME-2003-01, 2003.

[51]

S. UCHIHASHI, J. FOOTE, A. GIRGENSOHN, et al. (1999). Video Manga: generating semantically meaningful video summaries, Proceedings of the 7th ACM international conference on Multimedia, Orlando, Florida, USA, 383--392, 1999.

Digital Library

[52]

B. VIOLA (1980). Will there be condominiums in data space? Video 80(5).

[53]

H. D. WACTLAR (1996). The next generation electronic library--capturing the experience. ACM Computing Surveys 28(4es).

Digital Library

[54]

H. D. WACTLAR, M. G. CHRISTEL, A. G. HAUPTMANN, et al. (1999). Informedia Experience-on-Demand: capturing, integrating and communicating experiences across people, time and space. ACM Computing Surveys 31(2es).

Digital Library

[55]

L. YE, C. HU, X. ZHU, et al. (2000). A Unified Framework for Semantics and Feature Based Relevance Feedback in Image Retrieval Systems., In: Proc. ACM MM2000, 31--38.

Digital Library

[56]

B. L. YEO and M. YEUNG (1998). Classification, Simplification and Dynamic Visualization of Scene Transition Graphs for Video Browsing, Proc. SPIE Storage and Retrieval of Image and Video Databases VI, San Jose CA, Feb. 1998., 1998.

[57]

B.-L. YEO and B. LIU (1995). Rapid scene analysis on compressed video. IEEE Transactions on Circuits and Systems for Video Technology 5(6): 533--544.

Digital Library

[58]

M. YEUNG and B.-L. YEO (1996). Time-constrained clustering for segmentation of video into story units, Proceedings of the 13th International Conference on Pattern Recognition, 1996, 375--380, Aug. 1996.

Digital Library

[59]

M. YEUNG and B.-L. YEO (1997). Video visualization for compact presentation and fast browsing of pictorial content. IEEE Transactions on Circuits and Systems for Video Technology 7(5): 771--785.

Digital Library

Cited By

Pereira LSalazar AVergara L(2023)A Comparative Study on Recent Automatic Data Fusion MethodsComputers10.3390/computers1301001313:1(13)Online publication date: 30-Dec-2023
https://doi.org/10.3390/computers13010013
Kim HLew MZhang ZZhang ZJain RZhuang Y(2010)Social webconsciousnessProceedings of the 1st ACM international workshop on Connected multimedia10.1145/1877911.1877915(3-8)Online publication date: 29-Oct-2010
https://dl.acm.org/doi/10.1145/1877911.1877915
Atrey PHossain MEl Saddik AKankanhalli M(2010)Multimodal fusion for multimedia analysisMultimedia Systems10.1007/s00530-010-0182-016:6(345-379)Online publication date: 1-Nov-2010
https://dl.acm.org/doi/10.1007/s00530-010-0182-0
Show More Cited By

Index Terms

Recommendations

Understanding the most satisfying and unsatisfying user experiences: Emotions, psychological needs, and context

The aim of this research was to study the structure of the most satisfying and unsatisfying user experiences in terms of experienced emotions, psychological needs, and contextual factors. 45 university students wrote descriptions of their most ...
An overview of context types within multimedia and social computing
EANN '15: Proceedings of the 16th International Conference on Engineering Applications of Neural Networks (INNS)

The importance of contextual information is widely acknowledged and has become a major topic of interest, investigation, and experimentation for quite some time, generating numerous papers and many scientific works. In this position paper we attempt to ...
Analysing user experience of personal mobile products through contextual factors
MUM '10: Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia

Context plays a critical role in the user experience (UX) of mobile products. In UX studies, context is often seen as a single entity such as mobile context or car context. More fine-grained categorization of contextual factors is needed when we want to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ETP '03: Proceedings of the 2003 ACM SIGMM workshop on Experiential telepresence

November 2003

127 pages

ISBN:1581137753

DOI:10.1145/982484

Conference Chairs:
Hari Sundaram
Arizona State University
,
Gopal Pingali
IBM T.J. Watson Research Center
,
Frank Nack
CWI, The Netherlands

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
843
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pereira LSalazar AVergara L(2023)A Comparative Study on Recent Automatic Data Fusion MethodsComputers10.3390/computers1301001313:1(13)Online publication date: 30-Dec-2023
https://doi.org/10.3390/computers13010013
Kim HLew MZhang ZZhang ZJain RZhuang Y(2010)Social webconsciousnessProceedings of the 1st ACM international workshop on Connected multimedia10.1145/1877911.1877915(3-8)Online publication date: 29-Oct-2010
https://dl.acm.org/doi/10.1145/1877911.1877915
Atrey PHossain MEl Saddik AKankanhalli M(2010)Multimodal fusion for multimedia analysisMultimedia Systems10.1007/s00530-010-0182-016:6(345-379)Online publication date: 1-Nov-2010
https://dl.acm.org/doi/10.1007/s00530-010-0182-0
Bolchini CCurino CQuintarelli ESchreiber FTanca L(2009)Context information for knowledge reshapingInternational Journal of Web Engineering and Technology10.1504/IJWET.2009.0250155:1(88-103)Online publication date: 1-May-2009
https://dl.acm.org/doi/10.1504/IJWET.2009.025015
Joung YZarki MJain R(2009)A user model for personalization services2009 Fourth International Conference on Digital Information Management10.1109/ICDIM.2009.5356766(1-6)Online publication date: Nov-2009
https://doi.org/10.1109/ICDIM.2009.5356766
Bolchini CCurino CQuintarelli ESchreiber FTanca L(2007)A data-oriented survey of context modelsACM SIGMOD Record10.1145/1361348.136135336:4(19-26)Online publication date: 1-Dec-2007
https://dl.acm.org/doi/10.1145/1361348.1361353
Candan KDönderler MRamamoorthy JKim JMase K(2006)Clustering and indexing of experience sequences for popularity-driven recommendationsProceedings of the 3rd ACM workshop on Continuous archival and retrival of personal experences10.1145/1178657.1178671(75-84)Online publication date: 28-Oct-2006
https://dl.acm.org/doi/10.1145/1178657.1178671
Singh RJain R(2006)From Information-Centric to Experiential EnvironmentsInteractive Computation10.1007/3-540-34874-3_13(323-351)Online publication date: 2006
https://doi.org/10.1007/3-540-34874-3_13
Sridharan HMani ASundaram HBrungart JBirchfield D(2005)Context-Aware Dynamic Presentation Synthesis for Exploratory Multimodal Environments2005 IEEE International Conference on Multimedia and Expo10.1109/ICME.2005.1521596(1014-1017)Online publication date: 2005
https://doi.org/10.1109/ICME.2005.1521596
Adams BVenkatesh S(2005)Injection, Detection and Repair of Aesthetics in Home Movies2005 IEEE International Conference on Multimedia and Expo10.1109/ICME.2005.1521353(33-36)Online publication date: 2005
https://doi.org/10.1109/ICME.2005.1521353
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents