Abstract
As multimodal data becomes easier to record and store, the question arises as to what practical use can be made of archived corpora, and in particular what tools allowing efficient access to it can be built. We use the AMI Meeting Corpus as a case study to build an automatic content linking device, i.e. a system for real-time data retrieval. The corpus provides not only the data repository, but is used also to simulate ongoing meetings for development and testing of the device. The main features of the corpus are briefly described, followed by an outline of data preparation steps prior to indexing, and of the methods for building queries from ongoing meeting discussions, retrieving elements from the corpus and accessing the results. A series of user studies based on prototypes of the content linking device have confirmed the relevance of the concept, and methods for task-based evaluation are under development.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Hart, P.E., Graham, J.: Query-free information retrieval. IEEE Expert: Intelligent Systems and Their Applications 12(5), 32–37 (1997)
Rhodes, B.J., Starner, T.: The Remembrance Agent: A continuously running information retrieval system. In: PAAM 1996 (1st International Conference on Practical Applications of Intelligent Agents and Multi-Agent Technology), London, pp. 486–495 (1996)
Rhodes, B.J., Maes, P.: Just-in-time information retrieval agents. IBM Systems Journal 39(3-4), 685–704 (2000)
Budzik, J., Hammond, K.J.: User interactions with everyday applications as context for just-in-time information access. In: IUI 2000 (5th International Conference on Intelligent User Interfaces), New Orleans, LA (2000)
Henziker, M., Chang, B.W., Milch, B., Brin, S.: Query-free news search. World Wide Web: Internet and Web Information Systems 8, 101–126 (2005)
Popescu-Belis, A., Lalanne, D.: Reference resolution over a restricted domain: References to documents. In: ACL 2004 Workshop on Reference Resolution and its Applications, Barcelona, pp. 71–78 (2004)
Mekhaldi, D., Lalanne, D., Ingold, R.: From searching to browsing through multimodal documents linking. In: ICDAR 2005 (8th International Conference on Document Analysis and Recognition), Seoul, pp. 924–928 (2005)
Nijholt, A., Rienks, R., Zwiers, J., Reidsma, D.: Online and off-line visualization of meeting information and meeting support. The Visual Computer 22(12), 965–976 (2006)
Rhodes, B.J.: The wearable Remembrance Agent: A system for augmented memory. Personal Technologies: Special Issue on Wearable Computing 1, 218–224 (1997)
Franz, A., Milch, B.: Searching the Web by voice. In: Coling 2002 (19th International Conference on Computational Linguistics), Taipei, pp. 11–15 (2002)
Chang, E., Seide, F., Meng, H.M., Chen, Z., Shi, Y., Li, Y.C.: A system for spoken query information retrieval on mobile devices. IEEE Transactions on Speech and Audio Processing 10(8), 531–541 (2002)
Garofolo, J.S., Auzanne, C.G.P., Voorhees, E.M.: The TREC spoken document retrieval track: A success story. In: RIAO 2000 (6th International Conference on Computer-Assisted Information Retrieval), Paris, pp. 1–20 (2000)
Lew, M., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: State of the art and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) 2(1), 1–19 (2006)
AMI Consortium: The AMI Meeting Corpus, http://corpus.amiproject.org (accessed November 18, 2008)
Carletta, J.: Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus. Language Resources and Evaluation Journal 41(2), 181–190 (2007)
Carletta, J., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Karaiskos, V., Kraaij, W., Kronenthal, M., Lathoud, G., Lincoln, M., Lisowska, A., McCowan, I., Post, W., Reidsma, D., Wellner, P.: The AMI Meeting Corpus: A pre-announcement. In: Renals, S., Bengio, S. (eds.) MLMI 2005. LNCS, vol. 3869, pp. 28–39. Springer, Heidelberg (2006)
AMI Consortium: The AMI multimodal meeting database – infrastructure, data and management. Deliverable 2.2, AMI (Augmented Multi-party Interaction) Integrated Project FP 6506811 (August 2005)
Whittaker, S., Tucker, S., Swampillai, K., Laban, R.: Design and evaluation of systems to support interaction capture and retrieval. Personal and Ubiquitous Computing 12(3), 197–221 (2008)
AMI Consortium: Commercial component definition. Deliverable 7.2, AMIDA (Augmented Multi-party Interaction with Distance Access) Integrated Project IST 033812 (November 2007)
Carletta, J., Evert, S., Heid, U., Kilgour, J.: The NITE XML Toolkit: Data model and query language. Language Resources and Evaluation 39(4), 313–334 (2005)
Hain, T., Burget, L., Dines, J., Garau, G., Karafiat, M., Lincoln, M., Vepa, J., Wan, V.: The AMI system for the transcription of speech in meetings. In: ICASSP 2007 (32nd International Conference on Acoustics, Speech, and Signal Processing), Honolulu, pp. 357–360 (2007)
Garner, P.N., Dines, J., Hain, T., El Hannani, A., Karafiat, M., Korchagin, D., Lincoln, M., Wan, V., Zhang, L.: Real-time ASR from meetings. Technical report (2009)
Szoke, I., Schwarz, P., Matejka, P., Burget, L., Karafiat, M., Fapso, M., Cernocky, J.: Comparison of keyword spotting approaches for informal continuous speech. In: Eurospeech 2005 (9th European Conference on Speech Communication and Technology), Lisbon, pp. 633–636 (2005)
Popescu-Belis, A., Boertjes, E., Kilgour, J., Poller, P., Castronovo, S., Wilson, T., Jaimes, A., Carletta, J.: The AMIDA automatic content linking device: Just-in-time document retrieval in meetings. In: Popescu-Belis, A., Stiefelhagen, R. (eds.) MLMI 2008. LNCS, vol. 5237, pp. 272–283. Springer, Heidelberg (2008)
Wellner, P., Flynn, M., Guillemot, M.: Browsing recorded meetings with Ferret. In: Bengio, S., Bourlard, H. (eds.) MLMI 2004. LNCS, vol. 3361, pp. 12–21. Springer, Heidelberg (2005)
AMI Consortium: HCI evaluation of prototype applications. Deliverable 6.3, AMIDA (Augmented Multi-party Interaction with Distance Access) Integrated Project IST-033812 (October 2008)
AMI Consortium: AMIDA proof-of-concept system architecture. Deliverable 6.7, AMIDA (Augmented Multi-party Interaction with Distance Access) Integrated Project IST 033812 (March 2008)
Post, W.M., Elling, E., Cremers, A.H.M., Kraaij, W.: Experimental comparison of multimodal meeting browsers. In: Smith, M.J., Salvendy, G. (eds.) HCII 2007. LNCS, vol. 4558, pp. 118–127. Springer, Heidelberg (2007)
AMI Consortium: Meeting browser evaluation. Deliverable 6.4, AMI (Augmented Multi-party Interaction) Integrated Project FP 6506811 (December 2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Popescu-Belis, A., Carletta, J., Kilgour, J., Poller, P. (2009). Accessing a Large Multimodal Corpus Using an Automatic Content Linking Device. In: Kipp, M., Martin, JC., Paggio, P., Heylen, D. (eds) Multimodal Corpora. MMCorp 2008. Lecture Notes in Computer Science(), vol 5509. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04793-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-04793-0_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04792-3
Online ISBN: 978-3-642-04793-0
eBook Packages: Computer ScienceComputer Science (R0)