research-article

Retrieval of context-aware applications on mobile devices: how to evaluate?

Authors:

Stefano Mizzaro,

Luca VassenaAuthors Info & Claims

IIiX '08: Proceedings of the second international symposium on Information interaction in context

Pages 65 - 71

https://doi.org/10.1145/1414694.1414710

Published: 14 October 2008 Publication History

Abstract

This paper discusses the issue of evaluation of context-aware retrieval applications. We begin by describing MoBe, a specific architecture that allows automatic download and execution of context-aware applications on mobile devices. In MoBe, the most relevant applications are selected by matching context and application descriptors. Since several alternatives for descriptors implementation exist, it is important to compare their effectiveness. To this aim, we develop a TREC-like benchmark, in which the collection is made up by a set of application descriptors and the topics are made up by context descriptors. We then use the benchmark to evaluate the effectiveness of different descriptors components, and of structured and unstructured (i.e., free text) data, finding results that are somehow useful for future MoBe development. We also discuss the issue of the evaluation methodology for highly interactive and novel applications like context-aware retrieval systems, and MoBe in particular.

References

[1]

G. D. Abowd, A. K. Dey, P. J. Brown, N. Davies, M. Smith, and P. Steggles. Towards a better understanding of context and context-awareness. In HUC '99: Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing, pages 304--307, London, UK, 1999. Springer-Verlag.

Digital Library

[2]

M. Baldauf and S. Dustdar. A survey on context-aware systems. International Journal of Ad Hoc and Ubiquitous Computing, 2(4), 2007.

Digital Library

[3]

M. Bazire and P. Brezillon. Understanding context before using it. In CONTEXT 2005, pages 29--40, 2005.

Digital Library

[4]

A. Broder. Computational advertising. In SODA '08: Proceedings of the 19th ACM-SIAM symposium on Discrete algorithms, pages 992--992, Philadelphia, PA, USA, 2008.

Digital Library

[5]

P. J. Brown, J. D. Bovey, and X. Chen. Context-aware applications: from the laboratory to the marketplace. IEEE Personal Communications, 4(5):58--64, 1997.

[6]

P. J. Brown and G. J. F. Jones. Context-aware retrieval: Exploring a new environment for information retrieval and information filtering. Personal Ubiquitous Comput., 5(4):253--263, 2001.

Digital Library

[7]

G. Chen and D. Kotz. A survey of context-aware mobile computing research. Technical Report TR2000-381, Dartmouth College, Nov. 2000.

Digital Library

[8]

P. Coppola, V. Della Mea, L. Di Gaspero, S. Mizzaro, I. Scagnetto, A. Selva, L. Vassena, and P. Zandegiacomo Riziò. Information filtering and retrieving of context-aware applications within the MoBe framework. In International Workshop on Context-Based Information Retrieval (CIR-05), CEUR Workshop Proceedings, 2005.

[9]

A. K. Dey. Context-aware computing: The CyberDesk project. In AAAI 1998 Spring Symposium on Intelligent Environments, pages 51--54, Palo Alto, 1998. AAAI Press.

[10]

D. J. Harper and D. G. Hendry. Evaluation light. In Proceedings of the second MIRA workshop, pages 53--56, 1997. Technical Report TR-1997-2, Department of Computing Science, University of Glasgow, Glasgow. http://www.dcs.gla.ac.uk/mira/workshops/padua_procs/.

[11]

G. Hripcsak and A. S. Rothschild. Agreement, the f-measure, and reliability in information retrieval. Journal of American Medical Informatics Association, 12(3):296--298, 2005.

[12]

G. J. F. Jones and P. J. Brown. Context-aware retrieval for ubiquitous computing environments. In Mobile HCI Workshop on Mobile and Ubiquitous Information Access, pages 227--243, 2003.

[13]

M. Korkea-aho. Context-aware applications survey. Technical Report Internetworking Seminar (Tik-110.551), Helsinki University of Technology, 2000.

[14]

B. J. Rhodes. Just-In-Time Information Retrieval. PhD thesis, MIT Media Laboratory, Cambridge, MA, May 2000.

Digital Library

[15]

B. Schilit and M. Theimer. Disseminating active map information to mobile hosts. IEEE Network, 8(5):22--32, 1994.

Digital Library

[16]

T. Sohn, K. A. Li, W. G. Griswold, and J. D. Hollan. A diary study of mobile information needs. In CHI 2008 Proceedings, pages 433--442, Florence, Italy, Apr. 2008.

Digital Library

[17]

E. M. Voorhees. The philosophy of information retrieval evaluation. In CLEF '01: Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems, pages 355--370, London, UK, 2002. Springer-Verlag.

Digital Library

Cited By

Solvoll T(2019)Mobile Communication in HospitalsHandbook of Research on Strategic Communication, Leadership, and Conflict Management in Modern Organizations10.4018/978-1-5225-8516-9.ch014(278-301)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-8516-9.ch014
Mizzaro SVassena L(2018)A social approach to context-aware retrievalWorld Wide Web10.1007/s11280-011-0116-614:4(377-405)Online publication date: 25-Dec-2018
https://dl.acm.org/doi/10.1007/s11280-011-0116-6
Solvoll T(2016)Mobile Communication in HospitalsHandbook of Research on Effective Communication, Leadership, and Conflict Resolution10.4018/978-1-4666-9970-0.ch029(545-564)Online publication date: 2016
https://doi.org/10.4018/978-1-4666-9970-0.ch029
Show More Cited By

Index Terms

Retrieval of context-aware applications on mobile devices: how to evaluate?
1. Information systems
  1. Information retrieval

Recommendations

Context-aware retrieval going social
FDIA'09: Proceedings of the Third BCS-IRSG conference on Future Directions in Information Access

In this paper we present the Social Context-Aware Browser, a general purpose solution to Web content perusal by means of mobile devices. This is not just a new kind of application, but it is a novel approach for the information access based on the users'...
A social approach to context-aware retrieval

We present a general purpose solution to Web content and services perusal by means of mobile devices, named Social Context-Aware Browser. This is a novel approach for information access based on users' context, that exploits social and collaborative ...
A Graphical Model for Context-Aware Visual Content Recommendation

Existing recommender systems provide an elegant solution to the information overload in current digital libraries such as the Internet archive. Nowadays, the sensors that capture the user's contextual information such as the location and time are become ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

IIiX '08: Proceedings of the second international symposium on Information interaction in context

October 2008

173 pages

ISBN:9781605583105

DOI:10.1145/1414694

Editors:
Pia Borlund
Royal School of Library and Information Science, Denmark
,
Jesper W. Schneider
Royal School of Library and Information Science, Denmark
,
Mounia Lalmas
Queen Mary, University of London, UK
,
Anastasios Tombros
Queen Mary, University of London, UK
,
John Feather
Loughborough University, UK
,
Diane Kelly
University of North Carolina, USA
,
Arjen P. de Vries
CWI, The Netherlands

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Yahoo! Research
Information Retrieval Facility
British Computer Society: BCS

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IIiX08

Sponsor:

British Computer Society

IIiX08: International Interaction in Context Symposium

October 14 - 17, 2008

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 21 of 45 submissions, 47%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
531
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Solvoll T(2019)Mobile Communication in HospitalsHandbook of Research on Strategic Communication, Leadership, and Conflict Management in Modern Organizations10.4018/978-1-5225-8516-9.ch014(278-301)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-8516-9.ch014
Mizzaro SVassena L(2018)A social approach to context-aware retrievalWorld Wide Web10.1007/s11280-011-0116-614:4(377-405)Online publication date: 25-Dec-2018
https://dl.acm.org/doi/10.1007/s11280-011-0116-6
Solvoll T(2016)Mobile Communication in HospitalsHandbook of Research on Effective Communication, Leadership, and Conflict Resolution10.4018/978-1-4666-9970-0.ch029(545-564)Online publication date: 2016
https://doi.org/10.4018/978-1-4666-9970-0.ch029
Solvoll T(2015)Mobile Communication in HospitalsHealthcare Administration10.4018/978-1-4666-6339-8.ch034(688-702)Online publication date: 2015
https://doi.org/10.4018/978-1-4666-6339-8.ch034
Mizzaro SPavan MScagnetto IZanello ISaid ADe Luca EQuercia DBöhmer M(2014)A context-aware retrieval system for mobile applicationsProceedings of the 4th Workshop on Context-Awareness in Retrieval and Recommendation10.1145/2601301.2601305(18-25)Online publication date: 13-Apr-2014
https://dl.acm.org/doi/10.1145/2601301.2601305
Solvoll T(2013)Mobile Communication in HospitalsIntegrated Information and Computing Systems for Natural, Spatial, and Social Sciences10.4018/978-1-4666-2190-9.ch014(287-301)Online publication date: 2013
https://doi.org/10.4018/978-1-4666-2190-9.ch014
Bouidghaghen OTamine L(2012)Spatio-Temporal Based Personalization for Mobile SearchNext Generation Search Engines10.4018/978-1-4666-0330-1.ch017(386-409)Online publication date: 2012
https://doi.org/10.4018/978-1-4666-0330-1.ch017
Ghiata NCretu VMarcu M(2012)Context-aware unified communication2012 7th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI)10.1109/SACI.2012.6250019(297-301)Online publication date: May-2012
https://doi.org/10.1109/SACI.2012.6250019
Menegon DMizzaro SNazzi EVassena L(2011)Evaluating the Context Aware BrowserHandbook of Research on Mobility and Computing10.4018/978-1-60960-042-6.ch001(1-15)Online publication date: 2011
https://doi.org/10.4018/978-1-60960-042-6.ch001
Koshman S(2011)iSchool agendaProceedings of the 2011 iConference10.1145/1940761.1940790(211-216)Online publication date: 8-Feb-2011
https://dl.acm.org/doi/10.1145/1940761.1940790
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents