research-article

Quality and relevance of domain-specific search: A case study in mental health

Authors:

Thanh Tin Tang,

Kathy Griffiths,

Helen ChristensenAuthors Info & Claims

Information Retrieval, Volume 9, Issue 2

Pages 207 - 225

https://doi.org/10.1007/s10791-006-7150-5

Published: 01 March 2006 Publication History

Abstract

When searching for health information, results quality can be judged against available scientific evidence: Do search engines return advice consistent with evidence based medicine? We compared the performance of domain-specific health and depression search engines against a general-purpose engine (Google) on both relevance of results and quality of advice. Over 101 queries, to which the term ‘depression’ was added if not already present, Google returned more relevant results than those of the domain-specific engines. However, over the 50 treatment-related queries, Google returned 70 pages recommending for or against a well studied treatment, of which 19 strongly disagreed with the scientific evidence. A domain-specific index of 4 sites selected by domain experts was only wrong in 5 of 50 recommendations. Analysis suggests a tension between relevance and quality. Indexing more pages can give a greater number of relevant results, but selective inclusion can give better quality.

References

[1]

Aggarwal CC, Al-Garawi F, and Yu PS On the design of a learning crawler for topical resource discovery ACM Trans. Inf. Syst. 2001 19 3 286-309

[2]

Andrews G, Issakidis C, Andrews G, Issakidis C, and Carter G Shortfall in mental health service utilisation British Journal of Psychiatry 2001 179 417-25

[3]

American Psychiatric Association Diagnostic and statistical manual of mental disorders 1994 4th edition Washington, DC Author

[4]

Baker L, Wagner TH, Singer S, and Bundorf KM Use of the internet and e-mail for health care information Journal of the American Medical Association 2003 289 18 2400-2406

[5]

Bin L and Lun KC The retrieval effectiveness of medical information on the web International Journal of Medical Informatics 2001 62 155-163

[6]

Chakrabarti S, Berg M and Dom B (1999) Focused crawling: A new approach to topic-specific web resource discovery. In Proceeding of the 8th International World Wide Web Conference (WWW8).

[7]

Cho J, Garcia-Molina H and Page L (1998) Efficient crawling through url ordering. In Proceeding of the Seventh World Wide Web Conference.

[8]

Christensen H, Griffiths KM, and Jorm AF Delivering interventions for depression by using the internet: Randomised controlled trial BMJ 2004 328 7434 265-0

[9]

Clarke M and Oxman AD Cochrane Reviewers’ handbook 4.1.1 2001 Oxford The Cochrane Library

[10]

Eysebach G and Kohler C (2002) How do consumers search for and appraise health information on the world wide web? qualitative study using focus groups, usability tests, and in-depth interviews. BMJ, 324:573-577. http://bmj.com.

[11]

Gretchen BK, Elliot MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, Kanouse DE, Munoz JA, Puyol JA, Lara M, Watkins KE, Yang H, and McGlynn EA Health information on the internet: Accessibility, quality, and readability in English and Spanish JAMA 2001 285 20 2612-2621

[12]

Griffiths KM and Christensen H The quality and accessibility of australian depression sites on the world wide web MJA 2002 176 S97-S104

[13]

Hawking D, Craswell N, Bailey P and Griffiths K (2001) Measuring search engine quality. Information Retrieval, http://es.cmis.csiro.au/pubs/hawking_ir01.pdf, 4(1):33–59

[14]

Hersovici M, Jacovi M, Maarek YS, Pelleg D, Shtalhaim M and Ur S (1998) The shark-search algorithm, an application: Tailored web site mapping. In Proceeding of the Seventh World Wide Web Conference.

[15]

Ilic D, Bessell TL, Silagy CA, and Green S Specialized medical search-engines are no better than general search-engines in sourcing consumer information about androgen deficiency Human Reproduction 2003 18 3 557-561

[16]

Jarvelin K and Kekalainen J Cumulated gain-based evaluation of ir techniques ACM Trans. Inf. Syst. 2002 20 4 422-446

[17]

Jorm AF, Christensen H, Griffiths KM, Korten A, and Rodgers B Help for depression: What works (and what doesn’t) 2001 Canberra, Australia Centre for Mental Health Research

[18]

McCallum A, Nigam K, Rennie J and Sey-more K (1999) Building domain-specific search engines with machine learning technique. In Proceedings of AAAI Spring Symposium on Intelligents Engine in Cyberspace.

[19]

Menczer F, Pant G and Srinivasan P (2001) Evaluating topic-driven web crawlers. In Proceeding of the 24th Annual Intl. ACM SIGIR Conf. On Research and Development in Information Retrieval.

[20]

Mukherjea S (2000). Organizing topic-specific web information. In Proceedings of the eleventh ACM on Hypertext and hypermedia, ACM Press pp. 133–141.

[21]

Murray CJL and Lopez AD The global burden of disease and injury series 1996 Cambridge MA Harvard University Press

[22]

Sellen AJ, Murphy R and Shaw KL (2002). How knowledge workers use the web. In: Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press pp. 227–234.

[23]

Sormunen E (2000) A method for measuring wide range performance of Boolean queries in full-text databases. PhD thesis, University of Tampere. http://acta.uta.fi/teos.phtml?3786.

[24]

Spink A, Jansen BJ, Wolfram D, and Saracevic T From e-sex to e-commerce: Web search changes Computer 2002 35 3 107-109

[25]

Wu G and Li J Comparing web search engine performance in searching consumer health information: Evaluation and recommendations Bull Med Libr Assoc 1999 87 4 456-461

Cited By

Ghenai ASmucker MClarke CO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)A Think-Aloud Study to Understand Factors Affecting Online Health SearchProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377961(273-282)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3377961
Bounhas IElayeb BEvrard FSlimani Y(2015)Information Reliability EvaluationJournal on Computing and Cultural Heritage 10.1145/26938478:3(1-33)Online publication date: 13-Apr-2015
https://dl.acm.org/doi/10.1145/2693847
Hanbury ALupu M(2013)Toward a model of domain-specific searchProceedings of the 10th Conference on Open Research Areas in Information Retrieval10.5555/2491748.2491757(33-36)Online publication date: 15-May-2013
https://dl.acm.org/doi/10.5555/2491748.2491757
Show More Cited By

Index Terms

Quality and relevance of domain-specific search: A case study in mental health

Index terms have been assigned to the content through auto-classification.

Recommendations

Emerging opportunities in Domain Specific Search
ICTCS '16: Proceedings of the Second International Conference on Information and Communication Technology for Competitive Strategies

The cyber space is exploding in rapidity that nobody has ever imagined, it becomes very essential to search the web of cyber space efficiently and effectively. One solution came up as an resolution to this problem is search engines. By now a lot of ...
pybool_ir: A Toolkit for Domain-Specific Search Experiments
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Undertaking research in domain-specific scenarios such as systematic review literature search, legal search, and patent search can often have a high barrier of entry due to complicated indexing procedures and complex Boolean query syntax. Indexing and ...
Domain-Specific Web Search with Keyword Spices

Abstract--Domain-specific Web search engines are effective tools for reducing the difficulty experienced when acquiring information from the Web. Existing methods for building domain-specific Web search engines require human expertise or specific ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Information Retrieval

Information Retrieval Volume 9, Issue 2

Mar 2006

106 pages

ISSN:1386-4564

Issue’s Table of Contents

© Springer Science + Business Media, Inc. 2006.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 March 2006

Accepted: 28 January 2005

Revision received: 25 August 2004

Received: 29 March 2004

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ghenai ASmucker MClarke CO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)A Think-Aloud Study to Understand Factors Affecting Online Health SearchProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377961(273-282)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3377961
Bounhas IElayeb BEvrard FSlimani Y(2015)Information Reliability EvaluationJournal on Computing and Cultural Heritage 10.1145/26938478:3(1-33)Online publication date: 13-Apr-2015
https://dl.acm.org/doi/10.1145/2693847
Hanbury ALupu M(2013)Toward a model of domain-specific searchProceedings of the 10th Conference on Open Research Areas in Information Retrieval10.5555/2491748.2491757(33-36)Online publication date: 15-May-2013
https://dl.acm.org/doi/10.5555/2491748.2491757
Lupu MHanbury A(2013)Patent RetrievalFoundations and Trends in Information Retrieval10.1561/15000000277:1(1-97)Online publication date: 20-Feb-2013
https://dl.acm.org/doi/10.1561/1500000027
Hanbury AHersh WCallan JMaarek YSanderson M(2012)Medical information retrievalProceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval10.1145/2348283.2348537(1191-1192)Online publication date: 12-Aug-2012
https://dl.acm.org/doi/10.1145/2348283.2348537
Sondhi PVydiswaran VZhai C(2012)Reliability prediction of webpages in the medical domainProceedings of the 34th European conference on Advances in Information Retrieval10.1007/978-3-642-28997-2_19(219-231)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1007/978-3-642-28997-2_19

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents