Article

Improving query translation for cross-language information retrieval using statistical models

Authors:

Changning HuangAuthors Info & Claims

SIGIR '01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval

Pages 96 - 104

https://doi.org/10.1145/383952.383966

Published: 01 September 2001 Publication History

Abstract

Dictionaries have often been used for query translation in cross-language information retrieval (CLIR). However, we are faced with the problem of translation ambiguity, i.e. multiple translations are stored in a dictionary for a word. In addition, a word-by-word query translation is not precise enough. In this paper, we explore several methods to improve the previous dictionary-based query translation. First, as many as possible, noun phrases are recognized and translated as a whole by using statistical models and phrase translation patterns. Second, the best word translations are selected based on the cohesion of the translation words. Our experimental results on TREC English-Chinese CLIR collection show that these techniques result in significant improvements over the simple dictionary approaches, and achieve even better performance than a high-quality machine translation system.

References

[1]

Adriani, M. (2000). Using statistical term similarity for sense disambiguation in cross-language information retrieval. Information Retrieval. 2, 69-80.]]

Digital Library

[2]

Ballesteros, L., and Croft, W. B. (1997). Phrasal translation and query expansion techniques for cross-language information retrieval. In: Proceedings of the 20 th International Conference on Research and Development in Information Retrieval. 84-91.]]

Digital Library

[3]

Ballesteros, L., and Croft, W. B. (1998). Resolving ambiguity for cross-language retrieval. In Proceedings of the 21 st International Conference on Research and Development in Information Retrieval. Melbourne, Australia.]]

Digital Library

[4]

Brown, P. F., Della Pietra, S. A., Della Pietra, V. J., and Mercer, R.L. (1993). The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2): 263-311]]

Digital Library

[5]

Buckley, C. (1985). Implementation of the SMART information retrieval system, Technical report, #85-686, Cornell University.]]

Digital Library

[6]

Church, K., (1988) A stochastic parts program and noun phrase parser for unrestricted text. In Proceedings of the Second Conference on Applied Natural Language Processing, pages 136-143. Association of Computational Linguistics.]]

Digital Library

[7]

Davis, M. W., and Ogden, W. C. (1997). Free resources and advanced alignment for cross-language text retrieval. In: The Sixth Text Retrieval Conference (TREC-6). NIST, Gaithersbury, MD.]]

[8]

Fagan, J. (1988). Experiments in Automatic Phrase Indexing for Document Retrieval: A Comparison of Syntactic and Non-Syntactic methods. Computer Science, Cornell University.]]

Digital Library

[9]

Gao, J., Wang, H. F., Li, M. and Lee, K. F. (2000). A unified approach to statistical language modeling for Chinese. ICASSP-2000, Istanbul, Turkey, June.]]

[10]

Hiemstra, D., and Jong, F. (1999). Disambiguation strategies for cross-language information retrieval. In: Proceedings of the third European Conference on Research and Advanced Technology for Digital Libraries. 274-293.]]

Digital Library

[11]

Hull, D. (1993). Using statistical testing in the evaluation of retrieval experiments. In: Conference on Research and Development in Information Retrieval, ACM-SIGIR, 1993.]]

Digital Library

[12]

Hull, D. A. (1997). Using structured queries for disambiguation in cross-language information retrieval. In: AAAI Symposium on Cross-Language Text and Speech Retrieval.]]

[13]

Hull, D. A., and Grefenstette, G. (1996). Querying across languages: a dictionary-based approach to multilingual information retrieval. Research and Development in Information Retrieval, pp46-57.]]

Digital Library

[14]

Kowk, K. L. (2000). Exploiting a Chinese-English bilingual wordlist for English-Chinese cross language information retrieval. In: Fifth International Workshop on Information Retrieval with Asian Languages, IRAL-2000. Hong Kong, September 30 to October 1, 2000.]]

Digital Library

[15]

Marcus, M., Marcinkiewicx, M., and Santorini, B. (1993). Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19(2): 313-330.]]

Digital Library

[16]

Nie, J. Y., Simard, M., Isabelle, P., and Durand, R. (1999). Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the web. In: Conference on Research and Development in Information Retrieval, ACM-SIGIR, 1999.]]

Digital Library

[17]

Ramshaw, L. A., and Marcus, M., (1998). Text chunking using transformation-based learning. In Natural Language Processing Using Very large Corpora. Kluwer. Originally appeared in The second workshop on very large corpora WVLC'95, pp.82-94.]]

[18]

van Rijsbergen, D. J. (1979). Information retrieval, 2 nd ed. Butterworths, London.]]

Digital Library

[19]

Xu, J., and Weischedel, R. (2000a). Cross-lingual information retrieval using Hidden Markov models. In: Proceeding of the 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. Hong Kong, October 7-8, 2000.]]

Digital Library

[20]

Xu, J., and Weischedel, R. (2000b). TREC-9 crosslanguage retrieval at BBN. In: The Ninth Text Retrieval Conference (TREC-9). NIST, Gaithersbury, MD.]]

[21]

Xun, E. (1999). Incremental English parsing using combination of statistic and learning methods. Ph.D. thesis. Harbin Institute of Technology, China.]]

[22]

Xun, E., Zhou, M., and Huang, C. (2000). A unified statistical model for the identification of English base NP. In: The 38th Annual Meeting of the Association for Computational Linguistics, Hong Kong 3-6 October.]]

Digital Library

Cited By

Abka APratama MJatmiko W(2021)Cross-Lingual Summarization: English - Bahasa Indonesia2021 6th International Workshop on Big Data and Information Security (IWBIS)10.1109/IWBIS53353.2021.9631862(53-58)Online publication date: 23-Oct-2021
https://doi.org/10.1109/IWBIS53353.2021.9631862
Gemmell CRossetto FDalton JHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Relevance Transformer: Generating Concise Code Snippets with Relevance FeedbackProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401215(2005-2008)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401215
Patel JMakvana KShah P(2019)Cross-lingual Information Retrieval: application and Challenges for Indian Languages2019 IEEE 5th International Conference for Convergence in Technology (I2CT)10.1109/I2CT45611.2019.9033563(1-4)Online publication date: Mar-2019
https://doi.org/10.1109/I2CT45611.2019.9033563
Show More Cited By

Index Terms

Improving query translation for cross-language information retrieval using statistical models
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
    2. Retrieval models and ranking
2. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Program verification

Recommendations

Statistical query translation models for cross-language information retrieval

Query translation is an important task in cross-language information retrieval (CLIR), which aims to determine the best translation words and weights for a query. This article presents three statistical query translation models that focus on the ...
Preordering using a Target-Language Parser via Cross-Language Syntactic Projection for Statistical Machine Translation

When translating between languages with widely different word orders, word reordering can present a major challenge. Although some word reordering methods do not employ source-language syntactic structures, such structures are inherently useful for word ...
Towards a new possibilistic query translation tool for cross-language information retrieval

Approaches of query translation in Cross-Language Information Retrieval (CLIR) have frequently used dictionaries which suffer from translation ambiguity. Besides, a word-by-word query translation is not sufficient. In this paper, we propose, evaluate ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval

September 2001

454 pages

ISBN:1581133316

DOI:10.1145/383952

Chairmen:
Donald H. Kraft
Louisiana State Univ.
,
W. Bruce Croft
University of Massachusetts, (For the Americas)
,
David J. Harper
The Robert Gordon University, (For Europe and Africa)
,
Justin Zobel
RMIT University, (For Asia and Australasia)

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SIGIR01

Sponsor:

SIGIR

SIGIR01: 24th ACM/SIGIR International Conference on Research and Development in Information Retrieval

Louisiana, New Orleans, USA

Acceptance Rates

SIGIR '01 Paper Acceptance Rate 47 of 201 submissions, 23%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

80
Total Citations
View Citations
1,215
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Abka APratama MJatmiko W(2021)Cross-Lingual Summarization: English - Bahasa Indonesia2021 6th International Workshop on Big Data and Information Security (IWBIS)10.1109/IWBIS53353.2021.9631862(53-58)Online publication date: 23-Oct-2021
https://doi.org/10.1109/IWBIS53353.2021.9631862
Gemmell CRossetto FDalton JHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Relevance Transformer: Generating Concise Code Snippets with Relevance FeedbackProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401215(2005-2008)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401215
Patel JMakvana KShah P(2019)Cross-lingual Information Retrieval: application and Challenges for Indian Languages2019 IEEE 5th International Conference for Convergence in Technology (I2CT)10.1109/I2CT45611.2019.9033563(1-4)Online publication date: Mar-2019
https://doi.org/10.1109/I2CT45611.2019.9033563
Li YZhou D(2019)Research on Cross-Language Retrieval Using Bilingual Word Vectors in Different LanguagesData Science10.1007/978-981-15-0118-0_35(454-465)Online publication date: 13-Sep-2019
https://doi.org/10.1007/978-981-15-0118-0_35
Liu PZheng ZSu Q(2018)Cross-Language Information Retrieval Based on Multiple Information2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI)10.1109/WI.2018.00-26(623-626)Online publication date: Dec-2018
https://doi.org/10.1109/WI.2018.00-26
Elayeb BRomdhane WSaoud N(2018)Towards a new possibilistic query translation tool for cross-language information retrievalMultimedia Tools and Applications10.1007/s11042-017-4398-277:2(2423-2465)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1007/s11042-017-4398-2
Kumar VVerma AMittal NGromov S(2018)Anatomy of Preprocessing of Big Data for Monolingual Corpora Paraphrase Extraction: Source Language Sentence SelectionEmerging Technologies in Data Mining and Information Security10.1007/978-981-13-1501-5_43(495-505)Online publication date: 2-Sep-2018
https://doi.org/10.1007/978-981-13-1501-5_43
Chandra GDwivedi S(2017)Query expansion based on term selection for Hindi – English cross lingual IRJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2017.09.002Online publication date: Sep-2017
https://doi.org/10.1016/j.jksuci.2017.09.002
Li CMa TZhou YCheng JXu B(2017)Measuring Word Semantic Similarity Based on Transferred VectorsNeural Information Processing10.1007/978-3-319-70093-9_34(326-335)Online publication date: 24-Oct-2017
https://doi.org/10.1007/978-3-319-70093-9_34
Zhou GZeng ZHuang JHe TPerego RSebastiani FAslam JRuthven IZobel J(2016)Transfer Learning for Cross-Lingual Sentiment Classification with Weakly Shared Deep Neural NetworksProceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval10.1145/2911451.2911490(245-254)Online publication date: 7-Jul-2016
https://dl.acm.org/doi/10.1145/2911451.2911490
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents