Article

Free access

Distributional similarity models: clustering vs. nearest neighbors

Authors:

Fernando PereiraAuthors Info & Claims

ACL '99: Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics

Pages 33 - 40

https://doi.org/10.3115/1034678.1034694

Published: 20 June 1999 Publication History

Abstract

Distributional similarity is a useful notion in estimating the probabilities of rare joint events. It has been employed both to cluster events according to their distributions, and to directly compute averages of estimates for distributional neighbors of a target event. Here, we examine the tradeoffs between model size and prediction accuracy for cluster-based and nearest neighbors distributional models of unseen events.

References

[1]

Steven Abney. 1996. Partial parsing via finite-state cascades. In Proceedings of the ESSLLI '96 Robust Parsing Workshop.

Digital Library

[2]

L. Douglas Baker and Andrew Kachites McCallum. 1998. Distributional clustering of words for text classification. In 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '98), pages 96--103.

Digital Library

[3]

Peter F. Brown, Vincent J. DellaPietra, Peter V. deSouza, Jennifer C. Lai, and Robert L. Mercer. 1992. Class-based n-gram models of natural language. Computational Linguistics, 18(4): 467--479, December.

Digital Library

[4]

Peter Brucker. 1978. On the complexity of clustering problems. In Rudolf Henn, Bernhard H. Korte, and Werner Oettli, editors, Optimization and Operations Research, number 157 in Lecture Notes in Economics and Mathematical Systems. Springer-Verlag, Berlin.

[5]

Kenneth W. Church and William A. Gale. 1991. A comparison of the enhanced Good-Turing and deleted estimation methods for estimating probabilities of English bigrams. Computer Speech and Language, 5: 19--54.

[6]

Ido Dagan, Shaul Marcus, and Shaul Markovitch. 1995. Contextual word similarity and estimation from sparse data. Computer Speech and Language, 9: 123--152.

[7]

Ido Dagan, Lillian Lee, and Fernando Pereira. 1999. Similarity-based models of word cooccurrence probabilities. Machine Learning, 34(1--3): 43--69.

Digital Library

[8]

Thomas Hofmann, Jan Puzicha, and Michael I. Jordan. 1999. Learning from dyadic data. In Advances in Neural Information Processing Systems 11. MIT Press. To appear.

Digital Library

[9]

Nancy Ide and Jean Veronis. 1998. Introduction to the special issue on word sense disambiguation: The state of the art. Computational Linguistics, 24(1): 1--40, March.

Digital Library

[10]

Frederick Jelinek and Robert L. Mercer. 1980. Interpolated estimation of Markov source parameters from sparse data. In Proceedings of the Workshop on Pattern Recognition in Practice, Amsterdam, May. North Holland.

[11]

Slava M. Katz. 1987. Estimation of probabilities from sparse data for the language model component of a speech recognizer. IEEE Transactions on Acoustics, Speech and Signal Processing, ASSP-35(3): 400--401, March.

[12]

Lillian Lee. 1999. Measures of distributional similarity. In 37th Annual Meeting of the ACL, Somerset, New Jersey. Distributed by Morgan Kaufmann, San Francisco.

Digital Library

[13]

Jianhua Lin. 1991. Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1): 145--151.

Digital Library

[14]

Hermann Ney and Ute Essen. 1993. Estimating 'small' probabilities by leaving-one-out. In Third European Conference On Speech Communication and Technology, pages 2239--2242, Berlin, Germany.

[15]

Fernando C. N. Pereira, Naftali Tishby, and Lillian Lee. 1993. Distributional clustering of English words. In 31st Annual Meeting of the ACL, pages 183--190, Somerset, New Jersey. Association for Computational Linguistics. Distributed by Morgan Kaufmann, San Francisco.

Digital Library

[16]

C. Radhakrishna Rao. 1982. Diversity: Its measurement, decomposition, apportionment and analysis. Sankyhā: The Indian Journal of Statistics, 44(A): 1--22.

[17]

Hinrich Schütze. 1993. Word space. In S. J. Hanson, J. D. Cowan, and C. L. Giles, editors, Advances in Neural Information Processing Systems 5, pages 895--902. Morgan Kaufmann, San Francisco.

Digital Library

Cited By

Zagal JTomuro NLugmayr AFranssila HSotamaa OSafran CAaltonen T(2010)The aesthetics of gameplayProceedings of the 14th International Academic MindTrek Conference: Envisioning Future Media Environments10.1145/1930488.1930492(9-16)Online publication date: 6-Oct-2010
https://dl.acm.org/doi/10.1145/1930488.1930492
Ji HManandhar SKlapaftis I(2009)Cross-lingual predicate cluster acquisition to improve bilingual event extraction by inductive learningProceedings of the Workshop on Unsupervised and Minimally Supervised Learning of Lexical Semantics10.5555/1641968.1641972(27-35)Online publication date: 5-Jun-2009
https://dl.acm.org/doi/10.5555/1641968.1641972
Sze DFu TChung FLuk R(2007)Distributional Similarity Model for Multi-modality Clustering in Social MediaProceedings of the 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops10.5555/1339264.1339705(268-271)Online publication date: 2-Nov-2007
https://dl.acm.org/doi/10.5555/1339264.1339705
Show More Cited By

Distributional similarity models: clustering vs. nearest neighbors
1. Computing methodologies
  1. Artificial intelligence
2. Hardware
  1. Power and energy
    1. Power estimation and optimization

Recommendations

Measures of distributional similarity
ACL '99: Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics

We study distributional similarity measures for the purpose of improving probability estimation for unseen cooccurrences. Our contributions are three-fold: an empirical comparison of a broad range of measures; a classification of similarity functions ...
Distributional clustering of English words
ACL '93: Proceedings of the 31st annual meeting on Association for Computational Linguistics

We describe and evaluate experimentally a method for clustering words according to their distribution in particular syntactic contexts. Words are represented by the relative frequency distributions of contexts in which they appear, and relative entropy ...
The impact of random models on clustering similarity

Clustering is a central approach for unsupervised learning. After clustering is applied, the most fundamental analysis is to quantitatively compare clusterings. Such comparisons are crucial for the evaluation of clustering methods as well as other tasks ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image DL Hosted proceedings

ACL '99: Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics

June 1999

642 pages

ISBN:1558606093

Program Chairs:
Robert Dale,
Ken Church

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 20 June 1999

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 85 of 443 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
353
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)9

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zagal JTomuro NLugmayr AFranssila HSotamaa OSafran CAaltonen T(2010)The aesthetics of gameplayProceedings of the 14th International Academic MindTrek Conference: Envisioning Future Media Environments10.1145/1930488.1930492(9-16)Online publication date: 6-Oct-2010
https://dl.acm.org/doi/10.1145/1930488.1930492
Ji HManandhar SKlapaftis I(2009)Cross-lingual predicate cluster acquisition to improve bilingual event extraction by inductive learningProceedings of the Workshop on Unsupervised and Minimally Supervised Learning of Lexical Semantics10.5555/1641968.1641972(27-35)Online publication date: 5-Jun-2009
https://dl.acm.org/doi/10.5555/1641968.1641972
Sze DFu TChung FLuk R(2007)Distributional Similarity Model for Multi-modality Clustering in Social MediaProceedings of the 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops10.5555/1339264.1339705(268-271)Online publication date: 2-Nov-2007
https://dl.acm.org/doi/10.5555/1339264.1339705
Lin DPantel P(2002)Concept discovery from textProceedings of the 19th international conference on Computational linguistics - Volume 110.3115/1072228.1072372(1-7)Online publication date: 24-Aug-2002
https://dl.acm.org/doi/10.3115/1072228.1072372
Schiffman BMani IConcepcion KWebber B(2001)Producing biographical summariesProceedings of the 39th Annual Meeting on Association for Computational Linguistics10.3115/1073012.1073071(458-465)Online publication date: 6-Jul-2001
https://dl.acm.org/doi/10.3115/1073012.1073071
Even-Zohar YRoth DWiebe J(2000)A classification approach to word predictionProceedings of the 1st North American chapter of the Association for Computational Linguistics conference10.5555/974305.974322(124-131)Online publication date: 29-Apr-2000
https://dl.acm.org/doi/10.5555/974305.974322
Kim KChoi KIida H(2000)Dimension-reduced estimation of word co-occurrence probabilityProceedings of the 38th Annual Meeting on Association for Computational Linguistics10.3115/1075218.1075290(571-578)Online publication date: 3-Oct-2000
https://dl.acm.org/doi/10.3115/1075218.1075290

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents