article

Free access

Class-based n-gram models of natural language

Authors:

Peter F. Brown,

Peter V. deSouza,

Robert L. Mercer,

Vincent J. Della Pietra,

Jenifer C. LaiAuthors Info & Claims

Computational Linguistics, Volume 18, Issue 4

Pages 467 - 479

Published: 01 December 1992 Publication History

PDF eReader

Abstract

We address the problem of predicting a word from previous words in a sample of text. In particular, we discuss n-gram models based on classes of words. We also discuss several statistical algorithms for assigning words to classes based on the frequency of their co-occurrence with other words. We find that we are able to extract classes that have the flavor of either syntactically based groupings or semantically based groupings, depending on the nature of the underlying statistics.

References

[1]

Averbuch, A.; Bahl, L.; Bakis, R.; Brown, P.; Cole, A.; Daggett, G.; Das, S.; Davies, K.; Gennaro, S. De.; de Souza, P.; Epstein, E.; Fraleigh, D.; Jelinek, F.; Moorhead, J.; Lewis, B.; Mercer, R.; Nadas, A.; Nahamoo, D.; Picheny, M.; Shichman, G.; Spinelli, P.; Van Compernolle, D.; and Wilkens, H. (1987). "Experiments with the Tangora 20,000 word speech recognizer." In Proceedings, IEEE International Conference on Acoustics, Speech and Signal Processing. Dallas, Texas, 701--704.

Crossref

Google Scholar

[2]

Bahl, L. R.; Jelinek, F.; and Mercer, R. L. (1983). "A maximum likelihood approach to continuous speech recognition." IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-5(2), 179--190.

Digital Library

Google Scholar

[3]

Baum, L. (1972). "An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process." Inequalities, 3, 1--8.

Google Scholar

[4]

Brown, P. F.; Cocke, J.; DellaPietra, S. A.; DellaPietra, V. J.; Jelinek, F.; Lafferty, J. D.; Mercer, R. L.; and Roossin, P. S. (1990). "A statistical approach to machine translation." Computational Linguistics, 16(2), 79--85.

Digital Library

Google Scholar

[5]

Dempster, A.; Laird, N.; and Rubin, D. (1977). "Maximum likelihood from incomplete data via the EM algorithm." Journal of the Royal Statistical Society, 39(B), 1--38.

Google Scholar

[6]

Feller, W. (1950). An Introduction to Probability Theory and its Applications, Volume I. John Wiley & Sons, Inc.

Google Scholar

[7]

Gallagher, R. G. (1968). Information Theory and Reliable Communication. John Wiley & Sons, Inc.

Digital Library

Google Scholar

[8]

Good, I. (1953). "The population frequencies of species and the estimation of population parameters." Biometrika, 40(3--4), 237--264.

Google Scholar

[9]

Jelinek, F., and Mercer, R. L. (1980). "Interpolated estimation of Markov source parameters from sparse data." In Proceedings, Workshop on Pattern Recognition in Practice, Amsterdam, The Netherlands, 381--397.

Google Scholar

[10]

Kuçera, H., and Francis, W. (1967). Computational Analysis of Present Day American English. Brown University Press.

Google Scholar

[11]

Mays, E.; Damerau, F. J.; and Mercer, R. L. (1990). "Context-based spelling correction." In Proceedings, IBM Natural Language ITL. Paris, France, 517--522.

Google Scholar

Cited By

View all

Yan WMa HYang Z(2025)Segmented Sequence Prediction Using Variable-Order Markov Model EnsembleIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.352297537:3(1425-1438)Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1109/TKDE.2024.3522975
Razaque ABektemyssova GYoo JHariri SKhan MNalgozhina NHwang JKhan M(2025)Review of malicious code detection in data mining applications: challenges, algorithms, and future directionCluster Computing10.1007/s10586-024-05017-x28:3Online publication date: 1-Jun-2025
https://dl.acm.org/doi/10.1007/s10586-024-05017-x
Fang CMiao NSrivastav SLiu JZhang RFang RAsmita Tsang RNazari NWang HHomayoun HBalzarotti DXu W(2024)Large language models for code analysisProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3698947(829-846)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3698947
Show More Cited By

Recommendations

Multilingual stochastic n-gram class language models
ICASSP '96: Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 01

Stochastic language models are widely used in continuous speech recognition systems where a priori probabilities of word sequences are needed. These probabilities are usually given by n-gram word models, estimated on very large training texts. When n ...
Topic-Dependent-Class-Based $n$ -Gram Language Model

A topic-dependent-class (TDC)-based $n$-gram language model (LM) is a topic-based LM that employs a semantic extraction method to reveal latent topic information extracted from noun-noun relations. A topic of a given word sequence is decided on the ...
Multi class-based n-gram language model for new words using web data
ROCOM'11/MUSP'11: Proceedings of the 11th WSEAS international conference on robotics, control and manufacturing technology, and 11th WSEAS international conference on Multimedia systems & signal processing

Out-of-vocabulary (OOV) words cause a serious problem for automatic speech recognition (ASR) system. Not only it will be miss-recognized as an in-vocabulary word with similar phonetics, but the error will also affect nearby words to make errors. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

Computational Linguistics Volume 18, Issue 4

December 1992

175 pages

ISSN:0891-2017

EISSN:1530-9312

Issue’s Table of Contents

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 01 December 1992

Published in COLI Volume 18, Issue 4

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

609
Total Citations
View Citations
9,975
Total Downloads

Downloads (Last 12 months)413
Downloads (Last 6 weeks)118

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yan WMa HYang Z(2025)Segmented Sequence Prediction Using Variable-Order Markov Model EnsembleIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.352297537:3(1425-1438)Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1109/TKDE.2024.3522975
Razaque ABektemyssova GYoo JHariri SKhan MNalgozhina NHwang JKhan M(2025)Review of malicious code detection in data mining applications: challenges, algorithms, and future directionCluster Computing10.1007/s10586-024-05017-x28:3Online publication date: 1-Jun-2025
https://dl.acm.org/doi/10.1007/s10586-024-05017-x
Fang CMiao NSrivastav SLiu JZhang RFang RAsmita Tsang RNazari NWang HHomayoun HBalzarotti DXu W(2024)Large language models for code analysisProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3698947(829-846)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3698947
Esmaeili MFarokhnia Hamedani MZantedeschi DKhalesi C(2024)Election Interference and Online Propaganda Campaigns: Dynamic Interdependencies on Facebook, Google Trends, and the New York TimesACM Transactions on Management Information Systems10.1145/369082815:4(1-23)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3690828
Xu LGuan YXu Y(2024)NGram-Bayes, A Joint Model for Long Distance Context Dependency in Speech RecognitionProceedings of the 2024 7th International Conference on Signal Processing and Machine Learning10.1145/3686490.3686528(258-262)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3686490.3686528
Karas ZBansal AZhang YLi TMcMillan CHuang Y(2024)A Tale of Two Comprehensions? Analyzing Student Programmer Attention during Code SummarizationACM Transactions on Software Engineering and Methodology10.1145/366480833:7(1-37)Online publication date: 26-Aug-2024
https://dl.acm.org/doi/10.1145/3664808
Chang YWang XWang JWu YYang LZhu KChen HYi XWang CWang YYe WZhang YChang YYu PYang QXie X(2024)A Survey on Evaluation of Large Language ModelsACM Transactions on Intelligent Systems and Technology10.1145/364128915:3(1-45)Online publication date: 29-Mar-2024
https://dl.acm.org/doi/10.1145/3641289
Dashti SBardsiri AShahbazzadeh M(2024)Automatic real-word error correction in persian textNeural Computing and Applications10.1007/s00521-024-10045-036:29(18125-18149)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s00521-024-10045-0
Sagum JSagum RDela Fuente MInovero C(2023)Automated Filipino Language Treebank GeneratorProceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval10.1145/3639233.3639238(119-123)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3639233.3639238
Chang YNg SChen JLiang YHsu W(2023)Semantic Template-based Convolutional Neural Network for Text ClassificationACM Transactions on Asian and Low-Resource Language Information Processing10.1145/362782022:11(1-21)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3627820
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Recommendations

Multilingual stochastic n-gram class language models

Topic-Dependent-Class-Based $n$ -Gram Language Model

Multi class-based n-gram language model for new words using web data

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Share

Share this Publication link

Share on social media

Affiliations