research-article

An abstractive approach to sentence compression

Authors:

Mirella LapataAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 4, Issue 3

Article No.: 41, Pages 1 - 35

https://doi.org/10.1145/2483669.2483674

Published: 01 July 2013 Publication History

Abstract

In this article we generalize the sentence compression task. Rather than simply shorten a sentence by deleting words or constituents, as in previous work, we rewrite it using additional operations such as substitution, reordering, and insertion. We present an experimental study showing that humans can naturally create abstractive sentences using a variety of rewrite operations, not just deletion. We next create a new corpus that is suited to the abstractive compression task and formulate a discriminative tree-to-tree transduction model that can account for structural and lexical mismatches. The model incorporates a grammar extraction method, uses a language model for coherent output, and can be easily tuned to a wide range of compression-specific loss functions.

References

[1]

Aho, A. V. and Ullman, J. D. 1969. Syntax directed translations and the pushdown assembler. J. Comput. Syst. Sci. 3, 37--56.

Digital Library

[2]

Bannard, C. and Callison-Burch, C. 2005. Paraphrasing with bilingual parallel corpora. In Proceedings of the 43^rd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 597--604.

Digital Library

[3]

Barzilay, R. 2003. Information fusion for multi-document summarization: Paraphrasing and generation. Ph.D. thesis, Columbia University, New York.

Digital Library

[4]

Barzilay, R. and Elhadad, N. 2003. Sentence alignment for monolingual comparable corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. M. Collins and M. Steedman, Eds., Association for Computational Linguistics, 25--32.

Digital Library

[5]

Barzilay, R. and Lee, L. 2003. Learning to paraphrase: An unsupervised approach using multiple sequence alignment. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. 16--23.

Digital Library

[6]

Barzilay, R. and Mckeown, K. 2001. Extracting paraphrases from a parallel corpus. In Proceedings of the 39^th Annual Meeting of the Association for Computational Linguistics. N. Reithinger and G. Satta, Eds., Association for Computational Linguistics, 50--57.

Digital Library

[7]

Barzilay, R. and Mckeown, K. R. 2005. Sentence fusion for multidocument news summarization. Comput. Linguist. 31, 3, 297--327.

[8]

Bhagat, R. and Ravichandran, D. 2008. Large scale acquisition of paraphrases for learning surface patterns. In Proceedings of the 46^th Annual Meeting of the Association for Computational Linguistics with the Human Language Technology Conference. J. D. Moore, S. Teufel, J. Allan, and S. Furui, Eds., Association for Computational Linguistics, 674--682.

[9]

Bikel, D. 2002. Design of a multi-lingual, parallel-processing statistical parsing engine. In Proceedings of the 2^nd International Conference on Human Language Technology Research (HLT'02). Morgan Kaufmann Publishers, San Francisco, 24--27.

Digital Library

[10]

Brown, P. F., Pietra, S. A. D., Pietra, V. J. D., and Mercer, R. L. 1993. Mathematics of statistical machine translation: Parameter estimation. Comput. Linguist. 19, 2, 263--311.

Digital Library

[11]

Callison-Burch, C. 2007. Paraphrasing and translation. Ph.D. thesis, University of Edinburgh, U.K.

[12]

Callison-Burch, C. 2008. Syntactic constraints on paraphrases extracted from parallel corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. M. Lapata and H. T. Ng, Eds., Association for Computational Linguistics, 196--205.

Digital Library

[13]

Chandrasekar, R. and Srinivas, C. D. B. 1996. Motivations and methods for text simplification. In Proceedings of the 16^th International Conference on Computational Linguistics (COLING'96). 1041--1044.

Digital Library

[14]

Chiang, D. 2007. Hierarchical phrase-based translation. Comput. Linguist. 33, 2, 201--228.

Digital Library

[15]

Clarke, J. 2008. Global inference for sentence compression: An integer linear programming approach. Ph.D. thesis, University of Edinburgh.

[16]

Clarke, J. and Lapata, M. 2008. Global inference for sentence compression: An integer linear programming approach. J. Artif. Intell. Res. 31, 273--381.

Digital Library

[17]

Clarke, J. and Lapata, M. 2010. Discourse constraints for document compression. Comput. Linguist. 36, 3, 411--441.

Digital Library

[18]

Cohn, T. and Lapata, M. 2008. Sentence compression beyond word deletion. In Proceedings of the 22^nd International Conference on Computational Linguistics (COLING'08). D. Scott and H. Uszkoreit, Eds., 137--144.

Digital Library

[19]

Cohn, T. and Lapata, M. 2009. Sentence compression as tree transduction. J. Artif. Intell. Res. 34, 637--674.

[20]

Corston-Oliver, S. 2001. Text compaction for display on very small screens. In Proceedings of the NAACL Workshop on Automatic Summarization. J. Goldstein and C.-Y. Lin, Eds., Association for Computational Linguistics, 89--98.

[21]

Daume III, H. and Marcu, D. 2002. A noisy-channel model for document compression. In Proceedings of the 40^th Annual Meeting of the Association for Computational Linguistics. E. Charniak and D. Lin, Eds., Association for Computational Linguistics, 449--456.

Digital Library

[22]

Dorr, B., Zajic, D., and Schwartz, R. 2003. Hedge trimmer: A parse-and-trim approach to headline generation. In Proceedings of the HLT-NAACL Text Summarization Workshop. D. Radev and S. Teufel, Eds., Association for Computational Linguistics, 1--8.

Digital Library

[23]

Dras, M. 1999. Tree adjoining grammar and the reluctant paraphrasing of text. Ph.D. thesis, Macquarie University, Australia.

[24]

Eisner, J. 2003. Learning non-isomorphic tree mappings for machine translation. In Proceedings of the ACL Interactive Poster/Demonstration Sessions. Association for Computational Linguistics, 205--208.

Digital Library

[25]

Fellbaum, C., Ed. 1998. WordNet: An Electronic Database. MIT Press, Cambridge, MA.

[26]

Galley, M., Hopkins, M., Knight, K., and Marcu, D. 2004. What's in a translation rule&quest; In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL'04). Association for Computational Linguistics, 273--280.

[27]

Galley, M. and Mckeown, K. 2007. Lexicalized markov grammars for sentence compression. In Proceedings of the NAACL-HLT Conference of the North American Chapter of the Association for Computational Linguistics, Human Language Technologies. C. Sidner, T. Schultz, M. Stone, and C. Zhai, Eds., Association for Computational Linguistics, 180--187.

[28]

Ganitkevitch, J., Callison-Burch, C., Napoles, C., and Van Durme, B. 2011. Learning sentential paraphrases from bilingual parallel corpora for text-to-text generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1168--1179.

Digital Library

[29]

Grefenstette, G. 1998. Producing intelligent telegraphic text reduction to provide an audio scanning service for the blind. In Proceedings of the AAAI Symposium on Intelligent Text Summarization. E. Hovy and D. R. Radev, Eds., The AAAI Press, 111--117.

[30]

Habash, N. and Lavie, A., Eds. 2006. In Proceedings of the 7^th Conference of the Association for Machine Translation in the Americas (AMTA'06).

[31]

Hearst, M. and Ostendorf, M., Eds. 2003. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics.

Digital Library

[32]

Hirao, T., Suzuki, J., and Isozaki, H. 2009. A syntax-free approach to japanese sentence compression. In Proceedings of the Joint Conference of the 47^th Annual Meeting of the ACL and the 4^th International Joint Conference on Natural Language Processing of the AFNLP. Association for Computational Linguistics. 826--833.

Digital Library

[33]

Hori, C. and Furui, S. 2004. Speech summarization: an approach through word extraction and a method for evaluation. IEICE Trans. Inf. Syst. E87-D, 1, 15--25.

[34]

Huang, L., Knight, K., and Joshi, A. 2006. Statistical syntax-directed translation with extended domain of locality. In Proceedings of the 7^th Conference of the Association for Machine Translation in the Americas (AMTA'06). 66--73.

[35]

Jing, H. 2000. Sentence reduction for automatic text summarization. In Proceedings of the 6^th Applied Natural Language Processing Conference. S. Nirenburg, Ed., Association for Computational Linguistics, PA, 310--315.

Digital Library

[36]

Joachims, T. 2005. A support vector method for multivariate performance measures. In Proceedings of the 22^nd International Conference on Machine Learning. L. D. Raedt and S. Wrobel, Eds., ACM Press, New York, 377--384.

Digital Library

[37]

Keller, F., Gunasekharan, S., Mayo, N., and Corley, M. 2009. Timing accuracy of web experiments: A case study using the WebExp software package. Behav. Res. Methods 41, 1, 1--12.

[38]

Knight, K. and Marcu, D. 2002. Summarization beyond sentence extraction: A probabilistic approach to sentence compression. Artif. Intell. 139, 1, 91--107.

Digital Library

[39]

Knight, K., Ng, H. T., and Oflazer, K., Eds. 2005. Proceedings of the 43^rd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.

Digital Library

[40]

Koehn, P., Och, F. J., and Marcu, D. 2003. Statistical phrase-based translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics. 48--54.

Digital Library

[41]

Liang, P., Taskar, B., and Klein, D. 2006. Alignment by agreement. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (HLT-NAACL'06). R. C. Moore, J. Bilmes, J. Chu-Carroll, and M. Sanderson, Eds., Association for Computational Linguistics, 104--111.

Digital Library

[42]

Lin, C.-Y. 2003. Improving summarization performance by sentence compression — A pilot study. In Proceedings of the 6^th International Workshop on Information Retrieval with Asian Languages. J. Adachi and K.-F. Wong, Eds., Association for Computational Linguistics, 1--8.

Digital Library

[43]

Lin, D. and Pantel, P. 2001. Discovery of inference rules for question answering. Natural Lang. Engin. 7, 4, 342--360.

Digital Library

[44]

Liu, Y., Liu, Q., and Lin, S. 2006. Tree-to-string alignment template for statistical machine translation. In Proceedings of the 21^st International Conference on Computational Linguistics and 44^th Annual Meeting of the Association for Computational Linguistics. O. Kwong, Ed., Association for Computational Linguistics, 609--616.

Digital Library

[45]

Marcu, D. 1999. The automatic construction of large-scale corpora for summarization research. In Proceedings of the 22^nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'99). ACM Press, New York, 137--144.

Digital Library

[46]

Martins, A. F. T. and Smith, N. A. 2009. Summarization with a joint model for sentence extraction and compression. In Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing. Association for Computational Linguistics, 1--9.

Digital Library

[47]

Mcdonald, R. 2006. Discriminative sentence compression with soft syntactic constraints. In Proceedings of the 11^th Conference of the European Chapter of the Association for Computational Linguistics. D. McCarthy and S. Wintner, Eds., Association for Computational Linguistics, 297--304.

[48]

Mitchell, J. and Lapata, M. 2010. Composition in distributional models of semantics. Cogn. Sci. 34, 8, 1388--1429.

[49]

Nguyen, M. L., Shimazu, A., Horiguchi, S., Ho, T. B., and Fukushi, M. 2004. Probabilistic sentence reduction using support vector machines. In Proceedings of the 20^th International Conference on Computational Linguistics (COLING'04). 743--749.

Digital Library

[50]

Och, F. J. and Ney, H. 2004. The alignment template approach to statistical machine translation. Comput. Linguist. 30, 4, 417--449.

Digital Library

[51]

Pado, S., Cer, D., Galley, M., Jurafsky, D., and Manning, C. D. 2009. Measuring machine translation quality as semantic equivalence: A metric based on entailment features. Mach. Transl. 23, 2--3, 181--193.

Digital Library

[52]

Pang, B., Knight, K., and Marcu, D. 2003. Syntax-based alignment of multiple translations: Extracting paraphrases and generating new sentences. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics. 181--188.

Digital Library

[53]

Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40^th Annual Meeting of the Association for Computational Linguistics. E. Charniak and D. Lin, Eds., Association for Computational Linguistics, PA, 311--318.

Digital Library

[54]

Quirk, C., Brockett, C., and Dolan, W. 2004. Monolingual machine translation for paraphrase generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 142--149.

[55]

Riezler, S., King, T. H., Crouch, R., and Zaenen, A. 2003. Statistical sentence condensation using ambiguity packing and stochastic disambiguation methods for lexical-functional grammar. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics. 118--125.

Digital Library

[56]

Shieber, S. and Schabes, Y. 1990. Synchronous tree-adjoining grammars. In Proceedings of the 13^th International Conference on Computational Linguistics (COLING'90). Vol. 3. 253--258.

Digital Library

[57]

Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7^th Conference of the Association for Machine Translation in the Americas (AMTA'06). 223--231.

[58]

Stolcke, A. 2002. SRILM -- An extensible language modeling toolkit. In Proceedings of the 7^th International Conference on Spoken Language Processing. J. H. L. Hansen and B. Pellom, Eds., Casul Prod. Ltd., Denver, CO.

[59]

Su, K.-Y., Su, J., Wiebe, J., and Li, H., Eds. 2009. In Proceedings of the Joint Conference of the 47^th Annual Meeting of the ACL and the 4^th International Joint Conference on Natural Language Processing of the AFNLP. Association for Computational Linguistics.

Digital Library

[60]

Tsochantaridis, I., Joachims, T., Hofmann, T., and Altun, Y. 2005. Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6, 1453--1484.

Digital Library

[61]

Turner, J. and Charniak, E. 2005. Supervised and unsupervised learning for sentence compression. In Proceedings of the 43^rd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 290--297.

Digital Library

[62]

Vandeghinste, V. and Pan, Y. 2004. Sentence compression for automated subtitling: A hybrid approach. In Proceedings of the ACL Workshop on Text Summarization. Association for Computational Linguistics, 89--95.

[63]

Weiss, S. M. and Kulikowski, C. A. 1991. Computer Systems that Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning, and Expert Systems. Morgan Kaufmann, San Fransisco, CA.

Digital Library

[64]

Yamangil, E. and Nelken, R. 2008. Mining wikipedia revision histories for improving sentence compression. In Proceedings of the ACL-HLT Short Papers. Association for Computational Linguistics, 137--140.

Digital Library

[65]

Zajic, D. M., Dorr, B. J., Lin, J., and Schwartz, R. 2007. Multi-candidate reduction: Sentence compression as a tool for document summarization tasks. Inf. Process. Manag. 43, 1549--1570.

Digital Library

[66]

Zhao, S., Lan, X., Liu, T., and Li, S. 2009. Application-driven statistical paraphrase generation. In Proceedings of the Joint Conference of the 47^th Annual Meeting of the ACL and the 4^th International Joint Conference on Natural Language Processing of the AFNLP. Association for Computational Linguistics, 834--842.

Digital Library

Cited By

Lin YFu YLi YCai GZhou A(2021)Self-attention-based neural networks for refining the overlength product titlesMultimedia Tools and Applications10.1007/s11042-021-10908-xOnline publication date: 7-Jun-2021
https://doi.org/10.1007/s11042-021-10908-x
Alva-Manchego FScarton CSpecia L(2020)Data-Driven Sentence Simplification: Survey and BenchmarkComputational Linguistics10.1162/COLI_a_00370(1-87)Online publication date: 2-Jan-2020
https://doi.org/10.1162/COLI_a_00370
ISHIGAKI TTAKAMURA HOKUMURA M(2019)Extractive and Abstractive Summarization for Multiple-sentence Questions複数文質問を対象とした抽出型および生成型要約Journal of Natural Language Processing10.5715/jnlp.26.3726:1(37-58)Online publication date: 15-Mar-2019
https://doi.org/10.5715/jnlp.26.37
Show More Cited By

Index Terms

An abstractive approach to sentence compression
1. Computing methodologies
  1. Symbolic and algebraic manipulation
    1. Symbolic and algebraic algorithms

Recommendations

Sentence Compression by Removing Recursive Structure from Parse Tree
PRICAI '08: Proceedings of the 10th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence

Sentence compression is a task of generating a grammatical short sentence from an original sentence, retaining the most important information. The existing methods of removing the constituents in the parse tree of an original sentence cannot deal with ...
Steps Toward Knowledge-Based Machine Translation

This paper considers the possibilities for knowledge-based automatic text translation in the light of recent advances in artificial intelligence. It is argued that competent translation requires some reasonable depth of understanding of the source text, ...
Research on Element Sub-sentence in Chinese-English Patent Machine Translation
IALP '11: Proceedings of the 2011 International Conference on Asian Language Processing

This paper presents an approach to translate element sub-sentences which widely exist in Chinese patent documents. Element sub-sentence is a kind of language chunk in a sentence in which one part of sub-sentence is the headword and others are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 4, Issue 3

Special Sections on Paraphrasing; Intelligent Systems for Socially Aware Computing; Social Computing, Behavioral-Cultural Modeling, and Prediction

June 2013

435 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/2483669

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2013

Accepted: 01 November 2011

Revised: 01 July 2011

Received: 01 February 2011

Published in TIST Volume 4, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Engineering and Physical Sciences Research Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
307
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lin YFu YLi YCai GZhou A(2021)Self-attention-based neural networks for refining the overlength product titlesMultimedia Tools and Applications10.1007/s11042-021-10908-xOnline publication date: 7-Jun-2021
https://doi.org/10.1007/s11042-021-10908-x
Alva-Manchego FScarton CSpecia L(2020)Data-Driven Sentence Simplification: Survey and BenchmarkComputational Linguistics10.1162/COLI_a_00370(1-87)Online publication date: 2-Jan-2020
https://doi.org/10.1162/COLI_a_00370
ISHIGAKI TTAKAMURA HOKUMURA M(2019)Extractive and Abstractive Summarization for Multiple-sentence Questions複数文質問を対象とした抽出型および生成型要約Journal of Natural Language Processing10.5715/jnlp.26.3726:1(37-58)Online publication date: 15-Mar-2019
https://doi.org/10.5715/jnlp.26.37
Woodsend KLapata M(2018)Text rewriting improves semantic role labelingJournal of Artificial Intelligence Research10.5555/2750423.275042651:1(133-164)Online publication date: 20-Dec-2018
https://dl.acm.org/doi/10.5555/2750423.2750426
Sun FJiang PSun HPei COu WWang XCuzzocrea AAllan JPaton NSrivastava DAgrawal RBroder AZaki MCandan SLabrinidis ASchuster AWang H(2018)Multi-Source Pointer Network for Product Title SummarizationProceedings of the 27th ACM International Conference on Information and Knowledge Management10.1145/3269206.3271722(7-16)Online publication date: 17-Oct-2018
https://dl.acm.org/doi/10.1145/3269206.3271722
Belkebir RGuessoum A(2018)Concept generalization and fusion for abstractive sentence generationExpert Systems with Applications: An International Journal10.1016/j.eswa.2016.01.00753:C(43-56)Online publication date: 29-Dec-2018
https://dl.acm.org/doi/10.1016/j.eswa.2016.01.007
Effendi JSakti SNakamura S(2017)Creation of a multi-paraphrase corpus based on various elementary operations2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment (O-COCOSDA)10.1109/ICSDA.2017.8384465(1-6)Online publication date: Nov-2017
https://doi.org/10.1109/ICSDA.2017.8384465
Belkebir RGuessoum A(2017)TALAA-ATSF: A Global Operation-Based Arabic Text Summarization FrameworkIntelligent Natural Language Processing: Trends and Applications10.1007/978-3-319-67056-0_21(435-459)Online publication date: 18-Nov-2017
https://doi.org/10.1007/978-3-319-67056-0_21
Huang JZhao SDing SWu HSun MWang H(2016)Generating recommendation evidence using translation modelProceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence10.5555/3060832.3061014(2810-2816)Online publication date: 9-Jul-2016
https://dl.acm.org/doi/10.5555/3060832.3061014
Ye JMing ZChua T(2016)Generating Incremental Length Summary Based on Hierarchical Topic Coverage MaximizationACM Transactions on Intelligent Systems and Technology10.1145/28094337:3(1-33)Online publication date: 17-Feb-2016
https://dl.acm.org/doi/10.1145/2809433
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents