research-article

Extractive-abstractive summarization with pointer and coverage mechanism

Authors:

Weidong XiaoAuthors Info & Claims

ICBDT '18: Proceedings of the 1st International Conference on Big Data Technologies

Pages 69 - 74

https://doi.org/10.1145/3226116.3226126

Published: 18 May 2018 Publication History

Abstract

Neural sequence-to-sequence models have provided a viable new approach for abstractive text summarization. However, they are facing the challenges of low efficiency and accuracy when dealing with long text: their capability are not enough to handle very long input, they can not reproduce factual details accurately, and they tend to repeat themselves. In this paper, we propose an extractive and abstractive hybrid model. In the extractive part, we construct a graph model and propose a hybrid sentence similarity measure by combining sentence vector and Levenshtein. Then use this measure to rank and extract key sentences and concatenate the key sentences into a shorter text as the input of the summary generator. In the abstractive part, we make two improvement to the standard sequence-to-sequence attentional model. First, we use pointer mechanism to copy words from the source text, which helps the seq2seq generator to handle out-of-vocabulary (OOV) problem. Second, we use coverage mechanism to avoid repetition. We collect a financial news dataset and apply our model to the financial news summarization task, outperforming state-of-the-art method by at least 4.7 ROUGE points.

References

[1]

U Hahn and I Mani. The challenges of automatic summarization. Computer, 33(11):29--36, 2000.

Digital Library

[2]

Julian Kupiec. A trainable document summarizer. In International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 68--73, 1995.

Digital Library

[3]

Yishu Miao and Phil Blunsom. Language as a latent variable: Discrete generative models for sentence compression. pages 319--328, 2016.

[4]

Horacio Saggion and Thierry Poibeau. Automatic text summarization: Past, present and future. Theory & Applications of Natural Language Processing, pages 3--21, 2013.

[5]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. 4:3104--3112, 2014.

Digital Library

[6]

Sumit Chopra, Michael Auli, and Alexander M. Rush. Abstractive sentence ummarization with attentive recurrent neural networks. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 93--98, 2016.

[7]

Ramesh Nallapati, Bowen Zhou, Cicero Nogueira Dos Santos, Caglar Gulcehre, and Bing Xiang. Abstractive text summarization using sequence-tosequence rnns and beyond. 2016.

[8]

Alexander M. Rush, Sumit Chopra, and Jason Weston. A neural attention model for abstractive sentence summarization. Computer Science, 2015.

[9]

Haitao Mi, Baskaran Sankaran, Zhiguo Wang, and Abe Ittycheriah. Coverage embedding models for neural machine translation. 2016.

[10]

Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. Modeling coverage for neural machine translation. pages 76--85, 2016.

[11]

Quoc V Le and Tomas Mikolov. Distributed representations of sentences and documents. 4:II-1188, 2014.

Digital Library

[12]

V. I Levenshtein. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10(1):707--710, 1966.

[13]

Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. Computer Science, 2015.

[14]

Dragomir Radev, Timothy Allison, Sasha Blairgoldensohn, John Blitzer, Arda Celebi, Stanko Dimitrov, Elliott Drabek, Ali Hakim, Wai Lam, and Danyu Liu. Mead - a platform for multidocument multilingual text summarization. 2004.

[15]

Ani Nenkova, Lucy Vanderwende, and Kathleen Mckeown. A compositional context sensitive multi-document summarizer:exploring the factors that influence summarization. pages 573--580, 2006.

Digital Library

[16]

Erkan, Radev, and R Dragomir. Lexrank: graph-based lexical centrality as salience in text summarization. Journal of Qiqihar Junior Teachers College, 22:2004, 2011.

[17]

Rada Mihalcea and Paul Tarau. Textrank: Bringing order into texts. Unt Scholarly Works, pages 404--411, 2004.

[18]

L Page. The pagerank citation ranking : Bringing order to the web. Stanford Digital Libraries Working Paper, 9(1):1--14, 1998.

[19]

Sho Takase, Jun Suzuki, Naoaki Okazaki, Tsutomu Hirao, and Masaaki Nagata. Neural headline generation on abstract meaning representation. In Conference on Empirical Methods in Natural Language Processing, pages 1054--1059, 2016.

[20]

Marc' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. Computer Science, 2015.

[21]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. Computer Science, 2014.

[22]

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. Computer Science, pages 2048--2057, 2015.

Digital Library

[23]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 26:3111--3119, 2013.

Digital Library

[24]

Jun Suzuki and Masaaki Nagata. Rnn-based encoder-decoder approach with word frequency estimation. 2016.

[25]

Baotian Hu, Qingcai Chen, and Fangze Zhu. Lcsts: A large scale chinese short text summarization dataset. Computer Science, 2015.

[26]

Carlos Flick. Rouge: A package for automatic evaluation of summaries. In The Workshop on Text Summarization Branches Out, page 10, 2004.

[27]

Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K Li. Incorporating copying mechanism in sequence-to-sequence learning. pages 1631--1640, 2016.

[28]

Chris D. Paice. Constructing literature abstracts by computer: Techniques and prospects. Information Processing & Management, 26(1):171--186, 1990.

Digital Library

Cited By

Wang YZhou YWang MChen ZCai ZChen JLeung V(2024)Multidocument Aspect Classification for Aspect-Based Abstractive SummarizationIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.325272311:1(1483-1492)Online publication date: Feb-2024
https://doi.org/10.1109/TCSS.2023.3252723
Mou SXue QChen XChen JTakashima RTakiguchi TAriki Y(2024)Prefix tuning with prompt augmentation for efficient financial news summarizationJournal of Computational Social Science10.1007/s42001-024-00352-w8:1Online publication date: 26-Dec-2024
https://doi.org/10.1007/s42001-024-00352-w
Li HPeng QMou XWang YZeng ZBashir M(2023)Abstractive Financial News Summarization via Transformer-BiLSTM Encoder and Graph Attention-Based DecoderIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2023.330447331(3190-3205)Online publication date: 2023
https://doi.org/10.1109/TASLP.2023.3304473
Show More Cited By

Index Terms

Extractive-abstractive summarization with pointer and coverage mechanism
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Information extraction

Recommendations

Abstractive Summarization Improved by WordNet-Based Extractive Sentences
Natural Language Processing and Chinese Computing
Abstract
Recently, the seq2seq abstractive summarization models have achieved good results on the CNN/Daily Mail dataset. Still, how to improve abstractive methods with extractive methods is a good research direction, since extractive methods have their ...
Hybrid multi-document summarization using pre-trained language models
Abstract
Abstractive multi-document summarization is a type of automatic text summarization. It obtains information from multiple documents and generates a human-like summary from them. In this paper, we propose an abstractive multi-document ...
Highlights
- Introducing a multi-document summarizer, called HMSumm, based on pre-trained methods.
Assessing Abstractive and Extractive Methods for Automatic News Summarization
DocEng '24: Proceedings of the ACM Symposium on Document Engineering 2024

Automatic Text Summarization (ATS) is a research area that originated in the late 1950s and has gained increasing importance with the surge of text data available today. ATS approaches are generally classified into extractive and abstractive methods. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICBDT '18: Proceedings of the 1st International Conference on Big Data Technologies

May 2018

144 pages

ISBN:9781450364270

DOI:10.1145/3226116

Conference Chairs:
Chen Wei
Zhejiang University, China
,
Xiangxian Chen
Zhejiang University, China
,
William Wei Song
Dalarna University, Sweden
,
Program Chairs:
Qigang Gao
Dalhousie University, Canada
,
George Carutasu
Romanian American University, Romania

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICBDT '18

ICBDT '18: 2018 International Conference on Big Data Technologies

May 18 - 20, 2018

Hangzhou, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
200
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YZhou YWang MChen ZCai ZChen JLeung V(2024)Multidocument Aspect Classification for Aspect-Based Abstractive SummarizationIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.325272311:1(1483-1492)Online publication date: Feb-2024
https://doi.org/10.1109/TCSS.2023.3252723
Mou SXue QChen XChen JTakashima RTakiguchi TAriki Y(2024)Prefix tuning with prompt augmentation for efficient financial news summarizationJournal of Computational Social Science10.1007/s42001-024-00352-w8:1Online publication date: 26-Dec-2024
https://doi.org/10.1007/s42001-024-00352-w
Li HPeng QMou XWang YZeng ZBashir M(2023)Abstractive Financial News Summarization via Transformer-BiLSTM Encoder and Graph Attention-Based DecoderIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2023.330447331(3190-3205)Online publication date: 2023
https://doi.org/10.1109/TASLP.2023.3304473
Ahmed Zeyad ABiradar A(2023)Assessing BigBirdPegasus and BART Performance in Text Summarization: Identifying Right Methods2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN)10.1109/ICPCSN58827.2023.00297(1773-1778)Online publication date: Jun-2023
https://doi.org/10.1109/ICPCSN58827.2023.00297
Vanetik NPodkaminer ELitvak M(2023)Summarizing Financial Reports with Positional Language Model2023 IEEE International Conference on Big Data (BigData)10.1109/BigData59044.2023.10386704(2877-2883)Online publication date: 15-Dec-2023
https://doi.org/10.1109/BigData59044.2023.10386704
Vivek ADevi V(2023)SumBART - An Improved BART Model for Abstractive Text SummarizationNeural Information Processing10.1007/978-981-99-1639-9_26(313-323)Online publication date: 15-Apr-2023
https://doi.org/10.1007/978-981-99-1639-9_26
Vanetik NLitvak MKrimberg S(2022)Summarization of financial reports with TIBERMachine Learning with Applications10.1016/j.mlwa.2022.1003249(100324)Online publication date: Sep-2022
https://doi.org/10.1016/j.mlwa.2022.100324

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten