logo

SCIENCE CHINA Information Sciences, Volume 63 , Issue 8 : 182103(2020) https://doi.org/10.1007/s11432-019-2657-8

Topic-sensitive neural headline generation

More info
  • ReceivedMar 12, 2019
  • AcceptedSep 25, 2019
  • PublishedJul 15, 2020

Abstract

Neural models are being widely applied for text summarization, including headline generation, and are typically trained using a set of document-headline pairs. In a large document set, documents can usually be grouped into various topics, and documents within a certain topic may exhibit specific summarization patterns. Most existing neural models, however, have not taken the topic information of documents into consideration. This paper categorizes documents into multiple topics, since documents within the same topic have similar content and share similar summarization patterns. By taking advantage of document topic information, this study proposes a topic-sensitive neural headline generation model (TopicNHG). It is evaluated on a real-world dataset, large scale Chinese short text summarization dataset. Experimental results show that it outperforms several baseline systems on each topic and achieves comparable performance with the state-of-the-art system. This indicates that TopicNHG can generate more accurate headlines guided by document topics.


References

[1] Edmundson H P. New Methods in Automatic Extracting. J ACM, 1969, 16: 264-285 CrossRef Google Scholar

[2] Sutskever I, Vinyals O, Le Q V. Sequence to sequence learning with neural networks. In: Proceedings of NIPS 2014. Google Scholar

[3] Cheng J, Lapata M. Neural summarization by extracting sentences and words. In: Proceedings of ACL 2016. Google Scholar

[4] Nallapati R, Zhai F, Zhou B. Summarunner: a recurrent neural network based sequence model for extractive summarization of documents. In: Proceedings of AAAI 2017. Google Scholar

[5] Tan J, Wan X, Xiao J. Abstractive document summarization with a graph-based attentional neural model. In: Proceedings of ACL 2017. Google Scholar

[6] Rush A M, Chopra S, Weston J. A neural attention model for abstractive sentence summarization. In: Proceedings of EMNLP 2015. Google Scholar

[7] Gu J, Lu Z, Li H, et al. Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of ACL 2016. Google Scholar

[8] Zhou Q, Yang N, Wei F, et al. Selective encoding for abstractive sentence summarization. In: Proceedings of ACL 2017. Google Scholar

[9] Cao Z, Li W, Li S, et al. Retrieve, rerank and rewrite: soft template based neural summarization. In: Proceedings of ACL 2018. Google Scholar

[10] Jacobs R A, Jordan M I, Nowlan S J. Adaptive Mixtures of Local Experts.. Neural Computation, 1991, 3: 79-87 CrossRef PubMed Google Scholar

[11] Blei D M, Ng A Y, Jordan M I. Latent dirichlet allocation. Journal of machine Learning research 2003, 3: 993--1022. Google Scholar

[12] Hu B, Chen Q, Zhu F. Lcsts: a large scale chinese short text summarization dataset. In: Proceedings of EMNLP 2015. Google Scholar

[13] Nallapati R, Zhou B, dos Santos C. Abstractive text summarization using sequence-to-sequence rnns and beyond. In: Proceedings of CoNLL 2016. Google Scholar

[14] Mikolov T, Karafiát M, Burget L, et al. Recurrent neural network based language model. In: Proceedings of INTERSPEECH 2010. Google Scholar

[15] Schuster M, Paliwal K K. Bidirectional recurrent neural networks. IEEE Trans Signal Process, 1997, 45: 2673-2681 CrossRef ADS Google Scholar

[16] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: Proceedings of ICLR 2015. Google Scholar

[17] Ayana , Shen S Q, Lin Y K. Recent Advances on Neural Headline Generation. J Comput Sci Technol, 2017, 32: 768-784 CrossRef Google Scholar

[18] Chen Q, Zhu X, Ling Z, et al. Distraction-based neural networks for document summarization. In: Proceedings of IJCAI 2016. Google Scholar

[19] Li P, Lam W, Bing L, et al. Deep recurrent generative decoder for abstractive text summarization. In: Proceedings of EMNLP 2017. Google Scholar

[20] Wang L, Yao J, Tao Y, et al. A reinforced topic-aware convolutional sequence-to-sequence model for abstractive text summarization. In: Proceedings of IJCAI 2018. Google Scholar

[21] Lin C-Y. Rouge: a package for automatic evaluation of summaries. In: Text summarization branches out: Proceedings of the ACL-04 workshop 2004. Google Scholar

[22] Schluter N. The limits of automatic summarisation according to rouge. In: Proceedings of EACL 2017. Google Scholar

[23] Chen P, Wu F, Wang T, et al. A semantic qa-based approach for text summarization evaluation. In: Proceedings of AAAI 2018. Google Scholar

[24] Narayan S, Cohen S B, Lapata M. Ranking sentences for extractive summarization with reinforcement learning. In: Proceedings of NAACL 2018. Google Scholar

[25] Morris A H, Kasper G M, Adams D A. The Effects and Limitations of Automated Text Condensing on Reading Comprehension Performance. Inf Syst Res, 1992, 3: 17-35 CrossRef Google Scholar

[26] Mani I, Klein G, House D. SUMMAC: a text summarization evaluation. Nat Lang Eng, 2002, 8: 43-68 CrossRef Google Scholar

[27] Clarke J, Lapata M. Discourse Constraints for Document Compression. Comput Linguistics, 2010, 36: 411-441 CrossRef Google Scholar

[28] Gehring J, Auli M, Grangier D, et al. Convolutional sequence to sequence learning. 2017,. arXiv Google Scholar

[29] Rennie S J, Marcheret E, Mroueh Y, et al. Self-critical sequence training for image captioning. 2016,. arXiv Google Scholar

[30] Cao Z, Luo C, Li W, et al. Joint copying and restricted generation for paraphrase. In: Proceedings of AAAI 2017. Google Scholar

[31] Gulcehre C, Ahn S, Nallapati R, et al. Pointing the unknown words. In: Proceedings of ACL 2016. Google Scholar

[32] Yu L, Buys J, Blunsom P. Online segment to segment neural transduction. In: Proceedings of EMNLP 2016. Google Scholar

[33] Kikuchi Y, Neubig G, Sasano R, et al. Controlling output length in neural encoder-decoders. In: Proceedings of EMNLP 2016. Google Scholar

[34] Miao Y, Blunsom P. Language as a latent variable: discrete generative models for sentence compression. In: Proceedings of EMNLP 2016. Google Scholar

[35] Li P, Lam W, Bing L, et al. Deep recurrent generative decoder for abstractive text summarization. In: Proceedings of EMNLP 2017. Google Scholar

[36] Cao Z, Wei F, Li W, et al. Faithful to the original: fact aware neural abstractive summarization. In: Proceedings of AAAI 2018. Google Scholar

[37] Shen S, Cheng Y, He Z, et al. Minimum risk training for neural machine translation. In: Proceedings of ACL 2016. Google Scholar

[38] Li P, Bing L, Lam W. Actor-critic based training framework for abstractive summarization. 2018,. arXiv Google Scholar

[39] Celikyilmaz A, Hakkani-Tür D. Discovery of topically coherent sentences for extractive summarization. In: Proceedings of ACL 2011. Google Scholar

[40] Li J, Li S. A Novel Feature-based Bayesian Model for Query Focused Multi-document Summarization. Trans Association Comput Linguistics, 2013, 1: 89-98 CrossRef Google Scholar

[41] Li Y, Li S. Query-focused multi-document summarization: combining a topic model with graph-based semi-supervised learning. In: Proceedings of COLING 2014. Google Scholar

[42] Bairi R, Iyer R, Ramakrishnan G, et al. Summarization of multi-document topic hierarchies using submodular mixtures. In: Proceedings of ACL 2015. Google Scholar

Copyright 2020  CHINA SCIENCE PUBLISHING & MEDIA LTD.  中国科技出版传媒股份有限公司  版权所有

京ICP备14028887号-23       京公网安备11010102003388号