Abstract
Since the advent of World Wide Web, the world has seen an exponential growth in the information on the internet. In the web, today, a lot of digital articles are available for users to read. However, these documents lack a concise summary. Automatic systems that summarize such information are very rare. Moreover, these systems also lack the ability of getting information from multiple places and presenting the user with a coherent, informative summary. Summarization are of two types, viz. Extractive and Abstractive. While a lot of work has been done on extractive summary, abstractive summary generation is still an unexplored area. In the current work, the focus is essentially on producing abstractive summary of a document by first getting relevant and important sentences from the corpus and feeding these sentences as input in a trained deep neural network. For extracting salient sentences two models has been proposed, both these models not only consider syntactic similarity but also semantic similarity between sentences. To capture semantic similarity, embedding models like word2vec or paragraph vectors has been used. Next, to generate abstract version of these sentences a RNN autoencoder is used, which is responsible for the representation of longer sentences into shorter sentences without loosing semantic information. ROUGE evaluation technique has been used for evaluating the generated summary quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Continuous bag of words.
- 2.
- 3.
- 4.
In some context \(s_{t} \) and o is also denoted by \(h_t\) and Y respectively.
- 5.
- 6.
Sequence to sequence.
- 7.
- 8.
End of sentence token.
- 9.
Unknown token.
- 10.
Beginning of sentence.
References
Arora S, Liang Y, Ma T (2017) A simple but tough-to-beat baseline for sentence embeddings. In: International conference on learning representations. To Appear
Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473
Barzilay R, McKeown KR, Elhadad M (1999) Information fusion in the context of multi-document summarization. In: Proceedings of the 37th annual meeting of the association for computational linguistics on computational linguistics. Association for Computational Linguistics, pp 550–557
Burges C, Shaked T, Renshaw E, Lazier A, Deeds M, Hamilton N, Hullender G (2005) Learning to rank using gradient descent. In: Proceedings of the 22nd international conference on machine learning. ACM, New York, pp 89–96
Clarke J, Lapata M (2008) Global inference for sentence compression: an integer linear programming approach. J Artif Intell Res 31:399–429
Daumé III H, Marcu D (2006) Bayesian query-focused summarization. In: Proceedings of the 21st international conference on computational linguistics and the 44th annual meeting of the association for computational linguistics. Association for Computational Linguistics, pp 305–312
Edmundson HP (1969) New methods in automatic extracting. J ACM (JACM) 16(2):264–285
Erkan G, Radev DR (2004) Lexrank: graph-based lexical centrality as salience in text summarization. J Artif Intell Res 22:457–479
Fung P, Ngai G (2006) One story, one flow: hidden Markov story models for multilingual multidocument summarization. ACM Trans Speech Lang Process (TSLP) 3(2):1–16
Ganesan K, Zhai C, Han J (2010) Opinosis: a graph-based approach to abstractive summarization of highly redundant opinions. In: Proceedings of the 23rd international conference on computational linguistics. Association for Computational Linguistics, pp 340–348
Genest PE, Lapalme G (2011) Framework for abstractive summarization using text-to-text generation. In: Proceedings of the workshop on monolingual text-to-text generation. Association for Computational Linguistics, pp 64–73
Genest PE, Lapalme G (2012) Fully abstractive approach to guided summarization. In: Proceedings of the 50th annual meeting of the association for computational linguistics: short papers, vol 2. Association for Computational Linguistics, pp 354–358
Greenbacker CF (2011) Towards a framework for abstractive summarization of multimodal documents. In: Proceedings of the ACL 2011 student session. Association for Computational Linguistics, pp 75–80
Hagberg AA, Schult DA, Swart PJ (2008) Exploring network structure, dynamics, and function using NetworkX. In: Proceedings of the 7th python in science conference (SciPy2008). Pasadena, CA USA, pp 11–15
Haghighi A, Vanderwende L (2009) Exploring content models for multi-document summarization. In: Proceedings of human language technologies: the 2009 annual conference of the North American chapter of the association for computational linguistics. Association for Computational Linguistics, pp 362–370
Kupiec J, Pedersen J, Chen F (1995) A trainable document summarizer. In: Proceedings of the 18th annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 68–73
Lau JH, Baldwin T (2016) An empirical evaluation of doc2vec with practical insights into document embedding generation. arXiv:1607.05368
Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 1188–1196
Lin CY (2004) Rouge: a package for automatic evaluation of summaries. In: Proceedings of the ACL-04 workshop on text summarization branches out, vol 8. Barcelona, Spain
Lin CY, Hovy E (2000) The automated acquisition of topic signatures for text summarization. In: Proceedings of the 18th conference on computational linguistics, vol 1. Association for Computational Linguistics, pp 495–501
Liu F, Flanigan J, Thomson S, Sadeh N, Smith NA (2015) Toward abstractive summarization using semantic representations
Liu F, Liu Y (2009) From extractive to abstractive meeting summaries: can it be done by sentence compression? In: Proceedings of the ACL-IJCNLP 2009 conference short papers. Association for Computational Linguistics, pp 261–264
Luong MT, Sutskever I, Le QV, Vinyals O, Zaremba W (2014) Addressing the rare word problem in neural machine translation. arXiv:1410.8206
Moawad IF, Aref M (2012) Semantic graph reduction approach for abstractive text summarization. In: 2012 seventh international conference on computer engineering & systems (ICCES). IEEE, pp 132–138
Nallapati R, Zhou B, Gulcehre C, Xiang B et al (2016) Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv:1602.06023
Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical report, Stanford InfoLab
Qazvinian V, Radev DR (2008) Scientific paper summarization using citation summary networks. In: Proceedings of the 22nd international conference on computational linguistics, vol 1. Association for Computational Linguistics, pp 689–696
Radev DR, Jing H, Styś M, Tam D (2004) Centroid-based summarization of multiple documents. Inf Process Manag 40(6):919–938
Řehůřek R, Sojka P (2010) Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks. ELRA, Valletta, Malta, pp 45–50. http://is.muni.cz/publication/884893/en
Rush AM, Chopra S, Weston J (2015) A neural attention model for abstractive sentence summarization. arXiv:1509.00685
Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112
Svore KM, Vanderwende L, Burges CJ (2007) Enhancing single-document summarization by combining ranknet and third-party sources. In: Emnlp-conll, pp 448–457
Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. In: Advances in neural information processing systems, pp 2692–2700
Wiseman S, Rush AM (2016) Sequence-to-sequence learning as beam-search optimization. arXiv:1606.02960
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Dey, M., Das, D. (2020). A Deep Dive into Supervised Extractive and Abstractive Summarization from Text. In: Hemanth, J., Bhatia, M., Geman, O. (eds) Data Visualization and Knowledge Engineering. Lecture Notes on Data Engineering and Communications Technologies, vol 32. Springer, Cham. https://doi.org/10.1007/978-3-030-25797-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-25797-2_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25796-5
Online ISBN: 978-3-030-25797-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)