Advertisement

Single Document Extractive Text Summarization Using Neural Networks and Genetic Algorithm

  • Niladri ChatterjeeEmail author
  • Gautam Jain
  • Gurkirat Singh Bajwa
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 858)

Abstract

The presented paper proposes an extractive text summarization technique for single documents using Neural Networks and Genetic Algorithms. The Neural Network helps to define a fitness function to express mathematically the quality of the generated summary through six desired properties which are theme similarity, cohesion, sentiment, readability, aggregate similarity and sentence position. Genetic Algorithm maximizes the above-mentioned fitness function, and extracts the most important sentences to create the extractive summary. The results are compiled using DUC2002 data as a benchmark and calculated using the precision-recall technique. They are compared with techniques using Genetic Algorithm, Neural Network and a summarizer made by Microsoft. The comparison between the results clearly demonstrates the superiority of the technique and is very encouraging for future work in this area.

Keywords

Text summarization Neural networks Genetic algorithms Sentence extraction 

References

  1. 1.
    Li, S., Karatzoglou, A., Gentile, C.: Collaborative filtering bandits. In: The 39th ACM SIGIR, pp. 539–548 (2016)Google Scholar
  2. 2.
    Gentile, C., Li, S., Kar, P., Karatzoglou, A., Etrue, E., Zappella, G.: On context-dependent clustering of bandits. In: The 34th ICML, pp. 1253–1262 (2017)Google Scholar
  3. 3.
    Korda, N., Szorenyi, B., Li, S.: Distributed clustering of linear bandits in peer to peer networks. In: The 33rd ICML, pp. 1301–1309 (2016)Google Scholar
  4. 4.
    Mani, I.: Automatic Summarization. John Benjamins Publishing Company (2001)Google Scholar
  5. 5.
    Luhn, H.P.: The automatic creation of literature abstracts. IBM J. Res. Dev. 2(2), 159–165 (1958)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Edmundson, H.P.: New methods in automatic extracting. J. ACM 16(2), 264–285 (1969)CrossRefGoogle Scholar
  7. 7.
    Barzilay, R., Elhadad, M.: Using lexical chains for text summarization. In: Mani, I., Maybury, M.T. (eds.) Advances in Automatic Text Summarization, pp. 111–121. MIT Press (1999)Google Scholar
  8. 8.
    Fattah, M.A., Ren, F.: GA, MR, FFNN, PNN and GMM based models for automatic text summarization. Comput. Speech Lang. 23(1), 126–144 (2009)CrossRefGoogle Scholar
  9. 9.
    Chatterjee, N., Bhardwaj, A.: Single document text summarization using random indexing and neural networks. In: KEOD 2010, pp. 171–176 (2010)Google Scholar
  10. 10.
    Chatterjee, N., Mittal, A., Goyal, S.: Single document extractive text summarization using genetic algorithms. In: Third International Conference on Emerging Applications of Information Technology (EAIT) (2012)Google Scholar
  11. 11.
    Qazvinian, V., Hasaanabadi, L.S., Halavati, R.: Summarising text with a genetic algorithm-based sentence extraction. Int. J. Knowl. Manag. Stud. 2(4), 426–444 (2008)CrossRefGoogle Scholar
  12. 12.
    Yates, R.B., Neto, B.R.: Modern Information Retrieval. Addison Wesley (1999)Google Scholar
  13. 13.
    Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly Media (2009)Google Scholar
  14. 14.
    Mitra, M., Singhal, A., Buckley, C.: Automatic text summarization by paragraph extraction. In: ACL Workshop on Intelligent and Scalable Text Summarization, Madrid Spain, pp. 39–46 (1997)Google Scholar
  15. 15.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L.: Introduction to Algorithms, 2nd edn. The MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  16. 16.
    Goldberg, D.E.: Genetic Algorithms. Addison Wiley Longman Inc. (1999)Google Scholar
  17. 17.
    Spears, W.M., Anand, V.: A study of crossover operators in genetic programming. In: Proceedings of the 6th International Symposium on Methodologies for Intelligent Systems, ser. ISMIS 1991, Springer, London, pp. 409–418 (1991)Google Scholar
  18. 18.
    Jizba, R.: Measuring Search Effectiveness, Precision and Recall. Creighton University (2007)Google Scholar
  19. 19.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  20. 20.
    Perumal, K., Chaudhuri, B.B.: Language independent sentence extraction based text summarization. In: ICON-2011: 9th International Conference on Natural Language Processing, pp. 213–217. Macmillan, India (2011)Google Scholar
  21. 21.
    Kupiec, J., Pedersen, J., Chen, F.: A trainable document summarizer. In: Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR 1995, pp. 68–73. ACM, New York (1995)Google Scholar
  22. 22.
    Nielsen, F.Å.: A new ANEW: evaluation of a word list for sentiment analysis in microblogs. In: Proceedings of the ESWC2011 Workshop on ‘Making Sense of Microposts’: Big Things Come in Small Packages 718 in CEUR Workshop Proceedings, pp. 93–98, May 2011Google Scholar
  23. 23.
    Ramanujam, N., Kaliappan, M.: An automatic multidocument text summarization approach based on Naïve Bayesian classifier using timestamp strategy. Sci. World J. 2016 (2016). Article ID 1784827CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Niladri Chatterjee
    • 1
    Email author
  • Gautam Jain
    • 1
  • Gurkirat Singh Bajwa
    • 1
  1. 1.Department of MathematicsIndian Institute of TechnologyNew DelhiIndia

Personalised recommendations