Skip to main content
SpringerLink
Log in

Trends in Document Analysis

  • Conference paper
  • First Online:
Data Management, Analytics and Innovation

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 808))

Abstract

Document analysis is one of the emerging area of research in the field of Information Retrieval. Many attempts have been made for retrieving information from a document using various machine learning algorithms. A concept of context vector is frequently used in information retrieval from document/s. Context Vector is an vector, which is used for various feature selection from documents, automatic classification of text documents, Subject Verb Agreement, etc. This paper discusses, the attempts made in the field of Information Retrieval (IR) from document using context vector. It also discuss about pros and cons of each attempt. This paper propose a system which can give “context vector” of the document set using Latent Semantic Analysis which is the most trending method in document analysis. The system is tested on BBC news dataset and proves to be successful.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Harris, S. C., Zheng, L., Kumar, V., & Kinshuk (2014). Multi-dimensional sentiment classification in online learning environment. In IEEE Sixth International Conference on Technology for Education.

    Google Scholar 

  2. Haribhakta, Y., Kalamkar, S., & Kulkarni, P. (2012). Feature annotation for text categorization. In Cube 2012.

    Google Scholar 

  3. Ye, L., Xu, R.-F., Xu, J. (2012). Emotion prediction of news articles from reader’s perspective based on multi-level classification. In Proceedings of the 2012 International Conference on Machine Learning and Cybernetics.

    Google Scholar 

  4. Khatavkar, V., & Kulkarni, P. (2016). Context vector machine for information retrieval. Atlantis Press, Advances in Intelligent Systems Research, 137, 375–379.

    Google Scholar 

  5. Khatavkar, V., & Kulkarni, P. (2017). Document context identification using latent semantic analysis. Presented in 3rd International Conference on Computing, Communication, Control and Automation, August 17–18, 2017, Pune, MS, India. (To be published on IEEE).

    Google Scholar 

  6. Han, J., & Kimber, M. (2000). Data mining: Concepts and techniques. Morgan Kaufmann.

    Google Scholar 

  7. Jain, A. K, Murty, M. N., & Flynn, P. J. (1999). Data clustering: A review. ACM Computing Surveys 31(3), 264–323.

    Google Scholar 

  8. Steinbach, M., Karypis, G., & Kumar, V. (2000). A comparison of document clustering techniques. KDD Workshop on Text Mining (Vol. 400).

    Google Scholar 

  9. Berkhin, P. (2004). Survey of clustering data mining techniques. Available at http://www.accrue.com/products/rp_cluster_review.pdf.

  10. Xu, R. (2005). Survey of clustering algorithms. IEEE Transactions on Neural Networks, 16(3), 634–678.

    Article  Google Scholar 

  11. Dou, Z., Jiang, Z., Hu, S., Wen, J.-R., & Song, R. (2015). Automatically mining facets for queries from their search results. IEEE Transactions on Knowledge and Data Engineering.

    Google Scholar 

  12. Nigam, K., Mccallum, A., Thrun, S., & Mitchell, T. (1998). Learning to classify text from labeled and unlabeled documents. In Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence.

    Google Scholar 

  13. Mccallum, A., & Nigam, K. (2003). A comparison of event models for naive bayes text classification. In Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics (Vol. 1).

    Google Scholar 

  14. Nigam, K., & Mccallum, A. (1998). Employing Em and pool-based active learning for text classification. In Proceedings of the Fifteenth International Conference on Machine Learning.

    Google Scholar 

  15. Puzicha, J., Hofmann, T., & Buhmann, J. M. (2000). A theory of proximity based clustering: structure detection by optimization. Pattern Recognition 33(4). 12.

    Google Scholar 

  16. Boughanem, M., Brini, A., & Dubois, D. (2009). Possibilistic networks for information retrieval. International Journal of Approximate Reasoning (Special Section on Graphical Models and Information Retrieval) 50(7). (Elsevier).

    Google Scholar 

  17. Salakhutdinov, R., Hinton, G. (2009). Semantic hashing. International Journal Of Approximate Reasoning (Special Section On Graphical Models And Information Retrieval) 50(7). (Elsevier).

    Google Scholar 

  18. Haribhakta, Y., Malgaonkar, A., & Kulkarni, P. Dr. (2012). Unsupervised topic detection model and its application in text categorization. In Cube 2012.

    Google Scholar 

  19. Huang, A., Milne, D., Frank, E., & Witten, I. H. (2008). Clustering document with active learning using wikipedia. In Eighth IEEE International Conference on Data Mining.

    Google Scholar 

  20. Meena, Y. K., & Gopalani, D. Dr. (2014). Analysis of sentence scoring methods for extractive automatic text summarization. In International Conference on Information and Communication Technology for Competitive Strategies.

    Google Scholar 

  21. Chiang, I.-J., Liu, C. C.-H., Tsai, Y.-H., & Kumar, A. (2015). Discovering latent semantics in web documents using fuzzy clustering. IEEE Transactions on Fuzzy Systems.

    Google Scholar 

  22. Hatzivassiloglou, V., Klavans, J. L., & Eskin, E. (1999). Detecting text similarity over short passages: Exploring linguistic feature combinations via machine learning. In Proceedings of Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora.

    Google Scholar 

  23. Hofmann, T. (1999). Probabilistic latent semantic indexing. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.

    Google Scholar 

  24. Hofmann, T. (1999). The Cluster-Abstraction model: Unsupervised learning of topic hierarchies from text data. In International Joint Conferences on Artificial Intelligence.

    Google Scholar 

  25. Huang, A., Milne, D., Frank, E., Witten, I. H. (2009). Clustering documents using a wikipedia-based concept representation, advances in knowledge discovery and data mining. In 13th Pacific-Asia Conference, PAKDD 2009 Proceedings. 11.

    Google Scholar 

  26. Lee, C.-S., Kao, Y.-F., Kuo, Y.-H., & Wang, M.-H. (2007). Automated ontology construction for unstructured text documents. Data & Knowledge Engineering 60(3). (Elsevier).

    Google Scholar 

  27. Rushall, D. A., & Ilgen, M. R. (1996). DEPICT: Documents valuated as pictures visualizing information using context vectors and self organizing maps. IEEE.

    Google Scholar 

  28. Dahab, M. Y., Hassan, H. A., & Rafea, A. (2008). TextOntoEx: Automatic ontology construction from natural English text. Expert Systems with Applications 34(2). (Elsevier).

    Google Scholar 

  29. Villaverde, J., Persson, A., Godoy, D., Amandi, A. (2009). Supporting the discovery and labeling of non-taxonomic relationships in ontology learning. Expert Systems With Applications, 36(7). (Elsevier); Montes-y-Gomez, M., & Villasenor-Pineda, L. (2009). Representing context information for document retrieval. LNAI. Springer, Berlin, Heidelberg.

    Google Scholar 

  30. Abdalgader, K., & Skabar, A. (2012). Unsupervised similarity-based word sense disambiguation using context vectors and sentential word importance. ACM Transactions on Speech and Language Processing, 9(1), Article 2.

    Google Scholar 

  31. Oh, J.-H., & Choi, K.-S. (2002). Word sense disambiguation using static and dynamic sense vectors. In Proceedings of the 19th international conference on Computational linguistics (Vol. 1).

    Google Scholar 

  32. Farkas, J. (1996). Improving the classification accuracy of automatic text processing systems using context vectors and back-propagation algorithms. In CCECE’96. IEEE.

    Google Scholar 

  33. Zhang, J., Qu, D., Li, Z. (2014). An improved recurrent neural network language model with context vector features. IEEE.

    Google Scholar 

  34. Salehi, M., Khadivi, S., & Riahi, N. (2014). Confidence estimation for machine translation using context vectors. In 2014 7th International Symposium on Telecommunications (IST’2014).

    Google Scholar 

  35. Sharma, D., & Jain, S. Dr. (2015). Context-based weighting for vector space model to evaluate the relation between concept and context in information storage and retrieval system. In IEEE International Conference on Computer, Communication and Control (IC4-2015).

    Google Scholar 

  36. Wan, S., & Angryk, R. A. (2007). Measuring semantic similarity using wordnet-based context vectors. IEEE.

    Google Scholar 

  37. Rongali, S., Choudhury, A. R., Chandan, V., & Arya, V. (2015). A context vector regression based approach for demand forecasting in district heating networks. Innovative Smart Grid Technologies—Asia (ISGT ASIA). IEEE.

    Google Scholar 

  38. Kaufmann, S. (2002). Cohesion and collocation: Using context vectors in text segmentation. CSLI, Stanford University, May 2002.

    Google Scholar 

  39. Melucci, M. (2006). Ranking in context using vector spaces. In CIKM’06. ACM.

    Google Scholar 

  40. Erk, K., & Padó, S. (2008, October). A structured vector space model for word meaning in context. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (pp. 897–906).

    Google Scholar 

  41. Thater, S., & Furstenau, H., & Pinkal, M. (2010, July). Contextualizing semantic representations using syntactically enriched vector models. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (pp. 948–957).

    Google Scholar 

  42. Carrillo M., Villatoro-Tello E., López-López A., Eliasmith C., Montes-y-Gómez M., & Villaseñor-Pineda L. (2009). Representing context information for document retrieval. In: Andreasen, T., Yager, R. R., Bulskov, H., Christiansen, H., Larsen, H. L. (Eds.), Flexible Query Answering Systems. FQAS 2009. Lecture Notes in Computer Science, Vol 5822. Springer, Berlin, Heidelberg.

    Google Scholar 

  43. Milajevs, D., Sadrzadeh, M., & Roelleke, T. (2015). IR meets NLP: On the semantic similarity between subject-verb-object phrases. In ICTIR, 15. ACM 2015.

    Google Scholar 

  44. BBC news dataset available at: http://mlg.ucd.ie/datasets/bbc.html.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Information Technology, College of Engineering, Pune, Shivajinagar, Pune, Maharashtra, India

    Vaibhav Khatavkar & Parag Kulkarni

Authors
  1. Vaibhav Khatavkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Parag Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vaibhav Khatavkar .

Editor information

Editors and Affiliations

  1. Department of Automatics and Applied Software, Aurel Vlaicu University of Arad, Arad, Romania

    Valentina Emilia Balas

  2. Audyogik Tantra Shikshan Sanstha’s, IICMR, Pune, Maharashtra, India

    Neha Sharma

  3. Faculty of Engineering and Technology, A. K. Choudhury School of Information Technology, Kolkata, West Bengal, India

    Amlan Chakrabarti

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khatavkar, V., Kulkarni, P. (2019). Trends in Document Analysis. In: Balas, V., Sharma, N., Chakrabarti, A. (eds) Data Management, Analytics and Innovation. Advances in Intelligent Systems and Computing, vol 808. Springer, Singapore. https://doi.org/10.1007/978-981-13-1402-5_19

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-1402-5_19

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-1401-8

  • Online ISBN: 978-981-13-1402-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation