Skip to main content
SpringerLink
Log in

A Technical Evaluation of Neo4j and Elasticsearch for Mining Twitter Data

  • Conference paper
  • First Online:
Advances in Computing and Data Sciences (ICACDS 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 906))

Included in the following conference series:

Abstract

Knowledge mining on social media datasets especially twitter has been widely acknowledge due to recent prospects of intelligent systems for various purposes. It has been the most reliable indicator of the wider pulse of the world and whats happening around the globe. witter being one of the largest social media network has an average of 317 active monthly users as on January 2017. Considering the real-time activities of these users like tweets, the data to mine constitute the characteristics of Big Data. The process to handle this Big Data requires an efficient storage and retrial mechanism which the current implementation Redis lacks in some aspects.

Considering the technical requirements of social media data in general and twitter in particular, we present an evaluation of two big data technologies Elasticsearch and Neo4j. Initially we present a suitability analysis followed by experimental evaluation of these two implementations.

With the experimental results, it is concluded that Neo4j, a graph database has overthrown Elasticsearch in both aspects of storage and operations. The storage mechanism of Neo4j has proven to be efficient constituting only 45% of what required for Elasticsearch. In operational which is considered to be the strength of Elasticsearch, Neo4j was able to perform better in terms of data load and retrieval operations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kapase, H., Galande, K., Sonna, T., Pawar, D., Salunke, D.: A review on: sentiment polarity analysis on twitter data from different events (2018)

    Google Scholar 

  2. Yildiz, D., Munson, J., Vitali, A., Tinati, R., Holland, J.: Using Twitter data for population estimates (2017)

    Google Scholar 

  3. Lin, J., Cromley, R.G.: Inferring the home locations of Twitter users based on the spatiotemporal clustering of Twitter data. Trans. GIS 22(1), 82–97 (2018)

    Article  Google Scholar 

  4. Vioulès, M.J., Moulahi, B., Azé, J., Bringay, S.: Detection of suicide-related posts in Twitter data streams. IBM J. Res. Dev. 62(1), 1–7 (2018)

    Article  Google Scholar 

  5. Zou, L., Lam, N.S., Cai, H., Qiang, Y.: Mining Twitter data for improved understanding of disaster resilience. Ann. Am. Assoc. Geogr., 1–20 (2018)

    Google Scholar 

  6. Jones, A.S., Georgakis, P., Petalas, Y., Suresh, R.: Real-time traffic event detection using Twitter data. Infrastruct. Asset Manag., 1–33 (2018)

    Google Scholar 

  7. Tirumala, S.S., Shahamiri, S.R., Garhwal, A.S., Wang, R.: Speaker identification features extraction methods: a systematic review. Expert Syst. Appl. 90, 250–271 (2017)

    Article  Google Scholar 

  8. Ali, S., Tirumala, S.S., Sarrafzadeh, A.: SVM aggregation modelling for spatio-temporal air pollution analysis. In: IEEE 17th International Multi-Topic Conference (INMIC), pp. 249–254. IEEE (2014)

    Google Scholar 

  9. Ahuja, R., Malik, J., Tyagi, R., Brinda, R.: Role of open source software in big data storage. In: Handbook of Research on Big Data Storage and Visualization Techniques, pp. 123–150. IGI Global (2018)

    Google Scholar 

  10. Huang, K., Zhou, J., Huang, L., Shen, Y.: NVHT: an efficient key-value storage library for non-volatile memory. J. Parallel Distrib. Comput. 120, 339–354 (2018)

    Article  Google Scholar 

  11. Swami, D., Sahoo, S., Sahoo, B.: Storing and analyzing streaming data: a big data challenge. In: Big Data Analytics: Tools and Technology for Effective Planning, pp. 229–246 (2018)

    Google Scholar 

  12. Tirumala, S.S., Narayanan, A.: Hierarchical data classification using deep neural networks. In: Arik, S., Huang, T., Lai, W.K., Liu, Q. (eds.) ICONIP 2015. LNCS, vol. 9489, pp. 492–500. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-26532-2_54

    Chapter  Google Scholar 

  13. Roland, A., Tirumala, S.S., Babu, G.A.: Evaluating big data technologies for statistical homicide dataset. In: Second International Conference on Computing and Communications (IC3), India. Springer (2018)

    Google Scholar 

  14. Shapiro, M., Bieniusa, A., Preguiça, N., Balegas, V., Meiklejohn, C.: Just-right consistency: reconciling availability and safety. arXiv preprint arXiv:1801.06340 (2018)

  15. Brewer, E.: Pushing the cap: strategies for consistency and availability. Computer 45(2), 23–29 (2012)

    Article  Google Scholar 

  16. Reniers, V., Rafique, A., Van Landuyt, D., Joosen, W.: Object-NoSQL database mappers: a benchmark study on the performance overhead. J. Internet Serv. Appl. 8(1), 1 (2017)

    Article  Google Scholar 

  17. Katragadda, R., Tirumala, S.S., Nandigam, D.: ETL tools for Data Warehousing: an empirical study of Open Source Talend Studio versus Microsoft SSIS (2015)

    Google Scholar 

  18. Drakopoulos, G., Kanavos, A., Tsakalidis, A.K.: Evaluating Twitter influence ranking with system theory. In: WEBIST, vol. 1, pp. 113–120 (2016)

    Google Scholar 

  19. Langi, P.P., Najib, W., Aji, T.B., et al.: An evaluation of Twitter river and Logstash performances as elasticsearch inputs for social media analysis of Twitter. In: 2015 International Conference on Information & Communication Technology and Systems (ICTS), pp. 181–186. IEEE (2015)

    Google Scholar 

  20. DBEngine. System properties comparison Elasticsearch vs. Neo4j (1999). https://dbengines.com/en/system/Elasticsearch%3BNeo4j

  21. Shahi, D.: Apache Solr: an introduction. Apache Solr, pp. 1–9. Apress, Berkeley (2015). https://doi.org/10.1007/978-1-4842-1070-3_1

    Chapter  Google Scholar 

  22. Kononenko, O., Baysal, O., Holmes, R., Godfrey, M.W.: Mining modern repositories with elasticsearch. In: Proceedings of the 11th Working Conference on Mining Software Repositories, pp. 328–331. ACM (2014)

    Google Scholar 

  23. Gupta, S., Rani, R.: A comparative study of elasticsearch and CouchDB document oriented databases. In: International Conference on Inventive Computation Technologies (ICICT), vol. 1, pp. 1–4. IEEE (2016)

    Google Scholar 

  24. Montag, D.: Understanding Neo4j scalability. White Paper, Neotechnology (2013)

    Google Scholar 

  25. Sasaki, B.M.: Graph databases for beginners: acid vs. base explained (2015). https://neo4j.com/blog/acid-vs-baseconsistency-models-explained

  26. Marinescu, P., Parry, C., Pomarole, M., Tian, Y., Tague, P., Papagiannis, I.: IVD: automatic learning and enforcement of authorization rules in online social networks. In: IEEE Symposium on Security and Privacy (SP), pp. 1094–1109. IEEE (2017)

    Google Scholar 

  27. Crockford, D.: The application/json Media Type for JavaScript Object Notation (JSON) (2006). [Online]. Available: https://tools.ietf.org/html/rfc4627

Download references

Author information

Authors and Affiliations

  1. Unitec Institute of Technology, Auckland, New Zealand

    Janet Zhu & Sreenivas Sremath Tirumala

  2. Sri Venkateswara University, Tirupati, India

    G. Anjan Babu

Authors
  1. Janet Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sreenivas Sremath Tirumala
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G. Anjan Babu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sreenivas Sremath Tirumala .

Editor information

Editors and Affiliations

  1. University of KwaZulu-Natal, Durban, South Africa

    Mayank Singh

  2. Jaypee University of Information Technology, Solan, India

    P. K. Gupta

  3. Jaypee University of Engineering and Technology, Guna, Madhya Pradesh, India

    Vipin Tyagi

  4. Institute of Information Theory and Automation, Prague 8, Czech Republic

    Jan Flusser

  5. University of Ottawa, Ottawa, Canada

    Tuncer Ören

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhu, J., Tirumala, S.S., Anjan Babu, G. (2018). A Technical Evaluation of Neo4j and Elasticsearch for Mining Twitter Data. In: Singh, M., Gupta, P., Tyagi, V., Flusser, J., Ören, T. (eds) Advances in Computing and Data Sciences. ICACDS 2018. Communications in Computer and Information Science, vol 906. Springer, Singapore. https://doi.org/10.1007/978-981-13-1813-9_36

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-1813-9_36

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-1812-2

  • Online ISBN: 978-981-13-1813-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation