Advertisement

BIGGR: Bringing Gradoop to Applications

  • M. Ali RostamiEmail author
  • Matthias Kricke
  • Eric Peukert
  • Stefan Kühne
  • Moritz Wilke
  • Steffen Dienst
  • Erhard Rahm
Schwerpunktbeitrag
First Online:

Abstract

Analyzing large amounts of graph data, e.g., from social networks or bioinformatics, has recently gained much attention. Unfortunately, tool support for handling and analyzing such graph data is still weak and scalability to large data volumes is often limited. We introduce the BIGGR approach providing a novel tool for the user-friendly and efficient analysis and visualization of Big Graph Data on top of the open-source software KNIME and gradoop. Users can visually program graph analytics workflows, execute them on top of the distributed processing framework Apache Flink and visualize large graphs within KNIME. For visualization, we apply visualization-driven data reduction techniques by pushing down sampling and layouting to gradoop and Apache Flink. We also discuss an initial application of the tool for the analysis of patent citation graphs.

Keywords

Graph analysis Graph visualization Graph sampling Gradoop KNIME 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

The BIGGR project is joint work with KNIME and we thank Tobias Kötter und Mark Ortmann for assistance with technical parts of KNIME.

Funding

This work was funded by the German Federal Ministry of Education and Research within the projects BIGGR (BMBF 01IS16030B) and ScaDS Dresden/Leipzig (BMBF 01IS14014B).

References

  1. 1.
    Junghanns M, Petermann A, Neumann M, Rahm E (2017) Management and analysis of big graph data: current systems and open challenges. In: Handbook of big data technologies. Springer, Berlin, Heidelberg, pp 457–505  https://doi.org/10.1007/978-3-319-49340-4-14 CrossRefGoogle Scholar
  2. 2.
    Junghanns M, Petermann A, Gómez K, Rahm E (2015) Gradoop: scalable graph data management and analytics with Hadoop. arXiv preprint 150600548Google Scholar
  3. 3.
    Junghanns M, Kiessling M, Teichmann N, Gómez K, Petermann A, Rahm E (2018) Declarative and distributed graph analytics with GRADOOP. PVLDB 11:2006–2009.  https://doi.org/10.14778/3229863.3236246 Google Scholar
  4. 4.
    Rahm E, Nagel WE, Peukert E, Jäkel R, Gärtner F, Stadler PF, Wiegreffe D, Zeckzer D, Lehner W (2019) Big Data competence center ScaDS Dresden/Leipzig: Overview and selected research activities. Datenbank Spektrum 19(1).  https://doi.org/10.1007/s13222-018-00303-6 Google Scholar
  5. 5.
    Junghanns M, Petermann A, Teichmann N, Gómez K, Rahm E (2016) Analyzing extended property graphs with Apache Flink. In: Proc. ACM SIGMOD Workshop on Network Data Analytics (NDA).  https://doi.org/10.1145/2980523.2980527 Google Scholar
  6. 6.
    Junghanns M, Kiessling M, Averbuch A, Petermann A, Rahm E (2017) Cypher-based graph pattern matching in GRADOOP. In: Proc. 7th Int. Workshop on Graph Data Management Experiences & Systems (GRADES).  https://doi.org/10.1145/3078447.3078450 Google Scholar
  7. 7.
    Junghanns M, Petermann A, Rahm E (2017) Distributed grouping of property graphs with GRADOOP. In: Proc. Database systems for Business, Technology and Web (BTW), pp 103–122Google Scholar
  8. 8.
    Petermann A, Junghanns M, Rahm E (2017) DIMSpan: Transactional frequent subgraph mining with distributed in-memory dataflow systems. In: Proc. 4th IEEE/ACM Int. Conf. on Big Data Computing, Applications and Technologies (BDCAT), pp 237–246  https://doi.org/10.1145/3148055.3148064 Google Scholar
  9. 9.
    Berthold MR, Cebron N, Dill F, Gabriel TR, Kötter T, Meinl T, Ohl P, Thiel K, Wiswedel B (2009) KNIME-the Konstanz information miner: version 2.0 and beyond. ACM SIGKDD Explor Newsl 11(1):26–31.  https://doi.org/10.1145/1656274.1656280 CrossRefGoogle Scholar
  10. 10.
    Ludäscher B, Altintas I, Berkley C, Higgins D, Jaeger E, Jones M, Lee EA, Tao J, Zhao Y (2006) Scientific workflow management and the Kepler system: Research articles. Concurr Comput Pract Exper 18(10):1039–1065.  https://doi.org/10.1002/cpe.994 CrossRefGoogle Scholar
  11. 11.
    Hofmann M, Klinkenberg R (2013) Rapidminer: data mining use cases and business analytics applications. Chapman & Hall/CRC, Boca Raton, FLGoogle Scholar
  12. 12.
    Afgan E, Baker D, van den Beek M, Blankenberg D, Bouvier D, Cech M, Chilton J, Clements D, Coraor N, Eberhard C, Grüning BA, Guerler A, Hillman-Jackson J, Kuster GV, Rasche E, Soranzo N, Turaga N, Taylor J, Nekrutenko A, Goecks J (2016) The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2016 update. Nucleic Acids Res.  https://doi.org/10.1093/nar/gkw343 Google Scholar
  13. 13.
    da Silva RF, Filgueira R, Pietri I, Jiang M, Sakellariou R, Deelman E (2017) A characterization of workflow management systems for extreme-scale applications. Future Gener Comput Syst.  https://doi.org/10.1016/j.future.2017.02.026 Google Scholar
  14. 14.
    Wolstencroft K, Haines R, Fellows D, Williams A, Withers D, Owen S, Soiland-Reyes S, Dunlop I, Nenadic A, Fisher P, Bhagat J, Belhajjame K, Bacall F, Hardisty A, Nieva de la Hidalga A, Balcazar Vargas M, Sufi S, Goble C (2013) The Taverna workflow suite: designing and executing workflows of web services on the desktop, web or in the cloud. Nucleic Acids Res 41:W557–561.  https://doi.org/10.1093/nar/gkt328 CrossRefGoogle Scholar
  15. 15.
    Grunzke R, Jug F, Schuller B, Jäkel R, Myers G, Nagel WE (2016) Seamless HPC integration of data-intensive KNIME workflows via UNICORE. In: Euro-Par Workshops. Lecture Notes in Computer Science, vol 10104. Springer, Berlin, Heidelberg, pp 480–491  https://doi.org/10.1007/978-3-319-58943-5-39 Google Scholar
  16. 16.
    Riazi S, Norris B (2016) Graphflow: Workflow-based big graph processing. In: 2016 IEEE Int. Conf. on Big Data, pp 3336–3343  https://doi.org/10.1109/BigData.2016.7840993 CrossRefGoogle Scholar
  17. 17.
    Riazi S (2016) SparkGalaxy: Workflow-based Big Data processing. http://www.cs.uoregon.edu/Reports/DRP-201603-Riazi.pdf. Accessed 1 Mar 2019 (directed Research Proposal)Google Scholar
  18. 18.
    Herman I, Melançon G, Marshall MS (2000) Graph visualization and navigation in information visualization: a survey. IEEE Trans Vis Comput Graph 6(1):24–43.  https://doi.org/10.1109/2945.841119 CrossRefGoogle Scholar
  19. 19.
    Bikakis N, Sellis TK (2016) Exploration and visualization in the web of big linked data: a survey of the state of the art. CoRR abs/1601.08059Google Scholar
  20. 20.
    Caldarola EG, Picariello A, Rinaldi A, Sacco M (2016) Exploration and visualization of big graphs – the DBpedia case study. In: Proc. 8th Int. Conf. on Knowledge Discovery, Knowledge Engineering and Knowledge Management (KDIR)  https://doi.org/10.5220/0006046802570264 Google Scholar
  21. 21.
    Jugel U, Jerzak Z, Hackenbroich G, Markl V (2016) VDDA: automatic visualization-driven data aggregation in relational databases. VLDB J 25(1):53–77.  https://doi.org/10.1007/s00778-015-0396-z CrossRefGoogle Scholar
  22. 22.
    Rodriguez M, Neubauer P (2010) Constructions from dots and lines. Bull Am Soc Inf Sci Technol 36(6):35–41CrossRefGoogle Scholar
  23. 23.
    Rodriguez M, Neubauer P (2012) The graph traversal pattern. In: Graph Data Management: Techniques and Applications IGI Global, pp 29–46CrossRefGoogle Scholar
  24. 24.
    Kricke M, Peukert E, Rahm E (2019) Graph data transformations in gradoop. Proc BTW conf.Google Scholar
  25. 25.
    Hudak P (1989) Conception, evolution, and application of functional programming languages. ACM Comput Surv 21(3):359–411.  https://doi.org/10.1145/72551.72554 CrossRefGoogle Scholar
  26. 26.
    Seidman SB (1983) Network structure and minimum degree. Soc Networks 5(3):269–287MathSciNetCrossRefGoogle Scholar
  27. 27.
    Giatsidis C, Malliaros FD, Tziortziotis N, Dhanjal C, Kiagias E, Thilikos DM, Vazirgiannis M (2016) A k-core decomposition framework for graph clustering. CoRR abs/1607.02096Google Scholar
  28. 28.
    Hu P, Lau WC (2013) A survey and taxonomy of graph sampling. CoRR abs/1308.5865Google Scholar
  29. 29.
    Rostami MA, Saeedi A, Peukert E, Rahm E (2018) Interactive visualization of large similarity graphs and entity resolution clusters. In: Proc. Extending Database Technology (EDBT)  https://doi.org/10.5441/002/edbt.2018.86 Google Scholar
  30. 30.
    Kobourov SG (2012) Spring embedders and force directed graph drawing algorithms. Computing Research Repository (CoRR) abs/1201.3011Google Scholar

Copyright information

© Gesellschaft für Informatik e.V. and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute for InformaticsUniversity of LeipzigLeipzigGermany

Personalised recommendations

Cite article

Buy options