Advertisement

Polystore Data Management Systems for Managing Scientific Data-sets in Big Data Archives

  • Rashmi Girirajkumar Patidar
  • Shashank Shrestha
  • Subhash Bhalla
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11297)

Abstract

Large scale scientific data sets are often analyzed for the purpose of supporting workflow and querying. User need to query over different data sources. These systems manage intermediate results. Most prototypes are complex and have an ad hoc design. These require extensive modifications in case of growth of data and change of scale, in terms of data or number of users. New data sources may arise to further complicate the ad hoc design. The polystore data management approach provides ‘data independence’ for changes in data profile, including addition of cloud data resources. The users are often provided a quasi-relational query language. In many cases, the polystore systems support distinct tasks that are user defined workflow activity, in addition to providing a common view of data resources.

Keywords

Big data analytics Cloud-based databases Heterogeneous data Distributed data Polystore data management Query language support Scalability 

References

  1. 1.
    Duggan, J., et al.: The bigdawg polystore system. ACM Sigmod Rec. 44(2), 11–16 (2015)CrossRefGoogle Scholar
  2. 2.
    Saeed, M., et al.: Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II): a public-access intensive care unit database. Crit. Care Med. 39(5), 952 (2011)Google Scholar
  3. 3.
    Armbrust, M., et al.: Spark sql: relational data processing in spark. In: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. ACM (2015)Google Scholar
  4. 4.
  5. 5.
  6. 6.
    Kolev, B., Bondiombouy, C., Valduriez, P., Jiménez-Peris, R., Pau, R., Pereira, J.: The cloudmdsql multistore system. In: Proceedings of the 2016 International Conference on Management of Data. ACM (2016)Google Scholar
  7. 7.
    Law, N.M., et al.: The Palomar Transient Factory: system overview, performance, and first results. Publ. Astron. Soc. Pac. 121(886), 1395 (2009)Google Scholar
  8. 8.
  9. 9.
    Laher, R.R., et al.: IPAC image processing and data archiving for the Palomar Transient Factory. Publ. Astron. Soc. Pac. 126(941), 674 (2014)Google Scholar
  10. 10.
    Pence, W.D., et al.: Definition of the flexible image transport system (fits), version 3.0. Astronomy & Astrophysics 524, A42 (2010)CrossRefGoogle Scholar
  11. 11.
  12. 12.
    Robitaille, T.P., et al.: Astropy: a community Python package for astronomy. Astron. Astrophys. 558, A33 (2013)Google Scholar
  13. 13.
    Information on JS9 FITS image viewer. https://js9.si.edu/
  14. 14.
  15. 15.
    Gadepally, V., et al.: The BigDAWG polystore system and architecture. In: 2016 IEEE High Performance Extreme Computing Conference (HPEC), Waltham, MA, pp. 1–6 (2016)Google Scholar
  16. 16.
    Elmore, A., et al.: A demonstration of the BigDAWG polystore system. Proc. VLDB Endow. 8. 1908–1911 (2015).  https://doi.org/10.14778/2824032.2824098CrossRefGoogle Scholar
  17. 17.
    Kolev, B., Valduriez, P., Bondiombouy, C., Jiménez-Peris, R., Pau, R., Pereira, J.: CloudMdsQL: querying heterogeneous cloud data stores with a common language. Distrib. Parallel Databases 34(4), 463–503 (2016)CrossRefGoogle Scholar
  18. 18.
    Kolev, B., et al.: Design and Implementation of the CloudMdsQL Multistore System, 4 July 2016. https://hal-lirmm.ccsd.cnrs.fr/lirmm-01341172/document
  19. 19.
    O’Brien, K.: Polystore Systems for Complex Data Management. HPEC 2017. https://bigdawg.mit.edu/sites/default/files/documents/20170910r3-BigDAWG_Details.pdf
  20. 20.
    Sun, Jun: Information requirement elicitation in mobile commerce. Commun. CM (CACM) 46(12), 45–47 (2003)CrossRefGoogle Scholar
  21. 21.
    Shashank, S., et al.: PDSPTF: polystore database system for scalability and access to PTF time-domain astronomy data archives. In: International Workshop on Polystores and Other Systems for Heterogeneous Data (Poly’2018) co-located with VLDB 2018 (2018)Google Scholar
  22. 22.
    Tamer Özsu, M., Valduriez, P.: Principles of Distributed Database Systems. Springer, 2018-19Google Scholar
  23. 23.
    Valduriez, P., Danforth, S.: Functional SQL, an SQL Upward Compatible. Database Programming Language. Information Sciences (1992)Google Scholar
  24. 24.
    Khan, Y., Zimmermann, A., Jha, A., Rebholz-Schuhmann, D., Sahay, R.: Querying Web Polystores. In: 2017 IEEE International Conference on Big Data (Big Data), December 2017Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Rashmi Girirajkumar Patidar
    • 1
  • Shashank Shrestha
    • 1
  • Subhash Bhalla
    • 1
  1. 1.Department of Information SystemsUniversity of AizuFukushimaJapan

Personalised recommendations