Multi-domain and Sub-role Oriented Software Architecture for Managing Scientific Big Data

  • Qi Sun
  • Yue LiuEmail author
  • Wenjie Tian
  • Yike Guo
  • Jiawei Lu
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 911)


The existing Scientific Data Management Systems (SDMSs) usually focus on a single domain and the interaction pattern of each subsystem is complex. What’s more, the heterogeneity and multi-source of Scientific Big Data (SBD), resulting in a wide variety of databases, scientific devices and functional areas, make the incompatibility and conflict between system modules inevitable. In this context, the paper focuses on the design and technology requirements of a multi-domain and sub-role oriented software architecture. Through integrating multiple databases, third-party systems and related tools, this architecture realizes both the storage and the sharing of multi-domain and multi-type SBD. Particularly, this architecture is divided into four independent functional areas and corresponding roles are designed, which enhances the decoupling and extensibility of the architecture. In addition, this paper has a formal description of the partition design from the perspective of role. On this basis, this paper also shows the typical application scenarios under different roles. The rationality and comprehensiveness of the proposed architecture are proved by describing the architectures design and technology.


Software architecture Role REST Scientific big data 



This work is supported by the National Key Research and Development Plan of China (Grant No. 2016YFB1000600 and 2016YFB1000601).


  1. 1.
    Andreeva, J., Campana, S., Fanzago, F., Herrala, J.: High-energy physics on the grid: the atlas and CMS experience. J. Grid Comput. 6(1), 3–13 (2008)CrossRefGoogle Scholar
  2. 2.
    Bengtssonpalme, J., et al.: Strategies to improve usability and preserve accuracy in biological sequence databases. Proteomics 16(18), 2454–2460 (2016)CrossRefGoogle Scholar
  3. 3.
    Cook, C.E., Bergman, M.T., Cochrane, G., Apweiler, R., Birney, E.: The European bioinformatics institute in 2017: data coordination and integration. Nucleic Acids Res. 46(D1), D21 (2018)CrossRefGoogle Scholar
  4. 4.
    Dewitt, D.J., Kabra, N., Luo, J., Patel, J.M., Yu, J.B.: Client–server paradise. In: Proceedings of the 20th International Conference on Very Large Data Bases, pp. 558–569 (1994)Google Scholar
  5. 5.
    Dozier, J., Stonebraker, M., Frew, J.: Sequoia 2000: a next-generation information system for the study of global change. In: Proceedings Thirteenth IEEE Symposium on Mass Storage Systems. Towards Distributed Storage and Data Management Systems, pp. 47–53 (1994)Google Scholar
  6. 6.
    Gao, W., et al.: Data motif-based proxy benchmarks for big data and AI workloads. In: IISWC 2018 (2018)Google Scholar
  7. 7.
    Gao, W., et al.: Data motifs: a lens towards fully understanding big data and AI workloads. In: 27th International Conference on Parallel Architectures and Compilation Techniques, PACT 2018 (2018)Google Scholar
  8. 8.
    Ivanova, M., Nes, N., Goncalves, R., Kersten, M.: MonetDB/SQL meets skyserver: the challenges of a scientific database. In: International Conference on Scientific and Statistical Database Management, p. 13 (2007)Google Scholar
  9. 9.
    Ivezic, Z., et al.: LSST: from science drivers to reference design and anticipated data products. Am. Astron. Soc. 41, 366 (2008)Google Scholar
  10. 10.
    Jia, Z., et al.: Understanding big data analytics workloads on modern processors. IEEE Trans. Parallel Distrib. Syst. 28(6), 1797–1810 (2017)CrossRefGoogle Scholar
  11. 11.
    Jun, C., Wen, W., Zi-yang, L., An, L.: Landsat 5 satellite overview. Remote Sens. Inf. 43(3), 85–89 (2007)Google Scholar
  12. 12.
    Stonebraker, M.: Scientific data bases at scale and SciDB. Anal. Proc. 4, 199–206 (2013)Google Scholar
  13. 13.
    Suchanek, F.M., Weikum, G.: Knowledge bases in the age of big data analytics. VLDB Endowment (2014)Google Scholar
  14. 14.
    Szalay, A.S., Gray, J., Fekete, G., Kunszt, P.Z., Kukol, P., Thakar, A.: Indexing the sphere with the hierarchical triangular mesh. Microsoft Research (2007)Google Scholar
  15. 15.
    Team, C.T.P.: Paradise: a database system for gis applications. In: ACM SIGMOD International Conference on Management of Data, p. 485 (1995)Google Scholar
  16. 16.
    Wang, L., et al.: Bigdatabench: a big data benchmark suite from internet services. In: IEEE International Symposium on High Performance Computer Architecture, HPCA 2014 (2014)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.School of Computer Engineering and ScienceShanghai UniversityShanghaiChina

Personalised recommendations