Advertisement

Experimental Astronomy

, Volume 45, Issue 1, pp 41–55 | Cite as

A new archival infrastructure for highly-structured astronomical data

  • Erik Dovgan
  • Cristina Knapic
  • Massimo Sponza
  • Riccardo Smareglia
Original Article
  • 86 Downloads

Abstract

With the advent of the 2020 Radio Astronomy Telescopes era, the amount and format of the radioastronomical data is becoming a massive and performance-critical challenge. Such an evolution of data models and data formats require new data archiving techniques that allow massive and fast storage of data that are at the same time also efficiently processed. A useful expertise for efficient archiviation has been obtained through data archiving of Medicina and Noto Radio Telescopes. The presented archival infrastructure named the Radio Archive stores and handles various formats, such as FITS, MBFITS, and VLBI’s XML, which includes description and ancillary files. The modeling and architecture of the archive fulfill all the requirements of both data persistence and easy data discovery and exploitation. The presented archive already complies with the Virtual Observatory directives, therefore future service implementations will also be VO compliant. This article presents the Radio Archive services and tools, from the data acquisition to the end-user data utilization.

Keywords

Radioastronomy Archives Big data 

Notes

Acknowledgments

The authors acknowledge the financial support from the Slovenian Research Agency (research project funding No. Z2-7581, and research core funding No. P2-0209).

References

  1. 1.
    Arviset, C., Gaudet, S.: IVOA architecture, IVOA (2010)Google Scholar
  2. 2.
    De Marco, M., Knapic, C., Smareglia, R.: New archiving distributed infrastructure (NADIR): Status and evolution. Proc. Astron. Data Anal. Softw. Syst. XXIV 495, 461 (2015)ADSGoogle Scholar
  3. 3.
    Demleitner, M., Neves, M.C., Rothmaier, F., Wambsganss, J.: Virtual observatory publishing with DaCHS. Astron. Comput. 7–8, 27–36 (2014)ADSCrossRefGoogle Scholar
  4. 4.
    Dewdney, P.E., Hall, P.J., Schilizzi, R.T., Lazio, T.J.L.: The square kilometre array. Proc. IEEE 97(8), 1482–1496 (2009)ADSCrossRefGoogle Scholar
  5. 5.
    Dowler, P., Gaudet, S., Schade, D.: Infrastructure and the virtual observatory. Proc. Astron. Data Anal. Softw. Syst. XX 442, 603–606 (2011)ADSGoogle Scholar
  6. 6.
  7. 7.
    Dutka, L., Wrzeszcz, M., Lichon, T., Slota, R., Zemek, K., Trzepla, K., Opiola, L., Slota, R., Kitowski, J.: Onedata – a step forward towards globalization of data access for computing infrastructures. Proced. Comput. Sci. 51, 2843–2847 (2015)CrossRefGoogle Scholar
  8. 8.
    EGI, EGI-Engage: Engaging the EGI community towards an open science commons, H2020 project, https://wiki.egi.eu/wiki/EGI-Engage:Main_Page
  9. 9.
    FITS Working Group, Definition of the Flexible Image Transport System (FITS) International Astronomical Union (2016)Google Scholar
  10. 10.
    Gaudet, S., Dowler, P., Goliath, S., Hill, N., Kavelaars, J.J., Peddle, M: The canadian advanced network for astronomical research. Proc. Astron. Data Anal. Softw. Syst. XVIII 411, 185–188 (2009)ADSGoogle Scholar
  11. 11.
  12. 12.
    Hedges, M., Blanke, T., Hasan, A.: Rule-based curation and preservation of data: a data grid approach using iRODS. Fut. Gener. Comput. Syst. 25(4), 446–452 (2009)CrossRefGoogle Scholar
  13. 13.
    Holovaty, A., Kaplan-Moss, J.: The definitive guide to django: web development done right, 2nd edn. APress, Berkeley (2009)CrossRefGoogle Scholar
  14. 14.
    Italian National Institute for Astrophysics, Radioastronomical Institute, https://www.ira.inaf.it/Home.html
  15. 15.
    Lacy, M., Halstead, D., Hatz, M.: Data processing and archiving at the north american ALMA science center. Proc. Astron. Data Anal. Softw. Syst. XX. ASP Conf. Proceed. 442, 57–60 (2010)ADSGoogle Scholar
  16. 16.
    Leon, S., Espigares, V., Ruiz, J.E., Verdes-Montenegro, L., Mauersberger, R., Brunswig, W., Kramer, C., Santander-Vela, J.D., Wiesemeyer, H.: TAPAS – a VO archive at the IRAM 30-m telescope. Exper. Astron. 34, 65–88 (2012)ADSCrossRefGoogle Scholar
  17. 17.
    Louys, M., Tody, D., Dowler, P., Durand, D., Michel, L., Bonnarel, F., Micol, A.: And the IVOA DataModel working group, observation data model core components and its implementation in the table access frotocol, IOVA (2017)Google Scholar
  18. 18.
    Max Planck Institute for Radio Astronomy, http://www.mpifr-bonn.mpg.de/2169/en
  19. 19.
    Montaguti, S., Vittuari, L., Sarti, P., Negusini, M.: Medicina and Noto VLBI radiotelescopes: gravitational deformations evaluated with terrestrial laser scanning. Proc. of the 18th European VLBI for Geodesy and Astromety Working Meeting, pp. 4–9 (2007)Google Scholar
  20. 20.
    Muders, D., Polehampton, E., Hatchell, J.: Multi-Beam FITS raw data format, APEX (2015)Google Scholar
  21. 21.
    Salomoni, D., Campos Plasencia, I., Gaido, L., Donvito, G., Fuhrman, P., Marco, J., Lopez Garcia, A., Orviz Fernandez, P., Blanquer, I., Molto, G., Plociennik, M., Owsiak, M., Urbaniak, M., Hardt, M., Ceccanti, A., Wegh, B., Gomes, J., David, M., Aiftimiei, C., Dutka, L., Fiore, S., Aloisio, G., Barbera, R., Bruno, R., Fargetta, M., Giorgio, E., Reynaud, S., Schwarz, L.: INDIGO-Datacloud: foundations and architectural description of a Platform as a Service oriented to scientific computing. CoRR, vol. arXiv:1603.09536 (2016)
  22. 22.
    Santander-Vela, J.D., Garcia, E., Leon, S., Espigares, V., Ruiz, J.E., Verdes-Montenegro, L., Solano, E.: Building a VO-compliant Radio Astronomical Data Model for Single-dish radio telescopes (RADAMS). Exper. Astron. 34, 623–652 (2012)ADSCrossRefGoogle Scholar
  23. 23.
    Takahashi, F., Takahashi, Y., Kondo, T., Koyama, Y.: Very long baseline interferometer. IOS Press, Amsterdam (2000)Google Scholar
  24. 24.
    Tingay, S.J., Goeke, R., Bowman, J.D., Emrich, D., Ord, S.M., Mitchell, D.A., Morales, M.F., Booler, T., Crosse, B., Wayth, R.B., Lonsdale, C.J., Tremblay, S., Pallot, D., Colegate, T., Wicenec, A., Kudryavtseva, N., Arcus, W., Barnes, D., Bernardi, G., Briggs, F., Burns, S., Bunton, J.D., Cappallo, R.J., Corey, B.E., Deshpande, A., Desouza, L., Gaensler, B.M., Greenhill, L.J., Hall, P.J., Hazelton, B.J., Herne, D., Hewitt, J.N., Johnston-Hollitt, M., Kaplan, D.L., Kasper, J.C., Kincaid, B.B., Koenig, R., Kratzenberg, E., Lynch, M.J., Mckinley, B., Mcwhirter, S.R., Morgan, E., Oberoi, D., Pathikulangara, J., Prabu, T., Remillard, R.A., Rogers, A.E.E., Roshi, A., Salah, J.E., Sault, R.J., Udaya-Shankar, N., Schlagenhaufer, F., Srivani, K.S., Stevens, J., Subrahmanyan, R., Waterson, M., Webster, R.L., Whitney, A.R., Williams, A., Williams, C.L., Wyithe, J.S.: The murchison widefield array: the square kilometre array precursor at low radio frequencies. Publ. Astron. Soc. Aust. (PASA) 30, e007, 21 (2013)Google Scholar
  25. 25.
    Wicenec, A., Knudstrup, J.: ESO’s next generation archive system in full operation, The Messenger, no. 129 (2007)Google Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Astronomical Observatory of TriesteNational Institute for AstrophysicsTriesteItaly
  2. 2.Department of Intelligent SystemsJožef Stefan InstituteLjubljanaSlovenia

Personalised recommendations