Abstract
China’s self-developed GWAC is different from previous astronomical projects. GWAC consists of 40 wide-angle telescopes, collecting image data of the entire sky every 15 s, and requires data to be processed and alerted in real time within 15 s. These requirements are due to GWAC. Committed to discovering and timely capturing the development of short-time astronomical phenomena, such as supernova explosions, gamma blasts [1], and microgravity lenses, hoping that GWAC will be able to make timely warnings and early warnings in the early stages of these astronomical phenomena. The astronomical researchers are provided with detailed information on the event by scheduling a deep telescope to record the entire process of astronomical time development. Second, the GWAC project requires observations for up to 10 years of storage. These long-term stored data are provided to astronomers to help the astronomers get new discoveries after the technology and means are updated, as well as astronomy for astronomers. Big data mining provides support.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yuan, D., Yang, Y., Liu, X., et al.: On-demand minimum cost benchmarking for intermediate dataset storage in scientific cloud workflow systems. J. Parallel Distrib. Comput. 71(2), 316–332 (2011)
Boncz, P., Grust, T., Keulen, M.V., et al.: MonetDB/XQuery:a fast XQuery processor powered by a relational engine. In: ACM SIGMOD International Conference on Management of Data, pp. 479–490. ACM (2006)
Wan, M., Wu, C., Wang, J., et al.: Column store for GWAC: a high-cadence, high-density, large-scale astronomical light curve pipeline and distributed shared-nothing database. Publ. Astron. Soc. Pac. 128, 114501 (2016)
Apache kafka. http://kafka.apache.org
Apache hbase. http://hbase.apache.org
Redislab redis. http://redis.io
Meng, W.: GWACdbgen. http://github.com/wanmeng/gwac_dbgen
Jian, L.I., Cui, C.Z., Bo-Liang, H.E., et al.: Review and prospect of the astronomical database. Prog. Astron. 31(1), 1–16 (2013)
SDSS Skyserver. http://skyserver.org/
Zaharia, M., Chowdhury, M., Franklin, M.J., et al.: Spark: cluster computing with working sets. In: Usenix Conference on Hot Topics in Cloud Computing, p. 10. USENIX Association (2010)
Apache hbase. http://hadoop.apache.org
Meng, X., Meng, X., Meng, X., et al.: Spark SQL: relational data processing in spark. In: ACM SIGMOD International Conference on Management of Data, pp. 1383–1394. ACM (2015)
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 1–58 (2009)
Ahmad, S., Purdy, S.: Real-Time Anomaly Detection for Streaming Analytics (2016)
Acknowledgement
This work is partially supported by National Key R&D Program No. 2016YFB1000602.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, K., Guo, W., Cui, L., Xu, M., Li, Q. (2019). AstroBase: Distributed Long-Term Astronomical Database. In: Li, J., Meng, X., Zhang, Y., Cui, W., Du, Z. (eds) Big Scientific Data Management. BigSDM 2018. Lecture Notes in Computer Science(), vol 11473. Springer, Cham. https://doi.org/10.1007/978-3-030-28061-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-28061-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28060-4
Online ISBN: 978-3-030-28061-1
eBook Packages: Computer ScienceComputer Science (R0)