Abstract
Machine learning techniques offer a plethora of opportunities in tackling big data within the astronomical community. We present the set of Generalized Linear Models as a fast alternative for determining photometric redshifts of galaxies, a set of tools not commonly applied within astronomy, despite being widely used in other professions. With this technique, we achieve catastrophic outlier rates of the order of \(\sim 1\%\), that can be achieved in a matter of seconds on large datasets of size \(\sim 1,000,000\). To make these techniques easily accessible to the astronomical community, we developed a set of libraries and tools that are publicly available.
for the COIN collaboration
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brown, D., Rothery, P., et al.: Models in Biology: Mathematics, Statistics and Computing. John Wiley, New York (1993)
Candès, E.J., Li, X., Yi Ma, Wright J.: Robust principal component analysis? J. ACM 58(3), 11:1–11:37 (2011). ISSN 0004-5411. doi: 10.1145/1970392.1970395. http://doi.acm.org/10.1145/1970392.1970395
Carrasco Kind, M., Brunner, R.J.: Exhausting the information: novel Bayesian combination of photometric redshift PDFs. MNRAS 442, 3380–3399 (2014). doi: 10.1093/mnras/stu1098
de Souza, R.S., Maio, U., Biffi, V., Ciardi, B.: Robust PCA and MIC statistics of baryons in early minihaloes. MNRAS 440, 240–248 (2014). doi: 10.1093/mnras/stu274
Elliott, J., de Souza, R.S., Krone-Martins, A., Cameron, E., Ishida, E.E.O., Hilbe, J.: The overlooked potential of generalized linear models in astronomy-II: gamma regression and photometric redshifts. Astron. Comput. 10, 61–72 (2015). doi: 10.1016/j.ascom.2015.01.002
Green, J., Schechter, P., Baltay, C., Bean, R., Bennett, D., Brown, R., Conselice, C., Donahue, M., et al.: Wide-Field InfraRed Survey Telescope (WFIRST) Final Report (2012). arxiv:1208.4012
Hildebrandt, H., Arnouts, S., Capak, P., Moustakas, L.A., Wolf, C., Abdalla, F.B, Assef, R.J., Banerji, M., et. al.: PHAT: PHoto-z accuracy testing. A&A 523, A31 (2010). doi: 10.1051/0004-6361/201014885
Lindsey, J.K.: A review of some extensions to generalized linear models. Stat. Med. 18(17–18), 2223–2236 (1999). ISSN 0277-6715. http://view.ncbi.nlm.nih.gov/pubmed/10474135
LSST Science Collaboration, Abell, P.A., Allison, J., Anderson, S.F., Andrew, J.R., Angel, J.R.P., Armus, L., Arnett, D., Asztalos, S.J., Axelrod, T.S., et al.: LSST Science Book, Version 2.0 (2009). arxiv:0912.0201
Nelder, J.A., Wedderburn, R.W.M.: Generalized linear models. J. R. Stat. Soc., Ser. A, Gen. 135, 370–384 (1972)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Pindyck, R.S., Rubinfeld, D.L.: Econometric Models and Economic Forecasts, vol. 4. Irwin/McGraw-Hill, Boston (1998)
Refregier, A., Amara, A., Kitching, T.D., Rassat, A., Scaramella, R., Weller, J., Euclid Imaging Consortium, f. t.: Euclid Imaging Consortium Science Book (2010). arxiv:1001.0061
York, D.G., Adelman, J., Anderson, J.E., Jr., Anderson, S.F., Annis, J., Bahcall, N.A., Bakken, J.A., Barkhouser, R., et al., SDSS Collaboration: The sloan digital sky survey: technical summary. AJ 120, 1579–1587 (2000). doi: 10.1086/301513
Acknowledgements
We thank V. Busti, E. D. Feigelson, M. Killedar, J. Buchner, and A. Trindade for interesting suggestions and comments. JE, RSS and EEOI thank the SIM Laboratory of the Universidade de Lisboa for hospitality during the development of this work. Cosmostatistics Initiative (COIN)Footnote 6 is a non-profit organisation whose aim is to nourish the synergy between astrophysics, cosmology, statistics and machine learning communities. This work was partially supported by the ESA VA4D project (AO 1-6740/11/F/MOS). AKM thanks the Portuguese agency Fundação para Ciência e Tecnologia – FCT, for financial support (SFRH/BPD/74697/2010). EEOI is partially supported by the Brazilian agency CAPES (grant number 9229-13-2). Work on this paper has substantially benefited from using the collaborative website AWOB developed and maintained by the Max-Planck Institute for Astrophysics and the Max-Planck Digital Library. This work was written on the collaborative WriteLatex platform, and made use of the GitHub a web-based hosting service and git version control software. This work made use of the cloud based hosting platform ShinyApps.io. This work used the following public scientific Python packages scikit-learn v0.15 [11], seaborn v0.3.1, and statsmodels v0.6.0. Funding for SDSS-III has been provided by the Alfred P. Sloan Foundation, the Participating Institutions, the National Science Foundation, and the U.S. Department of Energy Office of Science.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Elliott, J., de Souza, R.S., Krone-Martins, A., Cameron, E., Ishida, E.E.O., Hilbe, J. (2016). Using Gamma Regression for Photometric Redshifts of Survey Galaxies. In: Napolitano, N., Longo, G., Marconi, M., Paolillo, M., Iodice, E. (eds) The Universe of Digital Sky Surveys. Astrophysics and Space Science Proceedings, vol 42. Springer, Cham. https://doi.org/10.1007/978-3-319-19330-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-19330-4_13
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19329-8
Online ISBN: 978-3-319-19330-4
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)