Censoring in Astronomical Data Due to Nondetections

  • Eric D. Feigelson


Astronomical surveys often involve observations of preselected samples of stars or galaxies at new wavebands. Due to limited sensitivities, some objects may be undetected, leading to upper limits in their derived luminosities. Statistically, these are left-censored data points. We review the nature of this problem in astronomy, the successes and limitations of using established “survival analysis” univariate and bivariate statistical techniques and discuss the need for further methodological development. In particular, astronomical censored data sets are often subject to experimentally known measurement errors (which are used to set censoring levels), may suffer simultaneous censoring in several variables, and may have particular “quasirandom” censoring patterns and parametric distributions.


Spiral Galaxy Luminosity Function Seyfert Galaxy Astronomical Data Astronomical Application 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Ak90]
    M. Akritas. Aligned Rank Tests for Regression With Censored Data. Technical Memorandum. Dept. Statistics. Penn State, 1990.Google Scholar
  2. [Au77]
    C. Auriemma. G.C. Perola, R. Ekers, R. Fanti, C. Lari, W.J. Jaffe, and M.H. Ulrich. A Determination of the Local Radio Luminosity Function of Elliptical Galaxies. Astron. Astrophys. 57, 41, 1977.ADSGoogle Scholar
  3. [Av80]
    Y. Avni, A. Soltan, H. Tananbaum, and G. Zamorani. A Method for Determining Luminosity Functions Incorporating Both Flux Measurements and Flux Upper Limits, with Applications to the Average X-ray to Optical Luminosity Radio for Quasars. Astrophys. J. 238. 800. 1980.ADSCrossRefGoogle Scholar
  4. [Av86]
    Y. Avni and H. Tananbaum. X-Ray Properties of Optically Selected QSOs. Astrophys. J. 305. 83, 1986.ADSCrossRefGoogle Scholar
  5. [Be69]
    P.R. Bevington. Data Reduction and Error Analysis for the Physical Sciences. McGraw-Hill, New York, 1969.Google Scholar
  6. [Br74]
    B.W.M. Brown. M. Hollander, and R.M. Korwar. In Reliability and Biometry (Eds. F. Proschan and R.J. Serfling), SIAM, Philadelphia, Pa., 1974.Google Scholar
  7. [Bu79]
    J. Buckley and I. James. Linear Regression with Censored Data. Biometrika 66, 429, 1979.MATHCrossRefGoogle Scholar
  8. [Ch87]
    P. Chameraux. A Statistical Method to Derive the True Distribution of an Astronomical Parameter Some Value of Which Are Known Only by Limits. Astron. Astrophys. 177, 326. 1987.ADSGoogle Scholar
  9. [Co72]
    D.R. Cox. Regression Models and Life-Tables. J. Roy. Stat. Soc. B 34, 187, 1972.MATHGoogle Scholar
  10. [Da88]
    O. Dahari and M.M. Robertis. Extinction, Profile Asymmetry and Tidal Effects in Seyfert and Starburst Galaxies. Astrophys. J. 331, 727, 1988.ADSCrossRefGoogle Scholar
  11. [Da89]
    B.J. Dain. D.H. Freeman, and J.J. Vredenburgh. Comparison of Difference Packages” Survival Test Results. In 1989 Proceedings of the Statatistical Computing Section, Washington, D.C., Amer. Stat. Assoc, 1989, p. 315.Google Scholar
  12. [De77]
    A.P. Dempster, N.M. Laird, and D.B. Rubin. Maximum-Likelihood from Incomplete Data via EM Algorithm. J. Roy. Stat. Soc. B 39, 1. 1977.MathSciNetMATHGoogle Scholar
  13. [Ef67]
    B. Efron. The Two Sample Problem in Censored Data. Proc. 5th Berkeley Symp. Math. Statist. 4 831, 1967.Google Scholar
  14. [Fa88]
    G. Fabbiano, I.M. Gioia, and G. Trinchieri. A Five-Band Study of Spiral Galaxies: X-Ray. Optical, Near- and Far Infrared, and Radio Continuum Correlations. Astrophys. J. 324, 749, 1988.ADSCrossRefGoogle Scholar
  15. [Fe83]
    E.D. Feigelson and C.J. Berg. X-Ray Observations of 20 3CR Radio Galaxies and Their Environs. Astrophys. J. 269, 400.Google Scholar
  16. [Fe85]
    E.D. Feigelson and P.I. Nelson. Statistical Methods for Astronomical Data with Upper Limits. I. Univariate Distributions. Astrophys. J. 293. 192, 1985.MathSciNetADSCrossRefGoogle Scholar
  17. [Fe90]
    E.D. Feigelson. Censored Data in Astronomy. In Errors, Bias and Uncertainties in Astronomy (Eds. C. Jaschek and F. Murtagh), Cambridge Univ. Press, Cambridge, U.K., 1990, p. 213.Google Scholar
  18. [He89]
    T.M. Heckman, L. Blitz, A.S. Wilson, L. Annus, and G.K. Miley. A Millimeter-Wave Survey of CO Emission in Seyfert Galaxies. Astrophys. J. 342, 735, 1989.ADSCrossRefGoogle Scholar
  19. [Ho89]
    I.D. Howarth and R.K. Prinja. The Stellar Winds of 203 Galactic O Stars: A Quantitative Ultraviolet Survey. Astrophys. J. Suppl. 69, 527, 1989.ADSCrossRefGoogle Scholar
  20. [Hu81]
    E. Hummel. The Radio Continuum Properties of Spiral Galaxies. Astron. Astrophys. 93, 93, 1981.ADSGoogle Scholar
  21. [Is86]
    T. Isobe, E.D. Feigelson, and P.I. Nelson. Statistical Methods for Astronomical Data with Upper Limits. II. Correlation and Regression. Astrophys. J. 306, 490, 1986.ADSCrossRefGoogle Scholar
  22. [Is90]
    T. Isobe and E.D. Feigelson. Bull. Amer. Astro. Soc. 22, 917, 1990.ADSGoogle Scholar
  23. [Je89]
    C.R. Jenkins. A Statistical Test for Comparing Luminosity Functions. Observatory 109, 69, 1989.ADSGoogle Scholar
  24. [Ka80]
    J.D. Kalbfleisch and R.L. Prentice. The Statistical Analysis of Failure Time Data, Wiley, New York, 1980.MATHGoogle Scholar
  25. [Ka58]
    E.L. Kaplan and P. Meier. Nonparametric Estimation from Incomplete Data. J. Am. Stat. Assoc. 53, 457, 1958.MathSciNetMATHCrossRefGoogle Scholar
  26. [Kn85]
    G.R. Knapp, E.L. Turner, and P.E. Cuniffe. The Statistical Distribution of the Neutral-Hydrogen Content of Elliptical Galaxies. Astron. J. 90, 454, 1985.ADSCrossRefGoogle Scholar
  27. [La82]
    J.F. Lawless. Statistical Models and Methods for Lifetime Data, Wiley, New York, 1982.MATHGoogle Scholar
  28. [Ma83]
    G.S. Maddala. Limited-Dependent and Qualitative Variables in Econometrics, Cambridge Univ. Press, Cambridge, U.K., 1983.MATHGoogle Scholar
  29. [Ma87]
    H.L. Marshall. The Radio Luminosity Function of Optically Selected Quasars. Astrophys. J. 316, 84, 1987.ADSCrossRefGoogle Scholar
  30. [Ma88]
    C. Magri, M.P. Haynes. W. Forman, C. Jones, and R. Giovanelli. The Pattern of Gas Defficiency in Cluster Spirals: The Correlation of H I and X-Ray Properties. Astrophys. J. 333, 136, 1988.ADSCrossRefGoogle Scholar
  31. [Mi88]
    G. Micela, S. Sciortino, G.S. Vaiana, J.H.M.M. Schmitt, R.A. Stern, F.R. Harnden Jr., and R. Rosner. The Einstein Observatory Survey of Stars in the Hyades Cluster Region. Astrophys. J. 325, 798, 1988.ADSCrossRefGoogle Scholar
  32. [Mi81]
    R.G. Miller, Jr. Survival Analysis, Wiley, New York, 1980.Google Scholar
  33. [Mo88]
    L.A. Molnar, M.J. Reid, and J.E. Grindlay. VLBI Observations of Expansion in Cyngus X-3. Astrophys. J. 331, 494, 1988.ADSCrossRefGoogle Scholar
  34. [Oa82]
    D. Oakes. A Concordance Test for Independence in the Presence of Censoring. Biometrics 38, 451, 1982.MATHCrossRefGoogle Scholar
  35. [Pf82]
    J. Pfleiderer and P. Krommidas. Statistics Under Incomplete Knowledge of Data. Mon. Not. Royal Astr. Soc. 200, 687, 1982.ADSGoogle Scholar
  36. [Pr86]
    W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling. Numerical Recipes, The Art of Scientific Computing, Cambridge Univ. Press. Cambridge. U.K., 1986, Chap. 13–14.MATHGoogle Scholar
  37. [Sa89]
    E.M. Sadler, C.R. Jenkins, and C.G. Kotanyi. Low-Luminosity Radio Sources in Early-Type Galaxies. Mon. Not. Royal Astro. Soc. 240, 591, 1989.ADSGoogle Scholar
  38. [Sc76]
    P. Schechter. An Analytic Expression for the Luminosity Function for Galaxies. Astrophys. J. 203, 297, 1976.ADSCrossRefGoogle Scholar
  39. [Sc79]
    J. Schmee and F.J. Hahn. Technometrics 21, 417, 1979.Google Scholar
  40. [Sc85]
    J.H.M.M. Schmitt. Statistical Analysis of Astronomical Data Containing Upper Bounds: General Methods and Examples Drawn from X-Ray Astronomy. Astrophys. J. 293, 178. 1985.MathSciNetADSCrossRefGoogle Scholar
  41. [Tr53]
    R.J. Trumpler and H.F. Weaver. Statistical Astronomy, U. Cal. Press, Berkeley, Calif., 1953.MATHGoogle Scholar
  42. [Ty82]
    D. Tytler. QSO Lyman Limit Absorption. Nature 298, 427, 1982.ADSCrossRefGoogle Scholar
  43. [Va85]
    Y. Vardi. Selection Bias Models. Ann. Stat. 13, 178, 1985.CrossRefGoogle Scholar
  44. [Wa85]
    A.E. Wagner and W.Q. Meeker, Jr. A Survey of Statistical Software for Life Data Analysis. In Proceedings of the Statistical Computing Section, Washington, D.C., Amer. Stat. Assoc, 1985, p. 441.Google Scholar
  45. [Wa89]
    D.E.P. Walsh, G.R. Knapp, J.M. Wrobel, and D.-W. Kim. Interstellar Matter in Early-Type Galaxies. III. Radio Emission and Star Formation. Astrophys. J. 337, 209, 1989.Google Scholar
  46. [Wa86]
    M. Wardle and G.R. Knapp. The Statistical Distribution of the Neutral-Hydrogen Content of SO Galaxies. Astron. J. 91. 23. 1989.ADSCrossRefGoogle Scholar
  47. [Wo79]
    M.S. Wolynetz. Applied Stat 28, 185, 1979 (and later corrigenda).MATHCrossRefGoogle Scholar
  48. [Wo87]
    D.M. Worrall, P. Giommi, H. Tananbaum, and G. Zamorani. X-Ray Studies of Quasars with the Einstein Observatory. IV. X-Ray Dependence on Radio Emission. Astrophys. J. 313, 596, 1987.ADSCrossRefGoogle Scholar
  49. [Yu87]
    K.N. Yu. The X-Ray Properties of Quasars Determined from an Iterative Multiple Regression Analysis with Censored Data. Astrophys. Space Sci. 137, 93, 1987.ADSCrossRefGoogle Scholar

Copyright information

© Springer-Verlag New York, Inc. 1992

Authors and Affiliations

  • Eric D. Feigelson
    • 1
  1. 1.Department of Astronomy & AstrophysicsPennsylvania State UniversityUniversity ParkUSA

Personalised recommendations