Sensitivity to Hidden Bias

  • Paul R. Rosenbaum
Part of the Springer Series in Statistics book series (SSS)


Cornfield, Haenszel, Hammond, Lilienfeld, Shimkin, and Wynder (1959) first conducted a formal sensitivity analysis in an observational study. Their paper is a survey of the evidence available in 1959 linking smoking with lung cancer. The paper asks whether the association between smoking and lung cancer is an effect caused by smoking or whether it is instead due to a hidden bias. Can lung cancer be prevented by not smoking? Or are the higher rates of lung cancer among smokers due to some other difference between smokers and nonsmokers?


Motor Neuron Disease Multiple Control Hide Bias Vaginal Cancer Concordant Pair 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aakvik, A. (2001) Bounding a matching estimator: The case of a Norwegian training program. Oxford Bulletin of Economics and Statistics, 63, 115–143.CrossRefGoogle Scholar
  2. Aiken, L. H., Smith, H. L., and Lake, E. T. (1994) Lower Medicare mortality among a set of hospitals known for good nursing care. Medical Care, 32, 771–787.CrossRefGoogle Scholar
  3. Armitage, P. (1977) Statistical Methods in Medical Research (fourth printing), Oxford, UK: Blackwell.Google Scholar
  4. Balke, A. and Pearl, J. (1997) Bounds on treatment effects from studies with imperfect compliance. Journal of the American Statistical Association, 92, 1172–1176.CrossRefGoogle Scholar
  5. Bickel, P. and van Zwet, (1978) Asymptotic expansions for the power of distribution free tests in the two-sample problem. Annals of Statistics, 6, 937–1004.MathSciNetzbMATHCrossRefGoogle Scholar
  6. Boston Collaborative Drug Pro j ect (1972) Excess of ampicillin rashes associated with allopurinol or hyperuricemia. New England Journal of Medicine, 286, 505–507.CrossRefGoogle Scholar
  7. Breslow, N. and Day, N. (1980) Statistical Methods in Cancer Research, I: The Analysis of Case-Control Studies. Lyon, France: International Agency for Research on Cancer.Google Scholar
  8. Bross, I. D. J. (1966) Spurious effects from an extraneous variable. Journal of Chronic Diseases, 19, 637–647.CrossRefGoogle Scholar
  9. Bross, I. D. J. (1967) Pertinency of an extraneous variable. Journal of Chronic Diseases, 20, 487–495.CrossRefGoogle Scholar
  10. Cameron, E. and Pauling, L. (1976) Supplemental ascorbate in the supportive treatment of cancer: Prolongation of survival times in terminal human cancer. Proceedings of the National Academy of Sciences (USA), 73, 3685–3689.CrossRefGoogle Scholar
  11. Copas, J. B. and Li, H. G. (1997) Inference for non-random samples (with discussion). Journal of the Royal Statistical Society, B, 59, 55–96.MathSciNetzbMATHCrossRefGoogle Scholar
  12. Cornfield, J., Haenszel, W., Hammond, E., Lilienfeld, A., Shimkin, M., and Wynder, E. (1959). Smoking and lung cancer: Recent evidence and a discussion of some questions. Journal of the National Cancer Institute, 22, 173–203.Google Scholar
  13. Davanzo, P., Thomas, M. A., Yue, K., Oshiro, T., Belin, T., Strober, M., and McCracken, J. (2001) Decreased anterior cingulate myoinositol/creatine spectoscopy resonance with lithium treatment in children with bipolar disorder. Neuropsychopharmacology, 24, 359 – 369.CrossRefGoogle Scholar
  14. Eaton, M. (1987) Lectures on Topics in Probability Inequalities. Amsterdam: Centrum voor Wiskunde en Informatica.zbMATHGoogle Scholar
  15. Eidson, M., Philen, R. M., Sewell, C. M., Voorhees, R., and Kilbourne, E. M. (1990) L-tryptophan and eosinophilia-myalgia syndrome in New Mexico. Lancet, 335, 645–648.CrossRefGoogle Scholar
  16. Fisher, R.A. (1958) Lung cancer and cigarettes? Nature, 182, July 12, 108.CrossRefGoogle Scholar
  17. Freidlin, B. and Gastwirth, J. L. (2000) Should the median test be retired from general use? American Statistician, 54, 161–164.Google Scholar
  18. Gastwirth, J. L. (1988) Statistical Reasoning in Law and Public Policy. New York: Academic.zbMATHGoogle Scholar
  19. Gastwirth, J. L. (1992a) Employment discrimination: A statistician’s look at analysis of disparate impact claims. Law and Inequality, 11, 151 – 179.Google Scholar
  20. Gastwirth, J. L. (1992b) Methods for assessing the sensitivity of statistical comparisons used in Title VII cases to omitte variahles Jurimetrics Journal, 19–34.Google Scholar
  21. Gastwirth, J. L. and Greenhouse, S. (1987) Estimating a common relative risk: Application in equal employment. Journal of the American Statistical Association, 82, 38–45.MathSciNetzbMATHCrossRefGoogle Scholar
  22. Gastwirth, J. L., Krieger, A. M. and Rosenbaum, P. R. (1998a) Cornfield’s inequality. In: Encyclopedia of Biostatistics, P. Armitage and T. Colton, eds., New York: Wiley, pp. 952–955.Google Scholar
  23. Gastwirth, J. L., Krieger, A. M., and Rosenbaum, P. R. (1998b) Dual and simultaneous sensitivity analysis for matched pairs. Biometrika, 85, 907–920.zbMATHCrossRefGoogle Scholar
  24. Gastwirth, J. L., Krieger, A. M., and Rosenbaum, P. R. (2000) Asymptotic separability in sensitivity analysis. Journal of the Royal Statistical Society, Series B, 62, 545–555.MathSciNetzbMATHCrossRefGoogle Scholar
  25. Gibbons, J. D. (1982) Brown-Mood median test. In: Encyclopedia of Statistical Sciences, Volume 1, S. Kotz and N. Johnson, eds., New York: Wiley, pp. 322–324.Google Scholar
  26. Graham, A. J., Macdonald, A. M., and Hawkes, C. H. (1997) British motor neuron disease twin study. Journal of Neurology, Neurosurgery and Psychiatry, 6, 562–569.CrossRefGoogle Scholar
  27. Greenhouse, S. (1982) Jerome Cornfield’s contributions to epidemiology. Biometrics, 38S, 33–46.CrossRefGoogle Scholar
  28. Greenland, S. (1996) Basic methods of sensitivity analysis of biases. International Journal of Epidemiology, 25, 1107–1116.Google Scholar
  29. Gu, X. S. and Rosenbaum, P. R. (1993) Comparison of multivariate matching methods: Structures, distances and algorithms. Journal of Computational and Grapical Statistics, 2, 405–420.Google Scholar
  30. Hájek, J., Sidák, Z. and Sen, P. K. (1999) Theory of Rank Tests (Second Edition) . New York: Academic.zbMATHGoogle Scholar
  31. Hakulinen, T. (1981) A Mantel-Haenszel statistic for testing the association between a polychotomous exposure and a rare outcome. American Journal of Epidemiology, 113 ,192–197.Google Scholar
  32. Hammond, E. C. (1964) Smoking in relation to mortality and morbidity: Findings in first thirty-four months of follow-up in a prospective study started in 1959. Journal of the National Cancer Institute, 32, 1161 – 1188.Google Scholar
  33. Hannan, J. and Harkness, W. (1963) Normal approximation to the distribution of two independent binomials, conditional on a fixed sum. Annals of Mathematical Statistics, 34, 1593–1595.MathSciNetzbMATHCrossRefGoogle Scholar
  34. Harkness, W. (1965) Properties of the extended hypergeometric distribution. Annals of Mathematical Statistics, 36, 938–945.MathSciNetzbMATHCrossRefGoogle Scholar
  35. Herbst, A., Ulfelder, H., and Poskanzer, D. (1971) Adenocarcinoma of the vagina: Association of maternal stilbestrol therapy with tumor appearance in young women. New England Journal of Medicine, 284, 878–881.CrossRefGoogle Scholar
  36. Hettmansperger, T. (1984) Statistical Inference Based on Ranks. New York: Wiley.zbMATHGoogle Scholar
  37. Hodges, J. and Lehmann, E. (1962) Rank methods for combination of individual experiments in the analysis of variance. Annals of Mathematical Statistics, 33, 482–497.MathSciNetzbMATHCrossRefGoogle Scholar
  38. Hodges, J. and Lehmann, E. (1963) Estimates of location based on rank tests. Annals of Mathematical Statistics, 34, 598–611.MathSciNetzbMATHCrossRefGoogle Scholar
  39. Hollander, M., Proschan, F., and Sethuraman, J. (1977) Functions decreasing in transposition and their applications in ranking problems. Annals of Statistics, 5, 722–733.MathSciNetzbMATHCrossRefGoogle Scholar
  40. Hollander, M. and Wolfe, D. (1973, 1999) Nonparametric Statistical Methods. New York: Wiley.zbMATHGoogle Scholar
  41. Holley, R. (1974) Remarks on the FKG inequalities. Communications in Mathematical Physics, 36, 227–231.MathSciNetCrossRefGoogle Scholar
  42. Jick, H., Miettinen, O., Neff, R., et al. (1973) Coffee and myocardial infarction. New England Journal of Medicine, 289, 63–77.CrossRefGoogle Scholar
  43. Johnson, N., Kotz, S., and Kemp, A. (1992) Univariate Discrete Distributions. New York: Wiley.zbMATHGoogle Scholar
  44. Kelsey, J. and Hardy, R. (1975) Driving of motor vehicles as a risk factor for acute herniated lumbar intervertebral disc. American Journal of Epidemiology, 102, 63–73.Google Scholar
  45. Kevekordes, S., Gebel, T. W., Hellwig, M., Dames, W., and Dunkelberg, H. (1998) Human effect monitoring in cases of occupational exposure to antineoplastic drugs: a method comparison. Occupational and Environmental Medicine, 55, 145–149.CrossRefGoogle Scholar
  46. Krieger, A. M. and Rosenbaum, P. R. (1994) A stochastic comparison for arrangement increasing functions. Combinatorics, Probability and Computing, 3, 345–348.MathSciNetzbMATHCrossRefGoogle Scholar
  47. Lehmann, E. (1975) Nonparametrics: Statistical Methods Based on Ranks. San Francisco: Holden-Day.zbMATHGoogle Scholar
  48. Lin, D. Y., Psaty, B. M., and Kronmal, R. A. (1998) Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics, 54, 948–963.zbMATHCrossRefGoogle Scholar
  49. Mack, T., Pike, M., Henderson, B., Pfeffer, R., Gerkins, V., Arthur, B., and Brown, S. (1976) Estrogens and endometrial cancer in a retirement community. New England Journal of Medicine, 294, 1262–1267.CrossRefGoogle Scholar
  50. Mann, H. and Whitney, D. (1947) On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics, 18, 50–60.MathSciNetzbMATHCrossRefGoogle Scholar
  51. Manski, C. (1990) Nonparametric bounds on treatment effects. American Economic Review, 319–323.Google Scholar
  52. Manski, C. (1995) Identification Problems in the Social Sciences. Cambridge, MA: Harvard University Press.Google Scholar
  53. Mantel, N. and Haenszel, W. (1959) Statistical aspects of retrospective studies of disease. Journal of the National Cancer Institute, 22, 719–748.Google Scholar
  54. Marcus, S. (1997) Using omitted variable bias to assess uncertainty in the estimation of an AIDS education treatment effect. Journal of Educational and Behavioral Statistics, 22, 193–202.Google Scholar
  55. Marshall, A. and Olkin, I. (1979) Inequalities: Theory of Majorization and Its Applications. New York: Academic.zbMATHGoogle Scholar
  56. McNemar, Q. (1947) Note on the sampling errorof the differences between correlated proportions or percentages. Psychometrika, 12, 153–157.CrossRefGoogle Scholar
  57. Miettinen, O. (1969) Individual matching with multiple controls in the case of all or none responses. Biometrics, 22, 339–355.MathSciNetCrossRefGoogle Scholar
  58. Ming, K. and Rosenbaum, P. R. (2000) Substantial gains in bias reduction from matching with a variable number of controls. Biometrics, 56, 118–124.zbMATHCrossRefGoogle Scholar
  59. Moertel, C., Fleming, T., Creagan, E., Rubin, J., O’Connell, M., and Ames, M. (1985) High-dose vitamin C vs placebo in the treatment of patients with advanced cancer who have had no prior chemotherapy: A randomized double-blind comparison. New England Journal of Medicine, 312, 137–141.CrossRefGoogle Scholar
  60. Morton, D., Saah, A., Silberg, S., Owens, W., Roberts, M., and Saah, M. (1982) Lead absorption in children of employees in a lead related industry. American Journal of Epidemiology, 115, 549–555.Google Scholar
  61. Mosteller, F. and Tukey, J. (1977) Data Analysis and Regression. Reading, MA: Addison-Wesley.Google Scholar
  62. Nicholson, W., Selikoff, I., Seidman, H., Lilis, R., and Formby, P. (1979) Long-term mortality experience of chrysotile miners and millers in Thetford Mines, Quebec. Annals of the New York Academy of Sciences, 330, 11–21.CrossRefGoogle Scholar
  63. Normand, S. T., Landrum, M. B., Guadagnoli, E., Ayanian, J. Z., Ryan, T. J., Cleary, P. D., and McNeil, B. J. (2001) Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: A matched analysis using propensity scores. Journal of Clinical Epidemiology, 54, 387–398.CrossRefGoogle Scholar
  64. O’Brien, P. C. and Fleming, T. R. (1987) A paired Prentice-Wilcoxon test for censored paired data. Biometrics, 43, 169–180.CrossRefGoogle Scholar
  65. Pearl, J. (2000) Causality. New York: Cambridge University Press.zbMATHGoogle Scholar
  66. Peters, J. M., Preston-Martin, S., and Yu, M. C. (1981) Brain tumors in children and occupational exposure of parents. Science, 213, 235–236.CrossRefGoogle Scholar
  67. Plackett, R. L. (1981) The Analysis of Categorical Data (second edition). New York: Macmillan.zbMATHGoogle Scholar
  68. Psaty, B. M., Koepsell, T. D., Lin, D., Weiss, N., Siscovick, D. S., Rosendaal, F. R., Pahor, M., and Furberg, C. D. (1999) Assessment and control for confounding by indication in observational studies. Journal of the American Geriatrics Society, 47, 749–754.Google Scholar
  69. Quade, D. (1979) Using weighted rankings in the analysis of complete blocks with additive block effects. Journal of the American Statistical Association, 74, 680–683.MathSciNetzbMATHCrossRefGoogle Scholar
  70. Rosenbaum, P. R. (1986) Dropping out of high school in the United States: An observational study. Journal of Educational Statistics, 11, 207–224.CrossRefGoogle Scholar
  71. Rosenbaum, P. R. (1987) Sensitivity analysis for certain permutation inferences in matched observational studies. Biometrika, 74, 13–26.MathSciNetzbMATHCrossRefGoogle Scholar
  72. Rosenbaum, P. R. (1988) Sensitivity analysis for matching with multiple controls. Biometrika, 75, 577–581.MathSciNetzbMATHCrossRefGoogle Scholar
  73. Rosenbaum, P. R. (1989) On permutation tests for hidden biases in observational studies: An application of Holley’s inequality to the Savage lattice. Annals of Statistics, 17, 643–653.MathSciNetzbMATHCrossRefGoogle Scholar
  74. Rosenbaum, P. R. (1991a) Sensitivity analysis for matched case-control studies. Biometrics, 47, 87–100.MathSciNetzbMATHCrossRefGoogle Scholar
  75. Rosenbaum, P. R. (1991b) Discussing hidden bias in observational studies. Annals of Internal Medicine, 115, 901–905.Google Scholar
  76. Rosenbaum, P. R. (1991c) A characterization of optimal designs for observational studies. Journal of the Royal Statistical Society, Series B,53, 597–610.MathSciNetzbMATHGoogle Scholar
  77. Rosenbaum, P. R. (1993) Hodges-Lehmann point estimates of treatment effect in observational studies. Journal of the American Statistical Association, 88, 1250–1253.MathSciNetzbMATHCrossRefGoogle Scholar
  78. Rosenbaum, P. R. (1995) Quantiles in nonrandom samples and observational studies. Journal of the American Statistical Association, 90, 11424–1431.MathSciNetCrossRefGoogle Scholar
  79. Rosenbaum, P. R. (1999a) Using combined quantile averages in matched observational studies. Applied Statistics, 48, 63–78.zbMATHGoogle Scholar
  80. Rosenbaum, P. R. (1999b) Reduced sensitivity to hidden bias at upper quantiles in observational studies with dilated effects. Biometrics, 55, 560–564.zbMATHCrossRefGoogle Scholar
  81. Rosenbaum, P. R. (1999) Holley’s inequality. In: Encyclopedia of Statistical Sciences, Update Volume 3, S. Kotz, C. B. Read, and D. L. Banks, eds., New York: Wiley, pp. 329–331.Google Scholar
  82. Rosenbaum, P. R. (2001) Effects attributable to treatment: Inference in experiments and observational studies with a discrete pivot. Biometrika, 88, 219–232.MathSciNetzbMATHCrossRefGoogle Scholar
  83. Rosenbaum, P. R. and Krieger, A. M. (1990) Sensitivity analysis for twosample permutation inferences in observational studies. Journal of the American Statistical Association, 85, 493–498.zbMATHCrossRefGoogle Scholar
  84. Rosenbaum, P. R. and Rubin, D. B. (1983) Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. Journal of the Royal Statistical Society, Series B,45, 212 –218.Google Scholar
  85. Rubin, D. B. (1978) Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6, 34–58.MathSciNetzbMATHCrossRefGoogle Scholar
  86. Savage, I. R. (1964) Contributions to the theory of rank order statistics: Applications of lattice theory. Review of the International Statistical Institute, 32, 52–63.MathSciNetzbMATHCrossRefGoogle Scholar
  87. Schlesselmann, J. J. (1978) Assessing the effects of confounding variables. American Journal of Epidemiology, 108, 3–8.Google Scholar
  88. Skerfving, S., Hansson, K., Mangs, C., Lindsten, J., and Ryman, N. (1974) Methylmercury-induced chromosome damage in man. Environmental Research, 7, 83–98.CrossRefGoogle Scholar
  89. Smith, H. L. (1997) Matching with multiple controls to estimate treatment effects in observational studies. Sociological Methodology, 27, 325 – 353.CrossRefGoogle Scholar
  90. Stevens, W. L. (1951) Mean and variance of an entry in a contingency table. Biometrika, 38, 468–470.MathSciNetzbMATHGoogle Scholar
  91. Tardif, S. (1987) Efficiency and optimality results for tests based on weighted rankings. Journal of the American Statistical Association, 82, 637–644.MathSciNetzbMATHCrossRefGoogle Scholar
  92. Tukey, J. W. (1957) Sums of random partitions of ranks. Annals of Mathematical Statistics, 28, 987–992.MathSciNetzbMATHCrossRefGoogle Scholar
  93. Wilcoxon, F. (1945) Individual comparisons by ranking methods. Biometrics, 1, 80–83.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2002

Authors and Affiliations

  • Paul R. Rosenbaum
    • 1
  1. 1.Department of Statistics, The Wharton SchoolUniversity of PennsylvaniaPhiladelphiaUSA

Personalised recommendations