The Utility of the Hui-Walter Paradigm for the Evaluation of Diagnostic Test in the Analysis of Social Science Data

  • Michael D. Sinclair
  • Joseph L. Gastwirth
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 114)


Just as in medical research, social scientists are concerned with the correct classification of individuals into well defined categories. Many economic policy decisions rely on the unemployment rate and related labor statistics. As the unemployment rate is the ratio of the estimated number of unemployed persons to the total labor force, misclassification of survey respondents may lead to an under or over estimate of it. Thus, estimating the accuracy of the original interview is quite important and the Census Bureau conducts a special reinterview study of about 20,000 respondents per year to monitor their error rates. In law, a large body of research (Hans and Vidmar; 1991, Blank and Rosenthal; 1991) has raised questions about how well the jury functions. The basic problem can be placed in the classification frame work. How well does the current system perform in correctly determining that a guilty party is found guilty and in not convicting an individual who should be acquitted ? This article reports some exploratory work we have carried out on extending and modifying the Hui-Walter methodology for evaluating the accuracy of diagnostic tests (see Vianna, 1995, for related work) to enable us to estimate the accuracy of the labor force data and to reanalyze a classic study (Kalven and Zeisel, 1966) of judge-jury agreements to estimate the accuracy of jury verdicts.


Error Rate Unemployment Rate Equal Error Rate Original Survey Taylor Series Approximation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bard, Y., Nonlinear Parameter Estimation, Academic Press, 1974.zbMATHGoogle Scholar
  2. Bailar, B.A., “Recent Research in Reinterview Procedures,” Journal of the American Statistical Association, 63 (1968), 41–63.MathSciNetCrossRefGoogle Scholar
  3. Blank, P.D., “What Empirical Research Tells us: Studying Judges and Juries’ Behavior,” American University Law Review, 40 (1991), 775.MathSciNetGoogle Scholar
  4. Brookmeyer, R. and Gail, M., Aids Epidemiology, a Quantitative Approach, 1994.Google Scholar
  5. Biemer, P.P., Groves, R.M., Lyberg, L.E. Mathiowetz, N.A. and Sudman, S., Measurement Errors in Surveys, New York: John Wiley & Sons, 1991.zbMATHGoogle Scholar
  6. Bureau of Labor Statics, “Overhauling the Population Survey,” Monthly Labor Review, Vol. 116, No. 9, Washington D.C.: U.S. Government Printing Office, 1993.Google Scholar
  7. Bureau of Labor Statistics, Employment and Earnings, 38(8) Washington DC: U.S. Government Printing Office, 1992.Google Scholar
  8. Chua, T.C. and Fuller, W.A., “A Model for Multinomial Response Error Applied to Labor Flows,” Journal of the American Statistical Association, New York: John Wiley & Sons (1991).Google Scholar
  9. Forsman, G. and Schreiner, I., “The Design and Analysis of Reinterview: An overview,” In Paul Biemer, et al., Eds., Measurement Errors in Surveys, New York: John Wiley & Sons (1991).Google Scholar
  10. Gastwirth, J.L., “The statistical precision of medical screening procedures: Application to polygraph and AIDS antibodies test data,” Statistical Science, 3 (1987), 213.MathSciNetCrossRefGoogle Scholar
  11. Gastwirth, J.L. and Sinclair, M. D., “The Potential Use of Diagnostic Test Methodology in the Design and Analysis of Judge-Jury Agreement Studies,” To appear in Jurimetrics in 1999.Google Scholar
  12. Gastwirth, J.L. and Sinclair, M.D, “Examination of the Kalven-Zeisel Study of Judge-jury Agreements Using Recent Contingency Table Analysis,” Technical Report, Department of Statistics, George Washington University (1987).Google Scholar
  13. Geisser, S., Predictive Inference: An Introduction, Chapman and Hall, New York, 1993, 170–172.zbMATHGoogle Scholar
  14. Hans, V.P, and Vidmar, N., “The American Jury at Twenty-five Years,” 16, Law and Social Inquiry (1991), 323.CrossRefGoogle Scholar
  15. Hausman, J.A. and Morton, S, S., “Misclassification of a Dependent Variables in a Discrete Response Setting,” working paper, Department of Economics, MIT, Cambridge (1994).Google Scholar
  16. Hui, S.L. and Walter, S.D., “Estimating the Error Rates of Diagnostic Tests,” Biometrics, 36 (1980), 167–17.zbMATHCrossRefGoogle Scholar
  17. Hughes, J.J. and Perlman, R., The Economics of Unemployment (New York: Cambridge Univ. Press), 1984.Google Scholar
  18. Kalven, H. and Zeisel, H., The American Jury, Little Brown and Company, 1966.Google Scholar
  19. Levy, P., and Lemeshow, S., Sampling for Health Professionals (California: Lifetime learning Publications ), 1980.Google Scholar
  20. Mcclish and Quade, “Improving Estimates of Prevalence by Repeated Testing,” Biometrics, 41 (1985), 81–89.CrossRefGoogle Scholar
  21. Mckenna, C.J., Uncertainty and the Labor Market: Recent Developments in Job Search Theory,New York: St. Martins Press.Google Scholar
  22. Rao, J.N.K. and Thomas, D.R., “Chi-squared tests with complex survey data subject to misclassification error,” in Paul Biemer, et al., Eds,“ Measurement Errors in Surveys,New York: Wiley.Google Scholar
  23. Schreiner, I., “Reinterview results from the CPS Independent Reconciliation Experiment (second quarter 1978 through third quarter 1989),” unpublished Bureau of the Census memorandum May 7, 1980.Google Scholar
  24. Shouka, M.M., and EdgeV.L., Statistical Methods for the Health Sciences, CRC Press, New York, Chap. 7, 1996.Google Scholar
  25. Sinclair, M.D. and Gastwirth, J.L., “Estimates of the Errors in Classification in the Labor Force Survey and their Effects on the Reported Unemployment Rate” (to appear in Survey Methodology in 1999 ).Google Scholar
  26. Sinclair, M.D. and Gastwirth, J.L., “On Procedures for Evaluating the Effectiveness of Reinterview Survey Methods: Application to Labor Force Data,” Journal of the American Statistical Association, 91 (435) (1996), 961–969.zbMATHCrossRefGoogle Scholar
  27. Singh, A.C., and J.N.K. Rao, “On the Adjustment of Gross Flow Estimate for Classification Error With Application to Data From the Canadian Labour Force Survey,” The Journal of the American Statistical Association, 90 (430) (1995), 478–488.CrossRefGoogle Scholar
  28. Vianna, M., “Statistical Assessment of Clinical Tests: Annotated Bibliography,” Technical Report, University of Illinois at Chicago, 1995.Google Scholar
  29. Weiter, W. and Diamond, S.S., “A Critical Review of the Jury Simulation Paradigm: The Case of Defendant Characteristics,” Law & Human Behavior, 3 (1979), 71.CrossRefGoogle Scholar
  30. U.S. Bureau of the Census, The Current Population Survey Reinterview Program: Some Notes and Discussion, Technical Paper No. 6, Washington, D.C.: U.S. Government Printing Office (1963).Google Scholar
  31. U.S. Bureau of the Census, Evaluating Censuses of Population and Housing, Statistical Training Document #ISP-TR-5 Washington, D.C.: U.S. Government Printing Office, 1985 (1963).Google Scholar
  32. Walter, S.D. and Irwig, L.M., “Estimation of Test Error Rates, Disease Prevalence, and Relative Risk from Misclassified Data: A Review,” 41, Journal of. Clinical. Epidemiology (1988), 923.CrossRefGoogle Scholar
  33. Wolter, K.M., Introduction to Variance Estimation, Springer-Verlag, 1985, 221.zbMATHGoogle Scholar
  34. Vacek, P.M., “The Effect of Conditional Dependence on the Evaluation of Diagnostic Tests,” Biometrics, 41 (1985), 959.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Michael D. Sinclair
    • 1
  • Joseph L. Gastwirth
    • 2
  1. 1.Mathematica Policy ResearchPlainsboroUSA
  2. 2.Department of StatisticsThe George Washington UniversityUSA

Personalised recommendations