Computational Statistics

, Volume 15, Issue 3, pp 355–371 | Cite as

Bayesian probabilistic extensions of a deterministic classification model

  • Iwin Leenen
  • Iven Van Mechelen
  • Andrew Gelman


This paper extends deterministic models for Boolean regression within a Bayesian framework. For a given binary criterion variable Y and a set of k binary predictor variables X1,…, Xk, a Boolean regression model is a conjunctive (or disjunctive) logical combination consisting of a subset S of the X variables, which predicts Y. Formally, Boolean regression models include a specification of a k-dimensional binary indicator vector (θ1,…,θk) with θj = 1 iff XjS. In a probabilistic extension, a parameter π is added which represents the probability of the predicted value \({\hat y_i}\) and the observed value yi differing (for any observation i). Within a Bayesian framework, a posterior distribution of the parameters (θ1,…, θk, π) is looked for. The advantages of such a Bayesian approach include a proper account of the uncertainty in the model estimates and various possibilities for model checking (using posterior predictive checks). We illustrate this method with an example using real data.


Bayesian Estimation Boolean Regression Logical Rule Analysis Posterior Predictive Checks 


  1. Biswas, N.N. (1975). Introduction to logic and switching theory. New York: Gordon and Breach.zbMATHGoogle Scholar
  2. Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (1995). Bayesian data analysis. London: Chapman & Hall.CrossRefGoogle Scholar
  3. Gelman, A., Leenen, I., Van Mechelen, I., & De Boeck, P. (1999). Bridges between deterministic and probabilistic models for binary data. Manuscript submitted for publication.Google Scholar
  4. Gelman, A., Meng, X.L., & Stern, H.S. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statistica Sinica, 6, 733–807.MathSciNetzbMATHGoogle Scholar
  5. Gelman, A., & Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511.CrossRefGoogle Scholar
  6. Haider, A.K. (1978). Grouping table for the minimization of n-variable Boolean functions. Proceedings of the Institution of Electric Engineers London, 125, 474–482.CrossRefGoogle Scholar
  7. Leenen, I., & Van Mechelen, I. (1998). A branch-and-bound algorithm for Boolean regression. In: I. Balderjahn, R. Mathar, & M. Schader (Eds.), Data Highways and Information Flooding, a Challenge for Classification and Data Analysis (pp. 164–171). Berlin: Springer-Verlag.CrossRefGoogle Scholar
  8. McCluskey, E.J. (1965). Introduction to the theory of switching circuits. New York: McGraw-Hill.zbMATHGoogle Scholar
  9. McKenzie, D.M., Clarke, D.M., & Low, L.H. (1992). A method of constructing parsimonious diagnostic and screening tests. International Journal of Methods in Psychiatric Research, 2, 71–79.Google Scholar
  10. Mickey, M.R., Mundle, P., & Engelman, L. (1983). Boolean factor analysis. In W.J. Dixon (Ed.), BMDP statistical software (pp. 538–545, p. 692). Berkeley, CA: California Press.Google Scholar
  11. Ragin, C.C., Mayer, S.E., & Drass, K.A. (1984). Assessing discrimination: A Boolean approach. American Sociological Review, 49, 221–234.CrossRefGoogle Scholar
  12. Rubin, D.B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151–1172.MathSciNetCrossRefGoogle Scholar
  13. Sen, M. (1983). Minimization of Boolean functions of any number of variables using decimal labels. Information Sciences, 30, 37–45.MathSciNetCrossRefGoogle Scholar
  14. Sneath, P.H.A., & Sokal, R.R. (1973). Numerical taxonomy. San Francisco: Freeman.zbMATHGoogle Scholar
  15. Tversky, A. (1977). Features of similarity. Psychological Review, 84, 327–352.CrossRefGoogle Scholar
  16. Van Mechelen, I. (1988). Prediction of a dichotomous criterion variable by means of a logical combination of dichotomous predictors. Mathémathiques, Informatique et Sciences Humaines, 102, 47–54.MathSciNetzbMATHGoogle Scholar
  17. Van Mechelen, I., & De Boeck, P. (1990). Projection of a binary criterion into a model of hierarchical classes. Psychometrika, 55, 677–694.CrossRefGoogle Scholar
  18. Wierzbicka, A. (1992). Defining emotion concepts. Cognitive Science, 16, 539–581.CrossRefGoogle Scholar

Copyright information

© Physica-Verlag 2000

Authors and Affiliations

  • Iwin Leenen
    • 1
  • Iven Van Mechelen
    • 1
  • Andrew Gelman
    • 2
  1. 1.Department of PsychologyUniversity of LeuvenLeuvenBelgium
  2. 2.Department of StatisticsColumbia UniversityNew YorkUSA

Personalised recommendations