Abstract
This chapter is devoted to introducing various prior processes, their formulation, properties, inter-relationships, and their relative strengths and weaknesses. The sequencing of presentation of these priors reflects mostly the order in which they were discovered and developed. The Dirichlet process and its immediate generalizations—Dirichlet Invariant and Mixtures of Dirichlet—are presented first. The neutral to the right processes and the processes with independent increments, which form the basis for many other processes, are discussed next. They are key in the development of processes that include beta, gamma and extended gamma processes, proposed primarily to address specific applications in the reliability theory, are presented next. Beta-Stacy process which extends the Dirichlet process is discussed thereafter. Following that, tailfree and Polya tree processes are presented which are especially convenient for estimating density functions, and to place greater weights, where it is deemed appropriate, by selecting suitable partitions in developing the prior. In order to extend the nonparametric Bayesian analysis to covariate data, numerous extensions are proposed. They have origin in the Ferguson-Sethuraman infinite sum representation in which the weights are constructed by a stick-breaking construction. They are collectively called here as Ferguson-Sethuraman processes and include dependent and spatial Dirichlet processes, Pitman-Yor process, Chinese restaurant and Indian buffet processes, etc. They all are included in this chapter.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Part of the material of this and the next two subsections is based on Ferguson (1974), Ferguson and Phadia (1979) and Ferguson’s unpublished notes which clarify and provide further insight into the description of the posterior processes neutral to the right. I am grateful to Tom Ferguson for passing on his notes to me which helped in developing these sections.
References
Aalen, O. O. (1978). Nonparametric inference for a family of counting processes. Annals of Statistics, 6, 701–726.
Ammann, L. P. (1984). Bayesian nonparametric inference for quantal response data. Annals of Statistics, 12, 636–645.
Ammann, L. P. (1985). Conditional Laplace transforms for Bayesian nonparametric inference in reliability theory. Stochastic Processes and Their Applications, 20, 197–212.
Antoniak, C. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Annals of Statistics, 2, 1152–1174.
Basu, D., & Tiwari, R. C. (1982). A note on the Dirichlet process. In G. Kallianpur, P. R. Krishnaiah & J. K. Ghosh (Eds.), Statistics and probability: essays in honor of C. R. Rao (pp. 89–103).
Blackwell, D. (1973). Discreteness of Ferguson selections. Annals of Statistics, 1, 356–358.
Blackwell, D., & MacQueen, J. B. (1973). Ferguson distributions via Polya urn schemes. Annals of Statistics, 1, 353–355.
Blum, J., & Susarla, V. (1977). On the posterior distribution of a Dirichlet process given randomly right censored observations. Stochastic Processes and Their Applications, 5, 207–211.
Bulla, P., Muliere, P., & Walker, S. (2007). Bayesian nonparametric estimation of a bivariate survival function. Statistica Sinica, 17, 427–444.
Bulla, P., Muliere, P., & Walker, S. (2009). A Bayesian nonparametric estimator of a multivariate survival function. Journal of Statistical Planning and Inference, 139, 3639–3648.
Christensen, R., Hanson, T., & Jara, A. (2008). Parametric nonparametric statistics: an introduction to mixtures of finite Polya trees. The American Statistician, 62, 296–306.
Chung, Y., & Dunson, D. B. (2011). The local Dirichlet process. Annals of the Institute of Statistical Mathematics, 63, 59–80.
Cifarelli, D. M., & Regazzini, E. (1979a). Considerazioni generali sull’impostazione bayesiana di problemi non parametrici, Part I. Rivista di matematica per le scienze economiche e sociali, 2, 39–52.
Cifarelli, D. M., & Regazzini, E. (1979b). Considerazioni generali sull’impostazione bayesiana di problemi non parametrici, Part II. Rivista di matematica per le scienze economiche e sociali, 2, 95–111.
Connor, R. J., & Mosimann, J. E. (1969). Concept of independence for proportions with a generalization of the Dirichlet distribution. Journal of the American Statistical Association, 64, 194–206.
Dabrowska, D. M. (1988). Kaplan-Meier estimate on the plane. Annals of Statistics, 16, 1475–1489.
Dalal, S. R. (1979a). Dirichlet invariant processes and applications to nonparametric estimation of symmetric distribution functions. Stochastic Processes and Their Applications, 9, 99–107.
Dalal, S. R., & Hall, G. J. (1980). On approximating parametric Bayes models by nonparametric Bayes models. Annals of Statistics, 8, 664–672.
Dalal, S. R., & Phadia, E. G. (1983). Nonparametric Bayes inference for concordance in bivariate distributions. Communications in Statistics. Theory and Methods, 12(8), 947–963.
Damien, P., Laud, P. W., & Smith, A. F. M. (1995). Approximate random variate generation form infinitely divisible distributions with applications to Bayesian inference. Journal of the Royal Statistical Society. Series B. Methodological, 57, 547–563.
Damien, P., Laud, P. W., & Smith, A. F. M. (1996). Implementation of Bayesian non-parametric inference based on beta processes. Scandinavian Journal of Statistics, 23, 27–36.
Dey, D., Müller, P., & Sinha, D. (Eds.) (1998). Lecture notes in statistics. Practical nonparametric and semiparametric Bayesian statistics. New York: Springer.
Dey, J., Erickson, R. V., & Ramamoorthi, R. V. (2003). Some aspects of neutral to right priors. International Statistical Review, 71(2), 383–401.
Doksum, K. A. (1974). Tailfree and neutral random probabilities and their posterior distributions. Annals of Probability, 2, 183–201.
Doss, H. (1984). Bayesian estimation in the symmetric location problem. Zeitschrift Für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 68, 127–147.
Doss, H. (1985a). Bayesian nonparametric estimation of the median; part I: computation of the estimates. Annals of Statistics, 13, 1432–1444.
Doss, H. (1985b). Bayesian nonparametric estimation of the median; part II: asymptotic properties of the estimates. Annals of Statistics, 13, 1445–1464.
Dråghici, L., & Ramamoorthi, R. V. (2000). A note on the absolute continuity and singularity of Polya tree priors and posteriors. Scandinavian Journal of Statistics, 27, 299–303.
Duan, J. A., Guindani, M., & Gelfand, A. E. (2007). Generalized spatial Dirichlet process model. Biometrika, 94, 809–825.
Dubins, L. E., & Freedman, D. A. (1966). Random distribution functions. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, vol. 2: contributions to probability theory, part 1 (pp. 183–214).
Dunson, D. B., & Park, J. H. (2008). Kernel stick-breaking processes. Biometrika, 95, 307–323.
Dykstra, R. L., & Laud, P. (1981). A Bayesian nonparametric approach to reliability. Annals of Statistics, 9, 356–367.
Engen, S. (1975). A note on the geometric series as a species frequency model. Biometrika, 62, 697–699.
Engen, S. (1978). Stochastic abundance models with emphasis on biological communities and species diversity. London: Chapman & Hall.
Escobar, M. D. (1994). Estimating normal means with a Dirichlet process prior. Journal of the American Statistical Association, 89, 268–277.
Escobar, M. D., & West, M. (1995). Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90, 577–588.
Ewens, W. J. (1972). The sampling theory of selectively neutral alleles. Theoretical Population Biology, 3, 87–112.
Fabius, J. (1964). Asymptotic behavior of Bayes estimates. Annals of Mathematical Statistics, 35, 846–856.
Fabius, J. (1973). Neutrality and Dirichlet distributions. In Transactions of the 6th Prague conference on information theory, statistical decision functions and random processes (pp. 175–181).
Feller, W. (1966). An introduction to probability theory and its applications (Vol. II). New York: Wiley
Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1, 209–230.
Ferguson, T. S. (1974). Prior distributions on spaces of probability measures. Annals of Statistics, 2, 615–629.
Ferguson, T. S. (1983). Bayesian density estimation by mixtures of normal distributions. In H. Rizvi & J. S. Rustagi (Eds.), Recent advances in statistics (pp. 287–302). New York: Academic Press.
Ferguson, T. S., & Phadia, E. G. (1979). Bayesian nonparametric estimation based on censored data. Annals of Statistics, 7, 163–186.
Freedman, D. A. (1963). On the asymptotic behavior of Bayes estimates in the discrete case. Annals of Mathematical Statistics, 34, 1386–1403.
Gelfand, A. E., Kottas, A., & MacEachern, S. N. (2005). Bayesian nonparametric spatial modeling with Dirichlet process mixing. Journal of the American Statistical Association, 100, 1021–1035.
Ghahramani, Z., Griffiths, T. L., & Sollich, P. (2007). Bayesian nonparametric latent feature models (with discussion and rejoinder). In J. M. Bernardo et al. (Eds.), Bayesian statistics (Vol. 8). Oxford: Oxford University Press.
Ghosh, J. K., & Ramamoorthi, R. V. (2003). Springer series in statistics. Bayesian nonparametric. New York: Springer.
Ghosh, J. K., Hjort, N. L., Messan, C., & Ramamoorthi, R. V. (2006). Bayesian bivariate survival estimation. Journal of Statistical Planning and Inference, 136, 2297–2308.
Griffin, J. E., & Steel, M. F. J. (2006). Order-based dependent Dirichlet processes. Journal of the American Statistical Association, 101, 179–194.
Griffiths, R. C. (1980). Unpublished notes.
Griffiths, T. L., & Ghahramani, Z. (2006). Infinite latent feature models and the Indian buffet process. In Advances in neural information processing systems (Vol. 18). Cambridge: MIT.
Griffiths, T. L., & Ghahramani, Z. (2011). The Indian buffet process: an introduction and review. Journal of Machine Learning Research, 12, 1185–1224.
Hannum, R. C., Hollander, M., & Langberg, N. A. (1981). Distributional results for random functionals of a Dirichlet process. Annals of Probability, 9, 665–670.
Hanson, T. E. (2006). Inference for mixtures of finite Polya tree models. Journal of the American Statistical Association, 101, 1548–1565.
Hanson, T. E. (2007). Polya trees and their use in reliability and survival analysis. In Encyclopedia of statistics in quality and reliability (pp. 1385–1390). New York: Wiley.
Hanson, T. E., & Johnson, W. O. (2002). Modeling regression error with a mixture of Polya trees. Journal of the American Statistical Association, 97, 1020–1033.
Hanson, T. E., Branscum, A., & Gardner, I. (2008). Multivariate mixtures of Polya trees for modelling ROC data. Statistical Modelling, 8, 81–96.
Hjort, N. L. (1990). Nonparametric Bayes estimators based on beta processes in models for life history data. Annals of Statistics, 18(3), 1259–1294.
Ibrahim, J. L., Chen, M., & Sinha, D. (2001). Bayesian survival analysis. New York: Springer.
Ishwaran, H., & James, L. F. (2001). Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96, 161–173.
Ishwaran, H., & Zarepour, M. (2000). Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. Biometrika, 87, 371–390.
Ishwaran, H., & Zarepour, M. (2002). Exact and approximate sum representation for the Dirichlet process. Canadian Journal of Statistics, 30(2), 269–283.
James, L. F. (2006). Poisson calculus for spatial neutral to the right processes. Annals of Statistics, 34, 416–440.
Johnson, N. L., Kotz, S., & Balkrishnan, N. (1997). Multivariate Ewens distribution. In Discrete multivariate distributions (pp. 232–246). New York: Wiley.
Kalbfleisch, J. D. (1978). Nonparametric Bayesian analysis of survival data. Journal of the Royal Statistical Society. Series B. Methodological, 40, 214–221.
Kaplan, E. L., & Meier, P. (1958). Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53, 457–481.
Kim, Y. (1999). Nonparametric Bayesian estimators for counting processes. Annals of Statistics, 27, 562–588.
Kingman, J. F. C. (1967). Completely random measures. Pacific Journal of Mathematics, 21, 59–78.
Kingman, J. F. C. (1975). Random discrete distributions. Journal of the Royal Statistical Society. Series B. Methodological, 75, 1–22.
Kingman, J. F. C. (1993). Poisson processes. Oxford: Clarendon.
Korwar, R. M., & Hollander, M. (1973). Contributions to the theory of Dirichlet processes. Annals of Probability, 1, 705–711.
Kraft, C. H. (1964). A class of distribution function processes which have derivatives. Journal of Applied Probability, 1, 385–388.
Kraft, C. H., & van Eeden, C. (1964). Bayesian bioassay. Annals of Mathematical Statistics, 35, 886–890.
Lavine, M. (1992). Some aspects of Polya tree distributions for statistical modelling. Annals of Statistics, 20, 1222–1235.
Lavine, M. (1994). More aspects of Polya trees for statistical modelling. Annals of Statistics, 22, 1161–1176.
Lijoi, A., & Prünster, I. (2010). Models beyond the Dirichlet process. In N. L. Hjort et al. (Eds.), Cambridge series in statistical and probabilistic mathematics. Bayesian nonparametrics (pp. 80–136).
Lo, A. Y. (1981). Bayesian nonparametric statistical inference for shock models and wear processes. Scandinavian Journal of Statistics, 8, 237–242.
Lo, A. Y. (1982). Bayesian nonparametric statistical inference for Poisson point processes. Zeitschrift Für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 59, 55–66.
Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates; I. Density estimates. Annals of Statistics, 12, 351–357.
Lo, A. Y. (1986). Bayesian statistical inference for sampling a finite population. Annals of Statistics, 14, 1226–1233.
Lo, A. Y. (1987). A large sample study of the Bayesian bootstrap. Annals of Statistics, 15(1), 360–375.
Lo, A. Y. (1991). A characterization of the Dirichlet process. Statistics & Probability Letters, 12, 185–187.
Lo, A. Y. (1993a). A Bayesian bootstrap for censored data. Annals of Statistics, 21, 100–123.
Lo, A. Y. (1993b). A Bayesian method for weighted sampling. Annals of Statistics, 21, 2138–2148.
MacEachern, S. N. (1998). Computational methods for mixture of Dirichlet process models. In D. Dey, P. Müller, & D. Sinha (Eds.), Practical nonparametric and semiparametric Bayesian statistics (pp. 23–44).
MacEachern, S. N. (1999). Dependent nonparametric processes. In ASA proceedings of the section on Bayesian statistical science. Alexandria: Am. Statist. Assoc.
MacEachern, S. N. (2000). Dependent Dirichlet processes. Unpublished manuscript, The Ohio State University, Department of Statistics.
Mauldin, R. D., Sudderth, W. D., & Williams, S. C. (1992). Polya trees and random distributions. Annals of Statistics, 20, 1203–1221.
McCloskey, J. W. (1965). A model for the distribution of individuals by species in an environment. Unpublished Ph.D. thesis, Michigan State University.
Muliere, P., & Walker, S. (1997). A Bayesian non-parametric approach to survival analysis using Polya trees. Scandinavian Journal of Statistics, 24, 331–340.
Ongaro, A., & Cattaneo, C. (2004). Discrete random probability measures: a general framework for nonparametric Bayesian inference. Statistics & Probability Letters, 67, 33–45.
Patil, G. P., & Taillie, C. (1977). Diversity as a concept and its implications for random communities. Bulletin of the International Statistical Institute, 47, 497–515.
Perman, M., Pitman, J., & Yor, M. (1992). Size-biased sampling of Poisson point processes and excursions. Probability Theory and Related Fields, 92, 21–39.
Petrone, S. (1999). Random Bernstein polynomials. Scandinavian Journal of Statistics, 26, 373–393.
Phadia, E. G. (2007). On bivariate tailfree processes. In Proceedings of the 56th session of the International Statistical Institute, Lisbon, Portugal. Electronic version.
Phadia, E. G., & Susarla, V. (1983). Nonparametric Bayesian estimation of a survival curve with dependent censoring mechanism. Annals of the Institute of Statistical Mathematics, 35, 389–400.
Pitman, J. (1995). Exchangeable and partially exchangeable random partitions. Probability Theory and Related Fields, 102, 145–158.
Pitman, J. (1996a). Random discrete distributions invariant under size-biased permutation. Advances in Applied Probability, 28, 525–539.
Pitman, J. (1996b). Some developments of the Blackwell-MacQueen urn scheme. In T. S. Ferguson, L. S. Shapley & J. B. MacQueen (Eds.), Statistics, probability and game theory. Papers in honor or David Blackwell (pp. 245–267). Hayward: IMS.
Pitman, J., & Yor, M. (1997). The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability, 25, 855–900.
Pruiit, R. C. (1988). An inconsistent Bayes estimate in bivariate survival curve analysis. Preprint, University of Minnesota.
Reich, B. J., & Fuentes, M. (2007). A multivariate semiparametric Bayesian spatial modeling framework for hurricane surface wind fields. Annals of Applied Statistics, 1, 240–264.
Rodriguez, A., Dunson, D. B., & Gelfand, A. E. (2010). Latent stick-breaking processes. Journal of the American Statistical Association, 105, 647–659.
Salinas-Torres, V. H., Pereira, C. A. B., & Tiwari, R. C. (2002). Bayesian nonparametric estimation in a series system or a competing-risks model. Journal of Nonparametric Statistics, 14, 449–458.
Sethuraman, J. (1994). A constructive definition of the Dirichlet process prior. Statistica Sinica, 2, 639–650.
Sethuraman, J., & Tiwari, R. C. (1982). Convergence of Dirichlet measures and the interpretation of their parameter. In S. Gupta & J. Berger (Eds.), Statistical decision theory and related topics III (Vol. 1, pp. 305–315).
Sinha, D. (1997). Time-discrete beta-process model for interval-censored survival data. Canadian Journal of Statistics, 25, 445–456.
Susarla, V., & Van Ryzin, J. (1976). Nonparametric Bayesian estimation of survival curves from incomplete observations. Journal of the American Statistical Association, 71, 897–902.
Susarla, V., & Van Ryzin, J. (1978b). Large sample theory for a Bayesian nonparametric survival curve estimator based on censored samples. Annals of Statistics, 6, 755–768.
Susarla, V., & Van Ryzin, J. (1980). Addendum to “Large sample theory for a Bayesian nonparametric survival curve estimator based on censored samples”. Annals of Statistics, 8, 693.
Teh, Y. W., & Gorur, D. (2010). Indian buffet processes with power-law behavior. In Advances in neural information processing systems (Vol. 22).
Teh, Y. W., & Jordan, M. I. (2010). Hierarchical Bayesian nonparametric models with applications. In N. L. Hjort et al. (Eds.), Cambridge series in statistical and probabilistic mathematics. Bayesian nonparametrics.
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2004). Hierarchical Dirichlet processes. In Advances in neural information processing systems (Vol. 17). Cambridge: MIT Press.
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006). Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101, 1566–1581.
Teh, Y. W., Gorur, D., & Ghahramani, Z. (2007). Stick-breaking construction for the Indian buffet process. In M. Meila & X. Shen (Eds.), Proceedings of the international conference on artificial intelligence and statistics (Vol. 11, pp. 556–563). Brookline: Microtome Publishing.
Thibaux, R., & Jordan, M. I. (2007). Hierarchical beta processes and the Indian buffet process. In M. Meila & X. Shen (Eds.), Proceedings of the international conference on artificial intelligence and statistics (Vol. 11, pp. 564–571). Brookline: Microtome Publishing.
Tiwari, R. C. (1981). A mathematical study of the Dirichlet process. Ph.D. dissertation, Florida State University, Department of Statistics.
Tiwari, R. C. (1988). Convergence of the Dirichlet invariant measures and the limits of Bayes estimates. Communications in Statistics. Theory and Methods, 17(2), 375–393.
Tsai, W. Y. (1986). Estimation of survival curves from dependent censorship models via a generalized self-consistent property with nonparametric Bayesian estimation application. Annals of Statistics, 14, 238–249.
Walker, S. G., & Damien, P. (1998). A full Bayesian nonparametric analysis involving a neutral to the right process. Scandinavian Journal of Statistics, 25, 669–680.
Walker, S. G., & Mallick, B. K. (1997b). Hierarchical generalized linear models and frailty models with Bayesian nonparametric mixing. Journal of the Royal Statistical Society. Series B, Statistical Methodology, 59, 845–860.
Walker, S. G., & Mallick, B. K. (1999). Semiparametric accelerated life time models. Biometrics, 55, 477–483.
Walker, S. G., & Muliere, P. (1997a). Beta-Stacy processes and a generalization of the Polya-urn scheme. Annals of Statistics, 25(4), 1762–1780.
Walker, S. G., & Muliere, P. (1997b). A characterization of Polya tree distributions. Statistics & Probability Letters, 31, 163–168.
Walker, S. G., & Muliere, P. (1999). A characterization of a neutral to the right prior via an extension of Johnson’s sufficientness postulate. Annals of Statistics, 27(2), 589–599.
Walker, S. G., & Muliere, P. (2003). A bivariate Dirichlet process. Statistics & Probability Letters, 64, 1–7.
Walker, S. G., Damien, P., Laud, P., & Smith, A. F. M. (1999). Bayesian nonparametric inference for random distributions and related functions. Journal of the Royal Statistical Society. Series B, Statistical Methodology, 61, 485–527.
West, M. (1992). Modelling with mixtures (with discussion and rejoinder). In J. M. Bernardo et al. (Eds.), Bayesian statistics (Vol. 4). Oxford: Oxford University Press.
Wild, C. J., & Kalbfleisch, J. D. (1981). A note on a paper by Ferguson and Phadia. Annals of Statistics, 9, 1061–1065.
Wolpart, R. L., & Ickstadt, K. (1998). Poisson/gamma random field models for spatial statistics. Biometrika, 85, 251–267.
Yamato, H. (1977a). Relations between limiting Bayes estimates and the U-statistics for estimable parameters of degree 2 and 3. Communications in Statistics. Theory and Methods, 6, 55–66.
Yamato, H. (1977b). Relations between limiting Bayes estimates and the U-statistics for estimable parameters. Journal of the Japan Statistical Society, 7, 57–66.
Yamato, H. (1984). Characteristic functions of means of distributions chosen from a Dirichlet process. Annals of Probability, 12, 262–267.
Yamato, H. (1986). Bayes estimates of estimable parameters with a Dirichlet invariant process. Communications in Statistics. Theory and Methods, 15(8), 2383–2390.
Yamato, H. (1987). Nonparametric Bayes estimates of estimable parameters with a Dirichlet invariant process and invariant U-statistics. Communications in Statistics. Theory and Methods, 16(2), 525–543.
Yang, M., Hanson, T., & Christensen, R. (2008). Nonparametric Bayesian estimation of a bivariate density with interval censored data. Computational Statistics & Data Analysis, 52(12), 5202–5214.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Phadia, E.G. (2013). Prior Processes. In: Prior Processes and Their Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39280-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-39280-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39279-5
Online ISBN: 978-3-642-39280-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)