Abstract
Dirichlet processes (DP) are widely applied in Bayesian nonparametric modeling. However, in their basic form they do not directly integrate dependency information among data arising from space and time. In this paper, we propose location dependent Dirichlet processes (LDDP) which incorporate nonparametric Gaussian processes in the DP modeling framework to model such dependencies. We develop the LDDP in the context of mixture modeling, and develop a mean field variational inference algorithm for this mixture model. The effectiveness of the proposed modeling framework is shown on an image segmentation task.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
\(q(\theta _k)\) is problem-specific, so we ignore it here.
References
Hjort, N., Holmes, C., Müller, P., Walker, S. (eds.): Bayesian Nonparametrics: Principles and Practice. Cambridge University Press, Cambridge (2010)
Orbanz, P., Teh, Y.W.: Bayesian nonparametric models. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 81–89. Springer, New York (2010). doi:10.1007/978-0-387-30164-8_66
Teh, Y.W.: Dirichlet processes. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 280–287. Springer, New York (2010). doi:10.1007/978-0-387-30164-8_219
Ferguson, T.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)
Sun, X., Yung, N., Lam, E.: Unsupervised tracking with the doubly stochastic Dirichlet process mixture model. IEEE Trans. Intell. Transp. Syst. 17, 2594–2599 (2016)
Zhu, F., Chen, G., Hao, J., Heng, P.A.: Blind image denoising via dependent Dirichlet process tree. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1–14 (2016)
MacEachern, S.: Dependent nonparametric processes. In: ASA Proceedings of the Section on Bayesian Statistical Science, pp. 50–55 (1999)
Griffin, J., Steel, M.: Order-based dependent Dirichlet processes. J. Am. Stat. Assoc. 101, 179–194 (2006)
Duan, J., Guindani, M., Gelfand, A.: Generalized spatial Dirichlet process models. Biometrika 94, 809–825 (2007)
Sudderth, E., Jordan, M.: Shared segmentation of natural scences using dependent Pitman-Yor processes. Adv. Neural Inf. Process. Syst. 21, 1585–1592 (2008)
Griffin, J.: The Ornstein-Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference. J. Stat. Plann. Infer. 141, 3648–3664 (2011)
Blei, D., Frazier, P.: Distance dependent Chinese restaurant processes. J. Mach. Learn. Res. 12, 2461–2488 (2011)
Ghosh, S., Ungureunu, A., Sudderth, E., Blei, D.: Spatial distance dependent Chinese restaurant processes for image segmentation. In: Advances in Neural Information Processing Systems, pp. 1476–1484 (2011)
Ren, L., Wang, Y., Dunson, D., Carin, L.: The kernel beta process. Adv. Neural Inf. Process. Syst. 24, 963–971 (2011)
RodrÃguez, A., Dunson, D.: Nonparametric Bayesian models through probit stick-breaking processes. Bayesian Anal. 6, 145–178 (2011)
Paisley, J., Wang, C., Blei, D.: The discrete infinite logistic normal distribution. Bayesian Anal. 7, 235–272 (2012)
Sethuraman, J.: A constructive definition of Dirichlet priors. Stat. Sin. 4, 639–650 (1994)
Ishwaran, H., Zarepour, M.: Dirichlet prior sieves in finite normal mixtures. Stat. Sin. 12, 941–963 (2002)
Rasmussen, C.E.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Bishop, C.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Blei, D.: Build, compute, critique, repeat: data analysis with latent variable models. Ann. Rev. Stat. Appl. 1, 203–232 (2014)
Williams, C., Seeger, M.: Using the Nyström method to speed up kernel machines. Adv. Neural Inf. Process. Syst. 13, 682–688 (2011)
Kumar, S., Mohri, M., Talwalkar, A.: Sampling methods for the Nyström method. J. Mach. Learn. Res. 13, 981–1006 (2012)
Dunson, D., Park, J.H.: Kernel stick-breaking processes. Biometrika 95, 307–323 (2008)
An, Q., Wang, C., Shterev, I., Wang, E., Carin, L., Dunson, D.: Hierarchical kernel stick-breaking process for multi-task image analysis. In: Proceedings of the International Conference on Machine Learning, pp. 17–24 (2008)
Foti, N., Williamson, S.: Slice sampling normalized kernel-weighted completely random measure mixture models. Adv. Neural Inf. Process. Syst. 25, 2240–2248 (2012)
Foti, N., Futoma, J., Rockmore, D., Williamson, S.: A unifying representation for a class of dependent random measures. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 20–28 (2013)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 888–905 (2000)
Teh, Y.: A hierarchical Bayesian language model based on Pitman-Yor processes. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 985–992 (2006)
Uetz, R., Behnke, S.: Large-scale object recognition with CUDA-accelerated hierarchical neural networks. In: Proceedings of the IEEE International Conference on Intelligent Computing and Intelligent Systems, pp. 1–6 (2009)
Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward objective evaluation of image segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 29, 929–944 (2007)
Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001)
Acknowledgments
The corresponding author Shiliang Sun would like to thank supports from the National Natural Science Foundation of China under Projects 61673179 and 61370175, Shanghai Knowledge Service Platform Project (No. ZF1213), and the Fundamental Research Funds for the Central Universities.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Sun, S., Paisley, J., Liu, Q. (2017). Location Dependent Dirichlet Processes. In: Sun, Y., Lu, H., Zhang, L., Yang, J., Huang, H. (eds) Intelligence Science and Big Data Engineering. IScIDE 2017. Lecture Notes in Computer Science(), vol 10559. Springer, Cham. https://doi.org/10.1007/978-3-319-67777-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-67777-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67776-7
Online ISBN: 978-3-319-67777-4
eBook Packages: Computer ScienceComputer Science (R0)