Skip to main content
SpringerLink
Log in

Location Dependent Dirichlet Processes

  • Conference paper
  • First Online:
Intelligence Science and Big Data Engineering (IScIDE 2017)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10559))

Abstract

Dirichlet processes (DP) are widely applied in Bayesian nonparametric modeling. However, in their basic form they do not directly integrate dependency information among data arising from space and time. In this paper, we propose location dependent Dirichlet processes (LDDP) which incorporate nonparametric Gaussian processes in the DP modeling framework to model such dependencies. We develop the LDDP in the context of mixture modeling, and develop a mean field variational inference algorithm for this mixture model. The effectiveness of the proposed modeling framework is shown on an image segmentation task.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    \(q(\theta _k)\) is problem-specific, so we ignore it here.

References

  1. Hjort, N., Holmes, C., Müller, P., Walker, S. (eds.): Bayesian Nonparametrics: Principles and Practice. Cambridge University Press, Cambridge (2010)

    Google Scholar 

  2. Orbanz, P., Teh, Y.W.: Bayesian nonparametric models. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 81–89. Springer, New York (2010). doi:10.1007/978-0-387-30164-8_66

    Google Scholar 

  3. Teh, Y.W.: Dirichlet processes. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 280–287. Springer, New York (2010). doi:10.1007/978-0-387-30164-8_219

    Google Scholar 

  4. Ferguson, T.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  5. Sun, X., Yung, N., Lam, E.: Unsupervised tracking with the doubly stochastic Dirichlet process mixture model. IEEE Trans. Intell. Transp. Syst. 17, 2594–2599 (2016)

    Article  Google Scholar 

  6. Zhu, F., Chen, G., Hao, J., Heng, P.A.: Blind image denoising via dependent Dirichlet process tree. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1–14 (2016)

    Article  Google Scholar 

  7. MacEachern, S.: Dependent nonparametric processes. In: ASA Proceedings of the Section on Bayesian Statistical Science, pp. 50–55 (1999)

    Google Scholar 

  8. Griffin, J., Steel, M.: Order-based dependent Dirichlet processes. J. Am. Stat. Assoc. 101, 179–194 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Duan, J., Guindani, M., Gelfand, A.: Generalized spatial Dirichlet process models. Biometrika 94, 809–825 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  10. Sudderth, E., Jordan, M.: Shared segmentation of natural scences using dependent Pitman-Yor processes. Adv. Neural Inf. Process. Syst. 21, 1585–1592 (2008)

    Google Scholar 

  11. Griffin, J.: The Ornstein-Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference. J. Stat. Plann. Infer. 141, 3648–3664 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Blei, D., Frazier, P.: Distance dependent Chinese restaurant processes. J. Mach. Learn. Res. 12, 2461–2488 (2011)

    MathSciNet  MATH  Google Scholar 

  13. Ghosh, S., Ungureunu, A., Sudderth, E., Blei, D.: Spatial distance dependent Chinese restaurant processes for image segmentation. In: Advances in Neural Information Processing Systems, pp. 1476–1484 (2011)

    Google Scholar 

  14. Ren, L., Wang, Y., Dunson, D., Carin, L.: The kernel beta process. Adv. Neural Inf. Process. Syst. 24, 963–971 (2011)

    Google Scholar 

  15. Rodríguez, A., Dunson, D.: Nonparametric Bayesian models through probit stick-breaking processes. Bayesian Anal. 6, 145–178 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Paisley, J., Wang, C., Blei, D.: The discrete infinite logistic normal distribution. Bayesian Anal. 7, 235–272 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Sethuraman, J.: A constructive definition of Dirichlet priors. Stat. Sin. 4, 639–650 (1994)

    MathSciNet  MATH  Google Scholar 

  18. Ishwaran, H., Zarepour, M.: Dirichlet prior sieves in finite normal mixtures. Stat. Sin. 12, 941–963 (2002)

    MathSciNet  MATH  Google Scholar 

  19. Rasmussen, C.E.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  20. Bishop, C.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  21. Blei, D.: Build, compute, critique, repeat: data analysis with latent variable models. Ann. Rev. Stat. Appl. 1, 203–232 (2014)

    Article  Google Scholar 

  22. Williams, C., Seeger, M.: Using the Nyström method to speed up kernel machines. Adv. Neural Inf. Process. Syst. 13, 682–688 (2011)

    Google Scholar 

  23. Kumar, S., Mohri, M., Talwalkar, A.: Sampling methods for the Nyström method. J. Mach. Learn. Res. 13, 981–1006 (2012)

    MathSciNet  MATH  Google Scholar 

  24. Dunson, D., Park, J.H.: Kernel stick-breaking processes. Biometrika 95, 307–323 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  25. An, Q., Wang, C., Shterev, I., Wang, E., Carin, L., Dunson, D.: Hierarchical kernel stick-breaking process for multi-task image analysis. In: Proceedings of the International Conference on Machine Learning, pp. 17–24 (2008)

    Google Scholar 

  26. Foti, N., Williamson, S.: Slice sampling normalized kernel-weighted completely random measure mixture models. Adv. Neural Inf. Process. Syst. 25, 2240–2248 (2012)

    Google Scholar 

  27. Foti, N., Futoma, J., Rockmore, D., Williamson, S.: A unifying representation for a class of dependent random measures. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 20–28 (2013)

    Google Scholar 

  28. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 888–905 (2000)

    Article  Google Scholar 

  29. Teh, Y.: A hierarchical Bayesian language model based on Pitman-Yor processes. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 985–992 (2006)

    Google Scholar 

  30. Uetz, R., Behnke, S.: Large-scale object recognition with CUDA-accelerated hierarchical neural networks. In: Proceedings of the IEEE International Conference on Intelligent Computing and Intelligent Systems, pp. 1–6 (2009)

    Google Scholar 

  31. Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward objective evaluation of image segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 29, 929–944 (2007)

    Article  Google Scholar 

  32. Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The corresponding author Shiliang Sun would like to thank supports from the National Natural Science Foundation of China under Projects 61673179 and 61370175, Shanghai Knowledge Service Platform Project (No. ZF1213), and the Fundamental Research Funds for the Central Universities.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, East China Normal University, 3663 North Zhongshan Road, Shanghai, 200062, China

    Shiliang Sun & Qiuyang Liu

  2. Department of Electrical Engineering, Columbia University, New York City, USA

    John Paisley

Authors
  1. Shiliang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Paisley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiuyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiliang Sun .

Editor information

Editors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Yi Sun

  2. Dalian University of Technology, Dalian, China

    Huchuan Lu

  3. Dalian University of Technology, Dalian, China

    Lihe Zhang

  4. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

  5. Beijing Institute of Technology, Beijing, China

    Hua Huang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Sun, S., Paisley, J., Liu, Q. (2017). Location Dependent Dirichlet Processes. In: Sun, Y., Lu, H., Zhang, L., Yang, J., Huang, H. (eds) Intelligence Science and Big Data Engineering. IScIDE 2017. Lecture Notes in Computer Science(), vol 10559. Springer, Cham. https://doi.org/10.1007/978-3-319-67777-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67777-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67776-7

  • Online ISBN: 978-3-319-67777-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation