Gaussian Process Deep Belief Networks: A Smooth Generative Model of Shape with Uncertainty Propagation

  • Alessandro Di MartinoEmail author
  • Erik Bodin
  • Carl Henrik Ek
  • Neill D. F. Campbell
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11364)


The shape of an object is an important characteristic for many vision problems such as segmentation, detection and tracking. Being independent of appearance, it is possible to generalize to a large range of objects from only small amounts of data. However, shapes represented as silhouette images are challenging to model due to complicated likelihood functions leading to intractable posteriors. In this paper we present a generative model of shapes which provides a low dimensional latent encoding which importantly resides on a smooth manifold with respect to the silhouette images. The proposed model propagates uncertainty in a principled manner allowing it to learn from small amounts of data and providing predictions with associated uncertainty. We provide experiments that show how our proposed model provides favorable quantitative results compared with the state-of-the-art while simultaneously providing a representation that resides on a low-dimensional interpretable manifold.


Shape models Unsupervised learning Gaussian processes Deep belief networks 



This work was supported by the EPSRC CAMERA (EP/M023281/1) grant and the Royal Society.

Supplementary material (12.3 mb)
Supplementary material 1 (zip 12611 KB)


  1. 1.
    Abadi, M., et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems (2015). Software available from
  2. 2.
    Bengio, Y., LeCun, Y., et al.: Scaling learning algorithms towards AI. Large-Scale Kernel Mach. 34(5), 1–41 (2007)Google Scholar
  3. 3.
    Campbell, N.D.F., Kautz, J.: Learning a manifold of fonts. ACM Trans. Graph. (SIGGRAPH) 33(4), 91 (2014)CrossRefGoogle Scholar
  4. 4.
    Carreira-Perpiñán, M.Á., Hinton, G.E.: On contrastive divergence learning. In: Cowell, R.G., Ghahramani, Z. (eds.) Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, AISTATS 2005, Bridgetown, Barbados, 6–8 January 2005. Society for Artificial Intelligence and Statistics (2005)Google Scholar
  5. 5.
    Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In: Lee, D.D., Sugiyama, M., von Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, Barcelona, Spain, 5–10 December 2016, pp. 2172–2180 (2016)Google Scholar
  6. 6.
    Damianou, A.C., Lawrence, N.D.: Deep Gaussian processes. In: Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2013, Scottsdale, AZ, USA, 29 April–1 May 2013. JMLR Workshop and Conference Proceedings, vol. 31, pp. 207–215. (2013)Google Scholar
  7. 7.
    Davies, R., Twining, C., Taylor, C.: Statistical Models of Shape: Optimisation and Evaluation. Springer, Heidelberg (2008). Scholar
  8. 8.
    Elhabian, S.Y., Whitaker, R.T.: ShapeOdds: variational Bayesian learning of generative shape models. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 2185–2196. IEEE Computer Society (2017)Google Scholar
  9. 9.
    Eslami, S.M.A., Williams, C.K.I.: A generative model for parts-based object segmentation. In: Bartlett, P.L., Pereira, F.C.N., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a Meeting Held Lake Tahoe, Nevada, USA, 3–6 December 2012, pp. 100–107 (2012)Google Scholar
  10. 10.
    Eslami, S.A., Heess, N., Williams, C.K., Winn, J.: The shape Boltzmann machine: a strong model of object shape. Int. J. Comput. Vis. 107(2), 155–176 (2014)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Goodfellow, I.J., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 8–13 December 2014, pp. 2672–2680 (2014)Google Scholar
  12. 12.
    Hinton, G.E.: A practical guide to training restricted Boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 599–619. Springer, Heidelberg (2012). Scholar
  13. 13.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2014)Google Scholar
  15. 15.
    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: International Conference on Learning Representations (ICLR) (2013)Google Scholar
  16. 16.
    Kirillov, A., Gavrikov, M., Lobacheva, E., Osokin, A., Vetrov, D.P.: Deep part-based generative shape model with latent variables. In: Wilson, R.C., Hancock, E.R., Smith, W.A.P. (eds.) Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, 19–22 September 2016. BMVA Press (2016)Google Scholar
  17. 17.
    Lawrence, N.D.: Probabilistic non-linear principal component analysis with Gaussian process latent variable models. J. Mach. Learn. Res. 6, 1783–1816 (2005)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Lawrence, N.D., Candela, J.Q.: Local distance preservation in the GP-LVM through back constraints. In: Cohen, W.W., Moore, A. (eds.) Machine Learning, Proceedings of the Twenty-Third International Conference (ICML 2006), Pittsburgh, Pennsylvania, USA, 25–29 June 2006. ACM International Conference Proceeding Series, vol. 148, pp. 513–520. ACM (2006)Google Scholar
  19. 19.
    Li, F., Fergus, R., Perona, P.: Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2004, Washington, DC, USA, 27 June–2 July 2004, p. 178. IEEE Computer Society (2004)Google Scholar
  20. 20.
    Li, W., Viola, F., Starck, J., Brostow, G.J., Campbell, N.D.F.: Roto++: accelerating professional rotoscoping using shape manifolds. ACM Trans. Graph. (SIGGRAPH) 35(4), 62 (2016)Google Scholar
  21. 21.
    Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: a continuous relaxation of discrete random variables. abs/1611.00712 (2016)Google Scholar
  22. 22.
    Prisacariu, V.A., Reid, I.D.: PWP3D: real-time segmentation and tracking of 3D objects. Int. J. Comput. Vis. 98(3), 335–354 (2012)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Prisacariu, V.A., Reid, I.D.: Nonlinear shape manifolds as shape priors in level set segmentation and tracking. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, Colorado Springs, CO, USA, 20–25 June 2011, pp. 2185–2192. IEEE Computer Society (2011)Google Scholar
  24. 24.
    Roy, A., Todorovic, S.: Combining bottom-up, top-down, and smoothness cues for weakly supervised image segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp. 7282–7291. IEEE Computer Society (2017)Google Scholar
  25. 25.
    Smolensky, P.: Information processing in dynamical systems: foundations of harmony theory. In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1. pp. 194–281 (1986)Google Scholar
  26. 26.
    Snoek, J., Adams, R.P., Larochelle, H.: Nonparametric guidance of autoencoder representations using label information. J. Mach. Learn. Res. 13, 2567–2588 (2012)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Tosi, A., Hauberg, S., Vellido, A., Lawrence, N.D.: Metrics for probabilistic geometries. In: Zhang, N.L., Tian, J. (eds.) Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI 2014, Quebec City, Quebec, Canada, 23–27 July 2014, pp. 800–808. AUAI Press (2014)Google Scholar
  28. 28.
    Tsogkas, S., Kokkinos, I., Papandreou, G., Vedaldi, A.: Semantic part segmentation with deep learning. arXiv preprint arXiv:1505.02438 (2015)
  29. 29.
    Turmukhambetov, D., Campbell, N.D.F., Goldman, D.B., Kautz, J.: Interactive sketch-driven image synthesis. Comput. Graph. Forum 34(8), 130–142 (2015)CrossRefGoogle Scholar
  30. 30.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  31. 31.
    Wu, Z., et al.: 3D ShapeNets: a deep representation for volumetric shapes. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, pp. 1912–1920. IEEE Computer Society (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Alessandro Di Martino
    • 1
    Email author
  • Erik Bodin
    • 2
  • Carl Henrik Ek
    • 2
  • Neill D. F. Campbell
    • 1
  1. 1.Department of Computer ScienceUniversity of BathBathUK
  2. 2.Department of Computer ScienceUniversity of BristolBristolUK

Personalised recommendations