Avoiding Implementation Pitfalls of “Matrix Capsules with EM Routing” by Hinton et al.

  • Ashley Daniel GritzmanEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1072)


The recent progress on capsule networks by Hinton et al. has generated considerable excitement in the machine learning community. The idea behind a capsule is inspired by a cortical minicolumn in the brain, whereby a vertically organised group of around 100 neurons receive common inputs, have common outputs, are interconnected, and may well constitute a fundamental computation unit of the cerebral cortex. However, Hinton’s paper on “Matrix Capsule with EM Routing” was unfortunately not accompanied by a release of source code, which left interested researchers attempting to implement the architecture and reproduce the benchmarks on their own. This has certainly slowed the progress of research building on this work. While writing our own implementation, we noticed several common mistakes in other open source implementations that we came across. In this paper we share some of these learnings, specifically focusing on three implementation pitfalls and how to avoid them: (1) parent capsules with only one child; (2) normalising the amount of data assigned to parent capsules; (3) parent capsules at different positions compete for child capsules. While our implementation is a considerable improvement over currently available implementations, it still falls slightly short of the performance reported by Hinton et al. (2018). The source code for this implementation is available on GitHub at the following URL:


Capsules EM routing Hinton CNN 


  1. 1.
    Calvano, G.S.: Some clarification on the convolution topology? July 2018.
  2. 2.
    Cruz, L., et al.: A statistically based density map method for identification and quantification of regional differences in microcolumnarity in the monkey brain. J. Neurosci. Methods 141(2), 321–332 (2005)CrossRefGoogle Scholar
  3. 3.
    Hinton, G., Krizhevsky, A., Jaitly, N., Tieleman, T., Tang, Y.: Does the brain do inverse graphics? In: Brain and Cognitive Sciences Fall Colloquium, vol. 2 (2012)Google Scholar
  4. 4.
    Hinton, G.E., Krizhevsky, A., Wang, S.D.: Transforming auto-encoders. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6791, pp. 44–51. Springer, Heidelberg (2011). Scholar
  5. 5.
    Hinton, G.E., Sabour, S., Frosst, N.: Lambda and margin, July 2018.
  6. 6.
    Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with EM routing. In: International Conference on Learning Representations (2018).
  7. 7.
    Hinton, G.E., Sabour, S., Frosst, N.: Regularization and learning rate? October 2018.
  8. 8.
    Huang, Y.: Capsules. GitHub, April 2019. Commit: e1f02d3
  9. 9.
    LeCun, Y., Cortes, C., Burges, C.J.: The MNIST database of handwritten digits (1998)Google Scholar
  10. 10.
    LeCun, Y., Huang, F.J., Bottou, L., et al.: Learning methods for generic object recognition with invariance to pose and lighting. In: CVPR, vol. 2, pp. 97–104. Citeseer (2004)Google Scholar
  11. 11.
    Lei, J.Y.: Matrix-capsules-em-pytorch. GitHub, March 2019. Commit: c4547bf
  12. 12.
    Sabour, S.: Code for capsule model used in dynamic routing between capsules. GitHub, January 2018. capsules. Commit: cac8804
  13. 13.
    Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems, pp. 3856–3866 (2017)Google Scholar
  14. 14.
    Zhang, S.: Matrix-capsules-em-tensorflow. GitHub, February 2018. Commit: 0196ead

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.IBM ResearchJohannesburgSouth Africa

Personalised recommendations