Abstract
In this paper, convolutional layers of pre-trained VGG16 model are analyzed. The analysis is based on the responses of neurons to the images of classes in ImageNet database. First, a visualization method is proposed in order to illustrate the learned content of each neuron. Next, single- and multi-faceted neurons are investigated based on the diversity of neuron responses to different category of objects. Finally, neuronal similarities at each layer are computed and compared. The results demonstrate that the neurons in lower layers exhibit a multi-faceted behavior, whereas the majority of neurons in higher layers comprise single-faceted property and tend to respond to a smaller number of concepts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Achille A, Soatto S (2017) Emergence of invariance and disentangling in deep representations. arXiv:1706.01350
Caywood MS, Willmore B, Tolhurst DJ (2004) Independent components of color natural scenes resemble V1 neurons in their spatial and color tuning. J Neurophysiol 91(6):2859–2873
Erhan D, Bengio Y, Courville A, Vincent P (2009) Visualizing higher-layer features of a deep network. University of Montreal 1341(3):1
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
Hyvärinen A, Karhunen J, Oja E (2001) Independent component analysis. Series on adaptive and learning systems for signal processing, communications, and control. Wiley
Kong J-H, Fish DR, Rockhill RL, Masland RH (2005) Diversity of ganglion cells in the mouse retina: unsupervised morphological classification and its limits. J Comp Neurol 489(3):293–310
Madhu N (2009) Note on measures for spectral flatness. Electron Lett 45(23):1195–1196
Nguyen A, Yosinski J, Clune J (2016) Multifaceted feature visualization: uncovering the different types of features learned by each neuron in deep neural networks. arXiv:1602.03616
Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I (2005) Invariant visual representation by single neurons in the human brain. Nature 435(7045):1102
Shwartz-Ziv R, Tishby N (2017) Opening the black box of deep neural networks via information. arXiv:1703.00810
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Stepanyants A, Chklovskii DB (2005) Neurogeometry and potential synaptic connectivity. Trends Neurosci 28(7):387–394
Stiefel KM, Sejnowski TJ (2007) Mapping function onto neuronal morphology. J Neurophysiol 98(1):513–526
Tishby N, Zaslavsky N (2015) Deep learning and the information bottleneck principle. In: Information theory workshop (ITW), 2015 IEEE, pp 1–5
Yosinski J, Clune J, Nguyen A, Fuchs T, Lipson H (2015). Understanding neural networks through deep visualization. arXiv:1506.06579
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In European conference on computer vision, pp 818–833)
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2014) Object detectors emerge in deep scene CNNs. arXiv:1412.6856
Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., & Oliva, A. (2014). Learning deep features for scene recognition using places database. In Advances in neural information processing systems (pp. 487–495)
Zintgraf LM, Cohen TS, Welling M (2016) A new method to visualize deep neural networks. arXiv:1603.02518
Zonoobi D, Kassim AA, Venkatesh YV (2011) Gini index as sparsity measure for signal reconstruction from compressive samples. IEEE J Sel Top Signal Process 5(5):927–932
Acknowledgements
This research has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skodowska-Curie grant agreement no. 665919.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sadeghi, Z. (2020). Conceptual Content in Deep Convolutional Neural Networks: An Analysis into Multi-faceted Properties of Neurons. In: Uddin, M.S., Bansal, J.C. (eds) Proceedings of International Joint Conference on Computational Intelligence. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-15-3607-6_2
Download citation
DOI: https://doi.org/10.1007/978-981-15-3607-6_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-3606-9
Online ISBN: 978-981-15-3607-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)