Navigational Affordance Cortical Responses Explained by Scene-Parsing Model
Deep Neural Networks (DNNs) are the leading models for explaining the population responses of neurons in the visual cortex. Recent studies show that responses of some task-specific brain regions can also be explained by a DNN trained for classification. In this work, we propose that responses of task-specific brain regions are better explained by DNNs trained on a similar task. We first show that responses of scene selective visual areas like parahippocampal place area (PPA) and Occipital Place Area (OPA) are better explained by a DNN trained for scene classification than one trained for object classification. Next, we consider a particular case of OPA which has been shown to encode navigational affordances. We argue that a scene parsing task, which predicts the class of each pixel in the scene is more related to navigational affordances than scene classification. Our results show that the responses in OPA are better explained by the scene parsing model than the scene classification model.
KeywordsDeep Neural Networks Representational similarity analysis Occipital Place Area Neural Encoding
This work was funded by the MOE SUTD SRG grant (SRG ISTD 2017 131). Kshitij Dwivedi was also funded by SUTD President’s Graduate Fellowship.
- 4.Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2009, pp. 248–255. IEEE (2009)Google Scholar
- 10.Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
- 11.Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
- 14.Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
- 16.Yamins, D.L.K., Hong, H., Cadieu, C.F., Solomon, E.A., Seibert, D., DiCarlo, J.J.: Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Nat. Acad. Sci. 111(23), 8619–8624 (2014). https://doi.org/10.1073/pnas.1403112111. http://www.pnas.org/cgi/doi/10.1073/pnas.1403112111CrossRefGoogle Scholar
- 19.Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Semantic understanding of scenes through the ADE20K dataset. arXiv preprint arXiv:1608.05442 (2016)
- 20.Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ADE20K dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 633–641 (2017)Google Scholar