Motion Selectivity of Neurons in Self-driving Networks

  • Baladitya YellapragadaEmail author
  • Alexander AndersonEmail author
  • Stella YuEmail author
  • Karl ZipserEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11133)


We investigated if optical flow filters were implicitly learned by a neural network trained to drive a vehicle. The network was not trained to predict optical flow across the frames, but, through a series of controlled experiments, we claim that optical flow filters are present in the network. However, this appears to be only the case for sideways flows more relevant for steering predictions. For motor throttle predictions, the network looks at the variance of the pixels over time rather than computing optical flow. In addition, the filters that are likely used for motor throttle predictions dominate primarily in the middle of the network.


Optical flow Motion selectivity Self-driving Autonomous driving Convolutional neural network Stereoscopic disparity 


  1. 1.
    Erhan, D., Courville, A., Bengio Y.: Understanding representations learned in deep architectures. Techreport (2010)Google Scholar
  2. 2.
    Iandola, F., Han, S., Moskewicz, M., Ashraf, K., Dally, W., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\) 0.5 MB model size. In: International Conference on Learning Representations (2017)Google Scholar
  3. 3.
    Lundquist, S., Paiton, D., Schultz, P., Kenyon, G.: Sparse encoding of binocular images for depth inference. In: IEEE (2016)Google Scholar
  4. 4.
    Meyer, S., Wang, O., Zimmer H., Grosse, M., Sorkine-Hornung, A.: Phase-based frame interpolation for video. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  5. 5.
    Saunders, J.: View rotation is used to perceive path curvature from optic flow. J. Vis. 10(13), 25 (2010)CrossRefGoogle Scholar
  6. 6.
    Yarkoni, T., Westfall, J.: Choosing prediction over explanation in psychology: lessons from machine learning. Perspect. Psychol. Sci. 12(6), 1100–1122 (2017)CrossRefGoogle Scholar
  7. 7.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). Scholar
  8. 8.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Object detectors emerge in deep scene CNNs. In: International Conference on Learning Representations (2015)Google Scholar
  9. 9.
    Zhou, B., Andonian, A., Oliva, A., Torralba, A.: Temporal Relational Reasoning in Videos. arXiv (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of California, BerkeleyBerkeleyUSA
  2. 2.International Computer Science InstituteBerkeleyUSA
  3. 3.Redwood Center for Theoretical NeuroscienceUniversity of CaliforniaBerkeleyUSA

Personalised recommendations