Motion Segmentation Using Spectral Clustering on Indian Road Scenes

  • Mahtab SandhuEmail author
  • Sarthak Upadhyay
  • Madhava Krishna
  • Shanti Medasani
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11133)


We propose a novel motion segmentation formulation over spatio-temporal depth images obtained from stereo sequences that segments multiple motion models in the scene in an unsupervised manner. The motion segmentation is obtained at frame rates that compete with the speed of the stereo depth computation. This is possible due to a decoupling framework that first delineates spatial clusters and subsequently assigns motion labels to each of these cluster with analysis of a novel motion graph model. A principled computation of the weights of the motion graph that signifies the relative shear and stretch between possible clusters lends itself to a high fidelity segmentation of the motion models in the scene.


Motion segmentation Object detection Spectral clustering 



The work described in this paper is supported by MathWorks. The opinions and views expressed in this publication are from the authors, and not necessarily that of the funding bodies.


  1. 1.
    Lenz, P., Ziegler, J., Geiger, A., Roser, M.: Sparse scene flow segmentation for moving object detection in urban environments. In: IV, pp. 926–932. IEEE (2011)Google Scholar
  2. 2.
    Reddy, N.D., Singhal, P., Chari, V., Krishna, K.M.: Dynamic body VSLAM with semantic constraints. In: IROS (2015)Google Scholar
  3. 3.
    Dewan, A., Caselitz, T., Tipaldi, G., Burgard, W.: Motion-based detection and tracking in 3D LiDAR scans. In: ICRA (2016)Google Scholar
  4. 4.
    Tourani, S., Krishna, K.M.: Using in-frame shear constraints for monocular motion segmentation of rigid bodies. JIRS 82, 237–255 (2016)Google Scholar
  5. 5.
    Namdev, R.K., Kundu, A., Krishna, K.M., Jawahar, C.: Motion segmentation of multiple objects from a freely moving monocular camera. In: ICRA, pp. 4092–4099 (2012)Google Scholar
  6. 6.
    Oniga, F., Nedevschi, S.: Processing dense stereo data using elevation maps: road surface, traffic isle, and obstacle detection. IEEE Trans. Veh. Technol. 59(3), 1172–1182 (2010)CrossRefGoogle Scholar
  7. 7.
    Hu, Z., Uchimura, K.: UV-disparity: an efficient algorithm for stereovision based scene analysis. In: 2015 IEEE Intelligent Vehicles Symposium Proceedings, pp. 48–54. IEEE (2005)Google Scholar
  8. 8.
    Lauer, F., Schnörr, C.: Spectral clustering of linear subspaces for motion segmentation. In: ICCV (2009)Google Scholar
  9. 9.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: CVPR, pp. 2790–2797 (2009)Google Scholar
  10. 10.
    Chen, G., Lerman, G.: Spectral curvature clustering (SCC). IJCV 81(3), 317–330 (2009)CrossRefGoogle Scholar
  11. 11.
    Jain, S., Madhav Govindu, V.: Efficient higher-order clustering on the grassmann manifold. In: ICCV, pp. 3511–3518 (2013)Google Scholar
  12. 12.
    Zappella, L., Provenzi, E., Lladó, X., Salvi, J.: Adaptive motion segmentation algorithm based on the principal angles configuration. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6494, pp. 15–26. Springer, Heidelberg (2011). Scholar
  13. 13.
    Kundu, A., Krishna, K., Sivaswamy, J.: Moving object detection by multi-view geometric techniques from a single camera mounted robot. In: IROS (2009)Google Scholar
  14. 14.
    Vidal, R., Sastry, S.: Optimal segmentation of dynamic scenes from two perspective views. In: CVPR, vol. 2 (2003)Google Scholar
  15. 15.
    Ochs, P., Brox, T.: Higher order motion models and spectral clustering. In: CVPR (2012)Google Scholar
  16. 16.
    Vertens, J., Valada, A., Burgard, W.: SMSnet: semantic motion segmentation using deep convolutional neural networks. In: IROS (2017)Google Scholar
  17. 17.
    Haque, N., Reddy, D., Krishna, M.: Joint semantic and motion segmentation for dynamic scenes using deep convolutional networks. In: VISAPP (2017)Google Scholar
  18. 18.
    Lin, T.H., Wang, C.C.: Deep learning of spatio-temporal features with geometric-based moving point detection for motion segmentation. In: ICRA, pp. 3058–3065 (2014)Google Scholar
  19. 19.
    Sandhu, M., Haque, N., Sharma, A., Krishna, K.M., Medasani, S.: Fast multi model motion segmentation on road scenes. In: 2018 IEEE Intelligent Vehicles Symposium (IV), pp. 2131–2136. IEEE (2018)Google Scholar
  20. 20.
    Furht, B.: Block matching. In: Furht, B. (ed.) Encyclopedia of Multimedia, pp. 55–56. Springer, Boston (2008). Scholar
  21. 21.
    Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information (2008)CrossRefGoogle Scholar
  22. 22.
    Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDDM (1996)Google Scholar
  23. 23.
    Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Geiger, A., Ziegler, J., Stiller, C.: Stereoscan: dense 3D reconstruction in real-time. In: Intelligent Vehicles Symposium (IV) (2011)Google Scholar
  25. 25.
    Kroeger, T., Timofte, R., Dai, D., Van Gool, L.: Fast optical flow using dense inverse search. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 471–488. Springer, Cham (2016). Scholar
  26. 26.
    Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: ICCV (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Mahtab Sandhu
    • 1
    Email author
  • Sarthak Upadhyay
    • 2
  • Madhava Krishna
    • 1
  • Shanti Medasani
    • 2
  1. 1.IIIT-HyderabadHyderabadIndia
  2. 2.MathWorksHyderabadIndia

Personalised recommendations