Abstract
It is important to enable autonomous robots to detect or segment moving objects in dynamic scenes as they must perform collision-free navigation. Motion segmentation from a moving platform is challenging due to the dual motion caused by the background and the moving objects. Existing approaches for motion segmentation either have long multistage pipelines which are inefficient for real-time application or utilize optical flow which is sensitive to environment. In this paper, this challenging task is tackled by constructing spatiotemporal features from two consecutive dense depth maps. Depth maps can be generated either by LiDaR scans data or stereo vision algorithms. The core of the proposed approach is a fully convolutional network with inserted Gated-Recurrent-Units, denoted as MotionRFCN. We also create a publicly available dataset (KITTI-MoSeg) which contains more than 2000 frames with motion annotations. Qualitative and quantitative evaluation of MotionRFCN are presented to demonstrate its state-of-the-art performance on the KITTI dataset. The basic MotionRFCN can run in real time and segment moving objects whether the platform is stationary or moving. To the best of our knowledge, the proposed method is the first to implement motion segmentation with only dense depth maps inputs.
Supported in part by the Natural Science Foundation of China under Grant U1613218 and 61722309.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cut. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1644–1659 (2005)
Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: Computer Vision and Pattern Recognition (2009)
Kundu, A., Krishna, K.M., Sivaswamy, J.: Moving object detection by multi-view geometric techniques from a single camera mounted robot. In: International Conference on Intelligent Robots and Systems (2009)
Sheikh, Y., Javed, O., Kanade, T.: Background subtraction for freely moving cameras. In: International Conference on Computer Vision (2009)
Lenz, P., Ziegler, J., Geiger, A., Roser, M.: Sparse scene flow segmentation for moving object detection in urban environments. In: Intelligent Vehicles Symposium (2011)
Namdev, R.K., Kundu, A., Krishna, K.M., Jawahar, C.V.: Motion segmentation of multiple objects from a freely moving monocular camera. In: International Conference on Robotics and Automation (2012)
Narayana, M., Hanson, A., Learned-Miller, E.: Coherent motion segmentation in moving camera videos using optical flow orientations. In: International Conference on Computer Vision (2013)
Romero-Cano, V., Nieto, J.I.: Stereo-based motion detection and tracking from a moving platform. In: Intelligent Vehicles Symposium (2013)
Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32, 1231–1237 (2013)
Reddy, N.D., Singhal, P., Krishna, K.M.: Semantic motion segmentation using dense CRF formulation. In: Proceedings of the Indian Conference on Computer Vision Graphics and Image Processing (2014)
Lin, T.H., Wang, C.C.: Deep learning of spatio-temporal features with geometric-based moving point detection for motion segmentation. In: International Conference on Robotics and Automation (2014)
Ochs, P., Malik, J., Brox, T.: Segmentation of moving objects by long term video analysis. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1187–1200 (2014)
Fragkiadaki, K., Arbelaez, P., Felsen, P., Malik, J.: Learning to segment moving objects in videos. In: Computer Vision and Pattern Recognition (2015)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Computer Vision and Pattern Recognition (2015)
Ballas, N., Yao, L., Pal, C., Courville, A.: Delving deeper into convolutional networks for learning video representations. arXiv preprint arXiv:1511.06432 (2015)
Qiu, F., Yang, Y., Li, H., Fu, M., Wang, S.: Semantic motion segmentation for urban dynamic scene understanding. In: International Conference on Automation Science and Engineering (2016)
Kao, J.Y., Tian, D., Mansour, H., Vetro, A., Ortega, A.: Moving object segmentation using depth and optical flow in car driving sequences. In: International Conference on Image Processing (2016)
Drayer, B., Brox, T.: Object detection, tracking, and motion segmentation for object-level video segmentation. arxiv:1608.03066 (2016)
Tourani, S., Krishna, K.M.: Using in-frame shear constraints for monocular motion segmentation of rigid bodies. J. Intell. Robotic Syst. 82(2), 237–255 (2016)
Chen, T., Lu, S.: Object-level motion detection from moving cameras. IEEE Trans. Circuits Syst. Video Technol. 27, 2333–2343 (2016)
Bideau, P., Learned-Miller, E.: It’s moving! a probabilistic model for causal motion segmentation in moving camera videos. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 433–449. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_26
Postica, G., Romanoni, A., Matteucci, M.: Robust moving objects detection in lidar data exploiting visual cues. In: International Conference on Intelligent Robots and Systems (2016)
Dewan, A., Caselitz, T., Tipaldi, G.D., Burgard, W.: Motion-based detection and tracking in 3D LiDAR scans. In: International Conference on Robotics and Automation (2016)
Haque, N., Reddy, N.D., Krishna, K.M.: Joint semantic and motion segmentation for dynamic scenes using deep convolutional networks. In: International Conference on Computer Vision Theory and Applications (2017)
Vertens, J., Valada, A., Burgard, W.: SMSnet: semantic motion segmentation using deep convolutional neural networks. In: International Conference on Intelligent Robots and Systems (2017)
Wehrwein, S., Szeliski, R.: Video segmentation with background motion models. In: British Machine Vision Conference (2017)
Tokmakov, P., Alahari, K., Schmid, C.: Learning motion patterns in videos. In: Computer Vision and Pattern Recognition (2017)
Haque, N., Reddy, N.D., Krishna, M.: Temporal semantic motion segmentation using spatio temporal optimization. In: Pelillo, M., Hancock, E. (eds.) EMMCVPR 2017. LNCS, vol. 10746, pp. 93–108. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78199-0_7
Siam, M., Mahgoub, H., Zahran, M., Yogamani, S., Jagersand, M.: Motion and appearance based moving object detection network for autonomous driving. arXiv:1709.04821v2 (2017)
He, K.M., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: International Conference on Computer Vision (2017)
Sun, D., Yang, X., Liu, M., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: Computer Vision and Pattern Recognition (2018)
Cordts, M., Omaran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R.: The cityscapes dataset for semantic urban scene understanding. In: Computer Vision and Pattern Recognition (2016)
Zheng, L., Wang, H., Chen, W.: A fast 3D object recognition pipeline in cluttered and occluded scenes. In: Huang, Y.A., Wu, H., Liu, H., Yin, Z. (eds.) ICIRA 2017. LNCS (LNAI), vol. 10463, pp. 588–598. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65292-4_51
Yang, B., Wang, H., Chen, W., Liang, Y.: Vision-based automatic hair follicular unit separation. In: Liu, H., Kubota, N., Zhu, X., Dillmann, R., Zhou, D. (eds.) ICIRA 2015. LNCS (LNAI), vol. 9246, pp. 273–284. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22873-0_24
Xu, L., Wang, H., Chen, W., Wang, J.: Light intensity optimization in trajectory planning of inspection robot. In: Liu, H., Kubota, N., Zhu, X., Dillmann, R., Zhou, D. (eds.) ICIRA 2015. LNCS (LNAI), vol. 9245, pp. 297–308. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22876-1_26
Wang, H., Lai, Y., Chen, W.: The time optimal trajectory planning with limitation of operating task for the Tokamak inspecting manipulator. Fusion Eng. Des. 113, 57–65 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Y., Wang, H. (2019). MotionRFCN: Motion Segmentation Using Consecutive Dense Depth Maps. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11671. Springer, Cham. https://doi.org/10.1007/978-3-030-29911-8_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-29911-8_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29910-1
Online ISBN: 978-3-030-29911-8
eBook Packages: Computer ScienceComputer Science (R0)