Joint Exploitation of Features and Optical Flow for Real-Time Moving Object Detection on Drones

  • Hazal Lezki
  • I. Ahu Ozturk
  • M. Akif Akpinar
  • M. Kerim YucelEmail author
  • K. Berker Logoglu
  • Aykut Erdem
  • Erkut Erdem
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11130)


Moving object detection is an imperative task in computer vision, where it is primarily used for surveillance applications. With the increasing availability of low-altitude aerial vehicles, new challenges for moving object detection have surfaced, both for academia and industry. In this paper, we propose a new approach that can detect moving objects efficiently and handle parallax cases. By introducing sparse flow based parallax handling and downscale processing, we push the boundaries of real-time performance with 16 FPS on limited embedded resources (a five-fold improvement over existing baselines), while managing to perform comparably or even improve the state-of-the-art in two different datasets. We also present a roadmap for extending our approach to exploit multi-modal data in order to mitigate the need for parameter tuning.


Moving object detection Optical flow UAV Drones Embedded vision Real-time vision 


  1. 1.
    Clarke, R.: Understanding the drone epidemic. Comput. Law Secur. Rev. 30(3), 230–246 (2014)CrossRefGoogle Scholar
  2. 2.
    Zhong, J., Lei, T., Yao, G.: Robust vehicle detection in aerial images based on cascaded convolutional neural networks. Sensors 17(12), 2720 (2017)CrossRefGoogle Scholar
  3. 3.
    Li, F., Li, S., Zhu, C., Lan, X., Chang, H.: Cost-effective class-imbalance aware cnn for vehicle localization and categorization in high resolution aerial images. Remote Sens. 9(5), 494 (2017)CrossRefGoogle Scholar
  4. 4.
    Tijtgat, N., Van Ranst, W., Volckaert, B., Goedemé, T., De Turck, F.: Embedded real-time object detection for a UAV warning system. In: The International Conference on Computer Vision, ICCV 2017, pp. 2110–2118 (2017)Google Scholar
  5. 5.
    Sommer, L.W., Schuchert, T., Beyerer, J.: Fast deep vehicle detection in aerial images. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 311–319. IEEE (2017)Google Scholar
  6. 6.
    Stek, T.D.: Drones over Mediterranean landscapes. The potential of small UAV’s (drones) for site detection and heritage management in archaeological survey projects: a case study from Le Pianelle in the Tappino Valley, Molise (Italy). J. Cultural Herit. 22, 1066–1071 (2016)CrossRefGoogle Scholar
  7. 7.
    Barekatain, M., et al.: Okutama-action: an aerial view video dataset for concurrent human action detection. In: 1st Joint BMTT-PETS Workshop on Tracking and Surveillance, CVPR, pp. 1–8 (2017)Google Scholar
  8. 8.
    Pestana, J., Sanchez-Lopez, J.L., Campoy, P., Saripalli, S.: Vision based GPS-denied object tracking and following for unmanned aerial vehicles. In: 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 1–6. IEEE (2013)Google Scholar
  9. 9.
    Dang, C.T., Pham, T.B., Truong, N.V., et al.: Vision based ground object tracking using AR drone quadrotor. In: 2013 International Conference on Control, Automation and Information Sciences (ICCAIS), pp. 146–151. IEEE (2013)Google Scholar
  10. 10.
    Chen, P., Dang, Y., Liang, R., Zhu, W., He, X.: Real-time object tracking on a drone with multi-inertial sensing data. IEEE Trans. Intell. Transp. Syst. 19(1), 131–139 (2018)CrossRefGoogle Scholar
  11. 11.
    Hsieh, M.R., Lin, Y.L., Hsu, W.H.: Drone-based object counting by spatially regularized regional proposal network. In: The IEEE International Conference on Computer Vision (ICCV), vol. 1 (2017)Google Scholar
  12. 12.
    Kanistras, K., Martins, G., Rutherford, M.J., Valavanis, K.P.: A survey of unmanned aerial vehicles (UAVs) for traffic monitoring. In: 2013 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 221–234. IEEE (2013)Google Scholar
  13. 13.
    Du, D., et al.: The unmanned aerial vehicle benchmark: object detection and tracking. arXiv preprint arXiv:1804.00518 (2018)
  14. 14.
    Wang, S., et al.: TorontoCity: seeing the world with a million eyes. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3028–3036. IEEE (2017)Google Scholar
  15. 15.
    Xia, G.S., et al.: DOTA: a large-scale dataset for object detection in aerial images. In: Proceedings of CVPR (2018)Google Scholar
  16. 16.
    Mueller, M., Smith, N., Ghanem, B.: A benchmark and simulator for UAV tracking. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 445–461. Springer, Cham (2016). Scholar
  17. 17.
    Berker Logoglu, K., et al.: Feature-based efficient moving object detection for low-altitude aerial platforms. In: The IEEE International Conference on Computer Vision (ICCV) Workshops, October 2017Google Scholar
  18. 18.
    Lam, D., et al.: xView: objects in context in overhead imagery. arXiv preprint arXiv:1802.07856 (2018)
  19. 19.
    Zhu, P., Wen, L., Bian, X., Ling, H., Hu, Q.: Vision meets drones: a challenge. arXiv preprint arXiv:1804.07437 (2018)
  20. 20.
    Yu, Q., Medioni, G.: A GPU-based implementation of motion detection from a moving platform (2008)Google Scholar
  21. 21.
    Kryjak, T., Komorkiewicz, M., Gorgon, M.: Real-time moving object detection for video surveillance system in FPGA. In: 2011 Conference on Design and Architectures for Signal and Image Processing (DASIP), pp. 1–8. IEEE (2011)Google Scholar
  22. 22.
    Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc. IEEE 90(7), 1151–1163 (2002)CrossRefGoogle Scholar
  23. 23.
    Eveland, C., Konolige, K., Bolles, R.C.: Background modeling for segmentation of video-rate stereo sequences. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 266–271. IEEE (1998)Google Scholar
  24. 24.
    Zhou, X., Yang, C., Yu, W.: Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 597–610 (2013)CrossRefGoogle Scholar
  25. 25.
    Suganuma, N., Kubo, T.: Fast dynamic object extraction using stereovision based on occupancy grid maps and optical flow. In: 2011 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), pp. 978–983. IEEE (2011)Google Scholar
  26. 26.
    Rodríguez-Canosa, G.R., Thomas, S., Del Cerro, J., Barrientos, A., MacDonald, B.: A real-time method to detect and track moving objects (DATMO) from unmanned aerial vehicles (UAVs) using a single camera. Remote Sens. 4(4), 1090–1111 (2012)CrossRefGoogle Scholar
  27. 27.
    Kimura, M., Shibasaki, R., Shao, X., Nagai, M.: Automatic extraction of moving objects from UAV-borne monocular images using multi-view geometric constraints. In: International Micro Air Vehicle Conference and Competition, IMAV 2014, Delft, The Netherlands, 12–15 August 2014, Delft University of Technology (2014)Google Scholar
  28. 28.
    Salgian, G., Bergen, J., Samarasekera, S., Kumar, R.: Moving target indication from a moving camera in the presence of strong parallax. Technical report, DTIC Document (2006)Google Scholar
  29. 29.
    Dey, S., Reilly, V., Saleemi, I., Shah, M.: Detection of independently moving objects in non-planar scenes via multi-frame monocular epipolar constraint. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 860–873. Springer, Heidelberg (2012). Scholar
  30. 30.
    Paragios, N., Deriche, R.: Geodesic active contours and level sets for the detection and tracking of moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 22(3), 266–280 (2000)CrossRefGoogle Scholar
  31. 31.
    Joshi, K.A., Thakore, D.G.: A survey on moving object detection and tracking in video surveillance system. Int. J. Soft Comput. Eng. 2(3), 44–48 (2012)Google Scholar
  32. 32.
    Cao, X., Lan, J., Yan, P., Li, X.: Vehicle detection and tracking in airborne videos by multi-motion layer analysis. Mach. Vis. Appl. 23(5), 921–935 (2012)CrossRefGoogle Scholar
  33. 33.
    Irani, M., Anandan, P.: A unified approach to moving object detection in 2D and 3D scenes. IEEE Trans. Pattern Anal. Mach. Intell. 20(6), 577–589 (1998)CrossRefGoogle Scholar
  34. 34.
    Kang, J., Cohen, I., Medioni, G., Yuan, C.: Detection and tracking of moving objects from a moving platform in presence of strong parallax. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 1, pp. 10–17. IEEE (2005)Google Scholar
  35. 35.
    Yuan, C., Medioni, G., Kang, J., Cohen, I.: Detecting motion regions in the presence of a strong parallax from a moving camera by multiview geometric constraints. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1627–1641 (2007)CrossRefGoogle Scholar
  36. 36.
    Kundu, A., Krishna, K.M., Sivaswamy, J.: Moving object detection by multi-view geometric techniques from a single camera mounted robot. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4306–4312, October 2009Google Scholar
  37. 37.
    Minaeian, S., Liu, J., Son, Y.J.: Effective and efficient detection of moving targets from a UAV’s camera. IEEE Trans. Intell. Transp. Syst. 19, 497–506 (2018)CrossRefGoogle Scholar
  38. 38.
    Castelli, T., Trémeau, A., Konik, H., Dinet, E.: Moving object detection for unconstrained low-altitude aerial videos, a pose-independant detector based on artificial flow. In: 2015 9th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 42–47. IEEE (2015)Google Scholar
  39. 39.
    Wu, Y., He, X., Nguyen, T.Q.: Moving object detection with a freely moving camera via background motion subtraction. IEEE Trans. Circuits Syst. Video Technol. 27(2), 236–248 (2017)CrossRefGoogle Scholar
  40. 40.
    Makino, K., Shibata, T., Yachida, S., Ogawa, T., Takahashi, K.: Moving-object detection method for moving cameras by merging background subtraction and optical flow methods. In: 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 383–387, November 2017Google Scholar
  41. 41.
    Ali, S., Shah, M.: COCOA: tracking in aerial imagery. In: Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications III, vol. 6209, p. 62090D. International Society for Optics and Photonics (2006)Google Scholar
  42. 42.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). Scholar
  43. 43.
    Lucas, B.D., Kanade, T., et al.: An iterative image registration technique with an application to stereo vision (1981)Google Scholar
  44. 44.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  45. 45.
    Collins, R., Zhou, X., Teh, S.K.: An open source tracking testbed and evaluation web site. In: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2005), vol. 2, p. 35 (2005)Google Scholar
  46. 46.
    Hasan, M.: Integrating geometric, motion and appearance constraints for robust tracking in aerial videos (2013)Google Scholar
  47. 47.
    Babenko, B., Yang, M.H., Belongie, S.: Visual tracking with online multiple instance learning. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 983–990. IEEE (2009)Google Scholar
  48. 48.
    Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. In: British Machine Vision Conference, vol. 1, p. 6 (2006)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Hazal Lezki
    • 1
    • 2
  • I. Ahu Ozturk
    • 1
  • M. Akif Akpinar
    • 1
    • 4
  • M. Kerim Yucel
    • 1
    • 3
    Email author
  • K. Berker Logoglu
    • 1
  • Aykut Erdem
    • 3
  • Erkut Erdem
    • 3
  1. 1.STM Defense Technologies and Trade Inc.AnkaraTurkey
  2. 2.Deparment of Electrical and Electronics EngineeringTOBB University of Economics and TechnologyAnkaraTurkey
  3. 3.Computer Vision Lab, Department of Computer EngineeringHacettepe UniversityAnkaraTurkey
  4. 4.Multimedia InformaticsMiddle East Technical UniversityAnkaraTurkey

Personalised recommendations