Skip to main content
Log in

Real-Time Model-Based Video Stabilization for Microaerial Vehicles

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The emerging branch of micro aerial vehicles (MAVs) has attracted a great interest for their indoor navigation capabilities, but they require a high quality video for tele-operated or autonomous tasks. A common problem of on-board video quality is the effect of undesired movements, so different approaches solve it with both mechanical stabilizers or video stabilizer software. Very few video stabilizer algorithms in the literature can be applied in real-time but they do not discriminate at all between intentional movements of the tele-operator and undesired ones. In this paper, a novel technique is introduced for real-time video stabilization with low computational cost, without generating false movements or decreasing the performance of the stabilized video sequence. Our proposal uses a combination of geometric transformations and outliers rejection to obtain a robust inter-frame motion estimation, and a Kalman filter based on an ANN learned model of the MAV that includes the control action for motion intention estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. The complete experimentation and associated results can be checked in [7].

References

  1. Aguilar WG, Angulo C (2012a) Compensación y aprendizaje de efectos generados en la imagen durante el desplazamiento de un robot. In: Proceedings of the 10th symposium CEA-Spanish Committee of Automatic Control, February 2012

  2. Aguilar WG, Angulo C (2012b) Compensación de los efectos generados en la imagen por el control de navegaciń del robot aibo ers 7. In: Proceedings of the 7th congress of science and technology ESPE 2012, June 2012

  3. Aguilar WG, Angulo C (2013) Estabilización robusta de vídeo basada en diferencia de nivel de gris. In: Proceedings of the 8th congress of science and technology ESPE 2013, June 2013

  4. Aguilar WG, Angulo C (2014a) Real-time video stabilization without phantom movements for micro aerial vehicles. EURASIP J Image Video Process 2014(1):46

  5. Aguilar WG, Angulo C (2014b) Optimization of robust video stabilization based on motion intention for micro aerial vehicles. In: 2014 International multi-conference on systems signals and devices (SSD), February 2014

  6. Aguilar WG, Angulo C (2014c) Estabilización de vídeo en micro vehículos aéreos y su aplicación en la detección de caras. In: Proceedings of the 9th congress of science and technology ESPE 2014, May 2014

  7. Aguilar WG, Angulo C, Costa R, Molina L (2014) Control autónomo de cuadricopteros para seguimiento de trayectorias. In: Proceedings of the 9th congress of science and technology ESPE 2014, May 2014

  8. Alahi A, Ortiz R, Vandergheynst P (2012) FREAK: fast retina keypoint. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR), 2012, pp 510–517

  9. Bailey SW, Bodenheimer B (2012) A comparison of motion capture data recorded from a vicon system and a Microsoft kinect sensor. In: Proceedings of the ACM symposium on applied perception, SAP ’12, 2012. ACM, New York, p 121

  10. Battiato S, Gallo G, Puglisi G, Scellato S (2007) SIFT features tracking for video stabilization. In: 14th International conference on image analysis and processing, 2007, ICIAP 2007, pp 825–830

  11. Bay H, Tuytelaars T, Gool L (2006) SURF: speeded up robust features. In: Leonardis A, Bischof H, Pinz A (eds) Computer vision—ECCV 2006, lecture notes in computer science, vol 3951. Springer Berlin Heidelberg, Berlin, pp 404–417

  12. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI–8(6):679–698

    Article  Google Scholar 

  13. Chang HC, Lai SH, Lu KR (2004) A robust and efficient video stabilization algorithm. In: 2004 IEEE international conference on multimedia and expo, 2004, ICME ’04, vol 1, pp 29–32

  14. Derpanis KG (2010) Overview of the RANSAC algorithm. Computer Science, York University

  15. Dimov D, Nikolov A (2014) Real time video stabilization for handheld devices. In: Proceedings of the 15th international conference on computer systems and technologies. ACM, pp 124–133

  16. Fang CL, Tsai TH, Chang CH (2012) Video stabilization with local rotational motion model. In: 2012 IEEE Asia Pacific conference on circuits and systems (APCCAS), pp 551–554

  17. Faugeras O, Luong QT, Papadopoulou T (2001) The geometry of multiple images: the laws that govern the formation of images of a scene and some of their applications. MIT Press, Cambridge

    MATH  Google Scholar 

  18. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  19. Forsyth DA, Ponce J (2002) Computer vision: a modern approach. Prentice Hall Professional Technical Reference, Upper Saddle River

    Google Scholar 

  20. Grundmann M, Kwatra V, Essa I (2011) Auto-directed video stabilization with robust l1 optimal camera paths. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 225–232

  21. Harris C, Stephens M (1988) A combined corner and edge detection. In: Proceedings of the fourth Alvey vision conference, pp 147–151

  22. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, New York

    MATH  Google Scholar 

  23. Hsu YF, Chou CC, Shih MY (2012) Moving camera video stabilization using homography consistency. In: 19th IEEE international conference on image processing (ICIP), 2012, pp 2761–2764

  24. Kang SJ, Wang TS, Kim DH, Morales A, Ko SJ (2012) Video stabilization based on motion segmentation. In: 2012 IEEE international conference on consumer electronics (ICCE), pp 416–417

  25. Kendoul F (2012) Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems. J Field Robot 29(2):315–378

    Article  Google Scholar 

  26. Lee KY, Chuang YY, Chen BY, Ouhyoung M (2009) Video stabilization using robust feature trajectories. In: IEEE 12th international conference on computer vision, 2009, pp 1397–1404

  27. Leutenegger S, Chli M, Siegwart RY (2011) BRISK: binary robust invariant scalable keypoints. In: 2011 IEEE international conference on computer vision (ICCV), pp 2548–2555

  28. Liu F, Gleicher M, Wang J, Jin H, Agarwala A (2011) Subspace video stabilization. ACM Trans Graph (TOG) 30(1):4

    Article  Google Scholar 

  29. Lowe D (1999) Object recognition from local scale-invariant features. In: The proceedings of the seventh IEEE international conference on computer vision, 1999, vol 2, pp 1150–1157

  30. Luo J, Oubong G (2009) A comparison of SIFT, PCA-SIFT and SURF. Int J Image Process (IJIP) 3(4):143–152

  31. Matsushita Y, Ofek E, Ge W, Tang X, Shum HY (2006) Full-frame video stabilization with motion inpainting. IEEE Trans Pattern Anal Mach Intell 28(7):1150–1163

    Article  Google Scholar 

  32. Mikolajczyk K, Schmid C (2004) Scale & affine invariant interest point detectors. Int J Comput Vis 60(1):63–86

  33. Niskanen M, Silven O, Tico M (2006) Video stabilization performance assessment. In: 2006 IEEE international conference on multimedia and expo, pp 405–408

  34. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: 2011 IEEE international conference on computer vision (ICCV), pp 2564–2571

  35. Song C, Zhao H, Jing W, Zhu H (2012) Robust video stabilization based on particle filtering with weighted feature points. IEEE Trans Consum Electron 58(2):570–577

    Article  Google Scholar 

  36. Tordoff B, Murray DW (2002) Guided sampling and consensus for motion estimation. In: Heyden A, Sparr G, Nielsen M, Johansen P (eds) Computer vision—ECCV 2002, lecture notes in computer science, vol 2350. Springer Berlin Heidelberg, Berlin, pp 82–96

  37. Wang C, Kim JH, Byun KY, Ni J, Ko SJ (2009) Robust digital image stabilization using the Kalman filter. IEEE Trans Consum Electron 55(1):6–14

    Article  Google Scholar 

  38. Xu J, Chang HW, Yang S, Wang M (2012) Fast feature-based video stabilization without accumulative global motion estimation. IEEE Trans Consum Electron 58(3):993–999

    Article  Google Scholar 

  39. Yang J, Schonfeld D, Mohamed M (2009) Robust video stabilization based on particle filter tracking of projected camera motion. IEEE Trans Circuits Syst Video Technol 19(7):945–954

    Article  Google Scholar 

  40. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):1–45

    Article  Google Scholar 

  41. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334

    Article  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by the Spanish Ministry of Economy and Competitiveness, through the PATRICIA Project (TIN 2012-38416-C03-01). The research fellow Wilbert G. Aguilar thanks the funding through a Grant from the Program “Convocatoria Abierta 2011” issued by the Secretary of Education, Science, Technology and Innovation SENESCYT of the Republic of Ecuador.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wilbert G. Aguilar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aguilar, W.G., Angulo, C. Real-Time Model-Based Video Stabilization for Microaerial Vehicles. Neural Process Lett 43, 459–477 (2016). https://doi.org/10.1007/s11063-015-9439-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-015-9439-0

Keywords

Navigation