Advertisement

Journal of Intelligent & Robotic Systems

, Volume 65, Issue 1–4, pp 295–308 | Cite as

Vision Based UAV Attitude Estimation: Progress and Insights

  • Abd El Rahman Shabayek
  • Cédric Demonceaux
  • Olivier Morel
  • David Fofi
Article

Abstract

Unmanned aerial vehicles (UAVs) are increasingly replacing manned systems in situations that are dangerous, remote, or difficult for manned aircraft to access. Its control tasks are empowered by computer vision technology. Visual sensors are robustly used for stabilization as primary or at least secondary sensors. Hence, UAV stabilization by attitude estimation from visual sensors is a very active research area. Vision based techniques are proving their effectiveness and robustness in handling this problem. In this work a comprehensive review of UAV vision based attitude estimation approaches is covered, starting from horizon based methods and passing by vanishing points, optical flow, and stereoscopic based techniques. A novel segmentation approach for UAV attitude estimation based on polarization is proposed. Our future insightes for attitude estimation from uncalibrated catadioptric sensors are also discussed.

Keywords

UAV Attitude estimation Vision Polarization Catadioptric Horizon Segmentation Vanishing points Optical flow Stereoscopic 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ettinger, S.M., Nechyba, M.C., Ifju, P.G., Waszak, M.: Vision-guided flight stability and control for micro air vehicles. In: IEEE/RSJ Int Conf on Robots and Systems, pp. 2134–2140 (2002)Google Scholar
  2. 2.
    Demonceaux, C., Vasseur, P., Pégard, C.: Omnidirectional vision on uav for attitude computation. In: ICRA, pp. 2842–2847 (2006)Google Scholar
  3. 3.
    Demonceaux, C., Vasseur, P., Pégard, C.: UAV attitude computation by omnidirectional vision in urban environment. In: ICRA, pp. 2017–2022 (2007)Google Scholar
  4. 4.
    Barrows, G.L., Chahl, J.S., Srinivasan, M.V.: Biomimetic visual sensing and flight control. In: Proceedings Seventeenth International Unmanned Air Vehicle Systems Conference (2002)Google Scholar
  5. 5.
    Moore, R.J.D., Thurrowgood, S., Bland, D., Soccol, D., Srinivasan, M.V.: A stereo vision system for uav guidance. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2009)Google Scholar
  6. 6.
    Hsieh, S.-C., Wang, L.K., Hsaio, F.-B., Huang, K.-Y., Tsai, F.-J.: Airborne attitude/ground target location determinations using unscented Kalman filter. In: IEEE Proceedings of Aerospace Conference, vol. 3(6), pp. 1561–1568 (2004)Google Scholar
  7. 7.
    Dusha, D., Boles, W., Walker, R.: Attitude estimation for a fixed-wing aircraft using horizon detection and optical flow. In: Proceedings of the 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications, DICTA ’07, pp. 485–492. IEEE Computer Society, Washington, DC (2007)Google Scholar
  8. 8.
    Hwangbo, M., Kanade, T.: Visual-inertial attitude estimation using urban scene regularities. In: IEEE International Conference on Robotics and Automation (2011, to appear)Google Scholar
  9. 9.
    Thurrowgood, S., Soccol, D., Moore, R.J.D., Bland, D., Srinivasan, M.V.: A vision based system for attitude estimation of UAVs. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2009)Google Scholar
  10. 10.
    Todorovic, S., Nechyba, M.C.: Sky/ground modeling for autonomous MAV flight. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1422–1427 (2003)Google Scholar
  11. 11.
    Demonceaux, C., Vasseur, P., Pégard, C.: Robust attitude estimation with catadioptric vision. In: IROS, pp. 3448–3453 (2006)Google Scholar
  12. 12.
    Mondragón, I.F., Olivares-Méndez, M.A., Campoy, P., Martínez, C., Mejias, L.: Unmanned aerial vehicles UAVs attitude, height, motion estimation and control using visual systems. Auton. Robots 29(1), 17–34 (2010)CrossRefGoogle Scholar
  13. 13.
    Mondragón, I.F., Campoy, P., Martinez, C., Olivares, M.: Omnidirectional vision applied to Unmanned Aerial Vehicles UAVs attitude and heading estimation. Robot. Auton. Syst. 58(6), 809–819 (2010)CrossRefGoogle Scholar
  14. 14.
    Cornall, T.D., Egan, G.K., Price, A.: Aircraft attitude estimation from horizon video. Electron. Lett. 42(13), 744–745 (2006)CrossRefGoogle Scholar
  15. 15.
    Taylor, B., Bil, C., Watkins, S.: Horizon sensing attitude stabilisation: a VMC autopilot. In: 18th International UAV Systems Conference (2003)Google Scholar
  16. 16.
    Fd-1665p polarization camera. http://www.fluxdata.com/products/fd-1665-polarization-camera/. Accessed 9 Jan 2011
  17. 17.
    Terrier, P., Devlaminck, V., Charbois, J.M.: Segmentation of rough surfaces using a polarization imaging system. J. Opt. Soc. Am. A 25, 423–430 (2008)CrossRefGoogle Scholar
  18. 18.
    Wolff, L.B.: Polarization-based material classification from specular reflection. IEEE Trans. Pattern Anal. Mach. Intell. 12, 1059–1071 (1990)CrossRefGoogle Scholar
  19. 19.
    Xie, B., Xiang, Z., Pan, H., Liu, J.: Polarization-based water hazards detection for autonomous off-road navigation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS 2007, 29 Oct–2 Nov 2007, pp. 3186–3190 (2007)Google Scholar
  20. 20.
    Bao, G., Zhou, Z., Xiong, S., Lin, X., Ye, X.: Towardsmicro air vehicle flight autonomy research on the method of horizon extraction. In: Proceedings of the 20th IEEE Instrumentation and Measurement Technology Conference (IMTC03), vol. 2, pp. 1387–1390 (2003)Google Scholar
  21. 21.
    Todorovic, S., Nechyba, M.C.: A vision system for intelligent mission profiles of micro air vehicles. IEEE Trans. Veh. Technol. 53, 1713–1725 (2004)CrossRefGoogle Scholar
  22. 22.
    Ettinger, S.M., Nechyba, M.C., Ifju, P.G., Waszak, M.: Vision-guided flight stability and control for micro air vehicles. Adv. Robot. 17(7), 617–640 (2003)CrossRefGoogle Scholar
  23. 23.
    Cornall, T.D., Egan, G.K.: Measuring horizon angle from video on a small unmanned air vehicle. In: 2nd International Conference on Autonomous Robots and Agents (2004)Google Scholar
  24. 24.
    Cornall, T.D., Egan, G.K.: Measuring horizon angle from video on a small unmanned airborne vehicle. In: 2nd International Conference on Autonomous Robots and Agents. Palmerston North, New Zealand (2004)Google Scholar
  25. 25.
    Cornall, T.D., Egan, G.K.: Measuring horizon angle from video on a small unmanned air vehicle. Technical report, MONASH University, Department of Electrical and Computer Systems Engineering (2005)Google Scholar
  26. 26.
    Baker, S., Nayar, S.K.: A theory of catadioptric image formation. In: International Conference on Computer Vision (ICCV03), pp. 1351–1358 (2003)Google Scholar
  27. 27.
    Bazin, J.C., Kweon, I.S., Demonceaux, C., Vasseur, P.: UAV attitude estimation by combining horizon-based and homography-based approaches for catadioptric image. In: 6th IFAC/EURON Intelligent Autonomous Vehicles (IAV07). Toulouse, France (2007)Google Scholar
  28. 28.
    Antone, M.E., Teller, S.: Automatic recovery of relative camera rotations for urban scenes. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR00), pp. 282–289. Head Island, SC, USA (2000)Google Scholar
  29. 29.
    Bazin, J.C., Kweon, I., Demonceaux, C., Vasseur, P.: UAV attitude estimation by vanishing points in catadioptric images. In: ICRA, pp. 2743–2749 (2008)Google Scholar
  30. 30.
    Tarhan, M., Altug, E.: EKF based attitude estimation and stabilization of a quadrotor UAV using vanishing points in catadioptric images. J. Intell. Robot. Syst. 62(3–4) (2011)Google Scholar
  31. 31.
    Shufelt, J.A.: Performance evaluation and analysis of vanishing point detection techniques. IEEE Trans. Pattern Anal. Mach. Intell. 21(3), 282–288 (1999)CrossRefGoogle Scholar
  32. 32.
    Denis, P., Elder, J.H., Estrada, F.J.: Efficient edge-based methods for estimating Manhattan frames in urban imagery. In: Proceedings of the 10th European Conference on Computer Vision: Part II, pp. 197–210. Springer, Berlin (2008)Google Scholar
  33. 33.
    Barnard, S.T.: Interpreting perspective images. Artif. Intell. 21, 435–462 (1984)CrossRefGoogle Scholar
  34. 34.
    Rother, C.: A new approach for vanishing point detection in architectural environments. In: Proc. 11th British Machine Vision Conference, pp. 382–391 (2000)Google Scholar
  35. 35.
    Wang, L.K., Hsieh, S.-C., Hsueh, E.C.-W., Hsaio, F.-B., Huang, K.-Y.: Complete pose determination for low altitude Unmanned Aerial Vehicle using stereo vision. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005 (IROS 2005), pp. 108–113 (2005)Google Scholar
  36. 36.
    Eynard, D., Vasseur, P., Demonceaux, C., Fremont, V.: UAV altitude estimation by mixed stereoscopic vision. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 646–651 (2010)Google Scholar
  37. 37.
    Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton Mifflin, Boston (1950)Google Scholar
  38. 38.
    Soatto, S., Frezza, R., Perona, P.: Motion estimation via dynamic vision. IEEE Trans. Automat. Contr. 41, 393–413 (1996)MATHMathSciNetCrossRefGoogle Scholar
  39. 39.
    Soatto, S., Perona, P.: Recursive 3-d visual motion estimation using subspace constraints. Int. J. Comput. Vis. 22, 235–259 (1997)CrossRefGoogle Scholar
  40. 40.
    Gurfil, P., Rotstein, H.: Partial aircraft state estimation from visual motion using the subspace constraints approach. J. Guid. Control Dyn. 24, 1016–1028 (2001)CrossRefGoogle Scholar
  41. 41.
    Webb, T., Prazenica, R., Kurdila, A., Lind, R.: Vision-based state estimation for autonomous micro air vehicles. In: Proc. of the AIAA Guidance, Navigation, and Control Conference, p. 5249 (2004)Google Scholar
  42. 42.
    Webb, T., Prazenica, R., Kurdila, A., Lind, R.: Vision-based state estimation for uninhabited aerial vehicles. In: Proc. of the AIAA Guidance, Navigation, and Control Conference, p. 5869 (2005)Google Scholar
  43. 43.
    Gebert, G., Snyder, D., Lopez, J., Siddiqi, N., Evers, J.: Optical flow angular rate determination. In: Proc. of the International Conference on Image Processing, vol. 1, pp. 49–52 (2003)Google Scholar
  44. 44.
    Iyer, R.V., He, Z., Chandler, P.R.: On the computation of the ego-motion and distance to obstacles for a micro air vehicle. In: Proc. of the IEEE American Control Conference (2006)Google Scholar
  45. 45.
    Kehoe, J., Causey, R., Arvai, A., Lind, R.: Partial aircraft state estimation from optical flow using non-model-based optimization. In: Proc. of the IEEE American Control Conference (2006)Google Scholar
  46. 46.
    Kehoe, J.J., Watkins, A.S., Causey, R.S., Lind, R.: State estimation using optical flow from parallax-weighted feature tracking. In: Proceedings of the AIAA Guidance, Navigation, and Control Conference (2006)Google Scholar
  47. 47.
    Moore, R.J.D., Thurrowgood, S., Soccol, D., Bland, D., Srinivasan, M.V.: A bio-inspired stereo vision system for guidance of autonomous aircraft. In: Advances in Theory and Applications of Stereo Vision (2010)Google Scholar
  48. 48.
    Stowers, J., Bainbridge-Smith, A., Hayes, M., Mills, S.: Optical flow for attitude estimation of a quadrotor helicopter. In: European Micro Air Vehicle Conference (2009)Google Scholar
  49. 49.
    Moore, R.J.D., Thurrowgood, S., Bland, D., Soccol, D., Srinivasan, M.V.: UAV altitude and attitude stabilisation using a coaxial stere ovision system. In: IEEE International Conference on Robotics and Automation (2010)Google Scholar
  50. 50.
    Hedden, C.: Vision-based UAV Aerodynamic Attitude Estimation in the Presence of Dynamic Obstacles. University of Kansas, Lawrence (2010)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Abd El Rahman Shabayek
    • 1
  • Cédric Demonceaux
    • 1
  • Olivier Morel
    • 1
  • David Fofi
    • 1
  1. 1.Le2i - UMR CNRS 5158, IUT Le CreusotUniversité de BourgogneDijonFrance

Personalised recommendations