Visual feedback control of quadrotor by object detection in movies

Abstract

In the previous report, the authors developed a quadrotor mission planning function using a smartphone and an autopilot function using GPS signals. In this paper, a visual feedback control is considered so that the system can automatically recognize the existence of an object in images or movies acquired by a quadrotor with a surveillance camera and this enables the quadrotor to track the object within the operational environment without using GPS signals. The visual feedback control is realized by automatic object recognition based on techniques of computer vision, in which the center of gravity position in the image coordinate system can be obtained from color or shape information. A visual feedback control application for flight control of DJI’s quadrotor Matrice100, and image analysis software are developed using Xcode and MATLAB, respectively. The validity and effectiveness are evaluated through experiments.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. 1.

    Hattori H (2008) Stereo vision technology for automotive applications. IEEE Intell Veh Sympos 63(5):48–51

    Google Scholar 

  2. 2.

    Serizawa K, Ladig R, Shimonomura K (2019) Movable camera and stereoscopic image system for teleoperation of aerial robot working in high place. In: Proceedings of the 2019 JSME conference on robotics and mechatronics, 1P2-N08

  3. 3.

    Otsuka H, Yajima R, Nagatani K, Kubo D (2015) Localization of a small multi-rotor UAV using dropping AR Markersh. In: Proceedings of the 2015 JSME conference on robotics and mechatronics, 2A2-G02

  4. 4.

    Saito H, Takubo T, Ueno A, Cai K, Miyamoto R, Hara S (2018) Recognizing own UAV position and measuring position of a person using camera image. In: Proceedings of the 2018 JSME conference on robotics and mechatronics, 1P1-C08

  5. 5.

    Lu Z, Nagata F, Watanabe K, Habib MK (2017) iOS application for quadrotor UAV remote control—implementation of basic functions with iPhone. Artif Life Robot 22(3):374–379

    Article  Google Scholar 

  6. 6.

    Lu Z, Nagata F, Watanabe K (2018) Mission planning of iOS application for a quadrotor UAV. Artif Life Robot 23(3):428–433

    Article  Google Scholar 

  7. 7.

    Gomez-Avila J, Lopez-Franco C, Alanis AY, Arana-Daniel N, Lopez-Franco M (2018) Ground vehicle tracking with a quadrotor using image based visual servoing. IFAC Papers Online, 2nd IFAC conference on modelling, identification and control of nonlinear systems MICNON 2018, vol 51, no 13, pp 344–349

  8. 8.

    Yang L, Liu Z, Wang X (2019) A new image-based visual servo control algorithm for target tracking problem of fixed-wing unmanned aerial vehicle. 2019 Chinese control conference (CCC), pp 8142–8147

  9. 9.

    Sheng H, Shi E, Zhang K (2019) Image-based visual servoing of a quadrotor with improved visibility using model predictive control. 2019 IEEE 28th international symposium on industrial electronics (ISIE), pp 551–556

  10. 10.

    Cao Z, Chen X, Yu Y, Yu J, Liu X, Zhou C, Tan M (2020) Image dynamics-based visual servoing for quadrotors tracking a target with a nonlinear trajectory observer. IEEE Trans Syst Man Cybern Syst 50(1):376–384

    Article  Google Scholar 

  11. 11.

    Shao L, Nagata F, Watanabe K (2019) Remote control application for a quadrotor supporting iOS and android. In: Proceedings of the 2019 JSME Conference on robotics and mechatronics, 1P1–N06

  12. 12.

    Bay H, Ess A, Tuytelaars T, Gool LV (2008) SURF: speeded up robust features. Comput Vis Image Underst (CVIU) 110(3):346–359

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Fusaomi Nagata.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was presented in part at the 25th International Symposium on Artificial Life and Robotics (Beppu, Oita, January 22–24, 2020).

About this article

Verify currency and authenticity via CrossMark

Cite this article

Shao, L., Nagata, F., Ochi, H. et al. Visual feedback control of quadrotor by object detection in movies. Artif Life Robotics (2020). https://doi.org/10.1007/s10015-020-00609-3

Download citation

Keywords

  • Visual feedback control
  • Quadrotor
  • iOS application
  • Image processing
  • Object detection