Skip to main content
SpringerLink
Log in

A Catadioptric and Pan-Tilt-Zoom Camera Pair Object Tracking System for UAVs

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Unmanned aerial vehicles (UAVs) are seeing widespread use in military, scientific, and civilian sectors in recent years. As the mission demands increase, these systems are becoming more complicated. Omnidirectional camera is a vision sensor that can captures 360° view in a single frame. In recent years omnidirectional camera usage has experienced a remarkable increase in many fields, where many innovative research has been done. Although, it is very promising, employment of omnidirectional cameras in UAVs is quite new. In this paper, an innovative sensory system is proposed, that has an omnidirectional imaging device and a pan tilt zoom (PTZ) camera. Such a system combines the advantages of both of the camera systems. The system can track any moving object within its 360° field of view and provide detailed images of it. The detection of the moving object has been accomplished by an adaptive background subtraction method implemented on the lowered resolution images of the catadioptric camera. A novel algorithm has also been developed to estimate the relative distance of the object with respect to the UAV, using tracking information of both of the cameras. The algorithms are implemented on an experimental system to validate the approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Castillo, P., Lozano, R., Dzul, A.E.: Modelling and Control of Mini-Flying Machines. Springer, London (2005)

    Google Scholar 

  2. Winters, N., Gaspar, J., Lacey, G., Santos-Victor, J.: Omni-directional vision for robot navigation. In: Proc. of IEEE Workshop on Omnidirectional Vision. Hilton Head Island, SC (2000)

    Google Scholar 

  3. Fiala, M., Basu A.: Robot navigation using panoramic tracking. Pattern Recogn. 37, 2195–2215 (2004)

    Article  Google Scholar 

  4. Gava, C.C., Vassallo, R.F., Roberti, F., Carelli, C., Bastos-Filho, T.F.: Nonlinear control techniques and omnidirectional vision for team formation on cooperative robotics. In: IEEE International Conference on Robotics and Automation, pp. 2409–2414. Roma, Italy (2007)

  5. Sun, Y., Cao, Q., Chen, W.: An object tracking and global localization method using omnidirectional vision system. In: Proceedings of the 5th World Congress on Intelligent Control and Automation, pp. 4730–4735. Hangzhou, Peoples Republic of China (2004)

    Google Scholar 

  6. Jamzad, M., Hadjkhodabakhshi, A.R., Mirrokni, V.S.: Object detection and localization using omnidirectional vision in the robocup environment. Sci. Iran. 14(6), 599–611 (2007)

    Google Scholar 

  7. Chen, Z., Chen, Y., Wang, Y.: Image-based control of an autonomous robot with omnidirectional vision. In: IEEE International Symposium on Industrial Embedded Systems. Lisbon (2007)

  8. J. Gluckman, S. K. Nayar: Ego-motion and omnidirectional cameras. In: Proc. International Conference on Computer Vision (ICCV98), pp. 999–1005 (1998)

  9. Bruss, A., Horn, B.K.P.: Passive navigation. Comput. Vis. Graph. Image Process. 21, 3–20 (1983)

    Article  Google Scholar 

  10. Hrabar, S., Sukhatme, G.S.: Omnidirectional vision for an autonomous helicopter. IEEE Int. Conf. Robot. Autom. 1, 558–563 (2003)

    Google Scholar 

  11. Hrabar, S., Sukhatme, G.S.: A comparison of two camera configurations for optic-flow based navigation of a UAV through urban canyons. In: Proceedings of the International Conference on Intelligent Robots and Systems, pp. 2673–2680 (2004)

  12. Demonceaux C., Vasseur P., Pégard C.: Robust attitude estimation with catadioptric vision. In: Proceedings of the International Conference on Intelligent Robots and Systems. Beijing, China (2006)

  13. Demonceaux C., Vasseur P., Pégard C.: Omnidirectional vision on UAV for attitude computation. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, pp. 2842–2847 (2006)

  14. Bazin, J.C., Kweon, I.S., Demonceaux, C., Vasseur, P.: UAV attitude estimation by combining horizon-based and homography-based approaches for catadioptric image. In: 6th IFAC/EURON Intelligent Autonomous Vehicles. Toulouse, France (2007)

  15. Demonceaux, C., Vasseur, P., Pégard, C.: UAV attitude computation by omnidirectional vision in urban environment. IEEE Int. Conf. Robot. Autom. (2007). doi:10.1109/ROBOT.2007.363618

    Google Scholar 

  16. Bazin, J.C., Kweon, I., Demonceaux, C., Vasseur, P.: UAV attitude estimation by vanishing points in catadioptric images. In: IEEE International Conference on Robotics and Automation, pp. 2743–2749. Pasadena, CA (2008)

    Chapter  Google Scholar 

  17. Tarhan, M., Altuğ, E.: Control of a quadrotor air vehicle by vanishing points in catadioptric images. In: International Symposium on Optomechatronic Technologies (ISOT). Istanbul Turkey (2009)

  18. Tarhan, M., Altuğ, E.: EKF based attitude estimation and stabilization of a quadrotor UAV using vanishing points in catadioptric images. J. Intell. Robot. Syst. (2010). doi:10.1007/s10846-010-9459-y

    Google Scholar 

  19. Sato, Y., Hashimoto, K., Shibata, Y.: A new networked surveillance video system by combination of omni-directional and network controlled cameras. In: Network-Based Information Systems, vol. 5186. Springer, Berlin (2008)

    Google Scholar 

  20. Scotti, G., Marcenaro, L., Coelho, C., Selvaggi, F., Regazzoni, C.S.: Dual camera intelligent sensor for high definition 360 degrees surveillance. IEE Proc., Vis. Image Signal Process. 152, 250–257 (2005)

    Article  Google Scholar 

  21. Yao, Y., Abidi, B., Abidi M.: Fusion of omnidirectional and PTZ cameras for accurate cooperative tracking. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS’06) (2006)

  22. Chen, C.H., Yao, Y., Page, D., Abidi, B., Koschan, A., Abidi, M.: Heterogeneous fusion of omnidirectional and PTZ cameras for multiple object tracking. IEEE Trans. Circuits Syst. Video Technol. 18(8), 1052–1063 (2008)

    Article  Google Scholar 

  23. Elhabian, Y., El-Sayed, K.M., Ahmed, S.H.: Moving object detection in spatial domain using background removal techniques—state-of-art. Recent Pat. Comput. Sci. 1, 32–54 (2008)

    Article  Google Scholar 

  24. Sugandi, B., Kim, H., Tan, J.K., Ishikawa, S.: Tracking of moving objects by using a low resolution image. In: Second International Conference on Innovative Computing, Information and Control, ICICIC (2007)

  25. Scotti, G., Cuocolo, A., Coelho, C., Marchesotti, L.: A novel pedestrian classification algorithm for a high definition dual camera 360 degrees surveillance system. In: IEEE International Conference on Image Processing, ICIP (2005)

  26. Matrox Imaging: http://www.matrox.com/imaging

  27. Vstone Corporation: http://www.vstone.co.jp/e/sensor.html

  28. Barreto, J.P., Araujo, H.: Geometric properties of central catadioptric line images and their application in calibration. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1327–1333 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. System Dynamics and Control Graduate Program, Istanbul Technical University, Istanbul, 34437, Turkey

    Metin Tarhan

  2. Department of Mechanical Engineering, Istanbul Technical University, Inönü Cad. No:65, Gümüşsuyu, Beyoğlu, Istanbul, 34437, Turkey

    Erdinç Altuğ

Authors
  1. Metin Tarhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erdinç Altuğ
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erdinç Altuğ.

Additional information

Supported in part by the project 107E211 funded by the Scientific and Technological Research Council of Turkey (TÜBİTAK).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tarhan, M., Altuğ, E. A Catadioptric and Pan-Tilt-Zoom Camera Pair Object Tracking System for UAVs. J Intell Robot Syst 61, 119–134 (2011). https://doi.org/10.1007/s10846-010-9504-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-010-9504-x

Keywords

Search

Navigation