Skip to main content
SpringerLink
Log in

Visual Servoing and Visual Tracking

  • Reference work entry
Springer Handbook of Robotics

Abstract

This chapter introduces visual servo control, using computer vision data in the servo loop to control the motion of a robot. We first describe the basic techniques that are by now well established in the field. We give a general overview of the formulation of the visual servo control problem, and describe the two archetypal visual servo control schemes: image-based and position-based visual servo control. We then discuss performance and stability issues that pertain to these two schemes, motivating advanced techniques. Of the many advanced techniques that have been developed, we discuss 2.5-D, hybrid, partitioned, and switched approaches. Having covered a variety of control schemes, we conclude by turning briefly to the problems of target tracking and controlling motion directly in the joint space.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 309.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

LQG:

linear quadratic Gaussian

References

  1. B. Espiau, F. Chaumette, P. Rives: A new approach to visual servoing in robotics, IEEE Trans. Robot. Autom. 8, 313–326 (1992)

    Article  Google Scholar 

  2. S. Hutchinson, G. Hager, P. Corke: A tutorial on visual servo control, IEEE Trans. Robot. Autom. 12, 651–670 (1996)

    Article  Google Scholar 

  3. L. Weiss, A. Sanderson, C. Neuman: Dynamic sensor-based control of robots with visual feedback, IEEE J. Robot. Autom. 3, 404–417 (1987)

    Article  Google Scholar 

  4. J. Feddema, O. Mitchell: Vision-guided servoing with feature-based trajectory generation, IEEE Trans. Robot. Autom. 5, 691–700 (1989)

    Article  Google Scholar 

  5. D. Forsyth, J. Ponce: Computer Vision: A Modern Approach (Prentice Hall, Upper Saddle River 2003)

    Google Scholar 

  6. Y. Ma, S. Soatto, J. Kosecka, S. Sastry: An Invitation to 3-D Vision:From Images to Geometric Models (Springer, New York 2003)

    Google Scholar 

  7. H. Michel, P. Rives: Singularities in the determination of the situation of a robot effector from the erspective view of three points. Tech. Rep. 1850, INRIA Res. Rep. (1993)

    Google Scholar 

  8. M. Fischler, R. Bolles: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Commun. ACM 24, 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  9. E. Malis: Improving vision-based control using efficient second-order minimization techniques, IEEE Int. Conf. Robot. Autom. (New Orleans 2004) pp. 1843–1848

    Google Scholar 

  10. P. Corke, S. Hutchinson: A new partitioned approach to image-based visual servo control, IEEE Trans. Robot. Autom. 17, 507–515 (2001)

    Article  Google Scholar 

  11. F. Chaumette: Potential problems of stability and convergence in image-based and position-based visual servoing. In: The Confluence of Vision and Control, LNCIS Series, Vol. 237, ed. by D. Kriegman, G. Hager, S. Morse (Springer, Heidelberg 1998) pp. 66–78

    Chapter  Google Scholar 

  12. E. Malis: Visual servoing invariant to changes in camera intrinsic parameters, IEEE Trans. Robot. Autom. 20, 72–81 (2004)

    Article  Google Scholar 

  13. A. Isidori: Nonlinear Control Systems, 3rd edn. (Springer Berlin, Heidelberg 1995)

    MATH  Google Scholar 

  14. G. Hager, W. Chang, A. Morse: Robot feedback control based on stereo vision: Towards calibration-free hand-eye coordination, IEEE Contr. Syst. Mag. 15, 30–39 (1995)

    Article  Google Scholar 

  15. M. Iwatsuki, N. Okiyama: A new formulation of visual servoing based on cylindrical coordinate system, IEEE Trans. Robot. Automation 21, 266–273 (2005)

    Google Scholar 

  16. F. Chaumette, P. Rives, B. Espiau: Classification and realization of the different vision-based tasks. In: Visual Servoing, Robotics and Automated Systems, Vol. 7, ed. by K. Hashimoto (World Scientific, Singapore 1993) pp. 199–228

    Google Scholar 

  17. A. Castano, S. Hutchinson: Visual compliance: task directed visual servo control, IEEE Trans. Robot. Autom. 10, 334–342 (1994)

    Article  Google Scholar 

  18. G. Hager: A modular system for robust positioning using feedback from stereo vision, IEEE Trans. Robot. Autom. 13, 582–595 (1997)

    Article  Google Scholar 

  19. F. Chaumette: Image moments: a general and useful set of features for visual servoing, IEEE Trans. Robot. Autom. 20, 713–723 (2004)

    Google Scholar 

  20. O. Tahri, F. Chaumette: Point-based and region-based image moments for visual servoing of planar objects, IEEE Trans. Robot. 21, 1116–1127 (2005)

    Article  Google Scholar 

  21. I. Suh: Visual servoing of robot manipulators by fuzzy membership function based neural networks. In: Visual Servoing, Robotics and Automated Systems, Vol. 7, ed. by K. Hashimoto (World Scientific, Singapore 1993) pp. 285–315

    Google Scholar 

  22. G. Wells, C. Venaille, C. Torras: Vision-based robot positioning using neural networks, Image Vision Comput. 14, 75–732 (1996)

    Article  Google Scholar 

  23. J.T. Lapresté, F. Jurie, F. Chaumette: An efficient method to compute the inverse jacobian matrix in visual servoing, IEEE Int. Conf. Robot. Autom. (New Orleans 2004) pp. 727–732

    Google Scholar 

  24. K. Hosada, M. Asada: Versatile visual servoing without knowledge of true jacobian, IEEE/RSJ Int. Conf. Intell. Robots Syst. (Munchen 1994) pp. 186–193

    Google Scholar 

  25. M. Jägersand, O. Fuentes, R. Nelson: Experimental evaluation of uncalibrated visual servoing for precision manipulation, IEEE Int. Conf. Robot. Autom. (Albuquerque 1997) pp. 2874–2880

    Google Scholar 

  26. J. Piepmeier, G.M. Murray, H. Lipkin: Uncalibrated dynamic visual servoing, IEEE Trans. Robot. Autom. 20, 143–147 (2004)

    Article  Google Scholar 

  27. K. Deguchi: Direct interpretation of dynamic images and camera motion for visual servoing without image feature correspondence, J. Robot. Mechatron. 9(2), 104–110 (1997)

    Google Scholar 

  28. W. Wilson, C. Hulls, G. Bell: Relative end-effector control using cartesian position based visual servoing, IEEE Trans. Robot. Autom. 12, 684–696 (1996)

    Article  Google Scholar 

  29. B. Thuilot, P. Martinet, L. Cordesses, J. Gallice: Position based visual servoing: Keeping the object in the field of vision, IEEE Int. Conf. Robot. Autom. (Washington 2002) pp. 1624–1629

    Google Scholar 

  30. D. Dementhon, L. Davis: Model-based object pose in 25 lines of code, Int. J. Comput. Vision 15, 123–141 (1995)

    Article  Google Scholar 

  31. D. Lowe: Three-dimensional object recognition from single two-dimensional images, Artif. Intell. 31(3), 355–395 (1987)

    Article  Google Scholar 

  32. E. Malis, F. Chaumette, S. Boudet: 2-1/2 D visual servoing, IEEE Trans. Robot. Autom. 15, 238–250 (1999)

    Article  Google Scholar 

  33. E. Malis, F. Chaumette: Theoretical improvements in the stability analysis of a new class of model-free visual servoing methods, IEEE Trans. Robot. Autom. 18, 176–186 (2002)

    Article  Google Scholar 

  34. J. Chen, D. Dawson, W. Dixon, A. Behal: Adaptive homography-based visual servo tracking for fixed camera-in-hand configurations, IEEE Trans. Contr. Syst. Technol. 13, 814–825 (2005)

    Article  Google Scholar 

  35. G. Morel, T. Leibezeit, J. Szewczyk, S. Boudet, J. Pot: Explicit incorporation of 2-D constraints in vision-based control of robot manipulators, Int. Symp. Exp. Robot. 250, 99–108 (2000), LNCIS Series

    Article  Google Scholar 

  36. F. Chaumette, E. Malis: 2 1/2 D visual servoing: a possible solution to improve image-based and position-based visual servoings, IEEE Int. Conf. Robot. Autom. (San Fransisco 2000) pp. 630–635

    Google Scholar 

  37. E. Cervera, A.D. Pobil, F. Berry, P. Martinet: Improving image-based visual servoing with three-dimensional features, Int. J. Robot. Res. 22, 821–840 (2004)

    Article  Google Scholar 

  38. F. Schramm, G. Morel, A. Micaelli, A. Lottin: Extended 2-D visual servoing, IEEE Int. Conf. Robot. Autom. (New Orleans 2004) pp. 267–273

    Google Scholar 

  39. N. Papanikolopoulos, P. Khosla, T. Kanade: Visual tracking of a moving target by a camera mounted on a robot: A combination of vision and control, IEEE Trans. Robot. Autom. 9, 14–35 (1993)

    Article  Google Scholar 

  40. K. Hashimoto, H. Kimura: LQ optimal and nonlinear approaches to visual servoing. In: Visual Servoing, Robot. Autom. Syst., Vol. 7, ed. by K. Hashimoto (World Scientific, Singapore 1993) pp. 165–198

    Google Scholar 

  41. B. Nelson, P. Khosla: Strategies for increasing the tracking region of an eye-in-hand system by singularity and joint limit avoidance, Int. J. Robot. Res. 14, 225–269 (1995)

    Article  Google Scholar 

  42. B. Nelson, P. Khosla: Force and vision resolvability for assimilating disparate sensory feedback, IEEE Trans. Robot. Autom. 12, 714–731 (1996)

    Article  Google Scholar 

  43. R. Sharma, S. Hutchinson: Motion perceptibility and its application to active vision-based servo control, IEEE Trans. Robot. Autom. 13, 607–617 (1997)

    Article  Google Scholar 

  44. E. Marchand, F. Chaumette, A. Rizzo: Using the task function approach to avoid robot joint limits and kinematic singularities in visual servoing, IEEE/RSJ Int. Conf. Intell. Robots Syst. (Osaka 1996) pp. 1083–1090

    Google Scholar 

  45. E. Marchand, G. Hager: Dynamic sensor planning in visual servoing, IEEE Int. Conf. Robot. Autom. (Leuven) (1998) pp. 1988–1993

    Google Scholar 

  46. N. Cowan, J. Weingarten, D. Koditschek: Visual servoing via navigation functions, IEEE Trans. Robot. Autom. 18, 521–533 (2002)

    Article  Google Scholar 

  47. N. Gans, S. Hutchinson: An asymptotically stable switched system visual controller for eye in hand robots, IEEE/RSJ Int. Conf. Intell. Robots Syst. (Las Vegas 2003) pp. 735–742

    Google Scholar 

  48. G. Chesi, K. Hashimoto, D. Prattichizio, A. Vicino: Keeping features in the field of view in eye-in-hand visual servoing: a switching approach, IEEE Trans. Robot. Autom. 20, 908–913 (2004)

    Google Scholar 

  49. K. Hosoda, K. Sakamato, M. Asada: Trajectory generation for obstacle avoidance of uncalibrated stereo visual servoing without 3-D reconstruction, IEEE/RSJ Int. Conf. Intell. Robots Syst., Vol. 3 (Pittsburgh 1995) pp. 29–34

    Google Scholar 

  50. Y. Mezouar, F. Chaumette: Path planning for robust image-based control, IEEE Trans. Robot. Autom. 18, 534–549 (2002)

    Article  Google Scholar 

  51. R. Basri, E. Rivlin, I. Shimshoni: Visual homing: Surfing on the epipoles, Int. J. Comput. Vision 33, 117–137 (1999)

    Article  Google Scholar 

  52. E. Malis, F. Chaumette, S. Boudet: 2 1/2 D visual servoing with respect to unknown objects through a new estimation scheme of camera displacement, Int. J. Comput. Vision 37, 79–97 (2000)

    Article  MATH  Google Scholar 

  53. O. Faugeras: Three-Dimensional Computer Vision: a Geometric Viewpoint (MIT Press, Cambridge 1993)

    Google Scholar 

  54. P. Corke, M. Goods: Controller design for high performance visual servoing, 12th World Congress IFACʼ93 (Sydney 1993) pp. 395–398

    Google Scholar 

  55. F. Bensalah, F. Chaumette: Compensation of abrupt motion changes in target tracking by visual servoing, IEEE/RSJ Int. Conf. Intell. Robots Syst. (Pittsburgh 1995) pp. 181–187

    Google Scholar 

  56. P. Allen, B. Yoshimi, A. Timcenko, P. Michelman: Automated tracking and grasping of a moving object with a robotic hand-eye system, IEEE Trans. Robot. Autom. 9, 152–165 (1993)

    Article  Google Scholar 

  57. K. Hashimoto, H. Kimura: Visual servoing with non linear observer, IEEE Int. Conf. Robot. Autom. (Nagoya 1995) pp. 484–489

    Google Scholar 

  58. A. Rizzi, D. Koditschek: An active visual estimator for dexterous manipulation, IEEE Trans. Robot. Autom. 12, 697–713 (1996)

    Article  Google Scholar 

  59. R. Ginhoux, J. Gangloff, M. de Mathelin, L. Soler, M.A. Sanchez, J. Marescaux: Active filtering of physiological motion in robotized surgery using predictive control, IEEE Trans. Robot. 21, 67–79 (2005)

    Article  Google Scholar 

  60. J. Gangloff, M. de Mathelin: Visual servoing of a 6 dof manipulator for unknown 3-D profile following, IEEE Trans. Robot. Autom. 18, 511–520 (2002)

    Article  Google Scholar 

  61. R. Tsai, R. Lenz: A new technique for fully autonomous efficient 3-D robotics hand-eye calibration, IEEE Trans. Robot. Autom. 5, 345–358 (1989)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Campus de Beaulieu, INRIA/IRISA, 35042, Rennes, France

    François Chaumette PhD

  2. Department of Electrical and Computer Engineering, University of Illinois, 61801, Urbana, IL, USA

    Seth Hutchinson Prof

Authors
  1. François Chaumette PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seth Hutchinson Prof
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to François Chaumette PhD or Seth Hutchinson Prof .

Editor information

Editors and Affiliations

  1. PRISMA Lab, Dipartimento di Informatica e Sistemistica, Universitá degli Studi di Napoli Federico II, 80125, Napoli, Italy, Via Claudio 21

    Bruno Siciliano Prof.

  2. Artificial Intelligence Laboratory, Department of Computer Science, Stanford University, 94305-9010, Stanford, CA, USA

    Oussama Khatib Prof.

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag

About this entry

Cite this entry

Chaumette, F., Hutchinson, S. (2008). Visual Servoing and Visual Tracking. In: Siciliano, B., Khatib, O. (eds) Springer Handbook of Robotics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30301-5_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-30301-5_25

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-23957-4

  • Online ISBN: 978-3-540-30301-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation