Skip to main content
SpringerLink
Log in
Book cover

The confluence of vision and control pp 24–40Cite as

Role of active vision in optimizing visual feedback for robot control

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 237))

Abstract

A purposeful change of camera parameters or “active vision” can be used to improve the process of extracting visual information. Thus if a robot visual servo loop incorporates active vision, it can lead to a better performance while increasing the scope of the control tasks. Although significant advances have been made in this direction, much of the potential improvement is still unrealized. This chapter discusses the advantages of using active vision for visual servoing. It reviews some of the past research in active vision relevant to visual servoing, with the aim of improving: (1) the measurement of image parameters, (2) the process of interpreting the image parameters in terms of the corresponding world parameters, and (3) the control of a robot in terms of the visual information extracted.

This is a preview of subscription content, log in via an institution.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. L. Abbott. A survey of selective fixation control for machine vision. IEEE Control Systems, 12(4):25–31, 1992.

    Article  MathSciNet  Google Scholar 

  2. A. L. Abbott and N. Ahuja. Surface reconstruction by dynamic integration of focus, camera vergence, and stereo. In Proc. IEEE International Conference on Computer Vision, pages 532–543, 1988.

    Google Scholar 

  3. S. Abrams, P. K. Allen, and K. A. Tarabanis. Dynamic sensor planning. In Proc. IEEE International Conference on Robotics and Automation, pages 605–610, 1993.

    Google Scholar 

  4. N. Ahuja and A. L. Abbott. Active stereo: Integrating disparity, vergence, focus, aperture, and calibration for surface estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(10):1007–1029, 1993.

    Article  Google Scholar 

  5. P. K. Allen, A. Timcenko, B. Yoshimi, and P. Michelman. Trajectory filtering and prediction for automatic tracking and grasping of a moving object. In Proc. IEEE International Conference on Robotics and Automation, pages 1850–1857, 1992.

    Google Scholar 

  6. J. Aloimonos and D. Tsakiris. On the visual mathematics of tracking. Image and Vision Computing, 9:235–251, 1991.

    Article  Google Scholar 

  7. J. Aloimonos, I. Weiss, and A. Bandyopadhyay. Active vision. International Journal of Computer Vision, 1:333–356, 1988.

    Article  Google Scholar 

  8. R. Bajcsy. Active perception. Proceedings of the IEEE, 78:996–1005, 1988.

    Google Scholar 

  9. D. H. Ballard and C. M. Brown. Principles of animate vision. CVGIP: Image Understanding, 56:3–21, 1992.

    Article  MATH  Google Scholar 

  10. D. H. Ballard and A. Ozcandarli. Eye fixation and early vision: Kinetic depth. In Proc. IEEE International Conference on Computer Vision, pages 524–531, 1988.

    Google Scholar 

  11. A. Bandopadhay and D. H. Ballard. Egomotion perception using visual tracking. Computational Intelligence, 7:39–47, 1991.

    Article  Google Scholar 

  12. A. Blake and A. Yuille. Active Vision. MIT Press, Cambridge, MA, 1992.

    Google Scholar 

  13. M. E. Bowman and A. K. Forrest. Visual detection of differential movement: Applications to robotics. Robotica, 6:7–12, 1988.

    Google Scholar 

  14. A. Cameron and H. Durrant-Whyte. A bayesian approach to optimal sensor placement. International Journal of Robotics Research, 9(5):70–88, 1990.

    Article  Google Scholar 

  15. R. Cipolla and A. Blake. The dynamic analysis of apparent contours. In Proc. IEEE International Conference on Computer Vision, pages 616–623, 1990.

    Google Scholar 

  16. J. J. Clark and N. J. Ferrier. Modal control of an attentive vision system. In Proc. IEEE International Conference on Computer Vision, pages 514–519, 1988.

    Google Scholar 

  17. J. J. Clark and N. J. Ferrier. Attentive visual servoing. In A. Blake and A. Yuille, editors, Active Vision, pages 137–154. MIT Press, Cambridge, MA, 1992.

    Google Scholar 

  18. W. F. Clocksin, J. S. E. Bromley, P. G. Davey, A. R. Vidler, and C. G. Morgan. An implementation of model-based visual feedback for robot arc welding of thin sheet steel. International Journal of Robotics Research, 4(1):13–26, 1985.

    Article  Google Scholar 

  19. H. Collewijn and E. Tamminga. Human smooth and saccadic eye movements during voluntary pursuit of different target motions on different backgrounds. Journal of Physiology, 351:217–250, 1984.

    Google Scholar 

  20. D. J. Coombs and C. M. Brown. Cooperative gaze holding in binocular vision. IEEE Control Systems, 11(4):24–33, 1991.

    Article  Google Scholar 

  21. P. I. Corke. Visual control of robot manipulators—a review. In K. Hashimoto, editor, Visual Servoing, pages 1–32. World Scientific, 1993.

    Google Scholar 

  22. C. G. Cowan. Automatic sensor placement from vision task requirements. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10:407–416, 1988.

    Article  Google Scholar 

  23. R. Curwen and A. Blake. Dynamic contours: Real-time active splines. In A. Blake and A. Yuille, editors, Active Vision, pages 39–58. MIT Press, Cambridge, MA, 1992.

    Google Scholar 

  24. S. Das and N. Ahuja. A comparative study of stereo, vergence, and focus as depth cues for active vision. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pages 194–199, 1993.

    Google Scholar 

  25. E. D. Dickmanns and W. Graefe. Applications of dynamic monocular machine vision. Machine Vision and Applications, 1:241–261, 1988.

    Article  Google Scholar 

  26. E. D. Dickmanns, B. Mysliwetz, and T. Christains. An integrated spatiotemporal approach to automatic visual guidance of autonomous vehicles. IEEE Transactions on Systems, Man, and Cybernetics, 20:1273–1284, 1990.

    Article  Google Scholar 

  27. B. Espiau, F. Chaumette, and P. Rives. A new approach to visual servoing in robotics. IEEE Transactions on Robotics and Automation, 8:313–326, 1992.

    Article  Google Scholar 

  28. B. B. et al. Qualitative target motion detection and tracking. In Proc. DARPA Image Understanding Workshop, pages 370–398, 1989.

    Google Scholar 

  29. J. T. Feddema, C. S. G. Lee, and O. R. Mitchell. Weighted selection of image features for resolved rate visual feedback control. IEEE Transactions on Robotics and Automation, 7:31–47, 1991.

    Article  Google Scholar 

  30. J. T. Feddema and O. R. Mitchell. Vision-guided servoing with feature-based trajectory generation. IEEE Transactions on Robotics and Automation, 5:691–700, 1989.

    Article  Google Scholar 

  31. C. Fermuller and Y. Aloimonos. Tracking facilitates 3-d motion estimation. Biological Cybernetics, 67:259–268, 1992.

    Article  Google Scholar 

  32. G. D. Hager. Task Directed Sensor Fusion and Planning. Kluwer Academic Publishers, 1990.

    Google Scholar 

  33. K. Hashimoto, T. Kimoto, T. Ebine, and H. Kimura. Manipulator control with image-based visual servo. In Proc. IEEE International Conference on Robotics and Automation, pages 2267–2271, 1991.

    Google Scholar 

  34. S. A. Hutchinson and A. C. Kak. Planning sensing strategies in a robot work cell with multi-sensor capabilities. IEEE Journal of Robotics and Automation, 5(6):765–782, 1989.

    Article  Google Scholar 

  35. M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. International Journal of Computer Vision, 1(4):321–331, 1987.

    Article  Google Scholar 

  36. P. K. Khosla, C. P. Neuman, and F. B. Prinz. An algorithm for seam tracking applications. International Journal of Robotics Research, 4(1):27–41, 1985.

    Article  Google Scholar 

  37. A. J. Koivo and N. Houshangi. Real-time vision feedback for servoing robotic manipulator with self-tuning controller. IEEE Transactions on Systems, Man, and Cybernetics, 21:134–142, 1991.

    Article  Google Scholar 

  38. E. P. Krotkov. Active Computer Vision by Cooperative Focus and Stereo. Springer-Verlag, Berlin, 1989.

    MATH  Google Scholar 

  39. K. N. Kutulakos and C. R. Dyer. Recovering shape by purposive viewpoint adjustment. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pages 16–22, 1992.

    Google Scholar 

  40. C. Laugier, A. Ijel, and J. Troccaz. Combining vision-based information and partial geometric models in automatic grasping. In Proc. IEEE International Conference on Robotics and Automation, pages 676–682, 1990.

    Google Scholar 

  41. M. Lei and B. K. Ghosh. Visually-guided robotic motion tracking. In Proc. Thirtieth Annual Allerton Conference on Communication, Control, and Computing, pages 712–721, 1992.

    Google Scholar 

  42. R. C. Luo and R. E. M. Jr. A modified optical flow approach for robotic tracking and acquisition. Journal of Robotic Systems, 6(5):489–508, 1989.

    Article  MATH  Google Scholar 

  43. L. Matthies, T. Kanade, and R. Szeliski. Kalman filter-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, 3:209–236, 1989.

    Article  Google Scholar 

  44. P. McLeod, J. Driver, Z. Dienes, and J. Crisp. Filtering by movements in visual search. Jrn. Experimental Psychology: Human Perception and Performance, 17(1):55–64, 1991.

    Article  Google Scholar 

  45. B. Nelson and P. K. Khosla. Strategies for increasing the tracking region of an eye-in-hand system by singularity and joint limit avoidance. International Journal of Robotics Research, 14(3):255–264, 1995.

    Article  Google Scholar 

  46. N. P. Papanikolopoulos and P. K. Khosla. Adaptive robot visual tracking: Theory and experiments. IEEE Transactions on Automatic Control, 38(3):429–445, 1993.

    Article  MATH  MathSciNet  Google Scholar 

  47. N. P. Papanikolopoulos, P. K. Khosla, and T. Kanade. Visual tracking of a moving target by a camera mounted on a robot: A combination of vision and control. IEEE Transactions on Robotics and Automation, 9(1):14–35, 1993.

    Article  Google Scholar 

  48. R. D. Rimey and C. M. Brown. Controlling eye movements with hidden markov models. International Journal of Computer Vision, 7(1):47–65, 1991.

    Article  Google Scholar 

  49. A. C. Sanderson and L. E. Weiss. Adaptive visual servo control of robots. In A. Pugh, editor, Robot Vision. IFS Publications, Bedford, UK, 1983.

    Google Scholar 

  50. S. A. Shafer. Automation and calibration for robot vision systems. Technical Report CMU-CS-88-147, Carnegie Mellon University, 1988.

    Google Scholar 

  51. R. Sharma and Y. Aloimonos. Early detection of independent motion from active control of normal image flow patterns. IEEE Transactions on Systems, Man, and Cybernetics, 26(1):42–52, February 1996.

    Google Scholar 

  52. R. Sharma and S. Hutchinson. Motion perceptibility and its application to active vision-based servo control. IEEE Transactions on Robotics and Automation, 13(4):607–617, August 1997.

    Article  Google Scholar 

  53. R. Sharma and H. Sutanto. A framework for robot motion planning with sensor constraints. IEEE Transactions on Robotics and Automation, 13(1):61–73, February 1997.

    Article  Google Scholar 

  54. N. Srinivasa and R. Sharma. Execution of saccades for active vision using a neurocontroller. IEEE Control Systems, 17(2):18–29, April 1997.

    Article  Google Scholar 

  55. H. Sutanto and R. Sharma. Global performance evaluation of image features for visual servo control. Journal of Robotic Systems, 13(4):243–258, April 1996.

    Article  MATH  Google Scholar 

  56. H. Sutanto, R. Sharma, and V. K. Varma. The role of exploratory movement in visual servoing without calibration. Robotics and Autonomous Systems, 1998. (to appear).

    Google Scholar 

  57. M. A. Taalebinezhaad. Direct robot vision by fixation. In Proc. IEEE International Conference on Robotics and Automation, pages 626–631, 1991.

    Google Scholar 

  58. K. Tarabanis and R. Y. Tsai. Sensor planning for robotic vision: A review. In O. Khatib, J. J. Craig, and T. Lozano-Pérez, editors, Robotics Review 2. MIT Press, Cambridge, MA, 1992.

    Google Scholar 

  59. K. A. Tarabanis, P. K. Allen, and R. Y. Tsai. A survey of sensor planning in computer vision. IEEE Transactions on Robotics and Automation, 11:86–104, 1995.

    Article  Google Scholar 

  60. D. Terzopoulos and D. Metaxas. Dynamic 3d models with local and global deformations: Deformable superquadrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7):703–714, 1991.

    Article  Google Scholar 

  61. R. Y. Tsai. Synopsis of recent progress on camera calibration for 3d machine vision. In Robotics Review 1. MIT Press, Cambridge, MA, 1989.

    Google Scholar 

  62. R. Y. Tsai and K. Tarabanis. Occlusion-free sensor placement planning. In H. Freeman, editor, Machine Vision, pages 301–339. Academic Press, 1990.

    Google Scholar 

  63. G. Verghese, K. L. Gale, and C. R. Dyer. Real-time motion tracking of three-dimensional objects. In Proc. IEEE International Conference on Robotics and Automation, pages 1998–2003, 1990.

    Google Scholar 

  64. D. Vernon and M. Tistarelli. Using camera motion to estimate range for robotic parts manipulation. IEEE Transactions on Robotics and Automation, 6(5):509–521, 1990.

    Article  Google Scholar 

  65. L. E. Weiss, A. C. Sanderson, and C. P. Neuman. Dynamic sensor-based control of robots with visual feedback. IEEE Journal of Robotics and Automation, 3:404–417, 1987.

    Article  Google Scholar 

  66. T. Yoshikawa. Analysis and control of robot manipulators with redundancy. In Robotics Research: The First Int. Symposium, pages 735–747. MIT Press, 1983.

    Google Scholar 

  67. J. Y. Zheng, Q. Chen, and A. Tsuji. Active camera guided manipulation. In Proc. IEEE International Conference on Robotics and Automation, pages 632–638, 1991.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, The Pennsylvania State University, 317 Pond Laboratory, 16802-6106, University Park, PA, USA

    Rajeev Sharma

Authors
  1. Rajeev Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

David J. Kriegman PhD Gregory D. Hager PhD A. Stephen Morse PhD

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag

About this paper

Cite this paper

Sharma, R. (1998). Role of active vision in optimizing visual feedback for robot control. In: Kriegman, D.J., Hager, G.D., Morse, A.S. (eds) The confluence of vision and control. Lecture Notes in Control and Information Sciences, vol 237. Springer, London. https://doi.org/10.1007/BFb0109661

Download citation

  • DOI: https://doi.org/10.1007/BFb0109661

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-85233-025-5

  • Online ISBN: 978-1-84628-528-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation