Journal of Intelligent and Robotic Systems

, Volume 48, Issue 4, pp 437–456 | Cite as

A Robust Approach to Control Robot Manipulators by Fusing Visual and Force Information

  • J. Pomares
  • G. J. García
  • F. Torres


In this paper, a method to combine visual and force information using the information obtained from the movement flow-based visual servoing system is proposed. This method allows not only to achieve a given desired position but also to specify the trajectory that the robot will follow from its initial position to its final one in the 3D space. This paper also extends the visual servoing system in order to increase the robustness when errors in the camera calibration parameters appear. Experiments using an eye-in-hand robotic system demonstrate the correct behaviour when important errors exist in the camera intrinsic parameters. After the description of this strategy, we then describe its application to an insertion task to be performed by the robotic system in which the joint use of visual and force information is required. To combine both sensorial systems, a position-based impedance-control system is implemented, which modifies the trajectory generated by the visual system depending on the robot’s interaction with its setting. This modification is performed without knowledge of the exact camera calibration parameters. Furthermore, the visual-force approach based on impedance control does not require having previous knowledge about the contact geometry.

Key words

force control impedance control intrinsics-free control movement flow visual-force control 2D visual servoing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hutchinson, S., Hager, G.D., Corke, P.I.: A tutorial on visual servo control. IEEE Trans. Robot. Autom. 12(5), 651–670 (1996)CrossRefGoogle Scholar
  2. 2.
    Mezouar, Y., Chaumette, F.: Path planning for robust image-based control. IEEE Trans. Robot. Autom. 18(4), 534–549 (2002)CrossRefGoogle Scholar
  3. 3.
    Pomares, J., Torres, F.: Movement-flow based visual servoing and force control fusion for manipulation tasks in unstructured environments. IEEE Trans. Syst. Man Cybern., Part C 35(1), 4–15 (2005)CrossRefGoogle Scholar
  4. 4.
    Nandi, G., Mitra, D.: Fusion strategies for minimizing sensing-level uncertainty in manipulator control. J. Intell. Robot. Syst. 43(1), 1–32 (2005)CrossRefGoogle Scholar
  5. 5.
    Baeten, J., De Schutter, J.: Hybrid vision/force control at corners in planar robotic-contour following. IEEE/ASME Trans. Mechatron. 7(2), 143–151 (2002)CrossRefGoogle Scholar
  6. 6.
    Nelson, B.J., Khosla, P.K.: Force and vision resolvability for assimilating disparate sensory feedback. IEEE Trans. Robot. Autom. 12(5), 714–731 (1996)CrossRefGoogle Scholar
  7. 7.
    Zhou, Y., Nelson, B.J., Vidramaditya, B.: Fusing force and vision feedback for micromanipulation. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1220–1225. Leuven, Belgium (1998)Google Scholar
  8. 8.
    von Collani, Y., Scheering, C., Zhang, J., Knoll, A.: A neuro-fuzzy solution for integrated visual and force control. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2965–2970. Detroit, MI (1999)Google Scholar
  9. 9.
    Morel, G., Malis, E., Boudet, S.: Impedance based combination of visual and force control. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1743–1748. Leuven, Belgium (1998)Google Scholar
  10. 10.
    Pomares, J., Torres, F., Puente, S.T.: Disassembly movements for geometrical objects through heuristic methods. In: Proceedings of the SPIE International Symposium on Intelligent Systems and Advanced Manufacturing, vol. 4569, pp. 71–80. Boston, MA (2002)Google Scholar
  11. 11.
    Pomares, J., Puente, S.T., Torres, F., Candelas, F.A., Gil, P.: Virtual disassembly of products based on geometric models. Comput. Ind. 55(1), 1–14 (2004)CrossRefGoogle Scholar
  12. 12.
    Nelson, B.J., Morrow, J., Khosla, P.K.: Robotic manipulation using high bandwith force and vision feedback. Math. Comput. Model. 24(5/6), 11–29 (1996)CrossRefGoogle Scholar
  13. 13.
    Whitney, D.E.: Force feedback control of manipulator fine motions. J. Dyn. Syst. Meas. Control 97–97 (1977)Google Scholar
  14. 14.
    Espiau, B., Merlet, J.P., Samson, C.: Force-feedbackcontrol and non-contact sensing: A unified approach. In: Proceedings of the 8th CISM-IFTOMM Symposium on Theory and Practice of Robots Manipulators (1990)Google Scholar
  15. 15.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, pp. 91–92. Cambridge University Press, Cambridge, UK (2000)MATHGoogle Scholar
  16. 16.
    Zhang, Z.: Three-dimensional Reconstruction under Varying Constraints on Camera Geometry for Robotic Navigation Scenarios, pp. 108–140. PhD thesis. (1996)Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2007

Authors and Affiliations

  1. 1.Department of Physics, System Engineering and Signal TheoryUniversity of AlicanteAlicanteSpain

Personalised recommendations