Artificial Life and Robotics

, Volume 23, Issue 1, pp 94–102 | Cite as

Behavioristic image-based pose control of mobile manipulators using an uncalibrated eye-in-hand vision system

  • Tsing-Iuan James Tsay
  • Pei-Jiun Hung
Original Article


In the execution of material handling, the mobile manipulator is controlled to reach a station by its mobile base. This study adopts an uncalibrated eye-in-hand vision system to provide visual information for the manipulator to pick up a workpiece on the station. A novel vision-guided control strategy with a behavior-based look-and-move structure is proposed. This strategy is based on six image features, predefined by image moment method. In the designed neural-fuzzy controllers with varying learning rate, each image feature error is taken to generate intuitively one DOF motion command relative to the camera coordinate frame using fuzzy rules, which define a particular visual behavior. These behaviors are then fused to produce a final command action to perform grasping tasks using the proposed behavior fusion scheme. Finally, the proposed control strategy is experimentally applied to control the end-effector to approach and grasp a workpiece in various locations on a station.


Behavior-based Behavior fusion Image moment Neural-fuzzy controller Vision-guided 



The authors would like to thank the National Science Council of the Republic of China, for financially supporting this research under Contract No. NSC 102-2221-E-006-295.


  1. 1.
    Hvilshøj M, Bøgh S, Nielsen OS et al (2012) Autonomous industrial mobile manipulation (AIMM): past, present and future. Ind Robot 39(2):120–135CrossRefGoogle Scholar
  2. 2.
    Hutchinson S, Hager GD, Corke PI (1996) A tutorial on visual servo control. IEEE Trans Robot Autom 12(5):651–670CrossRefGoogle Scholar
  3. 3.
    Wasik Z, Saffiotti A (2003) A hierarchical behavior-based approach to manipulation tasks. In: Proceedings of the IEEE international conference on robotics and automation, pp. 2780–2785Google Scholar
  4. 4.
    Wasik Z, Saffiotti A (2002) A fuzzy behavior-based control system for manipulation. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, pp. 1596–1601Google Scholar
  5. 5.
    Chaumette F (2004) Image moments: a general and useful set of features for visual servoing. IEEE Trans Robot 20(4):713–723CrossRefGoogle Scholar
  6. 6.
    Suh IH, Kim TW (1994) Fuzzy membership function based neural networks with applications to the visual servoing of robot manipulators. IEEE Trans Fuzzy Syst 2(3):203–220CrossRefGoogle Scholar
  7. 7.
    Kim TW, Yuh J (2005) Fast on-line neuro-fuzzy controller for autonomous underwater vehicles. In: Proceedings of the international offshore and polar engineering conference, pp. 289–294Google Scholar

Copyright information

© ISAROB 2017

Authors and Affiliations

  1. 1.National Cheng Kung UniversityTainanTaiwan

Personalised recommendations