Object Recognition and Visual Servoing: Two Case Studies of Employing Fuzzy Techniques in Robot Vision

  • Alois Knoll
  • Jianwei Zhang
  • Thorsten Graf
  • André Wolfram
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 52)


The capabilities of observing the world and manipulating objects based on visual information are basic requirements in robotic applications. Typically, for manipulating objects in real environments it is necessary to recognise and locate the substantial objects robustly. Due to external influences, e.g. partial occlusions of objects and illumination changes, as well as to internal influences, e.g. noisy imaging hardware, inaccurate measurements and quantization effects, the recognition systems have to cope with incomplete, uncertain and inaccurate information.

In the first part of this chapter we present the general framework of a robust approach for recognising partially occluded objects. It combines the popular concept of using invariant shape descriptions, i.e. descriptions of objects which remain unaffected by certain variations of the intrinsic and extrinsic camera parameters, with the flexibility and readability of rule-based fuzzy systems by applying invariant object shape descriptions in fuzzy if-then classification rules.

In the second part, we propose a fuzzy control approach for learning fine-positioning of parallel-jaw robot gripper using visual sensor data. The first component of the used model can be viewed as a perceptron network that projects high-dimensional input data into a low-dimensional eigenspace. The second component is a fuzzy controller serving as an interpolator whose input space is the eigenspace and whose outputs are the motion parameters. Instead of undergoing cumbersome hand-eye calibration processes, our system is trained in a supervised learning procedure using systematical perturbation motion around the optimal grasping pose.


Object Recognition Fuzzy Rule Training Image Fuzzy Controller Correction Angle 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Albus J.S.,A New Approach to Manipulator Control: The Cerebellar Model Articulation Contorller (CMAC)Transactions of ASME, Journal of Dynamic Systems Measurement and Control, Vol. 97, pp. 220–227, 1975 Google Scholar
  2. 2.
    Binford T.O. and Levitt T.S., Quasi-Invariants: Theory and Experiments,in: “Proceedings of ARPA Image Understanding Workshop” (Washington, DC, USA), pp. 819–829, 1994Google Scholar
  3. 3.
    Black M.J. and Jepson A.D., Eigen-Tracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation,in: “Proceedings of the European Conference on Computer Vision” (Cambridge, Great Britain), pp. 329–342, 1996Google Scholar
  4. 4.
    Blöchl B.,Fuzzy Control in Real-time for Vision Guided Autonomous Mobile Robotsin: “Fuzzy Logic in Artificial Intelligence — Proceedings of the Austrian Artificial Intelligence Conference (Klement E.P. and Slany W., eds.)” (Linz, Austria), pp. 114–125, 1993 Google Scholar
  5. 5.
    Canny J.F., A Computational Approach to Edge Detection,IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 8, No. 6, pp. 679–698, 1986Google Scholar
  6. 6.
    Chen Y.H.,Computer Vision for General Purpose Visual Inspection: A Fuzzy Logic ApproachOptics and Lasers in Engineering, Vol. 22, pp. 181–192, 1995 Google Scholar
  7. 7.
    Forsyth D.A., Mundy J.L., Zisserman A.P., Coelho C., Heller A. and Roth-well C.A., Invariant Descriptors for 3-D Object Recognition and Pose, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 10, pp. 971–991, 1991CrossRefGoogle Scholar
  8. 8.
    Farreny H. and Prade H.,On the Problem of Identifying an Object in a Robotics Scene from a Verbal Imprecise Descriptionin: “Advanced Software in Robotics (Danthine A. and Géradin M., eds.)”, Elsevier Science, pp. 343–351, 1984 Google Scholar
  9. 9.
    Graf T., Knoll A. and Wolfram A.,Recognition of Partially Occluded Objects through Fuzzy Invariant Indexingin: “Proceedings of the IEEE International Conference on Fuzzy Systems IEEE World Congress on Computational Intelligence” (Anchorage, Alaska, USA), Vol. 2, pp. 1566–1571, 1998 Google Scholar
  10. 10.
    Graf T., Knoll A. and Wolfram A.,Fuzzy Invariant Indexing: A General Indexing Scheme for Occluded Object Recognitionin: “Proceedings of the International Conference on Signal Processing” (Beijing, China), pp. 908–911, 1998 Google Scholar
  11. 11.
    Keller J.M., Fuzzy set theory in computer vision: A prospectus,Fuzzy Sets and Systems, Vol. 90, No. 2, pp. 177–182, 1997Google Scholar
  12. 12.
    Kamon I., Flash T. and Edelman S., Learning to Grasp Using Visual Information, in: “Proceedings of the IEEE International Conference on Robotics and Automation”, pp. 2470–2476, 1996Google Scholar
  13. 13.
    Knoll A., Hildebrandt B. and Zhang J.,Instructing Cooperating Assembly Robots through Situated Dialogues in Natural Languagein: “Proceedings of the IEEE International Conference on Robotics and Automation”, (Albuquerque, NM), pp. 888–894, 1997 Google Scholar
  14. 14.
    Kosako A. and Ralescu A.L.,Feature Based Parametric Eigenspace Method for Object Extractionin: “Proceedings of the IEEE International Conference on Fuzzy Systems” (Yokohama, Japan), pp. 1273–1278, 1995 Google Scholar
  15. 15.
    Lee K.-J. and Bien Z., A Model-Based Machine Vision System Using Fuzzy LogicInternational Journal of Approximate Reasoning, Vol. 16, pp. 119–135, 1997 Google Scholar
  16. 16.
    Lamdan Y., Schwartz J.T. and Wolfson H.J.,Object Recognition by Affine Invariant Matchingin: “Proceedings of Computer Vision and Pattern Recognition”, pp. 335–344, 1988 Google Scholar
  17. 17.
    Li W., Jiang X. and Wang Y.,Road Recognition for Vision Navigation of an Autonomous Vehicle by Fuzzy ReasoningFuzzy Sets and Systems, Vol. 93, pp. 275–280, 1998 Google Scholar
  18. 18.
    Mundy J.L., Huang C., Liu J., Hoffman W., Forsyth D.A., Rothwell C.A., Zisserman A., Utcke S. and Boumez O., MORSE: A 3D Object Recognition System Based on Geometric Invariants, in: “Proceedings of ARPA Image Understanding Workshop” (Monterey, California), pp. 1393–1402, 1994Google Scholar
  19. 19.
    Mundy J.L. and Zisserman A.,Introduction - Towards a New Framework for Visionin: “Geometric invariance in computer vision”, Cambridge, Mass. MIT Press, 1992 Google Scholar
  20. 20.
    Miller W.T.,Real-Time Application of Neural Networks for Sensor-Based Control of Robots with VisionIEEE Transactions on System, Man and Cybernetics, Vol. 19, pp. 825–831, 1989 Google Scholar
  21. 21.
    Mundy J.L. and Zisserman A., “Geometric invariance in computer vision”, Cambridge, Mass. MIT Press, 1992Google Scholar
  22. 22.
    Nayar S.K., Murase H. and Nene S.A.,Learning Positioning and Tracking Visual Appearancein: “Proceedings of the IEEE International Conference on Robotics and Automation”, pp. 3237–3244, 1994 Google Scholar
  23. 23.
    Pal S.K.,Uncertainty Management in Space Station Autonomous Research: Pattern Recognition PerspectiveInformation Sciences, Vol. 72, pp. 1–63, 1993 Google Scholar
  24. 24.
    R.othwell C.A., “Object recognition through invariant indexing”,Oxford University Press, 1995Google Scholar
  25. 25.
    Ray K.S. and Ghoshal J., Neuro-Fuzzy Reasoning for Occluded Object Recognition,Fuzzy Sets and Systems, Vol. 94, pp. 1–28, 1998Google Scholar
  26. 26.
    Sanger T., An Optimality Principle for Unsupervised Learning, in: “Advances in Neural Information Processing Systems”, Morgan Kaufmann, San Mateo, CA, 1989Google Scholar
  27. 27.
    Walker E.L., Perspectives on Fuzzy Systems in Computer Vision,in: “Proceedings of the Annual Conference of the North American Fuzzy Information Processing Society”, pp. 296–300, 1998Google Scholar
  28. 28.
    Wei G.-Q., Hirzinger G. and Brunner B, Sensorimotion Coordination and Sensor Fusion by Nneural Networks, in: “Proceedings of IEEE International Conference on Neural Networks” (San Francisco, USA), pp. 150–155, 1993Google Scholar
  29. 29.
    Zadeh L.A., Fuzzy sets, Information and Control, Vol. 8, pp. 338–353, 1965MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Zhang J. and Knoll A., Constructing Fuzzy Controllers with B-Spline Models - Principles and Applications International Journal of Intelligent Systems, Vol. 13, pp. 257–285, 1997Google Scholar
  31. 31.
    Zhang J., Schmidt R. and Knoll A., Appearance-Based Visual Learning in a Neuro-Fuzzy Model for Fine-Positioning of Manipulators, in: “Proceedings of the IEEE International Conference on Robotics and Automation” ( Detroit, USA ), 1999Google Scholar
  32. 32.
    Zhang J. and Knoll A., Situated Neuro-Fuzzy Control for Vision-Based Robot Localisation, Journal of Robotics and Autonomous Systems, Elsevier Science, Vol. 28, pp. 71–82, 1999MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Alois Knoll
    • 1
  • Jianwei Zhang
    • 1
  • Thorsten Graf
    • 1
  • André Wolfram
    • 1
  1. 1.Faculty of TechnologyUniversity of BielefeldBielefeldGermany

Personalised recommendations