Skip to main content

The MACS Project: An Approach to Affordance-Inspired Robot Control

  • Conference paper
Towards Affordance-Based Robot Control

Abstract

In this position paper, we present an outline of the MACS approach to affordance-inspired robot control. An affordance, a concept from Ecological Psychology, denotes a specific relationship between an animal and its environment. Perceiving an affordance means perceiving an interaction possibility that is specific for the animal’s perception and action capabilities. Perceiving an affordance does not include appearance-based object recognition, but rather feature-based perception of object functions. The central hypothesis of MACS is that an affordance-inspired control architecture enables a robot to perceive more interaction possibilities than a traditional architecture that relies on appearance-based object recognition alone. We describe how the concept of affordances can be exploited for controlling a mobile robot with manipulation capabilities. Particularly, we will describe how affordance support can be built into robot perception, how learning mechanisms can generate affordance-like relations, how this affordance-related information is represented, and how it can be used by a planner for realizing goal-directed robot behavior. We present both the MACS demonstrator and simulator, and summarize development and experiments that have been performed so far. By interfacing perception and goal-directed action in terms of affordances, we will provide a new way for reasoning and learning to connect with reactive robot control. We will show the potential of this new methodology by going beyond navigation-like tasks towards goal-directed autonomous manipulation in our project demonstrators.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton Mifflin, Boston, MA (1979), also: Lawrence Erlbaum Associates, London (1986)

    Google Scholar 

  2. Norman, D.A.: The Psychology of Everyday Things. Basic Books, New York (1988)

    Google Scholar 

  3. Chemero, A.: An outline of a theory of affordances. Ecological Psychology 15(2), 181–195 (2003)

    Article  Google Scholar 

  4. Steedman, M.: Formalizing affordance. In: Proceedings of the 24th Annual Meeting of the Cognitive Science Society. Conference Fairfax VA, August 2002, pp. 834–839. Lawrence Erlbaum, Washington D.C (2002)

    Google Scholar 

  5. Gibson, J.J.: The senses considered as perceptual systems. Houghton Mifflin, Boston (1966)

    Google Scholar 

  6. Warren, W.H.: Perceiving affordances: Visual guidance of stair climbing. Journal of Experimental Psychology 105(5), 683–703 (1984)

    Google Scholar 

  7. Warren, W.H., Whang, S.: Visual guidance of walking through apertures: body-scaled information for affordances. Journal of Experimental Psychology 13(3), 371–383 (1987)

    Google Scholar 

  8. Mark, L.S.: Eyeheight-scaled information about affordances: A study of sitting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance 13(3), 361–370 (1987)

    Article  MathSciNet  Google Scholar 

  9. Gibson, E.J., Riccio, G., Schmuckler, M.A., Stoffregen, T.A., Rosenberg, D., Taromina, J.: Detection of the traversability of surfaces by crawling and walking infants. Journal of Experimental Psychology 13(4), 533–544 (1987)

    Google Scholar 

  10. Kinsella-Shaw, J.M., Shaw, B., Turvey, M.T.: Perceiving walk-on-able slopes. Ecological Psychology 4(4), 223–239 (1992)

    Article  Google Scholar 

  11. Chemero, A.: What events are. Ecological Psychology 12(1), 37–42 (2000)

    Article  Google Scholar 

  12. Oudejans, R., Michaels, C., van Dort, B., Frissen, E.: To cross or not to cross: The effect of locomotion on street-crossing behavior. Ecological Psychology 8(3), 259–267 (1996)

    Article  Google Scholar 

  13. Arkin, R.C.: Behavior Based Robotics. MIT Press, Cambridge, MA (1998)

    Google Scholar 

  14. Duchon, A.P., Warren, W.H., Kaelbling, L.P.: Ecological robotics. Adaptive Behavior 6(3), 473–507 (1998)

    Article  Google Scholar 

  15. Brooks, R.A.: Intelligence without representation. Artificial Intelligence 47, 139–159 (1991)

    Article  Google Scholar 

  16. Brooks, R.A.: A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation 2(1), 14–23 (1986)

    Google Scholar 

  17. Lyons, D., Arbib, M.: A formal model of computation for sensory-based robotics. IEEE Transactions on Robotics and Automation 5(3), 280–293 (1989)

    Article  Google Scholar 

  18. Arkin, R.C., Balch, T.: AuRA: Principles and practice in review. Journal of Experimental and Theoretical Artificial Intelligence 9(2), 175–189 (1997)

    Article  Google Scholar 

  19. Murphy, R.R.: Case studies of applying Gibson’s ecological approach to mobile robots. IEEE Transactions on Systems, Man, and Cybernetics 29(1), 105–111 (1999)

    Article  Google Scholar 

  20. Norman, J.: Ecological psychology and the two visual systems: Not to worry! Ecological psychology 13(2), 135–145 (2001)

    Article  MathSciNet  Google Scholar 

  21. Gibson, E.J.: Perceptual learning in development: Some basic concepts. Ecological Psychology 12(4), 295–302 (2000)

    Article  Google Scholar 

  22. Neisser, U.: Multiple systems: A new approach to cognitive theory. The European Journal of Cognitive Psychology 6, 225–241 (1994)

    Article  Google Scholar 

  23. MacDorman, K.F.: Responding to affordances: Learning and projecting a sensorimotor mapping. In: Proc. of 2000 IEEE Int. Conf. on Robotics and Automation, San Fransisco, California, USA, pp. 3253–3259 (2000)

    Google Scholar 

  24. Connell, J.H.: SSS: a hybrid architecture applied to robot navigation. In: ICRA 1992. Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Nice, France, May 12–14, 1992, vol. 3, pp. 2719–2724. IEEE Computer Society Press, Los Alamitos, CA (1992)

    Google Scholar 

  25. Cooper, R., Glasspool, D.W.: Learning action affordances and action schemas. In: French, R.M., Sougne, J.P. (eds.) Connectionist Models of Learning, Development and Evolution. Sixth Neural Computation and Psychology Workshop, London. Perspectives in Neural Computing, pp. 133–142. Springer, Heidelberg (2001)

    Google Scholar 

  26. Cos-Aguilera, I., Canamero, L., Hayes, G.M.: Using a SOFM to learn object affordances. In: Proceedings of the 5th Workshop of Physical Agents, Girona, Catalonia, Spain (March 2004)

    Google Scholar 

  27. Fitzgerald, P., Metta, G., Natale, L., Rao, S., Sandini, G.: Learning about objects through action – initial steps towards artificial cognition. In: ICRA. Proceedings of the 2003 IEEE International Conference on Robotics and Automation, pp. 3140–3145 (2003)

    Google Scholar 

  28. Stoytchev, A.: Toward learning the binding affordances of objects: A behavior-grounded approach. In: Proceedings of AAAI Symposium on Developmental Robotics, Stanford University (March 2005)

    Google Scholar 

  29. Cos-Aguilera, I., Canamero, L., Hayes, G.M.: Motivation-driven learning of object affordances: First experiments using a simulated khepera robot. In: ICCM 2003. Proceedings of the 9th International Conference in Cognitive Modelling, Bamberg, Germany, pp. 57–62 (April 2003)

    Google Scholar 

  30. Stoytchev, A.: Behavior-grounded representation of tool affordances. In: ICRA. Proceedings of IEEE International Conference on Robotics and Automation, Barcelona, Spain, April 18–22, pp. 3071–3076 (2005)

    Google Scholar 

  31. Murphy, R.R., Arkin, R.C.: SFX: An architecture for action-oriented sensor fusion. In: IROS 1992. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Raleigh, NC, pp. 1079–1086 (July 1992)

    Google Scholar 

  32. Rome, E., Şahin, E., Breithaupt, R., Irran, J., Kintzler, F., Paletta, L., Çakmak, M., Uğur, E., Üçoluk, G., Doğar, M.R., Rudol, P., Fritz, G., Dorffner, G., Doherty, P., Wzoreck, M., Surmann, H., Lörken, C.: Evaluation of existing control architectures for using affordances. Technical Report MACS/2/2.2 v1, Fraunhofer Institut für Intelligente Analyse- und Informationssysteme (IAIS), Sankt Augustin, Germany (2006)

    Google Scholar 

  33. Doherty, P., Merz, T., Rudol, P., Wzorek, M.: Tentative proposal for a formal theory of affordances; Tentative proposal for an affordance support architecture; Prototype: Affordance-based motion planner. Technical Report MACS/4/2.1 v1, Linköpings Universitet, IDA Group, Linköping, Sweden (2005)

    Google Scholar 

  34. Turvey, M.: Affordances and prospective control: An outline of the ontology. Ecological Psychology 4, 173–187 (1992)

    Article  Google Scholar 

  35. Stoffregen, T.A.: Affordances are enough: Reply to chemero et al. Ecological Psychology 15(1), 29–36 (2003)

    Article  Google Scholar 

  36. Steedman, M.: Plans, affordances, and combinatory grammar. Linguistics and Philosophy 25(5–6), 723–753 (2002)

    Article  Google Scholar 

  37. Şahin, E., Çakmak, M., Doğar, M.R., Uğur, E., Üçoluk, G.: To afford or not to afford: A new formalization of affordances towards affordance-based robot control. Adaptive Behavior 15(4), 447–472 (2007)

    Article  Google Scholar 

  38. Lörken., C.: Introducing affordances into robot task execution. In: Kühnberger, K.-U., König, P., Ludewig, P. (eds.) Publications of the Institute of Cognitive Science (PICS), vol. 2, University of Osnabrück, Osnabrück, Germany, (May 2007) ISSN 1610-5389.

    Google Scholar 

  39. MacDorman, K.F.: Grounding symbols through sensorimotor integration. Journal of the RSJ (The Robotics Society of Japan) 17(1), 5 (1999)

    Google Scholar 

  40. McDermott, D.: PDDL – The planning domain definition language. Technical report, Yale University (1998)

    Google Scholar 

  41. Lörken, C., Hertzberg, J.: A specification for a propositional planner and its interface to the MACS execution control module. Deliverable MACS/2/3.2, University of Osnabrück, Institute of Computer Science, Osnabrück, Germany (2007)

    Google Scholar 

  42. Neisser, U.: Cognition and Reality: Principles and Implications of Cognitive Psychology. W.H. Freeman and Co., New York (1976)

    Google Scholar 

  43. Faillenot, I., Toni, I., Decety, J., Grégoire, M.-C., Jeannerod, M.: Visual pathways for object-oriented action and object recognition: functional anatomy with pet. Cerebral Cortex 7(9), 77–85 (1997)

    Article  Google Scholar 

  44. Fagg, A.H., Arbib, M.A.: Modeling parietal–premotor interactions in primate control of grasping. Neural Networks 11(7-8), 1277–1303 (1998)

    Article  Google Scholar 

  45. Edwards, M.G., Humphreys, G.W., Castiello, U.: Motor facilitation following action observation: a behavioural study in prehensile action. Brain Cognition 53, 495–502 (2003)

    Article  Google Scholar 

  46. Wheeler, D.S., Fagg, A.H., Grupen, R.A.: Learning prospective pick and place behavior. In: Proc. 2nd International Conference on Development and Learning, Cambridge, MA, June 2002, pp. 197–202. IEEE Computer Society Press, Los Alamitos (2002)

    Chapter  Google Scholar 

  47. Stark, L., Bowyer, K.W.: Function-based recognition for multiple object categories. Image Understanding 59(10), 1–21 (1994)

    Google Scholar 

  48. Rivlin, E., Dickinson, S.J., Rosenfeld, A.: Recognition by functional parts. Computer Vision and Image Understanding: CVIU 62(2), 164–176 (1995)

    Article  MATH  Google Scholar 

  49. Bogoni, L., Bajcsy, R.: Interactive recognition and representation of functionality. Computer Vision and Image Understanding: CVIU 62(2), 194–214 (1995)

    Article  MATH  Google Scholar 

  50. Paletta, L., Fritz, G., Rome, E., Dorffner, G.: A computational model for visual learning of affordance-like cues. In: ECVP 2006. Proc. 29th European Conference on Visual Perception, St. Petersburg, Russia (August 2006)

    Google Scholar 

  51. Fritz, G., Paletta, L., Breithaupt, R., Rome, E., Dorffner, G.: Learning predictive features in affordance-based robotic systems. In: IROS 2006. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, pp. 3642–3647. Springer, Heidelberg (October 2006)

    Google Scholar 

  52. Fritz, G., Paletta, L., Kumar, M., Dorffner, G., Breithaupt, R., Rome, E.: Visual learning of affordance based cues. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 52–64. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  53. Fritz, G., Paletta, L.: Reinforcement learning for the selection of predictive cues in affordance-based perception. In: ECVP 2006. Proc. 29th European Conference on Visual Perception, St. Petersburg, Russia (August 2006)

    Google Scholar 

  54. Frintrop, S.: VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search. LNCS (LNAI), vol. 3899. Springer, Heidelberg (2006) (also: PhD thesis, University of Bonn)

    Google Scholar 

  55. Frintrop, S., Hülse, M., Rome, E., Paletta, L.: Saliency detection with visual attention. Technical Report MACS/3/1.3 v1, Fraunhofer Institut für Autonome Intelligente Systeme, Sankt Augustin, Germany (2005)

    Google Scholar 

  56. May, S., Klodt, M., Rome, E., Breithaupt, R.: Gpu-accelerated affordance cueing based on visual attention. In: IROS 2007. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA (2007)

    Google Scholar 

  57. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110 (2004)

    Article  Google Scholar 

  58. Quinlan, J.R.: C4.5 Programs for Machine Learning, Morgan Kaufmann, San Mateo, CA (1993)

    Google Scholar 

  59. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). MIT Press, Cambridge, MA (1998)

    Google Scholar 

  60. Whitehead, S.D., Ballard, D.H.: Learning to perceive and act by trial and error. Machine Learning 7(1), 45–83 (1991)

    Google Scholar 

  61. Puterman, M.L.: Markov Decision Processes. John Wiley & Sons, New York (1994)

    MATH  Google Scholar 

  62. Paletta, L., Fritz, G.: Reinforcement learning of predictive features in affordance perception. In: Rome, E., Hertzberg, J., Dorffner, G. (eds.) Towards Affordance-based Robot Control – Proceedings of Dagstuhl Seminar 06231. LNCS (LNAI), vol. 4760, Springer, Heidelberg (February 2008)

    Google Scholar 

  63. Paletta, L., Fritz, G., Kintzler, F., Irran, J., Dorffner, G.: Learning to perceive affordances in a framework of developmental embodied cognition. In: ICDL 07. Proc. 6th International Conference on Development and Learning, London, UK (July 2007)

    Google Scholar 

  64. Janowicz, K.: Extending semantic similarity measurement by thematic roles. In: Rodríguez, M.A., Cruz, I., Levashkin, S., Egenhofer, M.J. (eds.) GeoS 2005. LNCS, vol. 3799, Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  65. Breithaupt, R., Frintrop, S., Hertzberg, J., Rome, E., Müller, B.S.: Specification of final demonstrator. Technical Report MACS/6/1.1 v2, Fraunhofer Institut für Autonome Intelligente Systeme, Sankt Augustin, Germany (2004)

    Google Scholar 

  66. Uğur, E., Doğar, M.R., Çakmak, M., Şahin, E.: The learning and use of traversability affordance using range images on a mobile robot. In: ICRA 2007. Proc. of the IEEE International Conference on Robotics and Automation, Conference: Rome, Italy, April 10–14, pp. 1721–1726. IEEE, Los Alamitos (2007)

    Google Scholar 

  67. Uğur, E., Doğar, M.R., Çakmak, M., Şahin, E.: The curiosity-driven learning of traversability affordance on a mobile robot. In: ICDL 2007. Proc. of the IEEE International Conference on Development and Learning, Conference: London, UK (July 11–13, 2007)

    Google Scholar 

  68. Breithaupt, R., Frintrop, S., Şahin, E., Hertzberg, J., Pölz, P., Rudol, P., Uğur, E., Doherty, P., Rome, E., Müller, B.S.: Report on experiment design. Technical Report MACS/6/4.1 v1, Fraunhofer Institut für Autonome Intelligente Systeme, Sankt Augustin, Germany (2004)

    Google Scholar 

  69. Paletta, L., Fritz, G., Kumar, M., Hertzberg, J., Schönherr, F.: Top-down and bottom-up symbol grounding. Technical Report MACS/3/1.1 v3, Joanneum Research Institute of Digital Image Processing Computational Perception (CAPE), Graz, Austria (2005)

    Google Scholar 

  70. Paletta, L., Fritz, G., Şahin, E., Kumar, M.: Affordance recognition from visual cues. Technical Report MACS/3/1.2 v1, Joanneum Research Institute of Digital Image Processing Computational Perception (CAPE), Graz, Austria (2005)

    Google Scholar 

  71. Paletta, L., Fritz, G., Rome, E., Frintrop, S., Hertzberg, J., Kumar, M.: Prototypical affordance based object detection for MACS scenario. Technical Report MACS/3/1.4 v1, Joanneum Research Institute of Digital Image Processing Computational Perception (CAPE), Graz, Austria (2005)

    Google Scholar 

  72. Rome, E., Paletta, L., Fritz, G., Surmann, H., May, S., Lörken, C.: Multi-sensor affordance recognition. Technical Report MACS/3/2.1 v2, Fraunhofer Institut für Autonome Intelligente Systeme, Sankt Augustin, Germany (2006)

    Google Scholar 

  73. Holz, D., Lörken., C.: Continuous 3d environment sensing for autonomous robot navigation and mapping. In: Proc. of the 9. Fachwissenschaftlicher Informatik-Kongress, Bonn, Germany, Lecture Notes in Informatics (LNI), pp.39–42 Gesellschaft für Informatik (GI) (March 2007)

    Google Scholar 

  74. Uğur, E.: Direct perception of traversability affordance on range images through learning on a mobile robot. M.Sc. thesis, Middle East Technical University, Kovan Laboratory, Ankara, Turkey (2006)

    Google Scholar 

  75. Doğar, M.R., Çakmak, M., Uğur, E., Şahin, E.: From primitive behaviors to goal-directed behavior using affordances. Technical Report METU-CENG-TR-2007-02, Middle East Technical University, Kovan Laboratory, Ankara, Turkey, Short version published for ICDL (2007)

    Google Scholar 

  76. Uğur, E., Doğar, M.R., Çakmak, M., Şahin, E.: From primitive behaviors to goal-directed behavior using affordances. In: IROS 2007. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA (2007)

    Google Scholar 

  77. Uğur, E., Doğar, M.R., Çakmak, M., Şahin, E.: Report on experimental results in simulator. Technical Report MACS/6/4.2 v1, Middle East Technical University Dept. of Computer Engineering, Ankara, Turkey (2007)

    Google Scholar 

  78. Doğar, M.R., Çakmak, M., Uğur, E., Şahin, E.: Report on experimental results in demonstrator. Technical Report MACS/6/4.3 v1, Middle East Technical University Dept. of Computer Engineering, Ankara, Turkey (2007)

    Google Scholar 

  79. Çakmak, M., Doğar, M.R., Uğur, E., Şahin, E.: Affordances as a framework for robot control. In: EpiRob 2007. Proc. of the International Conference on Epigenetic Robotics, Conference Piscataway, NJ, USA (November 5–7, 2007)

    Google Scholar 

  80. Çakmak, M.: Robot planning based on learned affordances. M.Sc. thesis, Middle East Technical University, Kovan Laboratory, Ankara, Turkey (2007)

    Google Scholar 

  81. Raubal, M.: Agent-based Simulation of Human Wayfinding: A Perceptual Model for Unfamiliar Buildings. PhD thesis, Institute for Geoinformation, Vienna University of Technology, Vienna, Austria (2001)

    Google Scholar 

  82. Schönherr, F.: Verankerung der Semantik veränderlicher Situations-Fakten und symbolischer Aktionen in der hybriden Roboterkontrollarchitekur DD&P, Fraunhofer series in information and communication technology 2004, 8. Shaker, Aachen (2004)

    Google Scholar 

  83. Neisser, U.: Cognitive Psychology. Prentice-Hall, Englewood Cliffs, NJ (1967)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Erich Rome Joachim Hertzberg Georg Dorffner

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rome, E. et al. (2008). The MACS Project: An Approach to Affordance-Inspired Robot Control. In: Rome, E., Hertzberg, J., Dorffner, G. (eds) Towards Affordance-Based Robot Control. Lecture Notes in Computer Science(), vol 4760. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77915-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-77915-5_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-77914-8

  • Online ISBN: 978-3-540-77915-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics