Abstract
Artificial intelligence is essential to succeed in challenging activities that involve dynamic environments, such as object manipulation tasks in indoor scenes. Most of the state-of-the-art literature explores robotic grasping methods by focusing exclusively on attributes of the target object. When it comes to human perceptual learning approaches, these physical qualities are not only inferred from the object, but also from the characteristics of the surroundings. This work proposes a method that includes environmental context to reason on an object affordance to then deduce its grasping regions. This affordance is reasoned using a ranked association of visual semantic attributes harvested in a knowledge base graph representation. The framework is assessed using standard learning evaluation metrics and the zero-shot affordance prediction scenario. The resulting grasping areas are compared with unseen labelled data to asses their accuracy matching percentage. The outcome of this evaluation suggest the autonomy capabilities of the proposed method for object interaction applications in indoor environments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ardón, P., Dragone, M., Erden, M.S.: Reaching and grasping of objects by humanoid robots through visual servoing. In: Prattichizzo, D., Shinoda, H., Tan, H.Z., Ruffaldi, E., Frisoli, A. (eds.) EuroHaptics 2018. LNCS, vol. 10894, pp. 353–365. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93399-3_31
Ardón, P., Pairet, È., Ramamoorthy, S., Lohan, K.S.: Towards robust grasps: using the environment semantics for robotic object affordances. In: Proceedings of the AAAI Fall Symposium on Reasoning and Learning in Real-World Systems for Long-Term Autonomy, pp. 5–12. AAAI Press (2018)
de Beeck, H.P.O., Torfs, K., Wagemans, J.: Perceived shape similarity among unfamiliar objects and the organization of the human object vision pathway. J. Neurosci. 28(40), 10111–10123 (2008)
Bonaiuto, J., Arbib, M.A.: Learning to grasp and extract affordances: the Integrated Learning of Grasps and Affordances (ILGA) model. Biol. Cybern. 109(6), 639–669 (2015)
Detry, R., et al.: Learning object-specific grasp affordance densities. In: IEEE 8th International Conference on Development and Learning, ICDL 2009, pp. 1–7. IEEE (2009)
Do, T.T., Nguyen, A., Reid, I.: AffordanceNet: an end-to-end deep learning approach for object affordance detection. In: International Conference on Robotics and Automation (ICRA) (2018)
Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1778–1785. IEEE (2009)
Geib, C., et al.: Object action complexes as an interface for planning and robot control. In: IEEE RAS International Conference on Humanoid Robots (2006)
Gibson, J.J.: The Ecological Approach to Visual Perception, Classic edn. Psychology Press, London (2014)
Gibson, J.: The theory of affordances. In: Shaw, R., Bransford, J. (eds.) Perceiving, Acting, and Knowing: Toward and Ecological Psychology, pp. 62–82. Erlbaum, Hillsdale, NJ (1977)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Krüger, N., et al.: Object-action complexes: grounded abstractions of sensory-motor processes. Robot. Auton. Syst. 59(10), 740–757 (2011)
Lai, K., Bo, L., Ren, X., Fox, D.: Detection-based object labeling in 3D scenes. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1330–1337. IEEE (2012)
Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. Int. J. Robot. Res. 34(4–5), 705–724 (2015)
Moldovan, B., Moreno, P., van Otterlo, M., Santos-Victor, J., De Raedt, L.: Learning relational affordance models for robots in multi-object manipulation tasks. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 4373–4378. IEEE (2012)
Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: Learning object affordances: from sensory-motor coordination to imitation. IEEE Trans. Robot. 24, 15–26 (2008)
Nguyen, A., Kanoulas, D., Caldwell, D.G., Tsagarakis, N.G.: Object-based affordances detection with convolutional neural networks and dense conditional random fields. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)
Pairet, È., Ardón, P., Broz, F., Mistry, M., Petillot, Y.: Learning and generalisation of primitives skills towards robust dual-arm manipulation. In: Proceedings of the AAAI Fall Symposium on Reasoning and Learning in Real-World Systems for Long-Term Autonomy, pp. 62–69. AAAI Press (2018)
Pairet, È., Ardón, P., Mistry, M., Petillot, Y.: Learning and composing primitive skills for dual-arm manipulation. In: Konstantinova, J., et al. (eds.) TAROS 2019. LNAI, vol. 11649, pp. 65–77 (2019)
Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 413–420. IEEE (2009)
Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Robot. Res. 27, 157–173 (2008)
Sridharan, M.: Integrating knowledge representation, reasoning, and learning for human-robot interaction. In: AAAI Fall Symposium. Artificial Intelligence for Human-Robot Interaction, pp. 69–76. AAAI Press (2017)
Stoytchev, A.: Toward learning the binding affordances of objects: a behavior-grounded approach. In: Proceedings of AAAI Symposium on Developmental Robotics, pp. 17–22 (2005)
Sung, J., Lenz, I., Saxena, A.: Deep multimodal embedding: manipulating novel objects with point-clouds, language and trajectories. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2794–2801. IEEE (2017)
Zhu, Y., Fathi, A., Fei-Fei, L.: Reasoning about object affordances in a knowledge base representation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 408–424. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10605-2_27
Acknowledgements
Thanks to the support of the EPSRC IAA 455791 along with ORCA Hub EPSRC (EP/R026173/1, 2017–2021) and consortium partners.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ardón, P., Pairet, È., Petrick, R., Ramamoorthy, S., Lohan, K. (2019). Reasoning on Grasp-Action Affordances. In: Althoefer, K., Konstantinova, J., Zhang, K. (eds) Towards Autonomous Robotic Systems. TAROS 2019. Lecture Notes in Computer Science(), vol 11649. Springer, Cham. https://doi.org/10.1007/978-3-030-23807-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-23807-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23806-3
Online ISBN: 978-3-030-23807-0
eBook Packages: Computer ScienceComputer Science (R0)