Scene Perception and Recognition for Human-Robot Co-operation

  • Nikhil Somani
  • Emmanuel Dean-León
  • Caixia Cai
  • Alois Knoll
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8158)


In this paper, an intuitive interface for collaborative tasks involving a human and a standard industrial robot is presented. The target for this interface is a worker who is experienced in manufacturing processes but has no experience in conventional industrial robot programming. Physical Human-Robot Interaction (pHRI) and interactive GUI control using hand gestures offered by this interface allows this novice user to instruct industrial robots with ease. This interface combines state of the art perception capabilities with first order logic reasoning to generate semantic description of the process plan. This semantic representation creates the possibility of including human and robot tasks in the same plan and also reduces the complexity of problem analysis by allowing process planning at semantic level, thereby isolating the problem description and analysis from the execution and scenario-specific parameters.


Perception HRI Reasoning 


  1. 1.
    Barequet, G., Har-Peled, S.: Efficiently approximating the minimum-volume bounding box of a point set in three dimensions. J. Algorithms 38, 91–109 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Delong, A., Osokin, A., Isack, H.N., Boykov, Y.: Fast approximate energy minimization with label costs. Int. J. Comput. Vision 96(1), 1–27 (2012), MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Fikes, R.E., Nilsson, N.J.: Strips: A new approach to the application of theorem proving to problem solving. Tech. Rep. 43R, AI Center, SRI International (May 1971)Google Scholar
  4. 4.
    Gonzalez, R.C., Woods, R.: Digital Image Processing, 2nd edn. Prentice Hall, New Jersey (2002)Google Scholar
  5. 5.
    Hastie, T., Tibshirani, R., Friedman, J.: 14.3.12 Hierarchical clustering The Elements of Statistical Learning, 2nd edn. Springer, New York (2009) ISBN 0-387-84857-6CrossRefGoogle Scholar
  6. 6.
    Hu, G.: 3-D object matching in the hough space. In: Intelligent Systems for the 21st Century Systems, Man and Cybernetics, vol. 3, pp. 2718–2723 (1995)Google Scholar
  7. 7.
    Kirsch, A., Kruse, T., Sisbot, E.A., Alami, R., Lawitzky, M., Brscic, D., Hirche, S., Basili, P., Glasauer, S.: Plan-based control of joint human-robot activities. Künstliche Intelligenz 24, 223–231 (2010)CrossRefGoogle Scholar
  8. 8.
    Leonardis, A., Gupta, A., Bajcsy, R.: Segmentation of range images as the search for geometric parametric models. Int. J. Comput. Vision 14(3), 253–277 (1995), CrossRefGoogle Scholar
  9. 9.
    Papazov, C., Haddadin, S., Parusel, S., Krieger, K., Burschka, D.: Rigid 3D geometry matching for grasping of known objects in cluttered scenes. International Journal of Robotic Research 31, 538–553 (2012)CrossRefGoogle Scholar
  10. 10.
    Rusu, R.B., Bradski, G., Thibaux, R., Hsu, J.: Fast 3D recognition and pose using the viewpoint feature histogram. In: 2010 IEEE/RSJ Intelligent Robots and Systems (IROS), pp. 2155–2162 (2010)Google Scholar
  11. 11.
    Schnabel, R., Wessel, R., Wahl, R., Klein, R.: Shape recognition in 3D point-clouds. In: Skala, V. (ed.) The 16th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2008. UNION Agency-Science Press (February 2008)Google Scholar
  12. 12.
    Sipiran, I., Bustos, B.: Harris 3D: A robust extension of the harris operator for interest point detection on 3D meshes. Vis. Comput. 27(11), 963–976 (2011), CrossRefGoogle Scholar
  13. 13.
    Somani, N., Dean, E., Cai, C., Knoll, A.: Perception and reasoning for scene understanding in human-robot interaction scenarios. In: Proceedings of the 2nd Workshop on Recognition and Action for Scene Understanding at the 15th International Conference on Computer Analysis of Images and Patterns (2013)Google Scholar
  14. 14.
    Somani, N., Dean, E., Cai, C., Knoll, A.: Scene perception and recognition in industrial environments for human-robot interaction. In: Proceedings of the 9th International Symposium on Visual Computing (2013)Google Scholar
  15. 15.
    Zhang, T., Hasanuzzaman, M., Ampornaramveth, V., Kiatisevi, P., Ueno, H.: Human-robot interaction control for industrial robot arm through software platform for agents and knowledge management. In: 2004 IEEE Systems, Man and Cybernetics, vol. 3, pp. 2865–2870 (October 2004)Google Scholar
  16. 16.
    Zhong, Y.: Intrinsic shape signatures: A shape descriptor for 3D object recognition. In: 2009 IEEE Computer Vision Workshops (ICCV Workshops), pp. 689–696 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Nikhil Somani
    • 1
  • Emmanuel Dean-León
    • 2
  • Caixia Cai
    • 1
  • Alois Knoll
    • 1
  1. 1.Fakultät für InformatikTechnische Universität MünchenGarching bei MünchenGermany
  2. 2.Cyber-Physical SystemsFortiss - An-Institut der Technischen Universität MünchenMünchenGermany

Personalised recommendations