Advertisement

Visual Data Simulation for Deep Learning in Robot Manipulation Tasks

  • Miroslav Surák
  • Karel KošnarEmail author
  • Miroslav Kulich
  • Viktor Kozák
  • Libor Přeučil
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11472)

Abstract

This paper introduces the usage of simulated images for training convolutional neural networks for object recognition and localization in the task of random bin picking. For machine learning applications, a limited amount of real world image data that can be captured and labeled for training and testing purposes is a big issue. In this paper, we focus on the use of realistic simulation of image data for training convolutional neural networks to be able to estimate the pose of an object. We can systematically generate varying camera viewpoint datasets with a various pose of an object and lighting conditions. After successful training and testing the neural network, we compare the performance of network trained on simulated images and images from a real camera capturing the physical object. The usage of the simulated data can speed up the complex and time-consuming task of gathering training data as well as increase robustness of object recognition by generating a bigger amount of data.

Keywords

Deep learning CNN Simulation Ray-tracing Robot manipulation Random bin picking 

Notes

Acknowledgement

Access to computing and storage facilities owned by parties and projects contributing to the National Grid Infrastructure MetaCentrum provided under the programme “Projects of Large Research, Development, and Innovations Infrastructures” (CESNET LM2015042), is greatly appreciated. This work has been supported from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688117 (SafeLog). The work was also supported by the Grant Agency of the Czech Technical University in Prague, grant No. SGS18/206/OHK3/3T/37.

References

  1. 1.
    Ciocarlie, M., Hsiao, K., Jones, E.G., Chitta, S., Rusu, R.B., Şucan, I.A.: Towards reliable grasping and manipulation in household environments. In: Khatib, O., Kumar, V., Sukhatme, G. (eds.) Experimental Robotics. Springer Tracts in Advanced Robotics, vol. 79. Springer, Heidelberg (2014).  https://doi.org/10.1007/978-3-642-28572-1_17CrossRefGoogle Scholar
  2. 2.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005 (2005).  https://doi.org/10.1109/CVPR.2005.177
  3. 3.
    Eitel, A., Springenberg, J.T., Spinello, L., Riedmiller, M., Burgard, W.: Multimodal deep learning for robust RGB-D object recognition. In: IEEE International Conference on Intelligent Robots and Systems (2015).  https://doi.org/10.1109/IROS.2015.7353446
  4. 4.
    Gualtieri, M., Pas, A.T., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: IEEE International Conference on Intelligent Robots and Systems (2016).  https://doi.org/10.1109/IROS.2016.7759114
  5. 5.
    Gupta, S., Arbeláez, P., Girshick, R., Malik, J.: Aligning 3D models to RGB-D images of cluttered scenes. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2015).  https://doi.org/10.1109/CVPR.2015.7299105
  6. 6.
    Hinterstoisser, S., et al.: Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In: Proceedings of the IEEE International Conference on Computer Vision (2011).  https://doi.org/10.1109/ICCV.2011.6126326
  7. 7.
    Mahler, J., et al.: Dex-Net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In: Robotics: Science and Systems (2017).  https://doi.org/10.15607/RSS.2017.XIII.058, http://arxiv.org/abs/1703.09312
  8. 8.
    Mahler, J., Matl, M., Liu, X., Li, A., Gealy, D., Goldberg, K.: Dex-Net 3.0: computing robust robot vacuum suction grasp targets in point clouds using a new analytic model and deep learning. In: 2018 IEEE International Conference on Robotics and Automation (IRCA) (2018).  https://doi.org/10.1109/ICRA.2018.8460887
  9. 9.
    Persistence of Vision Pty. Ltd.: POV-Ray - the persistence of vision raytracer (2004). http://www.povray.org/
  10. 10.
    Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50K tries and 700 robot hours. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 3406–3413, June 2016.  https://doi.org/10.1109/ICRA.2016.7487517
  11. 11.
    Sushkov, R.: Detection and pose determination of a part for bin picking. Master thesis, Czech Technical University in Prague, June 2017. https://dspace.cvut.cz/handle/10467/68468?show=full
  12. 12.
    Tzeng, E., et al.: Adapting deep visuomotor representations with weak pairwise constraints. In: The 12th International Workshop on the Algorithmic Foundations of Robotics (2016)Google Scholar
  13. 13.
    Varley, J., Weisz, J., Weiss, J., Allen, P.: Generating multi-fingered robotic grasps via deep learning. In: IEEE International Conference on Intelligent Robots and Systems (2015).  https://doi.org/10.1109/IROS.2015.7354004
  14. 14.
    Xie, Z., Singh, A., Uang, J., Narayan, K.S., Abbeel, P.: Multimodal blending for high-accuracy instance recognition. In: IEEE International Conference on Intelligent Robots and Systems, pp. 2214–2221 (2013).  https://doi.org/10.1109/IROS.2013.6696666
  15. 15.
    Zeng, A., et al.: Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge. In: Proceedings of IEEE International Conference on Robotics and Automation (2017).  https://doi.org/10.1109/ICRA.2017.7989165

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Miroslav Surák
    • 1
  • Karel Košnar
    • 2
    Email author
  • Miroslav Kulich
    • 2
  • Viktor Kozák
    • 2
  • Libor Přeučil
    • 2
  1. 1.Faculty of Electrical EngineeringCzech Technical University in PraguePragueCzech Republic
  2. 2.Czech Institute of Informatics, Robotics and CyberneticsCzech Technical University in PraguePragueCzech Republic

Personalised recommendations