Abstract
We propose a novel approach to multi-fingered grasp planning leveraging learned deep neural network models. We train a convolutional neural network to predict grasp success as a function of both visual information of an object and grasp configuration. We can then formulate grasp planning as inferring the grasp configuration which maximizes the probability of grasp success. We efficiently perform this inference using a gradient-ascent optimization inside the neural network using the backpropagation algorithm. Our work is the first to directly plan high quality multi-fingered grasps in configuration space using a deep neural network without the need of an external planner. We validate our inference method performing both multi-finger and two-finger grasps on real robots. Our experimental results show that our planning method outperforms existing planning methods for neural networks; while offering several other benefits including being data-efficient in learning and fast enough to be deployed in real robotic applications.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis-a survey. IEEE Trans. Robot. 30(2), 289–309 (2014)
Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., Dollar, A. M.: The YCB object and model set: towards common benchmarks for manipulation research. In: International Conference on Advanced Robotics (ICAR), pp. 510–517 (2015)
Ciocarlie, M., Goldfeder, C., Allen, P.K.: Dimensionality reduction for hand-independent dexterous robotic grasping. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3270–3275 (2007)
Dragiev, S., Toussaint, M., Gienger, M.: Gaussian process implicit surfaces for shape estimation and grasping. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2845–2850 (2011)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)
Grupen, R.A.: Planning grasp strategies for multifingered robot hands. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 646–651 (1991)
Gschwandtner, M., Kwitt, R., Uhl, A., Pree, W.: BlenSor: blender sensor simulation toolbox. In: Advances in visual computing, pp. 199–208 (2011)
Gualtieri, M., Ten Pas, A., Saenko, K., Platt, R.: High precision grasp pose detection in dense clutter. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 598–605 (2016)
Johns, E., Leutenegger, S., Davison. A.J.: Deep learning a grasp function for grasping under gripper pose uncertainty. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4461–4468 (2016)
Kappler, D., Bohg, J., Schaal, S.: Leveraging big data for grasp planning. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 4304–4311 (2015)
Kopicki, M., Detry, R., Adjigble, M., Stolkin, R., Leonardis, A., Wyatt, J.L.: One-shot learning and generation of dexterous grasps for novel objects. Int. J. Robot. Res. (IJRR) 35(8), 959–976 (2016)
Kumra, S., Kanan, C.: Robotic grasp detection using deep convolutional neural networks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)
Lenz, I., Lee, H., Saxena, A.: Deep learning for detecting robotic grasps. Int. J. Robot. Res. (IJRR) 34(4–5), 705–724 (2015)
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., Quillen, D.: Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Robot. Res. p. 0278364917710318 (2016)
Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Goldberg, K.: Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In: Robotics Science and Systems (RSS) (2017)
Murray, R.M., Li, Z., Sastry, S.S.: A Mathematical Introduction to Robotic Manipulation. CRC Press, Boca Raton (1994)
Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50K tries and 700 robot hours. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413 (2016)
Redmon, J., Angelova, A.: Real-time grasp detection using convolutional neural networks. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1316–1322 (2015)
Sahbani, A., El-Khoury, S., Bidaud, P.: An overview of 3D object grasp synthesis algorithms. Robot. Auton. Syst. 60(3), 326–336 (2012)
Saxena, A., Driemeyer, J., Ng, A.Y.: Robotic grasping of novel objects using vision. Int. J. Robot. Res.(IJRR), 27(2), 157–173 (2008)
Saxena, S., Wong, L.L.S., Ng, A.Y.: Learning grasp strategies with partial shape information. In: AAAI National Conference on Artificial Intelligence, pp. 1491–1494 (2008)
Singh, A., Sha, J., Narayan, K.S., Achim, T., Abbeel, P.: Bigbird: a large-scale 3D database of object instances. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 509–516 (2014)
Varley, J., Weisz, J., Weiss, J., Allen, P.: Generating multi-fingered robotic grasps via deep learning. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4415–4420 (2015)
Veres, M., Moussa, M., Taylor, G.W.: Modeling grasp motor imagery through deep conditional generative models. IEEE Robot. Autom. Lett. 2(2), 757–764 (2017)
Zhou, Y., Hauser, K.: 6DOF grasp planning by optimizing a deep learning scoring function. In: Robotics: Science and Systems (RSS) Workshop on Revisiting Contact - Turning a Problem into a Solution (2017)
Acknowledgements
Q. Lu and B. Sundaralingam were supported in part by NSF Award #1657596.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, Q., Chenna, K., Sundaralingam, B., Hermans, T. (2020). Planning Multi-fingered Grasps as Probabilistic Inference in a Learned Deep Network. In: Amato, N., Hager, G., Thomas, S., Torres-Torriti, M. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 10. Springer, Cham. https://doi.org/10.1007/978-3-030-28619-4_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-28619-4_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28618-7
Online ISBN: 978-3-030-28619-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)