Robotic Grasping and Manipulation Competition: Competitor Feedback and Lessons Learned

  • Joe FalcoEmail author
  • Yu Sun
  • Maximo Roa
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 816)


The First Robot Grasping and Manipulation Competition, held during IROS 2016, allowed researchers focused on the application of robot systems to compare the performance of hand designs as well as autonomous grasping and manipulation solutions across a common set of tasks. The competition was comprised of three tracks that included hand-in-hand grasping, fully autonomous grasping, and simulation. The hand-in-hand and fully autonomous tracks used 18 predefined manipulation tasks and 20 objects. Additionally, a bin picking operation was also performed within the hand-in-hand and fully autonomous tracks using a shopping basket and a subset of the objects. The simulation track included two parts. The first was a pick and place operation, where a simulated hand extracted as many objects as possible from a cluttered shelf and placed them randomly in a bin. The second part was a bin picking operation where a simulated robotic hand lifted as many balls as possible from a bin and deposited them into a second bin. This paper presents competitor feedback as well as an analysis of lessons learned towards improvements and advancements for the next competition at IROS 2017.


Robot Grasping Manipulation Competition Benchmarks 


  1. 1.
    IEEE RAS TC Robotic Hands Grasping and Manipulation. Accessed 30 June 2017
  2. 2.
    Yale-CMU-Berkeley Object Dataset. Access 30 Mar 2017
  3. 3.
    Amazon Picking Challenge. Accessed 30 June 2017
  4. 4.
    Klamp’t (Kris’ Locomotion and Manipulation Planning Toolbox). Accessed 30 June 2017
  5. 5.
    Messina, E., et al.: Statement of Requirements for Urban Search and Rescue Robot Performance Standards. DHS and NIST Preliminary Report, p. 27 (2005)Google Scholar
  6. 6.
    Falco, J., Van Wyk, K.: Grasping the performance: facilitating replicable performance measures via benchmarking and standardized methodologies. Robot. Autom. Mag. 22(4), 125–136 (2015)CrossRefGoogle Scholar
  7. 7.
    Performance Metrics and Benchmarks to Advance the State of Robotic Grasping Web Portal. Accessed 30 June 2017
  8. 8.
    Performance Metrics and Benchmarks to Advance the State of Robotic Assembly Web Portal. Accessed 30 June 2017
  9. 9.
    Huang, Y., Bianchi, M., Liarokapis, M., Sun, Y.: Recent data sets on object manipulation: a survey. Big Data 4(4), 197–216 (2016)CrossRefGoogle Scholar
  10. 10.
    Paulius, D., et al.: Functional object-oriented network for manipulation learning. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2655–2662 (2016)Google Scholar
  11. 11.
    Bianchi, M., Bohg, J., Sun, Y.: Latest datasets and technologies presented in the workshop on grasping and manipulation datasets. arXiv preprint arXiv:1609.02531 (2016)
  12. 12.
    Huang, Y., Sun, Y.: Datasets on object manipulation and interaction: a survey. arXiv preprint arXiv:1607.00442 (2016)
  13. 13.
    Sun, Y., Lin, Y., Huang, Y.: Robotic grasping for instrument manipulations. In: 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), pp. 302–304. IEEE (2016)Google Scholar
  14. 14.
    Lin, Y., Sun, Y.: Task-oriented grasp planning based on disturbance distribution. In: Inaba, M., Corke, P. (eds.) Robotics Research, pp. 577–592. Springer, Cham (2016). Scholar
  15. 15.
    Lin, Y., Sun, Y.: Task-based grasp quality measures for grasp synthesis. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 485–490. IEEE (2015)Google Scholar
  16. 16.
    Lin, Y., Sun, Y.: Grasp planning to maximize task coverage. Int. J. Robot. Res. 34(9), 1195–1210 (2015)CrossRefGoogle Scholar
  17. 17.
    Lin, Y., Sun, Y.: Robot grasp planning based on demonstrated grasp strategies. Int. J. Robot. Res. 34(1), 26–42 (2015)MathSciNetCrossRefGoogle Scholar

Copyright information

© This is a U.S. government work and its text is not subject to copyright protection in the United States; however, its text may be subject to foreign copyright protection 2018

Authors and Affiliations

  1. 1.National Institute of Standards and Technology (NIST)GaithersburgUSA
  2. 2.University of South FloridaTampaUSA
  3. 3.German Aerospace Center (DLR)CologneGermany

Personalised recommendations