Abstract
The Robot Grasping and Manipulation Competition, held during the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) in Daejeon, South Korea was sponsored by the IEEE Robotic and Automation Society (RAS) Technical Committee (TC) on Robotic Hands Grasping and Manipulation (RHGM) [1]. This competition was the first of a planned series of grasping and manipulation-themed events of increasing difficulty that are intended to spur technological developments and advance test methods and benchmarks so that they can be formalized for use by the community. The coupling of standardized performance testing with robot competitions will promote the use of unbiased evaluation methods to assess how well a robot system performs in a particular application space. A strategy is presented for a series of grasping and manipulation competitions that facilitate objective performance benchmarking of robotic assembly solutions. This strategy is based on test methods that can be used for more rigorous assessments and comparison of systems and components outside of the competition regime. While competitions have proven to be useful mechanisms for assessing the relative performance of robotic systems with measures of success, they often lack a methodical measurement science foundation. Consequently, scientifically sound and statistically significant metrics, measurement, and evaluation methods to quantify performance are missing. Using performance measurement methods in a condensed format will accommodate competition time limits while introducing the methods to the community as tools for benchmarking performance in the developmental and deployment phases of a robot system. The particular evaluation methods presented here are focused on the mechanical assembly process, an application space that is expected to accelerate with the new robot technologies coming to market.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Robotic Hands Grasping and Manipulation RAS TC. http://www.rhgm.org/. Accessed 30 June 2017
Robocup. http://www.robocup.org/. Accessed 30 June 2017
DARPA Robotics Challenge. http://archive.darpa.mil/roboticschallenge/. Accessed 30 June 2017
Amazon Picking Challenge. http://www.amazonrobotics.com/#/roboticschallenge/. Accessed 25 Aug 2017
Amigoni, F., Bonarini, A., Fontana, G., Matteucci, M., Schiaffonati, V.: To what extent are competitions experiments? A critical view. http://rockinrobotchallenge.eu/competitions_experiments.pdf. Accessed 30 June 2017
Anderson, M., Jenkins, O.C., Osentoski, S.: Recasting robotics challenges as experiments. IEEE Robot. Autom. Mag. 18(2), 10–11 (2011)
Jacoff, A., et al.: Using competitions to advance the development of standard test methods for response robots. Association of Computing Machinery (ACM) (2012). http://ws680.nist.gov/publication/get_pdf.cfm?pub_id=910662. Accessed 30 June 2017
Marvel, J., Hong, T., Messina, E.: 2011 solutions in perception challenge performance metrics and results. In: Proceedings of the Workshop on Performance Metrics for Intelligent Systems, pp. 59–63 (2012)
Falco, J., Van Wyk, K.: Grasping the performance: facilitating replicable performance measures via benchmarking and standardized methodologies. Robot. Autom. Mag. 22(4), 125–136 (2015)
Performance metrics and benchmarks to advance the state of robotic grasping web portal. https://www.nist.gov/programs-projects/performance-metrics-and-benchmarks-advance-state-robotic-grasping. Accessed 30 June 2017
Downs, A., Harrison, W., Schlenoff, C.: Test methods for robot agility in manufacturing. Ind. Robot Int. J. 43(5), 563–572 (2016)
Agile robotics for industrial automation competition (ARIAC). https://www.nist.gov/el/intelligent-systems-division-73500/agile-robotics-industrial-automation. Accessed 30 June 2017
Shneier, S., Messina, E., Schlenoff, C., Proctor, F., Kramer, T., Falco, J.: Measuring and representing the performance of manufacturing assembly robots, NIST Internal Report 8090, November 2015
International federation of robotics, World Robotics Report (2016). http://www.ifr.org/industrial-robots/statistics/. Accessed 30 June 2017
Price Waterhouse, The new hire: How a new generation of robots is transforming manufacturing. https://www.pwc.com/us/en/industrial-products/assets/industrial-robot-trends-in-manufacturing-report.pdf. Accessed 30 June 2017
Bostelman, R., Falco, J.: Survey of industrial manipulation technologies for autonomous assembly applications, National Institute of Standards and Technology Report, NISTIR 7844 (2012)
Tobe, F.: Why co-bots will be a huge innovation and growth driver for robotics industry. IEEE Spectrum, December 2015. http://spectrum.ieee.org/automaton/robotics/industrial-robots/collaborative-robots-innovation-growth-driver. Accessed 30 June 2017
Boothroyd, G., Dewhurst, P., Knight, W.: Product Design for Manufacture and Assembly, 3rd edn. CRC Press, Boca Raton (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 This is a U.S. government work and its text is not subject to copyright protection in the United States; however, its text may be subject to foreign copyright protection
About this paper
Cite this paper
Van Wyk, K., Falco, J., Messina, E. (2018). Robotic Grasping and Manipulation Competition: Future Tasks to Support the Development of Assembly Robotics. In: Sun, Y., Falco, J. (eds) Robotic Grasping and Manipulation. RGMC 2016. Communications in Computer and Information Science, vol 816. Springer, Cham. https://doi.org/10.1007/978-3-319-94568-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-94568-2_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94567-5
Online ISBN: 978-3-319-94568-2
eBook Packages: Computer ScienceComputer Science (R0)