Abstract
Within computer science, autonomous robotics takes the uneasy role of a discipline where the features of both systems (i.e., robots) and their operating environment (i.e., the physical world) conspire to make the application of the experimental scientific method most difficult. This is the reason why much experimental work in robotics is, from the methodological point of view, built on shaky grounds. In particular, scientifically sound benchmarking tools are still largely missing. This chapter starts from Rawseeds, a project focused precisely on benchmarking in robotics, to highlight the reasons for these difficulties and to propose strategies for overcoming some of them. The main result of Rawseeds is a Benchmarking Toolkit: a readily usable instrument to assess and compare algorithms for SLAM, localization, and mapping. Its most innovative aspects include a set of high-quality, validated, multi-sensor datasets, collected both in indoor and in outdoor locations and complemented by ground truth data, and the explicit definition of a set of quantitative performance metrics for the evaluation of algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Robotics Advancement through Web-publishing of Sensorial and Elaborated Extensive Data Sets (Rawseeds) [4] has been financed by the European Commission, within the 6th Framework Programme.
- 2.
Sensor fusion is the joint processing of more than one sensor datastreams. Self-localization is the process of finding one’s position on a map of the environment. Mapping is the set of operations required to build such a map, usually involving exploration. Finally, Self-Localization And Mapping (SLAM) requires to autonomously build a map of the environment and to keep track of one’s location on it.
- 3.
Nowadays the availability of systems for motion capture provides a means to precisely acquire the trajectory of a robot. This is still an expensive technology, especially when the capture area is large; however, it is readily available on the market and prices are getting lower.
References
Bonsignorio F, Hallam J, del Pobil AP (2007) Good experimental methodology - GEM guidelines. http://www.heronrobots.com/EuronGEMSig/downloads/GemSigGuidelinesBeta.pdf. Accessed July 2013
PerMIS (2010) Performance metrics for intelligent systems. http://www.nist.gov/el/isd/ks/permis.cfm, Accessed July 2013
Howard A, Roy N (2003) The robotics data set repository (radish). http://www.rawseeds.org/. Accessed July 2013
Rawseeds (2006) http://radish.sourceforge.net/. Accessed July 2013
Zinas N (2011) GPS network real time kinematic tutorial. Tech. Rep. TEKMON-001, Tekmon Geomatics LLP. http://tekmon.gr/tekmon-research/gps-network-rtk-tutorial/. Accessed July 2013
Ceriani S, Fontana G, Giusti A, Marzorati D, Matteucci M, Migliore D, Rizzi D, Sorrenti D, Taddei P (2009) Rawseeds ground truth collection systems for indoor self-localization and mapping. Auton Robot 27(4):353–371
Jacoff A, Messina E, Evans J (2002) Experiences in deploying test arenas for autonomous mobile robots. In: Proc 2011 PerMIS, Workshop, pp 87–94
Jacoff A, Messina E, Weiss BA, Tadokoro S, Nakagawa Y (2003) Test arenas and performance metrics for urban search and rescue robots. In: Proc 2003 IEEE/RSJ IROS, vol 3, pp 3396–3403
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 The Author(s)
About this chapter
Cite this chapter
Fontana, G., Matteucci, M., Sorrenti, D.G. (2014). Rawseeds: Building a Benchmarking Toolkit for Autonomous Robotics. In: Amigoni, F., Schiaffonati, V. (eds) Methods and Experimental Techniques in Computer Engineering. SpringerBriefs in Applied Sciences and Technology(). Springer, Cham. https://doi.org/10.1007/978-3-319-00272-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-00272-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-00271-2
Online ISBN: 978-3-319-00272-9
eBook Packages: EngineeringEngineering (R0)