Time-of-Flight Depth Datasets for Indoor Semantic SLAM

  • Vijaya K. Ghorpade
  • Dorit Borrmann
  • Paul ChecchinEmail author
  • Laurent Malaterre
  • Laurent Trassoudaine
Conference paper
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 10)


This paper introduces a medium-scale point cloud dataset for semantic SLAM (Simultaneous Localization and Mapping) acquired using a SwissRanger time-of-flight camera. An indoor environment with relatively unfluctuating lighting conditions is considered for mapping and localization. The camera is positioned on a mobile tripod and ready to capture images at prearranged locations in the environment. The prearranged locations are in fact used as ground truth for estimating the variance with poses calculated from SLAM, and also as initial pose estimates for the ICP algorithm (Iterative Closest Point). An interesting point is that, in this work, no type of Inertial Measurement Units or visual odometry techniques has been utilized, given the fact that, data from time-of-flight cameras is noisy and sensitive to external conditions (such as lighting, transparent surfaces, parallel overlapping surfaces etc.). Furthermore, a large collection of household objects is made in order to label the scene with semantic information. The whole SLAM dataset with pose files along with the point clouds of household objects is a major contribution in this paper apart from mapping and plane detection using a publicly available toolkit. Also, a novel metric, a context-based similarity score, for evaluating SLAM algorithms is presented.



This work is supported by the French government research program Investissements d’Avenir through the RobotEx Equipment of Excellence (ANR-10-EQPX-44) and the IMobS3 Laboratory of Excellence (ANR-10-LABX-16-01), by the European Union through the program Regional competitiveness and employment 2007–2013 (ERDF - Auvergne region), by the Auvergne region.


  1. 1.
    Bao, S.Y., Bagra, M., Chao, Y.W., Savarese, S.: Semantic structure from motion with points, regions, and objects. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2703–2710 (2012).
  2. 2.
    Borrmann, D., Elseberg, J., Lingemann, K., Nüchter, A., Hertzberg, J.: Globally consistent 3D mapping with scan matching. Robot. Auton. Syst. 56(2), 130–142 (2008).,
  3. 3.
    Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., Leonard, J.J.: Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Trans. Robot. 32(6), 1309–1332 (2016). Scholar
  4. 4.
    Cazorla, M., Viejo, D., Pomares, C.: Study of the sr4000 camera. In: Proceedings of XI Workshop of Physical Agents Fısicos, Valencia, Spain (2010)Google Scholar
  5. 5.
    Chiabrando, F., Chiabrando, R., Piatti, D., Rinaudo, F.: Sensors for 3d imaging: metric evaluation and calibration of a ccd/cmos time-of-flight camera. Sensors 9(12), 10080–10096 (2009)CrossRefGoogle Scholar
  6. 6.
    Chiabrando, F., Piatti, D., Rinaudo, F.: SR-4000 Tof camera: further experimental tests and first applications to metric surveys. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 38, 149–154 (2010)Google Scholar
  7. 7.
    Comport, A., Malis, E., Rives, P.: Real-time quadrifocal visual odometry. Int. J. Rob. Res. 29(2–3), 245–266 (2010).
  8. 8.
    Comport, A.I., Meill, M., Rives, P.: Real-time dense appearance-based SLAM for RGB-D sensors. In: Proceedings of the 2011 Australasian Conference on Robotics and Automation, pp. 100–109 (2011)Google Scholar
  9. 9.
    Cui, Y., Schuon, S., Chan, D., Thrun, S., Theobalt, C.: 3D shape scanning with a time-of-flight camera. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1173–1180 (2010).
  10. 10.
    Dib, A., Beaufort, N., Charpillet, F.: A real time visual SLAM for RGB-D cameras based on chamfer distance and occupancy grid. In: 2014 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 652–657 (2014).
  11. 11.
    Diebel, J., Thrun, S.: An application of Markov random fields to range sensing. In: Paper presented on NIPS, pp. 291–298. MIT Press, Cambridge (2005)Google Scholar
  12. 12.
    Donoho, D.L.: Denoising by soft-thresholding. IEEE Trans. Inf. Theory 41, 613–627 (1995).
  13. 13.
    Dopfer, A., Wang, H.H., Wang, C.C.: 3d active appearance model alignment using intensity and range data. Robot. Auton. Syst. 62(2), 168–176 (2014).,
  14. 14.
    Falie, D., Buzuloiu, V.: Noise characteristics of 3d time-of-flight cameras. In: 2007 International Symposium on Signals, Circuits and Systems, vol. 1, pp. 1–4 (2007).
  15. 15.
    Foix, S., Alenya, G., Torras, C.: Lock-in time-of-flight (tof) cameras: a survey. IEEE Sens. J. 11(9), 1917–1926 (2011). Scholar
  16. 16.
    Fuchs, S., May, S.: Calibration and registration for precise surface reconstruction with time-of-flight cameras. Int. J. Intell. Syst. Technol. Appl. 5(3/4), 274–284 (2008).
  17. 17.
    Ghorpade, V.K., Checchin, P., Malaterre, L., Trassoudaine, L.: Performance evaluation of 3d keypoint detectors for time-of-flight depth data. In: 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1–6 (2016).
  18. 18.
    Ghorpade, V.K., Checchin, P., Malaterre, L., Trassoudaine, L.: 3d shape representation with spatial probabilistic distribution of intrinsic shape keypoints. EURASIP J. Adv. Signal Process. 2017(1), 52 (2017).
  19. 19.
    Ghorpade, V.K., Checchin, P., Trassoudaine, L.: Line-of-sight-based tof camera’s range image filtering for precise 3d scene reconstruction. In: 2015 European Conference on Mobile Robots (ECMR), pp. 1–6 (2015).
  20. 20.
    Grisetti, G., Stachniss, C., Burgard, W.: Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE Trans. Robot. 23(1), 34–46 (2007). Scholar
  21. 21.
    Hansard, M., Lee, S., Choi, O., Horaud, R.P.: Time-of-flight cameras: principles, methods and applications. Springer Science & Business Media, London (2012)Google Scholar
  22. 22.
    He, Y., Liang, B., Zou, Y., He, J., Yang, J.: Depth errors analysis and correction for time-of-flight (tof) cameras. Sensors 17(1), 92 (2017)CrossRefGoogle Scholar
  23. 23.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments, pp. 477–491. Springer, Berlin (2014).
  24. 24.
    Hong, S., Ye, C., Bruch, M., Halterman, R.: Performance evaluation of a pose estimation method based on the swissranger sr4000. In: 2012 IEEE International Conference on Mechatronics and Automation, pp. 499–504 (2012).
  25. 25.
    Iddan, G.J., Yahav, G.: Three-dimensional imaging in the studio and elsewhere. In: Proceedings of the SPIE Three-Dimensional Image Capture and Applications IV. vol. 4298 (2001).
  26. 26.
    Jovanov, L., Pižurica, A., Philips, W.: Fuzzy logic-based approach to wavelet denoising of 3d images produced by time-of-flight cameras. Opt. Exp. 18, :22651–22676 (2010).
  27. 27.
    Kahlmann, T., Remondino, F., Ingensand, H.: Calibration for increased accuracy of the range imaging camera SwissRangerTM. In: Maas, H.G. (ed.) Proceedings of the ISPRS Commission V Symposium, International archives of photogrammetry, remote sensing and spatial information sciences, vol. 36, pp. 136–141. Institute of Photogrammetry and Remote Sensing, University of Technology, Dresden (2006)Google Scholar
  28. 28.
    Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012).,
  29. 29.
    Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR ’07, pp. 1–10. IEEE Computer Society, Washington, DC, USA (2007).
  30. 30.
    Kolb, A., Barth, E., Koch, R., Larsen, R.: Time-of-flight sensors in computer graphics. In: Pauly, M., Greiner, G. (eds.) Eurographics 2009 - State of the Art Reports. The Eurographics Association, Aire-la-Ville (2009).
  31. 31.
    Konolige, K., Agrawal, M., Bolles, R.C., Cowan, C., Fischler, M., Gerkey, B.: Outdoor Mapping and Navigation Using Stereo Vision, pp. 179–190. Springer, Berlin (2008).
  32. 32.
    Konolige, K., Bowman, J.: Towards lifelong visual maps. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1156–1163 (2009).
  33. 33.
    Kuffner, J.J.: Effective sampling and distance metrics for 3D rigid body path planning. In: 2004 IEEE International Conference on Robotics and Automation (ICRA), pp. 3993–3998. New Orleans, United States (2004)Google Scholar
  34. 34.
    Lange, R.: 3d time-of-flight distance measurement with custom solid-state image sensors in cmos/ccd-technology (2000).
  35. 35.
    Lu, F., Milios, E.: Globally consistent range scan alignment for environment mapping. Auton. Robot. 4(4), 333–349 (1997).
  36. 36.
    Magnusson, M., Andreasson, H., Nüchter, A., Lilienthal, A.J.: Automatic appearance-based loop detection from three-dimensional laser data using the normal distributions transform. J. Field Robot. 26(11–12), 892–914 (2009).
  37. 37.
    May, S., Droeschel, D., Fuchs, S., Holz, D., Nüchter, A.: Robust 3d-mapping with time-of-flight cameras. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1673–1678 (2009).
  38. 38.
    May, S., Droeschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., Hertzberg, J.: Three-dimensional mapping with time-of-flight cameras. J. Field Robot. 26(11–12), 934–965 (2009).
  39. 39.
    Montemerlo, M., Thrun, S., Koller, D., Wegbreit, B.: FastSLAM: a factored solution to the simultaneous localization and mapping problem. In: 18th National Conference on Artificial Intelligence, pp. 593–598. American Association for Artificial Intelligence, Menlo Park, CA, USA (2002).
  40. 40.
    Nister, D.: Preemptive RANSAC for live structure and motion estimation. In: Proceedings Ninth IEEE International Conference on Computer Vision, vol. 1, pp. 199–206 (2003).
  41. 41.
    Nüchter, A., Lingemann, K., Hertzberg, J., Surmann, H.: 6D SLAM - 3D mapping outdoor environments: research articles. J. Field Robot. 24(8–9), 699–722 (2007).
  42. 42.
    Nüchter, A., Lingemann, K.: Robotic 3D Scan Repository. (2017)
  43. 43.
    Pollefeys, M., Gool, L.V.: From images to 3D models. Commun. ACM 45(7), 50–55 (2002).
  44. 44.
    Pomerleau, F., Magnenat, S., Colas, F., Liu, M., Siegwart, R.: Tracking a depth camera: parameter exploration for fast ICP. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3824–3829 (2011).
  45. 45.
    Reynolds, M., Dobo, J., Peel, L., Weyrich, T., Brostow, G.J.: Capturing time-of-flight data with confidence. CVPR 2011, 945–952 (2011). Scholar
  46. 46.
    Robbins, S., Schroeder, B., Murawski, B., Heckman, N., Leung, J.: Photogrammetric calibration of the SwissRanger 3D range imaging sensor. In: Proceedings of the SPIE Optical Sensors, vol. 7003, p. 700320 (2008).
  47. 47.
    Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: Proceedings of Robotics: Science and Systems. Seattle, USA (2009).
  48. 48.
    Fuchs, S.: DLR institute of robotics and mechatronics: 3D mapping with ToF cameras. (2017)
  49. 49.
    Strasdat, H., Montiel, J., Davison, A.J.: Scale drift-aware large scale monocular SLAM. In: Robotics: Science and Systems VI. The MIT Press, Cambridge (2010)Google Scholar
  50. 50.
    Stühmer, J., Gumhold, S., Cremers, D.: Real-Time Dense Geometry from a Handheld Camera, pp. 11–20. Springer, Berlin (2010).
  51. 51.
    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 573–580. IEEE, Algarve (2012)Google Scholar
  52. 52.
    Sturm, J., Magnenat, S., Engelhard, N., Pomerleau, F., Colas, F., Cremers, D., Siegwart, R., Burgard, W.: Towards a benchmark for RGB-D SLAM evaluation. In: RGB-D Workshop on Advanced Reasoning with Depth Cameras at Robotics: Science and Systems Conference (RSS). Los Angeles, United States (2011).
  53. 53.
    Tamas, L., Jensen, B.: Robustness analysis of 3d feature descriptors for object recognition using a time-of-flight camera. In: 22nd Mediterranean Conference on Control and Automation, pp. 1020–1025 (2014).
  54. 54.
    Vivet, D., Gérossier, F., Checchin, P., Trassoudaine, L., Chapuis, R.: Mobile ground-based radar sensor for localization and mapping: an evaluation of two approaches. Int. J. Adv. Robot. Syst. 10(307), 12 (2013). Scholar
  55. 55.
    Vivet, D., Checchin, P., Chapuis, R.: Localization and mapping using only a rotating FMCW radar sensor. Sensors 13(4), 4527–4552 (2013).,
  56. 56.
    Weingarten, J.W., Gruener, G., Siegwart, R.: A state-of-the-art 3d sensor for robot navigation. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), vol. 3, pp. 2155–2160 (2004).
  57. 57.
    Ye, C., Bruch, M.: A visual odometry method based on the swissranger sr4000. In: Technical report, Arkansas University at Little rock, Little Rock (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Vijaya K. Ghorpade
    • 1
  • Dorit Borrmann
    • 2
  • Paul Checchin
    • 1
    Email author
  • Laurent Malaterre
    • 1
  • Laurent Trassoudaine
    • 1
  1. 1.Institut Pascal, SIGMA Clermont, CNRSUniversité Clermont AuvergneClermont-FerrandFrance
  2. 2.Robotics and Telematics GroupUniversity of WürzburgWürzburgGermany

Personalised recommendations