Skip to main content
SpringerLink
Log in

Robot Navigation in Domestic Environments: Experiences Using RGB-D Sensors in Real Homes

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Future home and service robots will require advanced navigation and interaction capabilities. In particular, domestic environments present open challenges that are hard to identify by conducting controlled tests in home-like settings: there is the need to test and evaluate navigation in the actual homes of users. This paper presents the experiences of operating a mobile robot with manipulation capabilities and an open set of tasks in extensive trials with real users, in their own homes. The main difficulties encountered are the requirement to move safely in cluttered 3D environments, the problems related to navigation in narrow spaces, and the need for an adaptive rather than fixed way to approach the users. We describe our solutions based on RGB-D perception and evaluate the integrated system for navigation in real home environments, pointing out remaining challenges towards more advanced commercial solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Vaussard, F., Fink, J., Bauwens, V., Rétornaz, P., Hamel, D., Dillenbourg, P., Mondada, F.: Lessons learned from robotic vacuum cleaners entering the home ecosystem. Robot. Auton. Syst. 62(3), 376–391 (2014)

    Article  Google Scholar 

  2. Gross, H.M., Mueller, S., Schroeter, C., Volkhardt, M., Scheidig, A., Debes, K., Richter, K., Doering, N.: Robot companion for domestic health assistance Implementation, test and case study under everyday conditions in private apartments. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), 5992–5999 (2015)

  3. Ruiz-Sarmiento, J.R., Galindo, C., González-Jiménez, J.: Robot@home, a robotic dataset for semantic mapping of home environments. Int. J. Robot. Res. 36(2), 131–141 (2017)

    Article  Google Scholar 

  4. Xu, Y., Qian, H., Wu, X.: Household service robotics. Intelligent systems. Academic Press, Cambridge (2015)

    Google Scholar 

  5. Fischinger, D., Einramhof, P., Papoutsakis, K., Wohlkinger, W., Mayer, P., Panek, P., Hofmann, S., Koertner, T., Weiss, A., Argyros, A., Hobbit, M. Vincze.: A care robot supporting independent living at home: First prototype and lessons learned. Robot. Auton. Syst. 75(Part A), 60–78 (2016)

    Article  Google Scholar 

  6. Coradeschi, S., Cesta, A., Cortellessa, G., Coraci, L., Galindo, C., Gonzalez, J., Karlsson, L., Forsberg, A., Frennert, S., Furfari, F., Loutfi, A., Orlandini, A., Palumbo, F., Pecora, F., von Rump, S., Stimec, A., Ullberg, J., Otslund, B.: Giraffplus: A system for monitoring activities and physiological parameters and promoting social interaction for elderly. In: Human-Computer systems interaction: backgrounds and applications 3, vol. 300, pp. 261–271. Springer International Publishing (2014)

  7. Görer, B.R., Salah, A.A., Akın, H.L.: A robotic fitness coach for the elderly. In: Proc. of ambient intelligence: 4th international joint conference, (AmI), pp. 124–139. Springer International Publishing (2013)

  8. Bajones, M., Fischinger, D., Weiss, A., de la Puente, P., Wolf, D., Vincze, M., Körtner, T., Weninger, M., Papoutsakis, D.M.K., Qammaz, A., Panteleris, P., Foukarakis, M., Adami, I., Ioannidi, D., Leonidis, A., Antona, M., Argyros, A., Mayer, P., Panek, P., Eftring, H., Frennert, S.: A year of field trials with the Hobbit Robot. Submitted to ACM Transactions on Human-Robot Interaction (2017)

  9. Breuer, T., Giorgana Macedo, G.R., Hartanto, R., Hochgeschwender, N., Holz, D., Hegger, F., Jin, Z., Müller, C., Paulus, J., Reckhaus, M., Álvarez Ruiz, J.A., Plöger, P.G., Kraetzschmar, G.K.: Johnny: An autonomous service robot for domestic environments. J. Intell. Robot. Syst. 66(1), 245–272 (2012)

    Article  Google Scholar 

  10. Marginean, M.T.: Distributed system for domestic robot operation using computer vision. PhD thesis (2016)

  11. Forlizzi, J., DiSalvo, C.: Service robots in the domestic environment A study of the roomba vacuum in the home. In: Proc. of ACM SIGCHI/SIGART conference on human-robot interaction, pp. 258–265 (2006)

  12. Schroeter, C., Mueller, S., Volkhardt, M., Einhorn, E., Huijnen, C., van den Heuvel, H., van Berlo, A., Bley, A., Gross, H.-M.: Realization and user evaluation of a companion robot for people with mild cognitive impairments. In: Proc. of IEEE international conference on robotics and automation (ICRA), pp. 1153–1159 (2013)

  13. Zsiga, K., Tóth, A., Pilissy, T., Péter, O., Dénes, Z., Fazekas, G.: Evaluation of a companion robot based on field tests with single older adults in their homes. Assist. Technol. 19, 1–8 (2017). https://doi.org/10.1080/10400435.2017.1322158

    Google Scholar 

  14. Iocchi, L., Ruiz-del Solar, J., van der Zant, T.: Domestic service robots in the real world. J. Intell. Robot. Syst. 66(1), 183–186 (2012)

    Article  Google Scholar 

  15. de la Puente, P., Bajones, M., Einramhof, P., Wolf, D., Fischinger, D., Vincze, M.: RGB-d sensor setup for multiple tasks of home robots and experimental results. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 2587–2594 (2014)

  16. Bajones, M., Fischinger, D., Weiss, A., et al.: Hobbit: providing fall detection and prevention for the elderly in the real world. J. Robot. 2018, 20pp (2018). https://doi.org/10.1155/2018/1754657

    Google Scholar 

  17. de la Puente, P., Bajones, M., Reuther, C., Fischinger, D., Wolf, D., Vincze, M.: Experiences with RGB-d based navigation in real home robotic trials. In: OAGM-ARW joint workshop 2016 on computer vision and robotics, pp. 153–165 (2016)

  18. Gates, B.: A robot in every home. Scientific American Magazine 296(1), 58–65 (2007)

    Article  Google Scholar 

  19. Fortunati, L.: Moving robots from industrial sectors to domestic spheres: a foreword. In: Esposito, A., Jain, L. (eds.) Toward Robotic Socially Believable Behaving Systems - Volume II. Intelligent Systems Reference Library, vol. 106. Springer, Cham (2016)

  20. Bogue, R.: Domestic robots: Has their time finally come?. Industrial Robot: An International Journal 44(2), 129–136 (2017)

    Article  Google Scholar 

  21. Kostavelis, I., Giakoumis, D., Malassiotis, S., Tzovaras, D.: Human aware robot navigation in semantically annotated domestic environments. In: Proc. of universal access in human-computer interaction, pp. 414–423, Springer International Publishing (2016)

  22. Kirsch, A.: Heuristic decision-making for human-aware navigation in domestic environments. In: Proc. of global conference on artificial intelligence (GCAI), vol. 41 of EPiC Series in Computing, pp. 200–213 (2016)

  23. Mount, J.: M Milford. 2d visual place recognition for domestic service robots at night. In: Proc. of IEEE international conference on robotics and automation (ICRA),pp. 4822–4829 (2016)

  24. Ruiz-Sarmiento, J.R., Galindo, C., Gonzalez-Jimenez, J.: Joint categorization of objects and rooms for mobile robots. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 2523–2528 (2015)

  25. Uršič, P., Leonardis, A., Skočaj, D., Kristan, M.: Learning part-based spatial models for laser-vision-based room categorization. Int. J. Robot. Res. 36(4), 379–402 (2015)

    Google Scholar 

  26. Sprute, D., Rasch, R., Tönnies, K., König, M.: A framework for interactive teaching of virtual borders to mobile robots. In: Proc. of IEEE/RSJ international symposium on robot and human interactive communication (RO-MAN), pp. 1175–1181 (2017)

  27. Holz, D., Kraetzschmar, G.K., Rome, E.: Robocup 2009. Chapter Robust and Computationally Efficient Navigation in Domestic Environments, pp 104–115. Springer, Berlin (2010)

    Google Scholar 

  28. Soroka, A.J., Qiu, R., Noyvirt, A., Ji, Z.: Challenges for service robots operating in non-industrial environments. In: Proc. of IEEE international conference on industrial informatics (INDIN), pp. 1152–1157 (2012)

  29. Prassler, E., Bischoff, R., Burgard, W., Haschke, R., Hägele, M., Lawitzky, G., Nebel, B., Plöger, P., Reiser, U., Zöllner, M.: Towards service robots for everyday environments. recent advances in designing service robots for complex tasks in everyday environments. Springer Tracts in Advanced Robotics, Springer, Berlin (2012)

    Book  Google Scholar 

  30. de Graaf, M.M.A., Ben Allouch, S., van Dijk, J.A.G.M.: Long-term evaluation of a social robot in real homes. Interact. Stud. 17(3), 462–491 (2016)

    Article  Google Scholar 

  31. Mast, M., Burmester, M., Berner, E., Facal, D., Pigini, L., Blasi, L.: Semi-autonomous teleoperated learning in-home service robots for elderly care A qualitative study on needs and perceptions of elderly people, family caregivers, and professional caregivers. In: Proc. of the 20th international conference on robotics and mechatronics, pp. 1–6 (2010)

  32. Papadopoulos, F., Küster, D., Corrigan, L.J., Kappas, A., Castellano, G.: Do relative positions and proxemics affect the engagement in a human-robot collaborative scenario?. Interact. Stud. 17(3), 321–347 (2016)

    Article  Google Scholar 

  33. Mead, R., Matarić, M.J.: Perceptual models of human-robot proxemics. In: Hsieh, M., Khatib, O., Kumar, V. (eds.) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 109. Springer, Cham (2016)

  34. Rossi, S., Ferland, F., Tapus, A.: User profiling and behavioral adaptation for HRI: a survey. Pattern Recogn. Lett. 99(Supplement C), 3–12 (2017)

    Article  Google Scholar 

  35. Koay, K.L., Syrdal, D.S., Ashgari-Oskoei, M., Walters, M.L., Dautenhahn, K.: Social roles and baseline proxemic preferences for a domestic service robot. Int. J. Soc. Robot. 6(4), 469–488 (2014)

    Article  Google Scholar 

  36. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng. A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)

  37. Einhorn, E., Langner, T., Stricker, R., Martin, C., Gross, H.: Mira - middleware for robotic applications. In: Proc. of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 2591–2598 (2012)

  38. Bohren, J., Cousins, S.: The smach high-level executive [ros news]. IEEE Robot. Autom. Mag. 17(4), 18–20 (2010)

    Article  Google Scholar 

  39. Hawes, N., Burbridge, C., Jovan, F., Kunze, L., Lacerda, B., Mudrová, L., Young, J., Wyatt, J., Hebesberger, D., Körtner, T., Ambrus, R., Bore, N., Folkesson, J., Jensfelt, P., Beyer, L., Hermans, A., Leibe, B., Aldoma, A., Fäulhammer, T., Zillich, M., Vincze, M., Chinellato, E., Al-Omari, M., Duckworth, P., Gatsoulis, Y., Hogg, D., Cohn, A., Dondrup, C., Pulido, J., Krajník, T., Santos, J., Duckett, T., Hanheide, M.: The STRANDS, project: Long-term autonomy in everyday environments. IEEE Robot. Autom. Mag. 24(3), 146–156 (2016)

    Article  Google Scholar 

  40. Hendrich, N., Bistry, H., Jianwei, Z.: PEIS, MIRA, and ROS: Three frameworks, one service robot -A tale of integration. In: 2014 IEEE international conference on robotics and biomimetics (ROBIO), pp. 1749–1756 (2014)

  41. Grisetti, G., Stachniss, C., Burgard, W.: Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE Trans. Robot. 23(1), 34–46 (2007)

    Article  Google Scholar 

  42. Trevor, A.J.B., Cosgun, A., Kumar, J., Christensen, H.I.: Interactive map labeling for service robots. In: IROS Workshop on Active Semantic Perception (2012)

  43. González-Jiménez, J., Galindo, C., Melendez-Fernandez, F., Ruiz-Sarmiento, J.R.: Building and exploiting maps in a telepresence robotic application. In: 10th international conference on informatics in control, automation and robotics (ICINCO), pp. 322–328 (2013)

  44. Panteleris, P., Argyros, A.: Vision-based SLAM And moving objects tracking for the perceptual support of a smart walker platform. In: Computer Vision - ECCV 2014 Workshops, vol. 8927 of Lecture Notes in Computer Science, pp. 407–423, Springer International Publishing (2015)

  45. Qammaz, A., Kyriazis, N., Argyros, A.A.: Boosting the performance of model-based 3d tracking by employing low level motion cues. In: Proc. of the British machine vision conference (BMVC), pp. 144.1–144.11 (2015)

  46. Fischinger, D., Weiss, A., Vincze, M.: Learning grasps with topographic features. Int. J. Robot. Res. 34(9), 1167–1194 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the contribution of all the project partners who made this work possible. Furthermore, Fig. 2. c) and d) were taken by Ammar Qammaz (Institute of Computer Science, FORTH) and Fig. 18. was obtained thanks to Susanne Frennert and Håkan Eftring (Certec, Lund University), who also recorded Online Resource 3. Lara Lammer (TU Wien) and Markus Weninger (AAF) additionally provided significant multimedia material. Astrid Weiss shared great discussions and advice. We are also very grateful to our elderly users. This work has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement No 288146.

Author information

Authors and Affiliations

  1. Centre for Automation and Robotics, UPM-CSIC, Universidad Politécnica de Madrid, Madrid, Spain

    Paloma de la Puente

  2. Vienna University of Technology (TU Wien), Vienna, Austria

    Paloma de la Puente, Markus Bajones, Daniel Wolf, David Fischinger & Markus Vincze

  3. Metralabs, Ilmenau, Germany

    Christian Reuther

  4. Aeolus Robotics, Inc., Vienna, Austria

    David Fischinger

Authors
  1. Paloma de la Puente
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Markus Bajones
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Reuther
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Fischinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Markus Vincze
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paloma de la Puente.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 16.8 MB)

(MP4 17.8 MB)

(MP4 18.9 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de la Puente, P., Bajones, M., Reuther, C. et al. Robot Navigation in Domestic Environments: Experiences Using RGB-D Sensors in Real Homes. J Intell Robot Syst 94, 455–470 (2019). https://doi.org/10.1007/s10846-018-0885-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0885-6

Keywords

Search

Navigation