Skip to main content
SpringerLink
Log in

Real-Time Multi-view Grid Map-Based Spatial Representation for Mixed Reality Applications

  • Conference paper
  • First Online:
Book cover Augmented Reality, Virtual Reality, and Computer Graphics (AVR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10850))

Abstract

In this paper it is presented a markerless multi-view vision-based system to create a spatial representation of an indoor environment. Mixed reality integrates real-world elements into a virtual world. The modeling of such elements should be accurate to avoid interaction inconsistencies. The goal of this work is to achieve the virtualization of a room and its elements, providing seamless user navigation and interaction. This spatial representation based on grid maps provides a clear distinction between occupied and free cells. Scene objects are segmented using clustering techniques. The proposed approach has several potential applications, such as interactive virtual visits to heritage museums, real estate, and training scenarios. Preliminary qualitative results evidence a visually accurate modeling of the considered scenes, identifying elements and free space, encouraging future work towards the identification and tracking of static and dynamic elements on the scene.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ricci, A., Piunti, M., Tummolini, L., Castelfranchi, C.: The mirror world: preparing for mixed-reality living. IEEE Pervasive Comput. 14(2), 60–63 (2015)

    Article  Google Scholar 

  2. Elfes, A.: Using occupancy grids for mobile robot perception and navigation. Computer 22(6), 46–57 (1989)

    Article  Google Scholar 

  3. Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., Fitzgibbon, A.: KinectFusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136, October 2011

    Google Scholar 

  4. Urmson, C., Anhalt, J., Bagnell, D., Baker, C., Bittner, R., Clark, M., Dolan, J., Duggins, D., Galatali, T., Geyer, C., et al.: Autonomous driving in urban environments: boss and the urban challenge. J. Field Robot. 25(8), 425–466 (2008)

    Article  Google Scholar 

  5. Montemerlo, M., Becker, J., Bhat, S., Dahlkamp, H., Dolgov, D., Ettinger, S., Haehnel, D., Hilden, T., Hoffmann, G., Huhnke, B., et al.: Junior: the stanford entry in the urban challenge. J. Field Robot. 25(9), 569–597 (2008)

    Article  Google Scholar 

  6. Laugier, C., Paromtchik, I.E., Perrollaz, M., Yong, M., Yoder, J.D., Tay, C., Mekhnacha, K., Nègre, A.: Probabilistic analysis of dynamic scenes and collision risks assessment to improve driving safety. IEEE Intell. Transp. Syst. Mag. 3(4), 4–19 (2011)

    Article  Google Scholar 

  7. Pfeiffer, D., Franke, U.: Efficient representation of traffic scenes by means of dynamic stixels. In: Intelligent Vehicles Symposium (IV), 2010 IEEE, pp. 217–224. IEEE (2010)

    Google Scholar 

  8. Lu, F., Milios, E.E.: Robot pose estimation in unknown environments by matching 2D range scans. In: 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–938, June 1994

    Google Scholar 

  9. Cole, D.M., Newman, P.M.: Using laser range data for 3D slam in outdoor environments. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006, pp. 1556–1563. IEEE (2006)

    Google Scholar 

  10. Leonard, J.J., Durrant-Whyte, H.F.: Simultaneous map building and localization for an autonomous mobile robot. In: IEEE/RSJ International Workshop on Intelligent Robots and Systems 1991. Intelligence for Mechanical Systems, Proceedings IROS 1991, pp. 1442–1447. IEEE (1991)

    Google Scholar 

  11. Dissanayake, M.G., Newman, P., Clark, S., Durrant-Whyte, H.F., Csorba, M.: A solution to the simultaneous localization and map building (slam) problem. IEEE Trans. Robot. Autom. 17(3), 229–241 (2001)

    Article  Google Scholar 

  12. Choset, H., Nagatani, K.: Topological simultaneous localization and mapping (slam): toward exact localization without explicit localization. IEEE Trans. Robot. Autom. 17(2), 125–137 (2001)

    Article  Google Scholar 

  13. Nuchter, A., Lingemann, K., Hertzberg, J., Surmann, H.: 6D slam with approximate data association. In: 12th International Conference on Advanced Robotics, 2005. ICAR 2005. Proceedings, pp. 242–249. IEEE (2005)

    Google Scholar 

  14. Montemerlo, M., Thrun, S., Koller, D., Wegbreit, B., et al.: FastSLAM: a factored solution to the simultaneous localization and mapping problem. In: AAAI/IAAI, pp. 593–598 (2002)

    Google Scholar 

  15. Siadat, A., Kaske, A., Klausmann, S., Dufaut, M., Husson, R.: An optimized segmentation method for a 2D laser-scanner applied to mobile robot navigation. IFAC Proc. Vol. 30(7), 149–154 (1997)

    Article  Google Scholar 

  16. Sack, D., Burgard, W.: A comparison of methods for line extraction from range data. IFAC Proc. Vol. 37(8), 728–733 (2004)

    Article  Google Scholar 

  17. Petrovskaya, A., Thrun, S.: Model based vehicle detection and tracking for autonomous urban driving. Auton. Robots 26(2–3), 123–139 (2009)

    Article  Google Scholar 

  18. Fayad, F., Cherfaoui, V.: Tracking objects using a laser scanner in driving situation based on modeling target shape. In: Intelligent Vehicles Symposium, 2007 IEEE, pp. 44–49. IEEE (2007)

    Google Scholar 

  19. Zhang, S., Adams, M., Tang, F., Xie, L.: Geometrical feature extraction using 2D range scanner. In: 4th International Conference on Control and Automation, 2003. ICCA 2003. Proceedings, pp. 901–905. IEEE (2003)

    Google Scholar 

  20. Pascoal, R., Santos, V., Premebida, C., Nunes, U.: Simultaneous segmentation and superquadrics fitting in laser-range data. IEEE Trans. Veh. Technol. 64(2), 441–452 (2015)

    Article  Google Scholar 

  21. Wang, C.C., Thorpe, C.: Simultaneous localization and mapping with detection and tracking of moving objects. In: IEEE International Conference on Robotics and Automation, 2002. Proceedings. ICRA 2002, vol. 3, pp. 2918–2924. IEEE (2002)

    Google Scholar 

  22. Vu, T.D.: Vehicle perception: localization, mapping with detection, classification and tracking of moving objects. Ph.D thesis, Institut National Polytechnique de Grenoble-INPG (2009)

    Google Scholar 

  23. Suma, E.A., Lange, B., Rizzo, A.S., Krum, D.M., Bolas, M.: FAAST: the flexible action and articulated skeleton toolkit. In: 2011 IEEE Virtual Reality Conference, pp. 247–248, March 2011

    Google Scholar 

  24. Maimone, A., Fuchs, H.: Reducing interference between multiple structured light depth sensors using motion. In: 2012 IEEE Virtual Reality Workshops (VRW), pp. 51–54, March 2012

    Google Scholar 

  25. Butler, D., Izadi, S., Hilliges, O., Molyneaux, D., Hodges, S., Kim, D., Butler, A., Molyneaux, D., Hilliges, O., Izadi, S., Hodges, S.: Shake’n’sense: reducing interference for overlapping structured light depth cameras. In: Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems, pp. 1933–1936, January 2012

    Google Scholar 

  26. Sarbolandi, H., Lefloch, D., Kolb, A.: Kinect range sensing: structured-light versus Time-of-Flight kinect. Comput. Vis. Image Underst. 139, 1–20 (2015)

    Article  Google Scholar 

  27. Martínez-Zarzuela, M., Pedraza-Hueso, M., Díaz-Pernas, F., González-Ortega, D., Antón-Rodríguez, M.: Indoor 3D Video Monitoring Using Multiple Kinect Depth-Cameras. arXiv preprint arXiv:1403.2895 (2014)

  28. Leoncini, P., Sikorski, B., Baraniello, V., Martone, F., Luongo, C., Guida, M.: Multiple NUI device approach to full body tracking for collaborative virtual environments. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10324, pp. 131–147. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60922-5_10

    Chapter  Google Scholar 

  29. Kowalski, M., Naruniec, J., Daniluk, M.: Livescan3D: a fast and inexpensive 3D data acquisition system for multiple kinect v2 sensors. In: 2015 International Conference on 3D Vision, pp. 318–325, October 2015

    Google Scholar 

  30. Blake, J., Echtler, F., Kerl, C.: libfreenect2 Project (2015). https://github.com/OpenKinect/libfreenect2

  31. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

  32. Bouguet, J.Y.: Camera calibration tool-box for matlab (2002). http://www.vision.caltech.edu/bouguetj/calib_doc/

  33. Rodehorst, V., Heinrichs, M., Hellwich, O.: Evaluation of relative pose estimation methods for multi-camera setups. In: International Archives of Photogrammetry and Remote Sensing (ISPRS 2008), pp. 135–140 (2008)

    Google Scholar 

  34. Garrote, L., Rosa, J., Paulo, J., Premebida, C., Peixoto, P., Nunes, U.J.: 3D point cloud Downsampling for 2D indoor scene modelling in mobile robotics. In: 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 228–233, April 2017

    Google Scholar 

  35. Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data: An Introduction to Cluster Analysis, vol. 344. John Wiley & Sons, Hoboken (2009)

    MATH  Google Scholar 

  36. Ester, M., Kriegel, H.P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)

    Google Scholar 

  37. Schubert, E., Sander, J., Ester, M., Kriegel, H.P., Xu, X.: Dbscan revisited, revisited: Why and how you should (still) use dbscan. ACM Trans. Database Syst. (TODS) 42(3), 19 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported by the project POCI-01-0247-FEDER-017644 HTPDIR - “Human Tracking and Perception in Dynamic Immersive Rooms” financed by the Portugal2020 program and European Union’s structural funds.

Author information

Authors and Affiliations

  1. Institute of Systems and Robotics, Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, Portugal

    Pedro Girão, João Paulo, Luís Garrote & Paulo Peixoto

Authors
  1. Pedro Girão
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Paulo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luís Garrote
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paulo Peixoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro Girão .

Editor information

Editors and Affiliations

  1. University of Salento, Lecce, Italy

    Lucio Tommaso De Paolis

  2. University of Paris-Sud, Orsay, France

    Patrick Bourdot

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Girão, P., Paulo, J., Garrote, L., Peixoto, P. (2018). Real-Time Multi-view Grid Map-Based Spatial Representation for Mixed Reality Applications. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2018. Lecture Notes in Computer Science(), vol 10850. Springer, Cham. https://doi.org/10.1007/978-3-319-95270-3_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-95270-3_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-95269-7

  • Online ISBN: 978-3-319-95270-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation