Occlusion Handling in Video-Based Augmented Reality Using the Kinect Sensor for Indoor Registration

  • Jesus Adrián Leal-Meléndrez
  • Leopoldo Altamirano-Robles
  • Jesus A. Gonzalez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8259)


Video-based Augmented Reality (VAR) aims to add 3D virtual objects (3D VOs) to a real world video sequence, in order to provide additional and useful information to facilitate some tasks, like computer aided surgery, simulation in a real environment, satellite positioning, interior design, among others. To achieve a consistent and convincing augmented scene, it is necessary that the VOs are properly occluded by real objects (Occlusion Problem in VAR); in this paper, we present a strategy based on the use of the Kinect sensor to solve this problem. In the occlusion stage we evaluate distances between real and VOs. Then, the parts of the VO occluded by a real object are calculated and removed. We found that the Kinect sensor is appropriate to be used for handling occlusions in indoor environments, dynamic scenarios and real-time applications. Experiments showed comparable results with the state of the art in both issues: occlusion handling and processing time.


occlusion handling video based augmented reality hidden surface removal kinect 


  1. 1.
    Dong, S., Feng, C., Kamat, V.R.: Occlusion Handling Method for Ubiquitous Augmented Reality Using Reality Capture Technology and GLSL. In: Proceedings of the 2011 ASCE International Workshop on Computing in Civil Engineering, Reston, VA, pp. 494–503 (2011)Google Scholar
  2. 2.
    Li, L., Guan, T., Ren, B.: Resolving occlusion between virtual and real scenes for augmented reality applications. In: Proceedings of the 12th International Conference on Human-Computer Interaction: Interaction Platforms and Techniques, Beijing, China, pp. 634–642 (2007)Google Scholar
  3. 3.
    Zhu, J., Pan, Z., Sun, C., Chen, W.: Handling Occlusions in Video-Based Augmented Reality Using Depth Information. Journal of Animation and Virtual Worlds: Wiley Online Library Computer 21, 509–521 (2010)CrossRefGoogle Scholar
  4. 4.
    Zhu, J., Pan, Z.: Occlusion registration in video-based augmented reality. In: Proceedings of The 7th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry, vol. 10, pp. 1–6 (2008)Google Scholar
  5. 5.
    Telea, A.: An Image Inpainting Technique Based on the Fast Marching Method. Journal of Graphics, Gpu, and Game Tools 9, 23–34 (2004)Google Scholar
  6. 6.
    Lepetit, V., Berger, M.-O.: Handling occlusion in augmented reality systems: a semi-automatic method. In: Proceedings EEE and ACM International Symposium on Augmented Reality (ISAR 2000), pp. 137–146 (2000)Google Scholar
  7. 7.
    Fischer, J., Bartz, D., StraBer, W.: Occlusion Handling for Medical Augmented Reality using a Volumetric Phantom Model. In: Journal of VRST 2004 Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 174–177 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jesus Adrián Leal-Meléndrez
    • 1
  • Leopoldo Altamirano-Robles
    • 1
  • Jesus A. Gonzalez
    • 1
  1. 1.Computer Science DepartmentNational Institute for Astrophysics, Optics and ElectronicsPueblaMexico

Personalised recommendations