Journal of Intelligent & Robotic Systems

, Volume 83, Issue 3–4, pp 359–373 | Cite as

A Framework for Augmented Reality using Non-Central Catadioptric Cameras

  • Tiago Dias
  • Pedro Miraldo
  • Nuno Gonçalves


This paper addresses the problem of augmented reality on images acquired from non-central catadioptric systems. We propose a solution which allows the projection of textured objects to images of these type of systems and, depending on the complexity of the objects, can run up to 20 fps, using a 1328×1048 image resolution. The main contributions are related with the image formation of the non-central catadioptric cameras: projection of the 3D segments onto the image of non-central catadioptric cameras; occlusions; and illumination/shading. To validate the proposed solution, we used a non-central catadioptric camera formed with a perspective camera and a spherical mirror. Also, to test the robustness of the proposed method, we used a regular object (a parallelepiped) and three well known irregular objects in computer graphics: “bunny”, “happy buddha” and “dragon”, from Stanford database.


Augmented reality Non-central catadioptric cameras Forward-projection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

10846_2016_349_MOESM1_ESM.avi (2.8 mb)
(AVI 2.84 MB)


  1. 1.
    Appel, A.: Some techniques for shading machine renderings of solids. In: Proceeding of the AFIPS (1968)Google Scholar
  2. 2.
    Azuma, R.T.: A survey of augmented reality. In: Presence: Teleoperators and virtual environments: MIT Press Journal (1997)Google Scholar
  3. 3.
    Fournier, A., Gunawan, A.S., Romanzin, C.: Common illumination between real and computer generated scenes. In: Proceeding of Graphics Interface (GI’93) (1993)Google Scholar
  4. 4.
    Sato, I., Sato, Y., Ikeuchi, K.: Acquiring a radiance distribution to superimpose virtual objects onto a real scene. IEEE Trans. Vis. Comput. Graph. 5(1), 1–12 (1999)CrossRefGoogle Scholar
  5. 5.
    Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. ACM SIGGRAPH 2008 classes. ACM, p. 32 (2008)Google Scholar
  6. 6.
    Santos, A.L., Lemos, D., Lindoso, J.E.F., Teichrieb, V.: Real time ray tracing for augmented reality. IEEE Symposium on Virtual and Augmented Reality (SVR) (2012)Google Scholar
  7. 7.
    Grossberg, M.D., Nayar, S.K.: A general imaging model and a method for finding its parameters. In: IEEE Proceedings of the International Conference on Computer Vision (ICCV) (2003)Google Scholar
  8. 8.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000)Google Scholar
  9. 9.
    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: A perspective on distortions. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2003)Google Scholar
  10. 10.
    Nalwa, V.S.: A True Omni-Directional Viewer. Technichal report, Bell Laboratories (1996)Google Scholar
  11. 11.
    Nayar, S.K., Baker, S.: Catadioptric image formation. In: DARPA Proceedings of the Image Understanding Workshop (1997)Google Scholar
  12. 12.
    Baker, S., Nayar, S.K.: A theory of single-viewpoint catadioptric image formation. Int. J. Comput. Vis. 35(2), 175–196 (1999)CrossRefGoogle Scholar
  13. 13.
    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: Caustics of catadioptric cameras. In: IEEE Proceedings of the International Conference on Computer Vision (ICCV) (2001)Google Scholar
  14. 14.
    Micusik, B., Pajdla, T.: Autocalibration & 3D reconstruction with non-central catadioptric cameras. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2004)Google Scholar
  15. 15.
    Gonçalves, N.: Noncentral Catadioptric Systems with Quadric Mirrors: Geometry and Calibration. Ph.D. dissertation, University of Coimbra (2008)Google Scholar
  16. 16.
    Perdigoto, L., Araujo, H.: Calibration of mirror position and extrinsic parameters in axial non-central catadioptric systems. Comput. Vis. Image Underst. 117(8), 909–921 (2013)CrossRefGoogle Scholar
  17. 17.
    Agrawal, A., Ramalingam, S.: Single image calibration of multi-axial imaging systems. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  18. 18.
    Goodrich, M.A., Schultz, A.C.: Human-Robot Interaction: a Survey. Foundations and Trends in Human-Computer Interaction (2007)Google Scholar
  19. 19.
    Chintamani, K., Cao, A., Ellis, R.D., Pandya, A.K.: Improved telemanipulator navigation during display-control misalignments using augmented reality cues. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 40(1), 29–39 (2010)CrossRefGoogle Scholar
  20. 20.
    Chen, I. Y.-H., MacDonald, B., Wunsche, B.: Mixed reality simulation for mobile robots. In: IEEE Proceedings of the International Conference on Robotics and Automation (ICRA) (2009)Google Scholar
  21. 21.
    Fuchs, H., Livingston, M.A., Raskar, R., Colucci, D., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., Meyer, A.A.: Augmented reality visualization for laparoscopic surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) (2009)Google Scholar
  22. 22.
    Gonçalves, N.: On the reflection point where light reflects to a known destination in quadric surfaces. Opt. Lett. 35(2), 100–102 (2010)CrossRefGoogle Scholar
  23. 23.
    Agrawal, A., Taguchi, Y., Ramalingam, S.: Beyond alhazen’ Problem: Analytical Projection Model for Non-Central Catadioptric Cameras with Quadric Mirrors. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  24. 24.
    Hughes, J., Dam, A.V., Mcguire, M., Skylar, D.F., Foley, J.D., Feiner, S.K., Akeley, K.: Computer Graphics: Principles and Practice, 3rd Ed. Addison-Wesley (2014)Google Scholar
  25. 25.
    Carpenter, L.: The A-buffer, an antialiased hidden surface method. ACM SIGGRAPH 18(3), 103–108 (1984)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Gouraud, H.: Continuous shading of curved surfaces. IEEE Trans. Comput. 100(6), 623–629 (1971)CrossRefMATHGoogle Scholar
  27. 27.
    Phong, B.T.: Illumination for Computer Generated Pictures. Commun. ACM 18(6), 311–317 (1975)CrossRefGoogle Scholar
  28. 28.
    Stanford University Computer Graphics Laboratory, Stanford Bunny, (1993)
  29. 29.
    Delaunay, B.: Sur la sphere vide. Izv. Akad. Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk 7(793-800), 1–2 (1934)MATHGoogle Scholar
  30. 30.
    Chen, C.-S., Chang, W.-Y.: On pose recovery for generalized visual sensors. IEEE Trans. Pattern Anal. Mach. Intell. 26(7), 848–861 (2004)CrossRefGoogle Scholar
  31. 31.
    Schweighofer, G., Pinz, A.: Globally Optimal O(n) Solution to the PnP Problem for General Camera Models. In: Proceedings of the British Machine Vision Conference (BMVC) (2008)Google Scholar
  32. 32.
    Miraldo, P., Araujo, H.: Pose estimation for non-central cameras using planes. In: IEEE International Conference on Autonomous Robot Systems & Competitions –ROBÓTICA (2014)Google Scholar
  33. 33.
    Miraldo, P., Araujo, H.: Planar pose estimation for general cameras using known 3D lines. In: IEEE/RSJ Proceedings of the International Conference on Intelligent Robots & Systems (IROS) (2014)Google Scholar
  34. 34.

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.Institute of Systems and Robotics, Department of Electrical and Computer EngineeringUniversity of CoimbraCoimbraPortugal
  2. 2.Institute for Systems and Robotics (LARSyS), Instituto Superior TécnicoUniversidade de LisboaLisboaPortugal

Personalised recommendations