Advertisement

LumiPath – Towards Real-Time Physically-Based Rendering on Embedded Devices

  • Laura Fink
  • Sing Chun Lee
  • Jie Ying Wu
  • Xingtong Liu
  • Tianyu Song
  • Yordanka Velikova
  • Marc Stamminger
  • Nassir Navab
  • Mathias UnberathEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11768)

Abstract

With the increasing computational power of today’s workstations, real-time physically-based rendering is within reach, rapidly gaining attention across a variety of domains. These have expeditiously applied to medicine, where it is a powerful tool for intuitive 3D data visualization. Embedded devices such as optical see-through head-mounted displays (OST HMDs) have been a trend for medical augmented reality. However, leveraging the obvious benefits of physically-based rendering remains challenging on these devices because of limited computational power, memory usage, and power consumption. We navigate the compromise between device limitations and image quality to achieve reasonable rendering results by introducing a novel light field that can be sampled in real-time on embedded devices. We demonstrate its applications in medicine and discuss limitations of the proposed method. An open-source version of this project is available at https://github.com/lorafib/LumiPath which provides full insight on implementation and exemplary demonstrational material.

Keywords

Light field Fibonacci Augmented reality 

Notes

Acknowledgements

The Titan V used for this research was donated by the Nvidia Corporation. The authors would like to thank Benjamin Keinert for helping to understand the inverse Fibonacci mapping and Arian Mehrfard for his help in acquiring screenshots.

Supplementary material

Supplementary material 1 (mp4 60444 KB)

References

  1. 1.
    Camahort, E., Lerios, A., Fussell, D.: Uniformly sampled light fields. In: Drettakis, G., Max, N. (eds.) Rendering Techniques 1998, pp. 117–130. Springer, Vienna (1998).  https://doi.org/10.1007/978-3-7091-6453-2_11CrossRefGoogle Scholar
  2. 2.
    Coretti, M.: Study and implementation of a decomposable virtual skull in 54 anatomically correct elements. ThesisGoogle Scholar
  3. 3.
    Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996, pp. 43–54. ACM, New York (1996)Google Scholar
  4. 4.
    Hajek, J., et al.: Closing the calibration loop: an inside-out-tracking paradigm for augmented reality in orthopedic surgery. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 299–306. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00937-3_35CrossRefGoogle Scholar
  5. 5.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR, pp. 5967–5976. IEEE (2017)Google Scholar
  6. 6.
    Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. arXiv preprint arXiv:1807.00734 (2018)
  7. 7.
    Keinert, B., Innmann, M., Sänger, M., Stamminger, M.: Spherical fibonacci mapping. ACM Trans. Graph. 34(6), 193:1–193:7 (2015)CrossRefGoogle Scholar
  8. 8.
    Kramida, G.: Resolving the vergence-accommodation conflict in head-mounted displays. IEEE Trans. Vis. Comput. Graphics 22(7), 1912–1931 (2015)CrossRefGoogle Scholar
  9. 9.
    Marques, R., Bouville, C., Ribardière, M., Santos, L.P., Bouatouch, K.: Spherical Fibonacci point sets for illumination integrals. Comput, Graph. Forum 32, 134–143 (2013)CrossRefGoogle Scholar
  10. 10.
    McMillan, L., Bishop, G.: Plenoptic modeling: an image-based rendering system. In: SIGGRAPH, pp. 39–46. Citeseer (1995)Google Scholar
  11. 11.
    Pharr, M., Jakob, W., Humphreys, G.: Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann, Burlington (2016)Google Scholar
  12. 12.
    Rademacher, P.: Ray tracing: graphics for the masses. XRDS 3(4), 3–7 (1997)CrossRefGoogle Scholar
  13. 13.
    Sampat, M.P., Wang, Z., Gupta, S., Bovik, A.C., Markey, M.K.: Complex wavelet structural similarity: a new image similarity index. IEEE Trans. Image Process. 18(11), 2385–2401 (2009)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Zhou, S.K., Engel, K.: Method and system for volume rendering based on 3D image filtering and real-time cinematic rendering. US Patent 9,984,493 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Laura Fink
    • 1
    • 2
  • Sing Chun Lee
    • 1
  • Jie Ying Wu
    • 1
  • Xingtong Liu
    • 1
  • Tianyu Song
    • 1
  • Yordanka Velikova
    • 1
  • Marc Stamminger
    • 2
  • Nassir Navab
    • 1
  • Mathias Unberath
    • 1
    Email author
  1. 1.Johns Hopkins UniversityBaltimoreUSA
  2. 2.Friedrich-Alexander University Erlangen-NurembergErlangenGermany

Personalised recommendations