Abstract
In this work, we propose a real-time foveated ray tracing system, which mimics the non-uniform and sparse characteristic of the human retina to reduce spatial sampling. Fewer primary rays are traced in the peripheral regions of vision, while sampling frequency for the fovea region traced by the eye tracker is maximised. Our GPU-accelerated ray tracer uses a sampling mask to generate a non-uniformly distributed set of pixels. Then, the regular Cartesian image is reconstructed based on the GPU-accelerated triangulation method with the barycentric interpolation. The temporal anti-aliasing is applied to reduce the flickering artefacts. We perform a user study in which people evaluate the visibility of artefacts in the peripheral region of vision where sampling is reduced. This evaluation is conducted for a number of sampling masks that mimic the sensitivity to contrast in the human eyes but also test different sampling strategies. The sampling that follows the gaze-dependent contrast sensitivity function is reported to generate images of the best quality. We test the performance of the whole system on the VR headset. The achieved frame-rate is twice higher in comparison to the typical Cartesian sampling and cause only barely visible degradation of the image quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Advanced Micro Devices, Inc.: Radeon-rays library, version 2.0 (2016). http://gpuopen.com/gaming-product/radeon-rays/
Akenine-Möller, T., Haines, E., Hoffman, N.: Real-Time Rendering, 3rd edn. A. K. Peters Ltd., Natick (2008)
Chwesiuk, M., Mantiuk, R.: Measurements of contrast detection thresholds for peripheral vision using non-flashing stimuli. In: Czarnowski, I., Howlett, R.J., Jain, L.C. (eds.) IDT 2017. SIST, vol. 73, pp. 258–267. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-59424-8_24
V Corporation: Openvr library, version 1.0.10 (2017). https://github.com/ValveSoftware/openvr
Fujita, M., Harada, T.: Foveated real-time ray tracing for virtual reality headset. Technical report, Light Transport Entertainment Research (2014)
Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 43–54. ACM (1996)
Guenter, B., Finch, M., Drucker, S., Tan, D., Snyder, J.: Foveated 3d graphics. ACM Trans. Graph. 31(6), 164:1–164:10 (2012)
Hunt, W.: Virtual reality: the next great graphics revolution. Keynote Talk HPG (2015)
Levoy, M., Whitaker, R.: Gaze-directed volume rendering. ACM SIGGRAPH Comput. Graph. 24(2), 217–223 (1990)
Loschky, L., McConkie, G., Yang, J., Miller, M.: The limits of visual resolution in natural scene viewing. Vis. Cogn. 12(6), 1057–1092 (2005)
Mantiuk, R., Bazyluk, B., Mantiuk, R.K.: Gaze-driven object tracking for real time rendering. Comput. Graph. Forum 32(2), 163–173 (2013)
Mantiuk, R.K., Tomaszewska, A., Mantiuk, R.: Comparison of four subjective methods for image quality assessment. Comput. Graph. Forum 31(8), 2478–2491 (2012)
Murphy, H.A., Duchowski, A.T., Tyrrell, R.A.: Hybrid image/model-based gaze-contingent rendering. ACM Trans. Appl. Percept. (TAP) 5(4), 22 (2009)
Ohshima, T., Yamamoto, H., Tamura, H.: Gaze-directed adaptive rendering for interacting with virtual space. In: 1996 Proceedings of the IEEE Conference on Virtual Reality Annual International Symposium, pp. 103–110. IEEE (1996)
Palmer, S.E.: Vision Science: Photons to Phenomenology, vol. 1. MIT Press, Cambridge (1999)
Patney, A., et al.: Perceptually-based foveated virtual reality. In: ACM SIGGRAPH 2016 Emerging Technologies, p. 17. ACM (2016)
Peli, E., Yang, J., Goldstein, R.B.: Image invariance with changes in size: the role of peripheral contrast thresholds. JOSA A 8(11), 1762–1774 (1991)
Roth, T., Weier, M., Hinkenjann, A., Li, Y., Slusallek, P.: An analysis of eye-tracking data in foveated ray tracing. In: IEEE Second Workshop on Eye Tracking and Visualization (ETVIS), pp. 69–73. IEEE (2016)
Siekawa, A.: Gaze-dependent ray tracing. In: Proceedings of CESCG 2014: The 18th Central European Seminar on Computer Graphics (Non-peer-reviewed) (2014)
Siekawa, A.: Image reconstruction from spatially non-uniform samples. In: Proceedings of CESCG 2017: The 21th Central European Seminar on Computer Graphics (Non-peer-reviewed) (2017)
Stengel, M., Magnor, M.: Gaze-contingent computational displays: boosting perceptual fidelity. IEEE Sig. Process. Mag. 33(5), 139–148 (2016)
Vaidyanathan, K., et al.: Coarse pixel shading. In: Proceedings of High Performance Graphics, pp. 9–18. Eurographics Association (2014)
Wandell, B.A.: Foundations of Vision, vol. 8. Sinauer Associates, Sunderland (1995)
Watson, B., Walker, N., Hodges, L.F., Worden, A.: Managing level of detail through peripheral degradation: effects on search performance with a head-mounted display. ACM Trans. Comput.-Hum. Interact. (TOCHI) 4(4), 323–346 (1997)
Weier, M., et al.: Perception-driven accelerated rendering. Comput. Graph. Forum 36(2), 611–643 (2017)
Acknowledgments
The project was funded by the Polish National Science Centre (decision number DEC-2013/09/B/ST6/02270).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Siekawa, A., Chwesiuk, M., Mantiuk, R., Piórkowski, R. (2019). Foveated Ray Tracing for VR Headsets. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, WH., Vrochidis, S. (eds) MultiMedia Modeling. MMM 2019. Lecture Notes in Computer Science(), vol 11295. Springer, Cham. https://doi.org/10.1007/978-3-030-05710-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-05710-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05709-1
Online ISBN: 978-3-030-05710-7
eBook Packages: Computer ScienceComputer Science (R0)