Abstract
The task of teleoperating a robot over a wireless video link is known to be very difficult. Teleoperation becomes even more difficult when the robot is surrounded by dense obstacles, or speed requirements are high, or video quality is poor, or wireless links are subject to latency. Due to high quality lidar data, and improvements in computing and video compression, virtualized reality has the capacity to dramatically improve teleoperation performance – even in high speed situations that were formerly impossible. In this paper, we demonstrate the conversion of dense geometry and appearance data, generated on-the-move by a mobile robot, into a photorealistic rendering database that gives the user a synthetic exterior line-of-sight view of the robot including the context of its surrounding terrain. This technique converts remote teleoperation into line-of-sight remote control. The underlying metrically consistent environment model also introduces the capacity to remove latency and enhance video compression. Display quality is sufficiently high that the user experience is similar to driving a video game where the surfaces used are textured with live video.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chatila, R., Lacroix, S., Simion, T., Herrb, M.: Planetary exploration by a mobile robot: mission teleprogramming and autonomous navigation. Autonomous Robots (1995)
Chen, S.E., Williams, L.: View interpolation for image synthesis. In: Proc. SIGGRAPH 1993, pp. 279–288 (1993)
Fong, T., Thorpe, C., Baur, C.: Advanced Interfaces for Vehicle Teleoperation: Collaborative Control, Sensor Fusion Displays, and Remote Driving Tools. Autonomous Robots 11, 77–85 (2001)
Fong, T., Pangels, H., Wettergreen, D.: Operator Interfaces and Network-Based Participation for Dante II. In: SAE 25th International Conference on Environmental Systems, San Diego, CA (July 1995)
Hine, B., Stoker, C., Sims, M., Rasmussen, D., Hontalas, P.: The Application of Telepresence and Virtual Reality to Subsea Exploration. In: Proceedings ROV 1994, the 2nd Workshop on Mobile Robots for Subsea Environments, Monterey, CA (May 1994)
Hine, B., Hontalas, P., Fong, T., Piguet, L., Nygren, E., Kline, A.: VEVI: A Virtual Environment Teleoperations Interface for Planetary Exploration. In: SAE 25th International Conference on Environmental Systems, San Diego, CA (July 1995)
Kanade, T., Rander, P., Narayanan, P.J.: Virtualized Reality: Constructing Virtual Worlds from Real Scenes. In: IEEE MultiMedia, vol. 4(1). IEEE Computer Society Press, Los Alamitos (1997)
Kelly, A., Stentz, A., Amidi, O., Bode, M., Bradley, D., Diaz-Calderon, A., Happold, M., Herman, H., Mandelbaum, R., Pilarski, T., Rander, P., Thayer, S., Vallidis, N., Warner, R.: Toward Reliable Off-Road Autonomous Vehicles Operating in Challenging Environments. The International Journal of Robotics Research 25(5-6), 449–483 (2006)
Kelly, A., Anderson, D., Capstick, E., Herman, H., Rander, P.: Photogeometric Sensing for Mobile Robot Control and Visualization Tasks. In: Symposium on New Frontiers in Human-Robot Interaction, Edinburgh, Scotland, April 8-9 (2009)
Kim, W.S.: Virtual Reality Calibration and Preview / Predictive Displays for Telerobotics. Presence: Teleoperators and Virtual Environments 5, 2, 173–190 (1996)
Matusik, W., Beuhler, C., Raskar, R., Gortler, S., McMillan, L.: Image-Based Visual Hulls. In: SIGGRAPH 2000, pp. 369–374 (July 2000)
Messuri, D.A., Klein, C.A.: Automatic Body Regulation for Maintaining Stability of a Legged Vehicle during Terrain Locomotion. IEEE Journal of Robotics and Automation, RA 1, 132–141 (1985)
Milgram, P., Yin, S., Grodski, J.: An Augmented Reality Based Teleoperation Interface For Unstructured Environments. In: Proc. American Nuclear Society (ANS) 7th Topical Meeting on Robotics and Remote Systems Augusta, Georgia, USA, April 27-May 1, pp. 966–973 (1997)
Rohlf, J., Helman, J.: IRIS Performer: A high performance multiprocessing toolkit for real-time 3D graphics. In: Glassner, A. (ed.) Proceedings of SIGGRAPH 1994, pp. 381–394 (1994)
Segal, M., Korobkin, C., van Widenfelt, R., Foran, J., aeberli, P.: Fast shadows and lighting effects using texture mapping. In: Proceedings of SIGGRAPH 1992, pp. 249–252 (1992)
Sheridan, T.: Human Supervisory Control of Robot Systems. In: Proceedings of the 1986 IEEE International Conference on Robotics and Automation (April 1986)
Stewart, B., Ko, J., Fox, D., Konolige, K.: The Revisiting Problem in Mobile Robot Map Building: A Hierarchical Bayesian Approach. In: Proc. of the Conference on Uncertainty in Artificial Intelligence (2003)
Terrien, G., Fong, T., Thorpe, C., Baur, C.: Remote driving with a multisensor user interface. In: Proceedings of the SAE ICES, Toulouse, France (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kelly, A., Capstick, E., Huber, D., Herman, H., Rander, P., Warner, R. (2011). Real-Time Photorealistic Virtualized Reality Interface for Remote Mobile Robot Control. In: Pradalier, C., Siegwart, R., Hirzinger, G. (eds) Robotics Research. Springer Tracts in Advanced Robotics, vol 70. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19457-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-19457-3_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19456-6
Online ISBN: 978-3-642-19457-3
eBook Packages: EngineeringEngineering (R0)