An UE4 Plugin to Develop CVE Applications Leveraging Participant’s Full Body Tracking Data
This technical paper describes a software framework that has been conceived in order to develop collaborative, immersive Virtual Reality applications with the distinctive ability to be fed by real-time full-body tracking data of the multiple collaborating participants. This feature has a key role both to improve the mutual awareness in the visually immersive CVE, and to exploit complete body poses for human-centered joint activities and related assessments. By following recent trends of VR research application development, the framework has been based on a popular game engine (Unreal Engine 4) and leverages its powerful built-in facilities for networked, multiplayer applications and for character skinning. The paper illustrates the implemented techniques for real-time size-independent mannequin animation that could be usefully applied for the same task to different game engines or VR development environments, and also describes the data management strategies devised for an efficient implementation of a CVE architecture dealing with full-body tracking data flows.
The knowledge gained during this engine-specific work will be useful for the future definition of an engine-independent communication framework for CVEs among single-user VR applications, based on heterogeneous game engine technologies, that is a strongly-felt need for promoting collaboration among VR research groups and in multi-partner industrial projects.
KeywordsCollaborative Virtual Environment Virtual reality Unreal Engine Skeletal animation Skeleton tracking Natural user interface Motion capture Motion tracking
- 1.Leoncini, P., Sikorski, B., Baraniello, V., Martone, F., Luongo, C., Guida, M.: Multiple NUI device approach to full body tracking for collaborative virtual environments. In: De Paolis, L.T., Bourdot, P., Mongelli, A. (eds.) AVR 2017. LNCS, vol. 10324, pp. 131–147. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60922-5_10CrossRefGoogle Scholar
- 2.McCaffrey, M.: Unreal Engine VR Cookbook: Developing Virtual Reality with UE4. Addison-Wesley Professional (2017). ISBN 978-01346491Google Scholar
- 3.Sherif, W., Whittle, S.: Unreal Engine 4 Scripting with C++ Cookbook. Packt Publishing (2016). ISBN 978-1785885549Google Scholar
- 4.Madathil, K.C., Greenstein, J.S.: An investigation of the efficacy of collaborative virtual reality systems for moderated remote usability testing. Elsevier (2017)Google Scholar
- 5.Sewell, B.: Blueprints Visual Scripting for Unreal Engine: Build professional 3D games with Unreal Engine 4’s Visual Scripting system. Packt Publishing (2015). ISBN 978-1785286018Google Scholar
- 6.Satheesh, P.V.: Unreal Engine 4 Game Development Essentials. Packt Publishing (2016). ISBN 978-1784391966Google Scholar
- 7.The Open Sound Control Protocol. http://opensoundcontrol.org/introduction-osc
- 8.TPCAST. https://www.tpcastvr.com
- 9.Unreal Engine 4 Programming Guide. https://docs.unrealengine.com/en-US/Programming