Real-Time Motion Capture on a Budget
The U.S. Army Research Laboratory’s Simulation & Training Technology Center, along with Cole Engineering Services, Inc. and the University of Central Florida have set out to leverage commercial technology with the goal of improving realism, and reducing cost for Army training tasks. The focus of this task is to establish a prototype functionality that allows a live person to take control of a virtual character. This is done using the Enhanced Dynamic Geo-Social Environment, which is an Army-owned simulation built upon the Unreal Engine 4.
Commercial games and movies make use of motion capture capabilities to animate characters. This functionality is needed in real-time to allow person-to-person interactions within a simulation. The goal is to have puppeteers that can take over Artificial Intelligence (AI) characters when in-depth interactions need to occur. While AAA games and movie budgets allow for more expensive systems, the goal of this team is to keep the cost well below $10,000.
A market analysis along with this team’s experience utilizing and integrating the market capabilities to meet these goals are described in this paper.
KeywordsVirtual training Avatar puppeteering Virtual humans Real-time motion capture
The ARL STTC, Cole Engineering Services, Inc., and the Institute for Simulation and Training and the University of Central Florida would like to express our appreciation for the support of the Defense Equal Opportunity Management Institute for resources and expertise provided to this technology area.
- 1.Griffith, T., Dwyer, T., Kinard, C., Flynn, J.R., Kirazian, V.: Research on the use of puppeteering to improve realism in army simulations and training games. In: Lackey, S., Shumaker, R. (eds.) VAMR 2016. LNCS, vol. 9740, pp. 386–396. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39907-2_37CrossRefGoogle Scholar
- 2.Bailey, K.: http://www.techcrunch.com, https://techcrunch.com/2017/02/25/conversational-ai-and-the-road-ahead/. Accessed 25 Feb 2017
- 3.Merritt, C.: http://www.towardsdatascience.com, https://towardsdatascience.com/a-driverless-world-c3baea61734b. Accessed 27 Sept 2017
- 4.Summers, N.: The real-time motion capture behind ‘Hellblade’. How a tiny team in Cambridge, England, brought Senua to life. https://www.engadget.com/2017/08/08/ninja-theory-hellblade-motion-capture-demo-video/. Accessed 8 Aug 2017
- 5.Peel, J.: www.pcgamesn.com, https://www.pcgamesn.com/hellblade-senuas-sacrifice/hellblade-unreal-engine-4. Accessed 27 Nov 2017
- 6.Boyle, E.: Tech Crunch. http://www.techradar.com/news/this-is-the-tech-that-will-make-games-look-better-and-cost-less. Accessed 8 Aug 2017
- 7.Souppouris, A.: http://www.Engadget.com, https://www.engadget.com/2015/09/16/ninja-theory-hellblade-indie-game. Accessed 16 Sept 2015
- 8.Hanson, B.: www.gameinformer.com, http://www.gameinformer.com/b/features/archive/2017/04/12/watch-us-push-hellblades-mocap-process-to-its-limits.aspx. Accessed 12 Apr 2017
- 9.Ninja Theory: Hellblade: Diary 21 - Making a Virtual Human | Real-time performance capture. https://www.youtube.com/watch?v=CmrXK4fNOEo&index=21&list=PLbpkF8TRYizaT6GfMcKBG-RoUOQ6BJRXp. Accessed 17 Mar 2016
- 11.Griffith, T.S., Ablanedo, J., Nenneman, M.: Taking it to the EDGE. In: Proceedings of the International Defense and Homeland Security Simulation Workshop, Bergeggi, Italy (2015)Google Scholar
- 12.Moore, M.E., Novak, J.: Game Industry Career Guide. Cengage Learning, Delmar (2010)Google Scholar
- 13.Glanville, S.: Object editor. In: Anim8 or Manual (2006)Google Scholar
- 14.Brier, D.: E3: New info on Microsoft’s Natal – how it works, multiplayer and PC versions. http://old.seattletimes.com/html/technologbrierdudleysblog/2009296568_e3_new_info_on_microsofts_nata.html. Accessed 3 June 2009
- 15.Marquand, R.: Star Wars Episode VI: Return of the Jedi; George Lucas Commentary (Film). 20th Century Fox (2004)Google Scholar
- 16.da Silva, D.B., Vidal, C.A., Cavalcante-Neto, J.B., Pessoa, I.N.S., Nunes, R.F.: A robust balance strategy applied to real-time animation data with Kinect sensor. In: 2017 19th Symposium on Virtual and Augmented Reality (SVR), pp. 8–17 (2017)Google Scholar
- 17.Originating Technology/NASA Contribution: Inertial Motion-Tracking Technology for Virtual 3-D (2005). https://spinoff.nasa.gov/Spinoff2005/ch_10.html
- 18.Baraniuk, C.: Faceshift: Apple buys Star Wars motion-capture company. BBC News. http://www.bbc.com/news/technology-34920548. Accessed 25 Nov 2015
- 20.McAlone, N.: See the amazing tech behind the original Jabba the Hutt puppet - which managed to fit 3 people inside of it. http://www.businessinsider.com/the-tech-of-the-jabba-the-hutt-puppet-2015–11. Accessed 10 Nov 2015
- 21.Ninja Theory: Hellblade Development Diary 5: Business of Creation (2014). https://www.youtube.com/watch?v=_f3NMMrQFEE