Data-Driven Hand Animation Synthesis

Reference work entry
First Online:

Abstract

As virtual characters are becoming more and more realistic, the need for recording and synthesizing detailed animations for their hands is increasing. Whether we watch virtual characters in a movie, communicate with an embodied conversational agent in real time, or steer an agent ourselves in a virtual reality application or in a game, detailed hand motions have an impact on how we perceive the character. In this chapter, we give an overview of current methods to record and synthesize the subtleties of hand and finger motions. The approaches we present include marker-based and markerless optical systems, depth sensors, and sensored gloves to capture and record hand motions and data-driven algorithms to synthesize movements when only the body or arm motions are known. We furthermore describe the complex anatomy of the hand and how it is being simplified and give insights on our perception of hand motions to convey why creating realistic hand motions is challenging.

Keywords

Hand motions Fingers Character animation Data-driven animation Virtual characters Motion capture 
This is a preview of subscription content, log in to check access

References

  1. Andrews S, Jarvis M, Kry PG (2013) Data-driven fingertip appearance for interactive hand simulation. In: Proceedings of motion on games, MIG ‘13, Dublin, pp 155:177–155:186Google Scholar
  2. Argelaguet F, Hoyet L, Trico M, Lecuyer A (2016) The role of interaction in virtual embodiment: effects of the virtual hand representation. In: IEEE virtual reality (VR), Greenville, pp 3–10Google Scholar
  3. Aydin Y, Nakajima M (1999) Database guided computer animation of human grasping using forward and inverse kinematics. Comput Graph 23(1):145–154.  https://doi.org/10.1016/S0097-8493(98)00122-8CrossRefGoogle Scholar
  4. Braido P, Zhang X (2004) Quantitative analysis of finger motion coordination in hand manipulative and gestic acts. Hum Mov Sci 22(6):661–678.  https://doi.org/10.1016/j.humov.2003.10.001CrossRefGoogle Scholar
  5. Chaminade T, Hodgins J, Kawato M (2009) Anthropomorphism influences perception of computer-animated characters’ actions. Soc Cogn Affect Neurosci 2(3):206–216CrossRefGoogle Scholar
  6. Ciocarlie M, Goldfeder C, Goldfeder C (2007) Dimensionality reduction for hand-independent dexterous robotic grasping. In: IEEE/RSJ international conference on intelligent robots and systems, IROS 2007, San Diego, pp 3270–3275Google Scholar
  7. Cutting J, Kozlowski L (1977) Recognizing friends by their walk: gait perception without familiarity cues. Bull Psychon Soc 9(5):353–356CrossRefGoogle Scholar
  8. de La Gorce M, Paragios N, Fleet DJ (2008) Model-based hand tracking with texture, shading and self-occlusions. In: IEEE conference on computer vision and pattern recognition, Anchorage, pp 1–8Google Scholar
  9. Dipietro L, Sabatini A, Dario P (2008) A survey of glove-based systems and their applications. IEEE Trans Syst Man Cybern Part C Appl Rev 38(4):461–482CrossRefGoogle Scholar
  10. Ebrahimi E, Babu SV, Pagano CC, Jörg S (2016) An empirical evaluation of visuo-haptic feedback on physical reaching behaviors during 3D interaction in real and immersive virtual environments. ACM Trans Appl Percept 13(4):19:1–19:21Google Scholar
  11. ElKoura G, Singh K (2003) Handrix: animating the human hand. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation, San Diego, pp 110–119Google Scholar
  12. Häger-Ross C, Schieber MH (2000) Quantifying the independence of human finger movements: comparisons of digits, hands, and movement frequencies. J Neurosci 20(22):8542–8550Google Scholar
  13. Hoyet L, Ryall K, McDonnell R, O’Sullivan C (2012) Sleight of hand: perception of finger motion from reduced marker sets. In: Proceedings of the ACM SIGGRAPH symposium on interactive 3D graphics and games, I3D ‘12, Costa Mesa, pp 79–86Google Scholar
  14. Huenerfauth M, Lu P (2010) Accurate and accessible motion-capture glove calibration for sign language data collection. ACM Trans Access Comput 3(1):2:1–2:32CrossRefGoogle Scholar
  15. Jörg S (2011) Perception of body and hand animations for realistic virtual characters. Ph thesis, University of Dublin, Trinity College, DublinGoogle Scholar
  16. Jörg S, O’Sullivan C (2009) Exploring the dimensionality of finger motion. In: Proceedings of the 9th Eurographics Ireland workshop (EGIE 2009), Dublin, pp 1–11Google Scholar
  17. Jörg S, Hodgins J, O’Sullivan C (2010) The perception of finger motions. In: Proceedings of the 7th symposium on applied perception in graphics and visualization (APGV 2010), Los Angeles, pp 129–133Google Scholar
  18. Jörg S, Hodgins JK, Safonova A (2012) Data-driven finger motion synthesis for gesturing characters. ACM Trans Graph 31(6):189:1–189:7CrossRefGoogle Scholar
  19. Kahlesz F, Zachmann G, Klein R (2004) Visual-fidelity dataglove calibration. In: Computer graphics international. IEEE Computer Society, Crete, pp 403–410Google Scholar
  20. Kang C, Wheatland N, Neff M, Zordan V (2012) Automatic hand-over animation for free-hand motions from low resolution input. In: Motion in games. Lecture notes in computer science, vol 7660. Springer, Berlin/Heidelberg, pp 244–253CrossRefGoogle Scholar
  21. Kendon A (2004) Gesture – visible action as utterance. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  22. Kitagawa M, Windsor B (2008) MoCap for artists: workflow and techniques for motion capture. Focal Press, Amsterdam/BostonGoogle Scholar
  23. Kozlowski LT, Cutting JE (1977) Recognizing the sex of a walker from a dynamic point-light display. Percept Psychophys 21(6):575–580CrossRefGoogle Scholar
  24. Kry PG, Pai DK (2006) Interaction capture and synthesis. ACM Trans Graph 25(3):872–880CrossRefGoogle Scholar
  25. Li P, Kry PG (2014) Multi-layer skin simulation with adaptive constraints. In: Proceedings of the 7th international conference on motion in games, MIG ‘14, Playa Vista, pp 171–176Google Scholar
  26. Lin L, Jörg S (2016) Need a hand?: how appearance affects the virtual hand illusion. In: Proceedings of the ACM symposium on applied perception, SAP ‘16, Anaheim, pp 69–76Google Scholar
  27. Liu CK (2008) Synthesis of interactive hand manipulation. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation, Dublin, pp 163–171Google Scholar
  28. Liu CK (2009) Dextrous manipulation from a grasping pose. ACM Trans Graph 28(3):3:1–3:6Google Scholar
  29. Lu P, Huenerfauth M (2009) Accessible motion-capture glove calibration protocol for recording sign language data from deaf subjects. In: Proceedings of the 11th international ACM SIGAC-CESS conference on computers and accessibility, pp 83–90Google Scholar
  30. Ma K, Hommel B (2015a) Body-ownership for actively operated non-corporeal objects. Conscious Cogn 36:75–86CrossRefGoogle Scholar
  31. Ma K, Hommel B (2015b) The role of agency for perceived ownership in the virtual hand illusion. Conscious Cogn 36:277–288CrossRefGoogle Scholar
  32. Majkowska A, Zordan VB, Faloutsos P (2006) Automatic splicing for hand and body animations. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation. Boston, MA, USA, pp 309–316Google Scholar
  33. McNeill D (1992) Hand and mind: what gestures reveal about thought. The University of Chicago Press, ChicagoGoogle Scholar
  34. Menache A (1999) Understanding motion capture for computer animation and video games. Morgan Kaufmann Publishers Inc., San FranciscoGoogle Scholar
  35. Mousas C, Newbury P, Anagnostopoulos CN (2014) Efficient hand-over motion reconstruction. In: Proceedings of the 22nd international conference in Central Europe on computer graphics, visualization and computer vision, WSCG ‘14. Plzen, Czech Republic, pp 111–120Google Scholar
  36. Mousas C, Anagnostopoulos CN, Newbury P (2015) Finger motion estimation and synthesis for gesturing characters. In: Proceedings of the 31st spring conference on computer graphics, SCCG ‘15. Smolenice, Slovakia, pp 97–104Google Scholar
  37. Napier J (1980) Hands. Pantheon Books, New YorkGoogle Scholar
  38. Neff M, Seidel HP (2006) Modeling relaxed hand shape for character animation. In: Articulated Motion and deformable objects. Lecture notes in computer science, vol 4069. Springer, Berlin/Heidelberg, pp 262–270Google Scholar
  39. Oshita M, Senju Y (2014) Generating hand motion from body motion using key hand poses. In: Proceedings of the 7th international conference on motion in games, MIG ‘14. Playa Vista, CA, USA, pp 147–151Google Scholar
  40. Palastanga N, Soames R (2012) Anatomy and human movement – structure and function, 6th edn. Butterworth Heinemann/Elsevier, Edinburgh/New YorkGoogle Scholar
  41. Parent R (2012) Computer animation: algorithms and techniques, 3rd edn. Morgan Kaufmann, BurlingtonGoogle Scholar
  42. Perani D, Fazio F, Borghese NA, Tettamanti M, Ferrari S, Decety J, Gilardi MC (2001) Different brain correlates for watching real and virtual hand actions. Neuroimage 14:749–758CrossRefGoogle Scholar
  43. Pollard NS, Zordan VB (2005) Physically based grasping control from example. In: Proceedings of the ACM SIGGRAPH/Eurographics symposium on computer animation. Los Angeles, CA, USA, pp 311–318Google Scholar
  44. Prachyabrued M, Borst CW (2014) Visual feedback for virtual grasping. In: IEEE symposium on 3D User Interfaces, 3DUI, 2014. Minneapolis, MN, USA, pp 19–26Google Scholar
  45. Samadani AA, DeHart BJ, Robinson K, Kulic D, Kubica E, Gorbet R (2011) A study of human performance in recognizing expressive hand movements. In: IEEE international symposium on robot and human interaction communication. Atlanta, GA, USAGoogle Scholar
  46. Santello M, Flanders M, Soechting JF (1998) Postural hand synergies for tool use. J Neurosci 18(23):10,105–10,115CrossRefGoogle Scholar
  47. Schröder M, Maycock J, Botsch M (2015) Reduced marker layouts for optical motion capture of hands. In: Proceedings of the 8th ACM SIGGRAPH conference on motion in games, MIG ‘15. Paris, France, pp 7–16Google Scholar
  48. Sturman DJ, Zeltzer D (1994) A survey of glove-based input. IEEE Comput Graph Appl 14(1):30–39CrossRefGoogle Scholar
  49. Wang Y, Neff M (2013) Data-driven glove calibration for hand motion capture. In: Proceedings of the 12th ACM SIGGRAPH/Eurographics symposium on computer animation, SCA ‘13. Anaheim, CA, USA, pp 15–24Google Scholar
  50. Wang RY, Popović J (2009) Real-time hand-tracking with a color glove. ACM Trans Graph 28(3):63Google Scholar
  51. Wang Y, Tree JEF, Walker M, Neff M (2016) Assessing the impact of hand motion on virtual character personality. ACM Trans Appl Percept 13(2):9:1–9:23CrossRefGoogle Scholar
  52. Wheatland N, Jörg S, Zordan V (2013): Automatic hand-over animation using principle component analysis. In: Proceedings of motion on games, MIG ‘13. Zürich, Switzerland, pp 175:197–175:202. ACMGoogle Scholar
  53. Wheatland N, Wang Y, Song H, Neff M, Zordan V, Jörg S (2015) State of the art in hand and finger modeling and animation. Comput Graph Forum 34(2):735–760CrossRefGoogle Scholar
  54. Ye Y, Liu CK (2012) Synthesis of detailed hand manipulations using contact sampling. ACM Trans Graph 31(4):245–254CrossRefGoogle Scholar
  55. Yuan Y, Steed A (2010) Is the rubber hand illusion induced by immersive virtual reality? Virtual Reality Conference (VR). IEEE Computer Soc. Waltham, MA, USA, pp 95–102Google Scholar
  56. Zhang J, Hommel B (2016) Body ownership and response to threat. Psychol Res 80(6):1020–1029Google Scholar
  57. Zhu Y, Ramakrishnan AS, Hamann B, Neff M (2013) A system for automatic animation of piano performances. Comput Anim Virtual Worlds 24(5):445–457CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of ComputingClemson UniversityClemsonUSA

Section editors and affiliations

  • Zhigang Deng
    • 1
  1. 1.Department of Computer Science,University of HoustonHoustonUSA

Personalised recommendations