Advertisement

Predicting Muscle Activity and Joint Angle from Skin Shape

  • Ryusuke SagawaEmail author
  • Ko Ayusawa
  • Yusuke Yoshiyasu
  • Akihiko Murai
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11129)

Abstract

Muscle of human body can be a clue to recognize the behavior and intention of a person. If the muscle activity is measured only by visual observation, it is useful to estimate the state of the muscle. In this paper, a method of predicting muscle activity and joint angle of human body from skin shape is proposed. Since the muscle activity and the joint angle affect the skin shape, the both factors should be considered simultaneously. The proposed method is a learning-based approach that uses the data set of the skin shape, the muscle activity and the joint angle. It trains a linear regressor for predicting muscle activity and joint angle from skin shape. The deformation of skin shape is calculated as the feature in the active regions, which are extracted from the training data and limits the regions of the skin shape that contribute to the prediction. We acquired a lower limb with simple motion to consider the small number of factors in this paper. In the experiment, the muscle activity and joint angle are predicted even in the case that the both factors change simultaneously. The skin regions that contributes to prediction are given as the result of learning, and the distribution is reasonable from the viewpoint of biomechanics.

Keywords

Skin shape Muscle activity Joint angle 
This is a preview of subscription content, log in to check access.

Supplementary material

478770_1_En_30_MOESM1_ESM.mp4 (22.1 mb)
Supplementary material 1 (mp4 22602 KB)

References

  1. 1.
    Delp, S., Loan, J.: A computational framework for simulating and analyzing human and animal movement. IEEE Comput. Sci. Eng. 2(5), 46–55 (2000)CrossRefGoogle Scholar
  2. 2.
    Nakamura, Y., Yamane, K., Fujita, Y., Suzuki, I.: Somatosensory computation for man-machine interface from motion-capture data and musculoskeletal human model. IEEE Trans. Robot. 21(1), 58–66 (2005)CrossRefGoogle Scholar
  3. 3.
    Luh, J., Walker, M., Paul, R.: On-line computational scheme for mechanical manipulators. ASME J. Dyn. Syst. Meas. Contr. 102(2), 69–76 (1980)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Gamage, S., Lasenby, J.: New least squares solutions for estimating the average centre of rotation and the axis of rotation. J. Biomech. 35(1), 87–93 (2002)CrossRefGoogle Scholar
  5. 5.
    Venture, G., Ayusawa, K., Nakamura, Y.: Optimal estimation of human body segments dynamics using realtime visual feedback. In: Proceedings of the IEEE/International Conference on Intelligent Robot System, pp. 1627–1632 (2009)Google Scholar
  6. 6.
    Rasmussen, J., Damsgaard, M., Voigt, M.: Muscle recruitment by the min/max criterion - a comparative numerical study. J. Biomech. 34(3), 409–415 (2001)CrossRefGoogle Scholar
  7. 7.
    Hill, A.: The heat of shortening and the dynamic constants of muscle. Proc. R. Soc. Lond. 126, 136–195 (1938)CrossRefGoogle Scholar
  8. 8.
    Stroeve, S.: Impedance characteristics of a neuro-musculoskeletal model of the human arm I: posture control. J. Biol. Cybern. 81, 475–494 (1999)CrossRefGoogle Scholar
  9. 9.
    Yamane, K., Fujita, Y., Nakamura, Y.: Estimation of physically and physiologically valid somatosensory information. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2624–2630 (2005)Google Scholar
  10. 10.
    Toshev, A., Szegedy, C.: DeepPose: human pose estimation via deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1653–1660 (2014)Google Scholar
  11. 11.
    Li, S., Chan, A.B.: 3D human pose estimation from monocular images with deep convolutional neural network. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9004, pp. 332–347. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16808-1_23CrossRefGoogle Scholar
  12. 12.
    Ramakrishna, V., Kanade, T., Sheikh, Y.: Reconstructing 3D human pose from 2D image landmarks. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 573–586. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33765-9_41CrossRefGoogle Scholar
  13. 13.
    Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., Black, M.J.: Keep it SMPL: automatic estimation of 3D human pose and shape from a single image. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 561–578. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_34CrossRefGoogle Scholar
  14. 14.
    Akhter, I., Black, M.J.: Pose-conditioned joint angle limits for 3D human pose reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1446–1455 (2015)Google Scholar
  15. 15.
    Zhou, X., Sun, X., Zhang, W., Liang, S., Wei, Y.: Deep kinematic pose regression. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 186–201. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-49409-8_17CrossRefGoogle Scholar
  16. 16.
    Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J.: End-to-end recovery of human shape and pose. In: Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  17. 17.
    Lee, D., Glueck, M., Khan, A., Fiume, E., Jackson, K.: Modeling and simulation of skeletal muscle for computer graphics: a survey. Found. Trends Comput. Graph. Vis. 7(4), 229–276 (2012)CrossRefGoogle Scholar
  18. 18.
    Robertini, N., Neumann, T., Varanasi, K., Theobalt, C.: Capture of arm-muscle deformations using a depth-camera. In: Proceedings of European Conference on Visual Media Production (CVMP), vol. 10 (2013)Google Scholar
  19. 19.
    Park, S., Hodgins, J.: Data-driven modeling of skin and muscle deformation. In: ACM TOG (2008)Google Scholar
  20. 20.
    Lewis, J., Anjyo, K., Rhee, T., Zhang, M., Pighin, F., Deng, Z.: Practice and theory of blendshape facial models. In: Proceedings of Eurographics 2014 - State the Art Reports (2014)Google Scholar
  21. 21.
    Sueda, S., Pai, D.K.: Dynamic simulation of the hand. In: Balasubramanian, R., Santos, V.J. (eds.) The Human Hand as an Inspiration for Robot Hand Development. STAR, vol. 95, pp. 267–288. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-03017-3_13CrossRefGoogle Scholar
  22. 22.
    Tsoli, A., Mahmood, N., Black, M.: Breathing life into shape: capturing, modeling and animating 3D human breathing. ACM Trans. Graph. 33(4), 52 (2014)CrossRefGoogle Scholar
  23. 23.
    Lee, S.H., Sifakis, E., Terzopoulos, D.: Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28(4), 99 (2009)CrossRefGoogle Scholar
  24. 24.
    Sagawa, R., Yoshiyasu, Y., Alspach, A., Ayusawa, K., Yamane, K., Hilton, A.: Analyzing muscle activity and force with skin shape captured by non-contact visual sensor. In: Bräunl, T., McCane, B., Rivera, M., Yu, X. (eds.) PSIVT 2015. LNCS, vol. 9431, pp. 488–501. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-29451-3_39CrossRefGoogle Scholar
  25. 25.
    Weise, T., Li, H., Van Gool, L., Pauly, M.: Face/off: live facial puppetry. In: Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 7–16 (2009)Google Scholar
  26. 26.
    Yeh, I.C., Lin, C.H., Sorkine, O., Lee, T.Y.: Template-based 3D model fitting using dual-domain relaxation. IEEE Trans. Visual. Comput. Graph. 17(8), 1178–1190 (2010)Google Scholar
  27. 27.
    Allen, B., Curless, B., Popović, Z.: The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22(3), 587–594 (2003)CrossRefGoogle Scholar
  28. 28.
    Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid ICP algorithms for surface registration. In: CVPR (2007)Google Scholar
  29. 29.
    Li, H., Sumner, R.W., Pauly, M.: Global correspondence optimization for non-rigid registration of depth scans. In: Proceedings of the Symposium on Geometry Processing, pp. 1421–1430 (2008)CrossRefGoogle Scholar
  30. 30.
    Huang, Q.X., Adams, B., Wicke, M., Guibas, L.J.: Non-rigid registration under isometric deformations. In: Proceedings of the Symposium on Geometry Processing, pp. 1449–1457 (2008)CrossRefGoogle Scholar
  31. 31.
    Tevs, A., Bokeloh, M., Wand, M., Schilling, A., Seidel, H.P.: Isometric registration of ambiguous and partial data. In: CVPR, pp. 1185–1192 (2009)Google Scholar
  32. 32.
    Liao, M., Zhang, Q., Wang, H., Yang, R., Gong, M.: Modeling deformable objects from a single depth camera. In: ICCV, pp. 167–174 (2009)Google Scholar
  33. 33.
    Papazov, C., Burschka, D.: Deformable 3D shape registration based on local similarity transforms. Comput. Graph. Forum 30, 1493–1502 (2011)CrossRefGoogle Scholar
  34. 34.
    Yoshiyasu, Y., Ma, W.C., Yoshida, E., Kanehiro, F.: As-conformal-as-possible surface registration. Comput. Graph. Forum 33(5), 257–267 (2014)CrossRefGoogle Scholar
  35. 35.
    Sagawa, R., Sakashita, K., Kasuya, N., Kawasaki, H., Furukawa, R., Yagi, Y.: Grid-based active stereo with single-colored wave pattern for dense one-shot 3D scan. In: 3DIMPVT, pp. 363–370 (2012)Google Scholar
  36. 36.
    Sagawa, R., Satoh, Y.: Illuminant-camera communication to observe moving objects under strong external light by spread spectrum modulation. In: Proceedings of CVPR (2017)Google Scholar
  37. 37.
    Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing. SGP 2006, pp. 61–70 (2006)Google Scholar
  38. 38.
    Merletti, R.: Standards for reporting EMG data. J. Electromyogr. Kinesiol. 9(1), III–IV (1999)Google Scholar
  39. 39.
    Stroeve, S.: Impedance characteristic of a neuromusculoskeletal model of the human arm I. Posture control. Biol. Cybern. 81(5), 475–494 (1999)CrossRefGoogle Scholar
  40. 40.
    Johansson, T., Meier, P., Blickhan, R.: A finite-element model for mechanical analysis of skeletal muscles. J. Theor. Biol. 206(1), 131–149 (2000)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Ryusuke Sagawa
    • 1
    Email author
  • Ko Ayusawa
    • 1
  • Yusuke Yoshiyasu
    • 1
  • Akihiko Murai
    • 1
  1. 1.National Institute of Advanced Industrial Science and TechnologyTsukubaJapan

Personalised recommendations

Cite paper

Buy options