Abstract
In the passive haptic system and other VR fields, there is a great demand for measuring the direction and magnitude of various forces applied to a grasped object. To acquire this force, there are mainly two kinds of methods; one is directly measuring it by the sensor attached or embedded to the object surface or the glove type device, and the other is estimating it by measuring the change of the surface electromyography (sEMG) of the forearm. The former is used a lot but has problems that it is difficult to stick the sensor due to the object’s surface, and sensors prevent fingers and objects from touching directly. The latter approach has a potential to release us from above problems. However, most of their works focused on estimating a certain direction of force, like a grip force, thus, there is no research to estimate multidirectional loading forces.
To solve these problems, we propose a solution for estimating direction and magnitude of the force applied to the grasped object using sEMG of the forearm with the convolutional neural network (CNN), as the gesture recognition field uses. By constructing training system that can measure sEMG signals and the force applied to the grasped object, we trained a model, and this model realized over 95% accurate directional estimation and the estimation of force magnitude with less than 7% NRMSE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tachi, S., Maeda, T., Hirata, R., Hoshino, H.A.: Construction method of virtual haptic space. In: Proceedings of the 4th International Conference on Artificial Reality and Tele-Existence, pp. 131–138 (1994)
Hinckley, K., Pausch, R., Goble, J.C., Kassell, N.F.: Passive real-world interface props for neurosurgical visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, vol. 452 (1994)
Azmandian, M., Hancock, M., Benko, H., Ofek, E., Wilson, A.D.: Haptic retargeting: dynamic repurposing of passive haptics for enhanced virtual reality experiences. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1968–1979 (2016)
Ban, Y., Kajinami, T., Narumi, T., Tanikawa, T., Hirose, M.: Modifying an identified angle of edged shapes using pseudo-haptic effects. In: Isokoski, P., Springare, J. (eds.) EuroHaptics 2012. LNCS, vol. 7282, pp. 25–36. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31401-8_3
Matsumoto, K., Ban, Y., et al.: Unlimited corridor: redirected walking techniques using visuo haptic interaction. In: ACM SIGGRAPH 2016 Emerging Technologies, p. 20 (2016)
Ban, Y., Narumi, T., et al.: Controlling perceived stiffness of pinched objects using visual feedback of hand deformation. In: Haptics Symposium (HAPTICS), pp. 557–562 (2014)
Kimura, T., Nojima, T.: Pseudo-haptic feedback on softness induced by grasping motion. In: Isokoski, P., Springare, J. (eds.) EuroHaptics 2012. LNCS, vol. 7283, pp. 202–205. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31404-9_36
Kawamoto, H., Sankai, Y.: Power assist system HAL-3 for gait disorder person. In: Miesenberger, K., Klaus, J., Zagler, W. (eds.) ICCHP 2002. LNCS, vol. 2398, pp. 196–203. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45491-8_43
Park, K.H., Lee, S.W.: Movement intention decoding based on deep learning for multiuser myoelectric interfaces. In: 2016 4th International Winter Conference Brain-Computer Interface (BCI), pp. 1–2 (2016)
Nitta, K., Sato, T., Koike, H., Nojima, T.: Photoelastic Ball: a touch detectable ball using photoelasticity. In: the 5th Augmented Human International Conference, vol. 16, pp. 1–8 (2014)
Sagisaka, T., Ohmura, Y., Nagakubo, A., Ozaki, K., Kuniyoshi, Y.: Development and applications of high-density tactile sensing glove. In: Isokoski, P., Springare, J. (eds.) EuroHaptics 2012. LNCS, vol. 7282, pp. 445–456. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31401-8_40
Misener, D.L., Morin, E.L.: An EMG to force model for the human elbow derived from surface EMG parameters. In: IEEE 17th Annual Conference Engineering in Medicine and Biology Society, vol. 2, pp. 1205–1206 (1995)
Doheny, E.P., Lowery, M.M., FitzPatrick, D.P., O’Malley, M.J.: Effect of elbow joint angle on force-EMG relationships in human elbow flexor and extensor muscles. J. Electromyogr. Kinesiol. 18(5), 760–770 (2008)
Jazlan, A., Sidek, S.N.: Development of a myoelectric interface for indirect hand grip force and wrist angle measurement/analysis. Int. J. Biomechatronics Biomed. Robot. 3(1), 42–53 (2014)
Oboe, R., Tonin, A., Yu, K., et al.: Weight estimation system using surface EMG armband. In: 2017 IEEE International Conference Industrial Technology (ICIT), pp. 688–693 (2017)
Atzori, M., Cognolato, M., Müller, H.: Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Front. Neurorobot. 10, 9 (2016)
Myo, Thalmic Labs Inc. (2013). https://www.myo.com/
Englehart, K., Hudgins, B., et al.: Classification of the myoelectric signal using time-frequency based representations. Med. Eng. Phys. 21(6), 431–438 (1999)
LeCun, Y., Bengio, Y.: Deep learning. Nature 521(7553), 436–444 (2015)
Silver, D., et al.: NIPS Workshop on “Inductive Transfer: 10 Years Later. Whistler (2005)
Côté-Allard, U., et al.: Transfer Learning for sEMG Hand Gestures Recognition Using Convolutional Neural Networks, Systems, Man, and Cybernetics (SMC) (2017). https://doi.org/10.1109/smc.2017.8122854
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Ban, Y. (2018). Estimating the Direction of Force Applied to the Grasped Object Using the Surface EMG. In: Prattichizzo, D., Shinoda, H., Tan, H., Ruffaldi, E., Frisoli, A. (eds) Haptics: Science, Technology, and Applications. EuroHaptics 2018. Lecture Notes in Computer Science(), vol 10894. Springer, Cham. https://doi.org/10.1007/978-3-319-93399-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-93399-3_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93398-6
Online ISBN: 978-3-319-93399-3
eBook Packages: Computer ScienceComputer Science (R0)