Advertisement

Journal of Medical and Biological Engineering

, Volume 39, Issue 4, pp 532–540 | Cite as

A Modular Data Glove System for Finger and Hand Motion Capture Based on Inertial Sensors

  • Bor-Shing Lin
  • I-Jung Lee
  • Pei-Ying Chiang
  • Shih-Yuan Huang
  • Chih-Wei PengEmail author
Open Access
Original Article
  • 1.5k Downloads

Abstract

This study proposes a modular data glove system to accurately and reliably capture hand kinematics. This data glove system’s modular design enhances its flexibility. It can provide the hand’s angular velocities, accelerations, and joint angles to physicians for adjusting rehabilitation treatments. Three validations—raw data verification, static angle verification, and dynamic angle verification—were conducted to verify the reliability and accuracy of the data glove. Furthermore, to ensure the wearability of the data glove, 15 healthy participants and 15 participants with stroke were recruited to test the data glove and fill out a questionnaire. The errors of the finger ROMs obtained from the fusion algorithm were less than 2°, proving that the fusion algorithm can measure the wearer’s range of motion accurately. The result of the questionnaire shows the participants’ high satisfaction with the data glove. Moreover, a comparison between the proposed data glove and related research shows that the proposed data glove is superior to other data glove systems.

Keywords

Motion capture Data glove Inertial sensor Joint measurement Rehabilitation 

1 Introduction

Capturing hand kinematics is important for several medical purposes, such as rehabilitation and assessment of manual dexterity [1, 6, 13]. By capturing the motion of hands, the physicians can record and accurately evaluate the condition of patients’ hands after neurological diseases or hand surgery. Two types of hand motion capture technologies have been reported in the literature: non-contact systems and contact systems [1, 2, 3, 4, 5, 6, 7, 9, 11, 12, 15]. Typical non-contact systems capture hand motions by using camera-based devices and image processing technology [9, 11, 15]. The users are not required to wear any devices, but such systems are easily affected by environmental conditions, such as illumination and occlusion. By contrast, contact-based designs are more practical in medical settings [1, 2, 3, 4, 5, 6, 7, 12]. Data gloves are the most popular type of contact-based design. Various sensors, such as optical fiber sensors, resistance sensors, and inertial measurement units (IMUs), can be used in data gloves to capture hand motions. IMU-based data glove designs are reported to be the most practical designs [14]. An IMU-based data glove can record hand kinematics accurately and provide useful parameters, such as acceleration or angular velocity. In addition, most IMU-based gloves are light and can be worn easily.

Several IMU-based data gloves have been proposed [2, 3, 4, 5, 6, 7]. In 2014, Kortier et al. presented a data glove with inertial magnetic sensors to assess hand kinematics [6]. The disadvantage of this work was that it is time-consuming to attach the sensors on the user’s fingers; furthermore, the method of attaching sensors did not result in stable attachments, and the measurements of motion were often inaccurate. Moreover, this design featured universal serial bus (USB) transmission over a wire, which limited the mobility of the user’s hand. In 2016, Choi et al. presented a low-cost data glove with multiple IMUs [2]. However, this design was not suitable for all users. For users with small hands, the sensors tended to constrain finger motion; also, the attachments of the sensors to the tops of the fingers were unstable and, thus, the glove could not accurately measure the attitude of the fingers. Moreover, Choi et al. conducted the evaluation only in the static state without considering the dynamic situation, which was not practical for routine use. In 2017, Fang et al. proposed a wearable device and a sensor fusion algorithm to measure hand kinematics [4]. This device was tested using both static and dynamic evaluations to validate the accuracy of the algorithm. However, the wearable device was not a modular design, and the wires on the data glove may have impaired the wearability and mobility. Furthermore, all the aforementioned data gloves can be affected by the magnetic environment because they all use magnetometers.

To overcome the disadvantages of the previous devices, including loss of modulation and flexibility, a data glove system with 6-axis IMU sensors to capture hand kinematics was proposed. This proposed data glove includes a modular system that adapts to different hand sizes and enhances maintainability. Moreover, the sensors are attached to the surfaces of fingers stably using a novel method; therefore, the sensors accurately measure finger attitudes.

2 System Architecture and Design

2.1 Overview of Data Glove System

A data glove system was developed as a tool for capturing hand motion during physical rehabilitation. It contains a motion-capture board (MCB), five flexible finger units (FFUs), an arm board, and a host system. The architecture of the system is shown in Fig. 1.
Fig. 1

Architecture of the data glove system

The data glove consists of 16 IMU sensors (Fig. 2). Each IMU sensor includes a 3-axis accelerometer and a 3-axis gyroscope and measures parameters such as angular velocity, acceleration, and range of motion (ROM). The parameters precisely represent the fingers’ movements while the patients perform the rehabilitation exercises. The sensors on the fingers can measure the ROM of the distal interphalangeal and proximal interphalangeal joints. The sensor on the back of the hand is regarded as a reference and used to measure the ROM of the metacarpophalangeal (MCP) joints with the sensors on the fingers. The sensor on the forearm collocated with the sensor on the back of the hand is used to measure the ROM of the wrist. The MCB (Fig. 3) contains a microcontroller unit (MCU), an IMU, a Bluetooth module, and a Li-polymer battery. The dimension of the MCB is 42 mm × 50 mm × 8 mm. The MCU (MSP430; Texas Instruments Inc., TX, United States) requests the data from all sensors through a serial peripheral interface. The sampling rate of the data glove system is 50 Hz. The data collected from the IMUs are encapsulated and transmitted through Bluetooth to the host system at 115,200 bps. A 1000-mAh Li-polymer battery is embedded in the data glove to provide power for at least 5 h, which is adequate for conducting various rehabilitation tasks. The C# program on the host system calculates the ROM of fingers and records the obtained parameters, enabling the physicians to conduct detailed evaluations of the patient.
Fig. 2

Positions of sensors on the data glove

Fig. 3

MCB and its components

2.2 Mechanical Design of the Data Glove

In previous studies, each data glove had a fixed size and could not fit hands of different sizes. However, the data glove proposed in this study is designed as a modular kit (Fig. 4). It comprises five FFUs, an MCB, and an arm board. The width of each FFU is 0.8 cm, which is limited by the IMU sensor and the layout, and four different lengths (6.5, 9.5, 10.5, and 11.5) are used (Fig. 5). Each FFU is independent and contains two or three IMU sensors. The weight can be reduced by directly embedding the sensors on the FFU to enhance the mobility of the fingers. When an IMU sensor on any FFU is broken, that FFU can be replaced quickly and easily because of the modular design, which enhances the maintainability. Moreover, masses of flame retardant 4 (FR4) are attached to the bottoms of the FFUs, where the sensors are located, to prevent solder cracking while the fingers are actively moving.
Fig. 4

Modular design of the data glove

Fig. 5

Four lengths of FFUs

The data glove contains three main mechanisms: the mechanism between the FFU and the glove, the mechanism between the MCB and the glove, and arm board module.

The mechanical design between the FFU and the glove is presented in Fig. 6. To enhance the mobility of the finger joints, the material over the positions of the interphalangeal joints was removed. The IMU sensors were attached on the transparent soft plastic on an elastic fabric by a strong double-sided tape (Polar Bear SR-6600 Double-Sided Tape; New Taipei City, Taiwan), which can be mounted and removed easily. Moreover, an elastic tape was used to connect the two edges of the elastic fabric as a circular ring, which fixed the sensor position. This design prevented the IMUs in the data glove from slipping, allowing accurate measurement of the attitude of the fingers.
Fig. 6

The mechanical design between the FFU and the glove

A patient’s hand may drag the wires of the MCB and the glove constantly; therefore, a flexible flat cable was used to connect the FFU and the MCB in this data glove. This design avoids instability of transmission caused by dragging the wires. Furthermore, to prevent the FFU wires from breaking when the FFU is folded, the contact point between the FFU and the connector was covered with special glue (Cemedine Super X8008 Adhesive AX-139; Tokyo, Japan) to form a soft substance that prevents folding of the base edge on the pad of the sensor.

The design of the arm board module is presented in Fig. 7. It contains a Li-polymer battery and an IMU sensor. The IMU can measure the attitude of the forearm, and the wrist ROM can be calculated from the attitude of the MCB and that of the arm board module. The arm board module is embedded into a tennis elbow support (Conwell 53070; Taichung, Taiwan), which can be worn easily. Its purpose is to measure the ROM of the wrist and move the bulky components to the forearm in order to enhance the hand mobility.
Fig. 7

The mechanical design between the FFU and the glove

3 System Software and Verification

3.1 Process of ROM Measurement

3.1.1 Quaternion Algorithm

This study adopted a stable quaternion algorithm, called Mahony’s complementary filter, proposed by Madgwick in 2011 [10] to fuse the data of the accelerometer and that of the gyroscope and get the quaternion of sensor’s attitude.

The quaternion q of the current attitude of the sensor is defined as \(q = (q_{0} , q_{1} , q_{2} , q_{3} )\). q can be calculated from Eq. (1).
$$q = q_{t - 1} + \left( {\frac{1}{2}q_{t - 1} \times \omega_{t} } \right) \cdot \Delta_{t}$$
(1)
where ωt is angular velocity measured at the current time t. \(\Delta_{t}\) is the sampling period of the sensor at t. For this data glove, \(\Delta_{t}\) is 0.02 because the sampling rate is 50 Hz. qt-1 is the estimated quaternion at the previous time.
ROM can be calculated from the Euler angles on the sensor frame. The conversion of q to Euler angles can be calculated from Eqs. (2), (3), and (4).
$$Pitch = \tan 2^{ - 1} \left( {\frac{{2\left( {q_{0} q_{2} - q_{1} q_{3} } \right)}}{{1 - 2q_{2}^{2} - 2q_{1}^{2} }}} \right)$$
(2)
$$Roll = \sin^{ - 1} 2\left( {q_{2} q_{3} + q_{0} q_{1} } \right)$$
(3)
$$Yaw = \tan 2^{ - 1} \left( {\frac{{2\left( {q_{1} q_{2} - q_{0} q_{3} } \right)}}{{2q_{0}^{2} + 2q_{1}^{2} - 1}}} \right)$$
(4)

3.1.2 Algorithm for Calculation of Finger ROM

The calculation of the ROM relies on two IMU sensors located on the proximal and distal sides of the interphalangeal joint (Fig. 8). The front sensor (FS) is an IMU sensor measuring the proximity to the fingertip, and the back sensor (BS) is an IMU sensor measuring the proximity to the metacarpophalangeal joint.
Fig. 8

The mechanical design between the FFU and the glove

The calculation process is presented in Fig. 9. The pitch angles of FS and BS are defined as Pitch(FSi) and Pitch(BSi), respectively. The variables i, j = 1, 2,…,16 are the indexes of selected sensors. ROMori is the original difference between Pitch(FSi) and Pitch(BSi). ROMij represents the final result of the ROM of sensor i and sensor j.
Fig. 9

Algorithm for calculating ROM

3.2 System Validation

3.2.1 Raw Data Validation

In the data glove system, ROMs of fingers are computed from acceleration and angular velocity. Therefore, it is necessary to verify the reliability of the acceleration and the angular velocity values obtained from the IMU sensors.

A reliable IMU device (LPMS-B; LP-Research, Tokyo, Japan) was used as a reference in this study. LPMS-B was attached beneath our IMU sensor, and both the sensors were rotated randomly and simultaneously. The raw data from these two devices were recorded and compared to prove the reliability of our IMU sensor.

3.2.2 Static Angle Verification

Static angle verification was used to verify the accuracy of the fusing algorithm in static situations. An FFU was attached to the static angle verification tool, which contained a protractor at a fixed angle (Fig. 10). This angle represented the actual value and was compared with the angle from the FFU.
Fig. 10

Tool used to verify the static angle

3.2.3 Dynamic Angle Verification

Dynamic angle verification was conducted after static angle verification to verify the ROM during finger movement. A servomotor (3001HB; Pololu Crop., Las Vegas, NV, United States) was used to produce repeated movements to simulate the movement of a real finger. The FFU was placed on the dynamic testing platform, and the servomotor was used to control the rotation of the platform (Fig. 11).
Fig. 11

Dynamic angle verification operation

4 Results

4.1 Raw Data Validation

The acceleration and angular velocity of each axis obtained from LPMS-B and the data glove are presented in Fig. 12. The average correlation of acceleration value was 0.957 and the average correlation of angular velocity value was 0.979.
Fig. 12

Verification of IMU sensor data using LPMS-B

4.2 Static Angle Verification

ROM and the angle of the protractor were compared, and the results are presented in Table 1. The process of testing was repeated with four different angles: 24, 43, 57, and 78°. The errors of the finger ROMs obtained from the algorithm were ≤ 2° (Table 1).
Table 1

Result of static angle verification

Angle of protractor (°)

Measured ROM (°)

Error (°)

24

23

1

43

42

1

57

56

1

78

76

2

4.3 Dynamic Angle Verification

The range of rotation was set to between 30 and 70°. The result of the ROM during the cycle from 30 to 70° (Fig. 13) shows the reliability and stability of the sensor fusion algorithm during ROM changes.
Fig. 13

Dynamic angle verification results

4.4 Questionnaire

After finishing the verification of the data glove system, the participants were asked to fill out an 8-item questionnaire containing questions related to the comfort and use of the data glove. The responses of all items could range from 1 (strongly disagree) to 5 (strongly agree). The questionnaire is shown in Table 2. The tasks mentioned in item 7 of the questionnaire is to ask the patients to perform the grip task, thumb task, and card turning task, proposed in [8]. The total time of conducting the three tasks are approximately 5.5 min, and real conducting time depends on the hand function of the subjects.
Table 2

Content of questionnaire used to evaluate data glove

Item

1. The glove was comfortable to put on.

2. The glove did not feel too cold or too hot.

3. The glove was comfortable to put on or remove.

4. I felt free to bend my fingers as though I were not wearing the glove.

5. The glove did not feel slippery during the tasks.

6. I used to do well in hand-related sports. (badminton, tennis, table tennis, etc.)

7. Wearing the glove during tasks was not painful.

8. I feel physically energetic.

Table 3 lists the feedback results from 15 healthy participants and 15 participants with stroke. All of the questionnaires were approved by the Institutional Review Board (IRB No.10102-019) of Chi Mei Hospital. A mean score for each question was computed across healthy participants and those with stroke. The scale resulted in an overall score of 1–5, with a higher score indicating a more favorable feedback.
Table 3

Statistical results of the questionnaire

Question

Healthy subjects

Stroke subjects

1

5.00

4.93

2

4.93

4.93

3

4.67

4.40

4

4.33

4.20

5

4.73

4.53

6

4.33

4.00

7

5.00

5.00

8

5.00

4.93

5 Discussion

This study presents a data glove with a modular design. This data glove can accurately record finger movements during physical rehabilitation tasks. Several verifications were conducted to evaluate the output of the data glove. The first evaluation was raw data validation. The average correlation between the acceleration and angular velocity of LPMS-B and the IMU sensors on the data glove were both over 0.9, which is a high correlation, indicating that the data from the data glove are reliable. The second evaluation was static angle validation, which revealed a low error rate (≤ 2°) between the measured ROM and the actual ROM, indicating that the ROM calculated from the algorithm was accurate and reliable. The final evaluation was dynamic angle validation, which established the stability and accuracy of the ROM measurement during repeated active finger movements. In the clinical setting, most rehabilitation tasks require repeated active movements. Thus, our data glove system can be used in actual rehabilitation settings.

After conducting the system validation, healthy participants and those with stroke were asked to fill out a questionnaire to gauge their comfort and experience while wearing the data glove. Although the mean scores of participants with stroke were lower than those of the healthy participants, the mean scores of all questions were higher than 4 points, showing that most of the participants were able to wear the data glove easily and comfortably. Item 6 (“I used to do well in sports.”) had the lowest score; however, item 6 only considered the ability to do exercise, which is different among participants, and not related to the design of the data glove. Therefore, the overall result shows that the data glove was comfortable, suitable for, and acceptable to all participants.

Table 4 compares our data glove system with other data glove systems. Kortier et al. developed a data glove to accurately assess the full kinematics of hands and fingers [6]. Their research was able to measure finger attitudes accurately during dynamic movement. However, their data glove was only able to transmit its data through a USB interface, which was prone to be restricted by the environment. Choi et al. developed a data glove with a fast orientation algorithm to effectively calculate the Euler angles [2]. But their data glove produced higher error than the proposed data glove. Furthermore, their data glove only conducted static verification and was not appropriate for rehabilitation. Fang et al. also proposed a data glove for motion capture [4], which was able to present the user’s motion in software easily. However, it was not modular and repair was difficult. The most crucial problem is that all of these three methods used magnetometer sensors, which are easily affected by environmental magnetic fields. However, our data glove system only contains accelerometers and gyroscopes and is thus independent of the environment, including small changes in the magnetic field. Moreover, most of the recently proposed data glove systems do not provide a modular design and cannot be repaired easily. Our system provides a modular design that enhances its flexibility and maintainability and has been fully verified to ensure its reliability.
Table 4

Comparison of our proposed data glove system with others

System

Kortier et al. [6]

Choi et al. [2]

Fang et al. [4]

Proposed system

Modular design

No

No

No

Yes

Transmission interface

USB

Bluetooth

Bluetooth

USB/Bluetooth

Easy to fix

No

No

No

Yes

Easy to wear

No

Yes

Yes

Yes

Not effected by environment

No

No

No

Yes

Validation

Dynamic

Static

Dynamic

Raw data, static, dynamic

The proposed modular data glove can be used in various medical applications, such as rehabilitation and hand function assessment. In 2017, this data glove had been used to assess the hand function of patients with stroke for verifying its practical use [8]. However, this is just one of the applications. In the future, the data glove will be applied in rehabilitation and hand function assessments for various kinds of diseases or injuries, such as Parkinson’s disease or essential tremor.

6 Conclusions

In this study, a data glove with 6-axis sensors for capturing hand kinematics was developed. This modular design can be adjusted to be worn by users with various hand sizes and characteristics. Full verification—including raw data, static angle validation, and dynamic angle validation—was conducted to verify the reliability and accuracy of the sensor fusion algorithm, showing high accuracy and usability in clinical settings. Also, results of the (healthy and stroke) participant questionnaire regarding the comfort of the data glove revealed that all of the participants had high opinions of the data glove. Our proposed data glove is superior to various recently proposed systems and provides more flexibility with its modular design.

Future studies using our data glove system should conduct clinical experiments and correlate motion parameters with clinical parameters. Such studies could be provided to physicians for evaluation. Moreover, the fusing algorithm can be improved to reach higher accuracy in real clinical settings

Notes

Acknowledgements

This research was partly supported by Ministry of Science and Technology in Taiwan, under grants MOST 106-2221-E-305-012. This research was also partly supported by University System of Taipei Joint Research Program, under grants USTP-NTPU-TMU-104-01. The authors thank the clinic staffs in Chi Mei Medical Center for their outstanding technical assistance. This manuscript was edited by Wallace Academic Editing.

References

  1. 1.
    Carbonaro, N., Mura, G. D., Lorussi, F., Paradiso, R., DeRossi, D., & Tognetti, A. (2014). Exploiting wearable goniometer technology for motion sensing gloves. IEEE Journal of Biomedical and Health Informatics, 18(6), 1788–1795.CrossRefGoogle Scholar
  2. 2.
    Choi, Y., Yoo, K., Kang, S. J., Seo, B., & Kim, S. K. (2016). Development of a low-cost wearable sensing glove with multiple inertial sensors and a light and fast orientation estimation algorithm. Journal of Supercomputing.  https://doi.org/10.1007/s11227-016-1833-5.Google Scholar
  3. 3.
    Djurić-Jovičić, M., Jovičić, N. S., Roby-Brami, A., Popović, M. B., Kostić, V. S., & Djordjević, A. R. (2017). Quantification of finger-tapping angle based on wearable sensors. Sensors (Switzerland), 17(2), 203.CrossRefGoogle Scholar
  4. 4.
    Fang, B., Sun, F., Liu, H., & Guo, D. (2017). Development of a wearable device for motion capturing based on magnetic and inertial measurement units. Scientific Programming.  https://doi.org/10.1155/2017/7594763.Google Scholar
  5. 5.
    Kitano, K., Ito, A., Tsujiuchi, N., &Wakida, S. (2016). Estimation of joint center and measurement of finger motion by inertial sensors. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 5668–5671).Google Scholar
  6. 6.
    Kortier, H. G., Sluiter, V. I., Roetenberg, D., & Veltink, P. H. (2014). Assessment of hand kinematics using inertial and magnetic sensors. Journal of NeuroEngineering and Rehabilitation, 11(1), 70.CrossRefGoogle Scholar
  7. 7.
    Lemos, J. D., Hernandez, A. M., & Soto-Romero, G. (2017). An instrumented glove to assess manual dexterity in simulation-based neurosurgical education. Sensors (Switzerland), 17(5), 988.CrossRefGoogle Scholar
  8. 8.
    Lin, B.-S., Hsiao, P.-C., Yang, S.-Y., Su, C.-S., & Lee, I.-J. (2017). Data Glove System Embedded with Inertial Measurement Units for Hand Function Evaluation in Stroke Patients. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(11), 2204–2213.CrossRefGoogle Scholar
  9. 9.
    Liu, H. (2011). Exploring human hand capabilities into embedded multifingered object manipulation. IEEE Transactions on Industrial Informatics, 7(3), 389–398.CrossRefGoogle Scholar
  10. 10.
    Madgwick, S. O. H., Harrison, A. J. L., &Vaidyanathan, R. (2011). Estimation of IMU and MARG orientation using a gradient descent algorithm. In IEEE International Conference on Rehabilitation Robotics (pp. 179–185).Google Scholar
  11. 11.
    Metcalf, C. D., Robinson, R., Malpass, A. J., Bogle, T. P., Dell, T. A., Harris, C., et al. (2013). Markerless motion capture and measurement of hand kinematics: Validation and application to home-based upper limb rehabilitation. IEEE Transactions on Biomedical Engineering, 60(8), 2184–2192.CrossRefGoogle Scholar
  12. 12.
    Park, Y., Lee, J., & Bae, J. (2015). Development of a wearable sensing glove for measuring the motion of fingers using linear potentiometers and flexible wires. IEEE Transactions on Industrial Informatics, 11(1), 198–206.CrossRefGoogle Scholar
  13. 13.
    Wang, Q., Chen, W., & Markopoulos, P. (2014). Literature review on wearable systems in upper extremity rehabilitation. In IEEE-EMBS International Conference on Biomedical and Health Informatics, (pp. 551–555).Google Scholar
  14. 14.
    Wang, Q., Markopoulos, P., Yu, B., Chen, W., & Timmermans, A. (2017). Interactive wearable systems for upper body rehabilitation: a systematic review. Journal of NeuroEngineering and Rehabilitation, 14(1), 20.CrossRefGoogle Scholar
  15. 15.
    Ye, M., Yang, C., Stankovic, V., Stankovic, L., & Kerr, A. (2016). A depth camera motion analysis framework for tele-rehabilitation: Motion Capture and person-centric kinematics analysis. IEEE Journal of Selected Topics in Signal Processing, 10(5), 877–887.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of Computer Science and Information EngineeringNational Taipei UniversityNew Taipei CityTaiwan
  2. 2.College of Electrical Engineering and Computer ScienceNational Taipei UniversityNew Taipei CityTaiwan
  3. 3.Department of Computer Science and Information EngineeringNational Taipei University of TechnologyTaipeiTaiwan
  4. 4.Department of Physical Medicine and Rehabilitation, School of Medicine, College of MedicineTaipei Medical UniversityTaipeiTaiwan
  5. 5.Department of Physical Medicine and RehabilitationTaipei Medical University HospitalTaipeiTaiwan
  6. 6.School of Biomedical Engineering, College of Biomedical EngineeringTaipei Medical UniversityTaipeiTaiwan
  7. 7.International Ph.D. Program in Biomedical Engineering, College of Biomedical EngineeringTaipei Medical UniversityTaipeiTaiwan

Personalised recommendations