Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements
- 4.6k Downloads
Myoelectric pattern recognition systems can decode movement intention to drive upper-limb prostheses. Despite recent advances in academic research, the commercial adoption of such systems remains low. This limitation is mainly due to the lack of classification robustness and a simultaneous requirement for a large number of electromyogram (EMG) electrodes. We propose to address these two issues by using a multi-modal approach which combines surface electromyography (sEMG) with inertial measurements (IMs) and an appropriate training data collection paradigm. We demonstrate that this can significantly improve classification performance as compared to conventional techniques exclusively based on sEMG signals.
We collected and analyzed a large dataset comprising recordings with 20 able-bodied and two amputee participants executing 40 movements. Additionally, we conducted a novel real-time prosthetic hand control experiment with 11 able-bodied subjects and an amputee by using a state-of-the-art commercial prosthetic hand. A systematic performance comparison was carried out to investigate the potential benefit of incorporating IMs in prosthetic hand control.
The inclusion of IM data improved performance significantly, by increasing classification accuracy (CA) in the offline analysis and improving completion rates (CRs) in the real-time experiment. Our findings were consistent across able-bodied and amputee subjects. Integrating the sEMG electrodes and IM sensors within a single sensor package enabled us to achieve high-level performance by using on average 4-6 sensors.
The results from our experiments suggest that IMs can form an excellent complimentary source signal for upper-limb myoelectric prostheses. We trust that multi-modal control solutions have the potential of improving the usability of upper-extremity prostheses in real-life applications.
KeywordsMyoelectric prosthesis Myoelectric control Inertial measurement unit Surface electromyography Hand motion classification
Extensor digitorum communis
Flexor digitorum superficialis
Inertial measurement unit
Linear discriminant analysis
Mean absolute value
Regulatory feedback network
Robot operating system
Sequential forward sensor selection
Support vector machine
Target achievement control
Upper-limb myoelectric prostheses aim at replacing the appearance and functionality of a missing limb. In academic research, pattern recognition-based systems have been very successful in decoding movement intent and have recently found their way into commercial products1. Classification methods have been extensively used for decoding grip type, wrist and individuated finger movement [1, 2, 3, 4, 5, 6]. Moreover, the real-time performance of such systems has been evaluated with able-bodied and amputee subjects [7, 8, 9]. Nevertheless, the acceptance of myoelectric systems by end users has been remarkably low . This is mainly due to limited classification robustness which can be partially improved by increasing the number of used electromyogram (EMG) sensors. The use of a large amount of sensors, however, may be impractical for the user. To further improve the clinical adoption of upper-limb prostheses, next generation myoelectric systems will have to exhibit multi-modal control and use a minimal amount of sensors .
The sEMG signal is inherently noisy and thus, not a robust source of input information for prosthetic systems . This is especially true for altered conditions such as sweat, fatigue, and electrode displacement . Therefore, it is imperative to move towards multi-modal control solutions.
One of the main issues associated with the use of the sEMG signal is the limb position effect which states that a system trained on a single arm position is likely to fail to generalise to different arm postures . Fougner et al.  proposed to address this issue by training decoders in multiple limb positions and also by using accelerometers placed on the forearm and the biceps muscle of the subjects to measure arm orientation. Their approach resulted in a substantial decrease in classification error from 18 to 5%. Geng et al.  later reproduced this finding in amputee subjects by training a two-stage (position/motion) classifier. Radmand et al.  demonstrated, however, that myoelectric decoding can benefit from the use of accelerometers only if training data are collected across many positions, which may be infeasible in practice. To overcome this issue, they proposed to collect training data with dynamic movements during which the (residual) arm is moved through the regions of interest. Khushaba et al.  investigated the combined effect of forearm orientation and muscular contraction level and verified that the use of accelerometers can be beneficial for classification performance. Finally, the combination of (EMG) and accelerometry has been also found to improve classification accuracy (CA) in lower-limb movement intent decoding . Nevertheless, all the aforementioned studies were limited to offline analyses. Yet, it is unclear whether the observed increase in offline CA can be associated with a performance improvement during real-time, task-oriented myoelectric control .
The goal of the current study has been threefold: 1) to investigate whether classification performance can further benefit from the use of additional inertial sensors, such as gyroscopes and magnetometers; 2) to assess whether an increase in offline CA can be translated to a performance improvement during real-time prosthetic control; 3) to investigate whether the inclusion of inertial measurements (IMs) can help reduce the number of sensors required to achieve robust classification performance. This last aspect is particularly important for real-life applications, where it is desirable to minimise the number of sensors used by the prosthesis.
Amputee subjects medical records
Type of amputation
Cause of amputation
Hand dominance (prior to amputation)
Cancer (epitheliod sarcoma)
Identical procedures were followed in the two sets of experiments for signal acquisition, conditioning (i.e. preprocessing and feature extraction), and motion/grasp classification. The main differences were with regards to the performed tasks and the metrics used to evaluate performance.
The sampling frequency was set to 2 kHz for myoelectric signals and 128 Hz for IM data. Since IM readings were used in their raw format, no calibration was required.
EMG-IM sensor placement
Equally spaced around forearm (3 cm below elbow)
Offline data collection
Data were collected offline with 20 able-bodied and two amputee subjects by adopting the NinaPro protocol [20, 21]. Subjects were asked to reproduce a series of 40 motions, including various individuated-finger, hand, wrist, grasping and functional movements (exercises B and C in ) instructed to them on a computer screen. Each movement was repeated six times and trials were interleaved with 5-s resting periods. The two amputee volunteers were instructed to perform bilateral imaginary mirrored movements3.
Following Gijsberts et al. , power line interference was suppressed from the myoelectric signals by applying a Hampel filter. The post-hoc relabeling procedure that was described in the same study was used to identify and refine the exact stimuli timings for each subject and trial in order to avoid introducing label-related noise in the classifiers. The cause of this type of noise is the natural variability introduced when subjects replicate movements instructed to them on a screen (i.e. onset delays, variability in trial lengths, etc.).
Real-time control pick and place experiment
Real-time experiment objects used and associated grip types
Credit card simulator
Compact Disc (CD)
Each session comprised a training and a testing phase. During the training phase, participants were required to perform five reach-to-grasp repetitions of each of the five poses/grips (classes 1-5 in Table 4). Throughout this stage, participants were instructed to move their arm at a steady pace and activate their muscles in a natural way, that is, without exerting excessive tension. The objects corresponding to the different poses were placed on a computer desk in front of the participants, however they were not able to physically grasp them, due to their fingers being constrained by the elastic bandage. During this phase, which was required to collect training data, the prosthetic hand was kept inactive. To indicate the motion being performed, participants were asked to press down with their contralateral hand a corresponding key on a computer keyboard, with a different key corresponding to each of the performed poses (i.e. 1-5). The amputee participant performed ten repetitions of each movement.
- I.sEMG data from all sensors.Table 5
Real-time experiment decoding conditions
Number of sensors
IM data from all sensors.
sEMG and IM data from all sensors.
sEMG and IM data from a selected subset of sensors.
During the testing phase, each trial consisted of picking and placing the three objects approximately 50 cm away from their initial position. A trial ended by pressing the space button on the computer keyboard by using the index pointer grip. Able-bodied subjects were given 60 s to accomplish the trials with the prosthetic hand and the amputee participant was given 75 s. The objects were presented to the subjects in a pseudo-randomised order, so that the sequence of required grasping motions varied across trials. Able-bodied subjects performed four trials for each of the four decoding conditions and the amputee participant performed six. When the prosthetic hand performed a different movement than the one intended by the user, for instance due to a motion misclassification, participants were asked to open the hand and try performing the intended movement again. The presentation order of the four decoders described above was counterbalanced across the able-bodied population in order to avoid favouring certain conditions over others, given the learning mechanisms taking place during prosthetic control [2, 19, 23]. The total duration of experiments including skin preparation, sensor placement, training, and testing was around 90 min for each subject.
A finite-state machine implementation was used for the real-time control of the prosthetic hand. A movement predicted by the classifier was triggered only if the most recently performed movement had terminated execution. To determine movement execution termination, the hand’s motor current readings were constantly monitored and compared to a fixed threshold. In addition, a control command was triggered only when it was predicted with high confidence, in other words when the posterior probability of the corresponding class exceeded a threshold. The probability threshold was set a priori to θ=0.995. For the real-time experiment, signal acquisition, pre-processing and control of the prosthetic hand were implemented in C++ and integrated into the Robot Operating System (ROS) . The communication between ROS and the hand was achieved via the CANBUS protocol.
Signal preprocessing and classifier training
Myoelectric and IM signals were synchronised via linear interpolation. By using a shifting window approach, four sEMG features were extracted from each channel, namely the mean absolute value (MAV), waveform length (WL), 4th-order auto-regressive (AR) coefficients and log-variance (LogVar). The selection of these features was based on previous studies demonstrating their efficacy in decoding hand motion intention [25, 26, 27]. Bearing in mind the need for low computational requirements during real-time control, we only considered time-domain (TD) sEMG features . The length of the shifting window was set to 256 ms and the increment to 50 ms (80% overlap). It has been previously shown that this selection offers a good compromise between classification performance and controller delay . For the real-time experiment, the average delay was 170 ms  which is within the acceptable range for the purposes of upper-limb myoelectric control . In order to match EMG features, IM data were also binned in 256 ms windows by extracting the mean value (MV) of the signals within the processing window. The total number of features contributed by each sensor was thus 16 (7 EMG, 9 IM features.)
The columns of the design matrix (i.e. input features) were standardised by subtracting the mean and dividing by standard deviation. For the offline experiment, mean subtraction and feature scaling followed cross-validation splitting and were thus performed by using training data only. For the real-time experiment, mean and standard deviation vectors for each subject were estimated on the entire training dataset and subsequently used to transform the input data during the testing phase. Training datasets collected for both types of experiments were included unchanged in the subsequent analyses steps, in other words, no segments of activity were manually removed.
For movement intent decoding from myoelectric and IM data, we employed a linear discriminant analysis (LDA) classifier. Discriminant analysis is a family of supervised dimensionality reduction algorithms for identifying feature projections that maximise class separability. These methods can be used for multi-class classification by assuming a class-conditional Gaussian model. The linear discriminant analysis (LDA) is a special case of this family which assumes a shared covariance matrix across the different classes resulting in linear decision boundaries (i.e. hyperplanes). In the context of myoelectric control, LDA and its variants have been extensively used, since they can achieve high decoding performance with minimal requirements [3, 4, 5, 32].
One particular advantage of discriminant analysis classifiers is that they are probabilistic, that is, they estimate a posterior probability distribution over classes as opposed, for instance, to support vector machines (SVMs) which only yield a classification decision. This feature was particularly important for our paradigm where confidence-based classification rejection was deployed at the final control stage. Another strong advantage of LDA is its efficiency at test time; both time and space complexities scale linearly with the input feature dimensionality .
For both experiments, the extracted sEMG and/or IM features were fed as input(s) to the classifiers and the vectors containing the stimulus time-series (i.e. grip performed) were used as targets. All four types of classifiers (Table 5) were trained and tested by using data from individual subjects.
Decoding performance assessment
Decoding performance was evaluated differently in the two experiments. For the offline experiment, five repetitions were used to train the decoders and the left-out repetition was used to assess decoding performance. The procedure was then repeated by using a different evaluation fold in each iteration, hence resulting in a 6-fold cross-validation. Following classification, the class distribution of the test folds was balanced by removing a large proportion of the instances corresponding to the rest class. This step was necessary to prevent CA scores from being biased by the aforementioned class. The identity of rest samples to be removed was determined according to their temporal distance from the nearest segment of muscle activity and, hence, the repeatability of our analysis was not affected by this procedure.
For the real-time experiment, we adopted two task-related metrics which are commonly used in the literature [8, 9]. The completion rate (CR) is defined as the ratio of successful to total number of trials, and completion time (CT) is defined as the time taken to complete a successful trial. A trial was considered successful only if it was accomplished within a given time frame (60 s for the able-bodied subjects and 75 s for the amputee participant).
Sequential forward sensor selection (SFSS)
One of the main aims of this study was to assess whether the use of inertial data measured with the same sensor packs that record EMG signals could help reduce the number of channels required to achieve high-level myoelectric control. Therefore, we investigated whether the use of an optimally selected subset of EMG-IM sensors could achieve the same level of decoding performance attained by the decoders when all available sensors were used.
A sensor selection method was developed which was based on the classic sequential forward feature selection algorithm [1, 34, 35]. Our adapted algorithm was initialised with an empty sensor set. In each iteration, the sensor which yielded the highest performance improvement was added to the pool. Decoding performance was assessed by including all input signals from the associated sensor, in other words 7 sEMG and 9 IM features. To increase the robustness of our method, cross-validation was used in each iteration and the sensor selection decision was based on a majority vote across the cross-validation folds. For consistency, the CA metric was used for assessing decoding performance in each step. The algorithm terminated execution once all sensors were included in the set, in other words when all available sensors were ranked according to their relative predictive power. For both the offline and real-time experiments we selected those sensors the addition of which yielded an improvement in CA larger than 1%. The sensor rankings varied across subjects, therefore a different subset was used for each subject. The size of the subset also varied across subjects. For the real-time experiment, sensor selection was performed by using the training data only and the sensor subset for each participant was kept fixed throughout the testing phase.
A Friedman test was performed to assess the effect of decoding condition on CA scores. A statistical significant effect was identified (p<10−3) and pair-wise comparisons were subsequently performed by using the Nemenyi test. For both populations, the performance of the sEMG-IM classifier was significantly higher than that of any other decoder (median balanced CA for this condition was 82.7% for able-bodied and 77.8% for amputee subjects). This is not surprising because this was the condition that all available sensorial information was used for classification. The second best performance was achieved by the IM decoder (81.7% able-bodied, 77.7% amputees), followed by the sEMG-IM subset condition (81.2% able-bodied, 76% amputees) although the differences between the latter two conditions were not statistically significant.
One of the motivations of this study was to identify whether the additional inclusion of gyroscope and magnetometer data beyond accelerometry would be beneficial for hand movement decoding. Our offline analysis provided evidence supporting this hypothesis, since it was found the sEMG-IM decoder performed significantly better than sEMG-accelerometer. That was also the case when we completely discarded myoelectric data, in other words the IM decoder significantly outperformed the accelerometer classifier. Importantly, all comparisons were consistent across the able-bodied and amputee populations.
Real-time control experiment
In terms of CTs, the performance of the four conditions was comparable. Nevertheless, condition III achieved marginally better results (i.e. lower mean CT) than the other three conditions.
For the amputee participant, we observed a slightly different pattern. The best decoding performance both in terms of CR and average CT was achieved with condition IV, when sEMG and IM data were used from a subset of sensors. Three sensors were used in this experiment, one of which targeted the FDS muscle, whilst the other two captured the activity of the extensor muscle group (sensors 1, 2 and 10 in Table 3).
The error bars in Fig. 7 represent 95% confidence intervals estimated via bootstrapping (100 iterations). Since there was only one amputee participant in this experiment, there was a single sample for CR (defined as the fraction of successful to total number of trials), thus no confidence interval was estimated for this measure. Similarly, for condition II, there was only one successful trial, therefore no confidence interval was estimated for the associated CT.
Reconstruction of EMG envelopes with IM
In this study, we investigated whether the performance of myoelectric decoders can benefit from the inclusion of additional information as measured by IMUs integrated within the EMG sensors. For this purpose, we collected a large dataset of 22 subjects performing a variety of movements and also conducted a real-time control pick and place experiment. The main contribution of this work has been threefold; including information from additional inertial sensors beyond accelerometers, minimizing the amount of sensors used for decoding and, most importantly, validating findings during real-time myoelectric control.
For our first (offline) experiment, the large number of classes makes gesture recognition a challenging task. We found that by including IM data, the CA increased by a significant factor. For the able-bodied group, we achieved a median CA of 82.7%, observing an increase in performance of 22.6% when compared to the sEMG-only case. For the amputee group, the same measure was 77.8% and the observed increase in performance was 37.1%. Remarkably, CA for the amputee group almost doubled when we included IM data in our decoders. To the best of our knowledge, this score is the highest ever reported for amputee subjects, given the large number of motions (i.e. 40 classes) in the dataset. For comparison, Atzori et al.  reported an average CA of 48% for the same set of movements. Other studies have achieved CAs as high as 96.6%, however for much smaller numbers of motions (i.e. 12 classes) .
Many studies have suggested that an observed increase in CA attained with purely offline analysis does not necessarily translate to performance improvement during online myoelectric control [19, 36]. Therefore, to validate our findings from the offline analysis, we conducted a real-time experiment by controlling a state-of-the-art commercial prosthetic hand. Comparing the real-time performance of all decoding schemes explored in our offline analysis would be impossible due to time constraints. By taking into account the results from the offline analysis, we decided to test the real-time performance of the four conditions presented in Table 5. Moreover, it is not practical to include 40 classes in a real-time control experiment, and perhaps not necessary from a clinical point of view. Therefore, we only included six classes (Table 4, Fig. 3) which have been previously identified as being the most useful from a user’s perspective . It is worth noting that our proposed experimental protocol bears strong similarities to the “object task” of the Southampton hand assessment procedure (SHAP) test5 that is commonly used in clinical environments.
In comparison with similar studies which previously employed the (TAC) test [7, 8, 9, 38], our experimental task was more challenging. Participants were required to trigger a sequence of control signals (seven in total including the required intermediate ‘open’ commands), rather than performing a single grasp motion. Additionally, participants were given a rather short time to accomplish trials (60 s for the able-bodied group, 75 s for the amputee subject). We considered this paradigm as a more realistic experiment that closely matches real-life applications.
Radmand et al.  demonstrated that integrating accelerometry data into myoelectric decoders can potentially decrease decoding performance unless training data are collected under most of the possible configurations in 3D space. They also showed that classifiers trained with static motions generalise poorly when used to decode hand gestures during dynamic movement. To overcome this limitation, and since collecting static training data in all possible positions would be practically impossible, they proposed a method for training classifiers with dynamic movements covering the regions of interest. Since our offline experiment involved static hand motions, we considered imperative to further validate our findings during real-time prosthetic control. During the training phase of the real-time experiment, participants were instructed to move their arms within a constrained workspace (60 cm × 50 cm × 30 cm) whilst performing the different grips. This was inspired by the work of Radmand et al. . Although this approach helped disambiguating muscle activity patterns under different postures, its potential to generalise to postures not present in the training set remains to be investigated. For instance, in our experiments training and testing used the same arm postures for same grip types, which is likely to favour the sEMG-IM decoders, since IM signals are posture dependent. Future work should test the trade-off between the benefit of using IM signals and generalisability under novel postures, i.e. by mixing grip types and arm postures. For clinical applications, acquisition of large and versatile datasets may be required to capture arm posture-related variability, and thereby ensure classification robustness.
Results from the real-time experiment were mostly in accordance with observations from the precedent offline analysis. We found that the inclusion of IM information resulted in significant improvement in CRs for the able-bodied group (median increase of 25%). One notable difference was that while offline analysis suggested that the use of IM data alone could achieve comparable CA to sEMG-IM classifiers (0.9% median difference), in our real-time experiment the hybrid decoders outperformed, although not significantly, IM classifiers (75.0% and 100.0% median CRs; 48.0 and 37.5 s median CTs for conditions (II) and (III), respectively). Such discrepancies between offline CA scores and task-related metrics have been previously reported in other studies [19, 36]. It has been commonly accepted that the latter should be regarded as more important than the former, since task-related metrics measuring the performance of real-time prosthetic systems are of greater clinical relevance than offline accuracy .
The trend observed for the amputee participant was slightly different to the one corresponding to the able-bodied pool. We seek to provide the following justification for this discrepancy: amputations are different and, consequently, skin conditions and positioning on the muscles can vary significantly. Therefore, there is a need to adopt a personalised approach for trans-radial amputees . Although we observed that for the specific amputee participant the IM decoder (condition II) did not achieve as high performance as the sEMG decoder (condition I), we found that the highest performance was achieved when all modalities were included (condition IV). This provides further support for our proposal for sensor fusion, as we believe that by including additional modalities it is more likely to capture a richer representation of the underlying muscular activity.
The best performance for the amputee participant both in terms of CRs and CTs was achieved when we combined sEMG and IM data but made use of a smaller subset of the available sensors. The performance was inferior when the whole set of sensors was used. One possible explanation for this observation is that the participant was able to develop a more efficient control strategy in the former case, due to the lower dimensionality of the input space . Nevertheless, the chance of observing a statistical error in this case, due to the small sample size, cannot be neglected.
A previous study reported high offline CA by discarding the EMG signal and using solely acceleration signals . We were able to replicate this finding (Fig. 4), and we additionally found that a high CA could also be achieved by using magnetometer data only. Importantly, we further demonstrated that efficient real-time control was feasible by using exclusively IM data (Fig. 7). It is worth noting, however, that CTs were slightly increased for this condition. The first commercial system using IM data as sensory input has recently appeared on the market4, although its working principle is fundamentally different. To the best of our knowledge, our study is the first to demonstrate that real-time prosthetic control can be achieved by using a biomimetic approach and IM data exclusively. This finding cannot be solely attributed to a potential association of arm postures to grips since in our experiments participants mainly employed two arm postures, each of them associated with two different grips; for the “cylindrical” and “lateral” classes, the palm of the prosthesis was required to be perpendicular to the surface, while for the “tripod” and “index pointer” classes it was required to be parallel to the surface. Moreover, following each object relocation, the “open” motion was required to be triggered in either postures, depending upon the object being relocated (Fig. 3).
We propose a different explanation for this rather surprising finding; since acceleration is recorded on the skin surface, the associated measurement could be an alternative manifestation of the underlying muscular activity process that also gives rise to the electric field measured over the skin with EMG sensors. This may also be true for magnetometer data, which by measuring the magnetic field around the muscle area could indirectly provide an alternative measurement of muscular activity. The relationship between the two fields stems directly from the Maxwell-Ampère law which states that a changing electric field, due to muscle contraction in our case, generates a magnetic field. To validate this speculation, we ran the following experiment; we hypothesised that if such relationship exists between sEMG, accelerometer and magnetometer data, then it should be possible to use one type of signal to estimate another and vice-versa. We trained linear regression models to reconstruct the sEMG envelopes signals from IM data and found that the use of both accelerometer and magnetometer data yielded surprisingly accurate reconstructions of sEMG envelopes (Fig. 11). Certainly, there is no reason to expect that the relationship between the sEMG, accelerometer and magnetometer data should be linear and, therefore, one would expect to achieve higher decoding accuracies by using non-linear regression models. Nevertheless, the results from this experiment demonstrate that sEMG and inertial signals are indeed closely related, which provides evidence that they might reflect different and perhaps complementary aspects and impacts of the same underlying phenomenon, that is, the muscular activity. Consequently, it should come as no surprise that the combined sEMG-IM based decoder yields more accurate hand gesture recognition (Figs. 4, 7). The fact that gyroscope data alone failed in decoding both hand gesture and sEMG envelopes provides further support for this hypothesis. Taking everything into consideration, we argue that the added benefit of using IMs can be attributed to both their ability to capture dynamic spatial information, as well as to increase the robustness of muscular activity estimation which is subsequently employed to decode movement intention.
Another particular focus of our study was to investigate whether by combining multi-modal input data it would be possible to reduce the number of sensors required for real-time decoding without compromising performance. In accordance with previous studies which were limited to offline analyses [1, 5, 6, 35], we found that a few sensors only were required to virtually achieve the CA attained by the whole set of sensors (Figs. 4, 9). In the real-time experiment, however, we did observe a small decrease in performance for the able-bodied population (Fig. 7). As expected from the variability of myoelectric signals across subjects , the number of selected sensors varied amongst participants but was typically in the range of 4 to 6 (Fig. 10). A common observed pattern was the selection of sensors 1, 2, 3 and 10. The first three channels captured the activity of the extensor muscle group, while sensor 10 targeted the FDS muscle. For the amputee participant, the SFSS algorithm yielded three sensors only and, remarkably, the sEMG-IM subset condition (IV) achieved the best overall performance with 83% CR and an average CT of less than 40 s. To the best of our knowledge, efficient real-time prosthetic control by an amputee subject with as few as three sensors has not been previously reported.
This work is a proof of principle for integrating IMs in myoelectric control. Throughout our study, we used raw sensor values from IMUs which measured proper acceleration (accelerometer), angular velocity (gyroscope) and magnetic field (magnetometer). An alternative would be to perform sensor fusion and work with a different representation, such as quaternions or Euler angles . Investigating the effect of different IM data representations in prosthetic control performance is subject of our current work.
In this study, we demonstrated that the concurrent use of sEMG with IMs recordings, in conjunction with adopting an appropriate training data collection paradigm, can improve the performance of classification-based prosthetic hand control. We collected a large dataset comprising recordings with 22 subjects (20 able-bodied, two amputees) performing a range of 40 movements. We also conducted a real-time control experiment with 12 volunteers (11 able-bodied, one amputee) by using a state-of-the-art commercial prosthetic hand. Our results suggest that both offline classification accuracy as well as real-time performance can be improved when IMs are integrated in the decoding process. Finally, we found that by combining sEMG and IM data we were able to significantly reduce the number of sensors required to achieve top-level performance, a highly-desirable feature for clinical myoelectric applications.
3 Data collection with one of the amputee subjects was interrupted early due to a power supply failure and as a result, the participant did not perform the final two movements.
The authors are grateful to the NinaPro team and especially Dr. Manfredo Atzori for sharing code, the NinaPro protocol and for providing advice regarding data collection. The authors are also thankful to the amputee volunteers for their participation in our experiments. Finally, the authors would like to thank Evangelos Vazaios for technical assistance.
AK is supported in part by grants EP/F500386/1 and BB/F529254/1 for the University of Edinburgh School of Informatics Doctoral Training Centre in Neuroinformatics and Computational Neuroscience from the UK Engineering and Physical Sciences Research Council (EPSRC), UK Biotechnology and Biological Sciences Research Council (BBSRC), and the UK Medical Research Council (MRC). IK is supported in part by the EPSRC Centre for Doctoral Training in Robotics and Autonomous Systems, funded by the UK Engineering and Physical Sciences Research Council (grant EP/L016834/1), the University of Edinburgh and Heriot-Watt University. KN is supported by the EPSRC grants EP/M025594/1 and EP/M025977/1. SV is supported by the Microsoft Research RAEng. Fellowship and EPSRC grant EP/H1012338/1.
Availability of data and materials
Data from the offline experiment are publicly available on the Ninapro repository (http://ninapro.hevs.ch/).
AK, IK, KN and SV conceptualised the study. AK and IK implemented the protocol. AK, IK and KN conducted the experiments. AK analyzed the data. All authors participated in writing the manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
All experiments were approved by the local Ethics Committees of the School of Informatics, University of Edinburgh and School of Electrical and Electronic Engineering, Newcastle University. Prior to the experiments, all subjects read and signed an informed consent participation form.
Consent for publication
Prosthesis hand control by an amputee. This video demonstrates an amputee subject using three sEMG-IM sensors (real-time experiment, condition III) to control the robo-limb™ prosthetic hand. (MP4 20685 kb)
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 24.Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Wheeler R, Ng AY. ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software.2009. p. 5. http://www.willowgarage.com/sites/default/files/icraoss09-ROS.pdf.
- 27.Krasoulis A, Vijayakumar S, Nazarpour K. Evaluation of regression methods for the continuous decoding of finger movement from surface emg and accelerometry. In: Proc. IEEE/EMBS Int. Conf. Neur. Eng.2015. p. 631–4. http://ieeexplore.ieee.org/abstract/document/7146702/.
- 30.Farrell TR. Determining delay created by multifunctional prosthesis controllers. J Rehabil Res Dev. 2011; 48(6):xxi-xxxviii. http://www.rehab.research.va.gov/jour/11/486/pagexxi.html.
- 33.Friedman J, Hastie T, Tibshirani R. The Elements of Statistical Learning. Berlin: Springer; 2001.Google Scholar
- 36.Ortiz-Catalan M, Rouhani F, Brånemark R, Håkansson B. Offline accuracy: A potentially misleading metric in myoelectric pattern recognition for prosthetic control. Conf Proc IEEE Eng Med Biol Soc. 2015:1140–3. http://ieeexplore.ieee.org/document/7318567/.
- 39.Vujaklija I, Roche AD, Hasenoehrl T, Sturma A, Amsuess S, Farina D, Aszmann OC. Translating research on myoelectric control into clinics–are the performance assessment methods adequate?Front Neurorobotics. 2017;11. http://journal.frontiersin.org/article/10.3389/fnbot.2017.00007/full.
- 42.Madgwick SOH, Harrison AJL, Vaidyanathan R. Estimation of IMU and MARG orientation using a gradient descent algorithm. In: Conf. Proc. IEEE Int. Conf. Rehabil. Robot.2011. p. 1–7. http://ieeexplore.ieee.org/document/5975346/.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.