Ranking Robot-Assisted Surgery Skills Using Kinematic Sensors
Assessing surgical skills is an essential part of medical performance evaluation and expert training. Since it is typically conducted as a subjective task by individuals, it may lead to misinterpretations of the skill performance and hence lead to suboptimal training and organization of the surgical activities. Therefore, objective assessment of surgical skills using computational intelligence techniques via sensory data has received attention from researchers in recent years. So far, the problem has been approached by employing a classification model where a query action for surgery is assigned to a predefined category that determines the level of expertise. In this study, we consider the skill assessment problem as a pairwise ranking task where we compare two input actions to identify better surgical performance. To this end, we propose a hybrid Siamese network that takes two kinematic motion data acquired from robot-assisted surgery sensors and report the probability of the first sample having a better skill than the second one. Experiments on annotated real surgery data reveals that the proposed framework has high accuracy and seems sufficiently accurate for use in practice. This approach may overcome the limitations of having consistent annotations to define skill levels and provide a more interpretable means for objective skill assessment.
KeywordsSkill assessment Ambient intelligence in education Ambient intelligence in health Robot-assisted surgery Siamese networks LSTM
Burçin Buket Oğul was financially supported by the Scientific and Technological Research Council of Turkey (TUBITAK) under 2214-A program.
- 1.Burges, C.J., Shaked, T., Renshaw, E., et al.: Learning to rank using gradient descent. In: International Conference on Machine Learning, pp. 89–96 (2005)Google Scholar
- 2.Doughty, H., Damen, D., Mayol-Cuevas, W.: Who’s better? Who’s best? Pairwise deep ranking for skill determination. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
- 4.Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Evaluating surgical skills from kinematic data using convolutional neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 214–221. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_25CrossRefGoogle Scholar
- 5.Funke, I., Mees, S.T., Weitz, J., Speidel, S.: Video-based surgical skill assessment using 3D convolutional neural networks. arXiv preprint arXiv:1903.02306 (2019)
- 6.Gao, Y., Vedula, S.S., Reiley, C.E., et al.: JHU-ISI gesture and skill assessment working set (JIGSAWS): a surgical activity dataset for human motion modelling. In: MICCAI Workshop (2014)Google Scholar
- 8.Graves, A., Fernández, S., Schmidhuber, J.: Bidirectional LSTM networks for improved phoneme classification and recognition. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 799–804. Springer, Heidelberg (2005). https://doi.org/10.1007/11550907_126CrossRefGoogle Scholar
- 10.Li, Z., Huang, Y., Cai, M., Sato, Y.: Manipulation-skill assessment from videos with spatial attention network. arXiv preprint arXiv:1901.02579 (2019)
- 13.Wang, Z., Fey, A.I.: SATR-DL: improving surgical skill assessment and task recognition in robot-assisted surgery with deep neural networks. In: IEEE Conference of the Engineering in Medicine and Biology Society, pp. 1793–1796 (2018)Google Scholar