Haptics-Enabled Surgical Training System with Guidance Using Deep Learning
In this paper, we present a haptics-enabled surgical training system integrated with deep learning for characterization of particular procedures of experienced surgeons to guide medical residents-in-training with quantifiable patterns. The prototype of virtual reality surgical system is built for open-heart surgery with specific steps and biopsy operation. Two abstract surgical scenarios are designed to emulate incision and biopsy surgical procedures. Using deep learning algorithm (autoencoder), the two surgical procedures were trained and characterized. Results show that a vector with 30 real-valued components can quantify both surgical patterns. These values can be used to compare how a resident-in-training performs differently as opposed to an experienced surgeon so that quantifiable corrective training guidance can be provided.
KeywordsVirtual surgical training system Haptic device Machine learning Deep learning algorithm Autoencoder Motion tracking and quantification
This project is sponsored by San Antonio Life Science Institute (SALSI) as part of the Medical Data Analytics and Visualization Cluster grant. We also appreciate the assistance of Sam Newman from Multi-media Lab at University of Texas Health Science Center at San Antonio, who generated digital representation of the virtual surgical room and the patient.
- 2.Valverde, H.H.: A review of flight simulator transfer of training studies. Hum. Factors: J. Hum. Factors Ergon. Soc. 15(6), 510–522 (1973)Google Scholar
- 4.R. N. Haber, “Flight Simulation,” Scientific American, 255(1), 1986Google Scholar
- 14.Larsen, C. R., Soerensen, J. L., Grantcharov, T. P., Dalsgaard, T., Schouenborg, L., Ottosen, C., Schroeder, T. V., Ottesen, B. S.: Effect of virtual reality training on laparoscopic surgery: randomised controlled trial, BMJ: British Medical Journal, 338, 2009Google Scholar
- 15.Entwistle, N.: Promoting deep learning through teaching and assessment: conceptual frameworks and educational contexts. In: Proceeding of TLRP Conference, Leicester, November 2000Google Scholar
- 16.Biglari, E., Feng, Y.: Interactive virtual reality driven learning framework for engineering and science education. In: Proceedings of American Society for Engineering Education Gulf-Southwest Conference, New Orleans, LA 2014Google Scholar
- 17.Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge (2012)Google Scholar