Multimedia Tools and Applications

, Volume 77, Issue 23, pp 30651–30682 | Cite as

Virtual reality training for assembly of hybrid medical devices

  • Nicholas HoEmail author
  • Pooi-Mun Wong
  • Matthew Chua
  • Chee-Kong Chui


Skill training in the medical device manufacturing industry is essential to optimize and expedite the efficiency level of new workers. This process, however, gives rise to many underlying issues such as contamination and safety risks, long training period, high skill and experience requirements of operators, and greater training costs. In this paper, we proposed and evaluated a novel virtual reality (VR) training system for the assembly of hybrid medical devices. The proposed system, which is an integration of Artificial Intelligence (AI), VR and gaming concepts, is self-adaptive and autonomous. This enables the training to take place in a virtual workcell environment without the supervision of a physical trainer. In this system, a sequential framework is proposed and utilized to enhance the training through its various “game” levels of familiarity-building processes. A type of hybrid medical device: carbon nanotubes-polydimethylsiloxane (CNT-PDMS) based artificial trachea prosthesis is used as a case study in this paper to demonstrate the effectiveness of the proposed system. Evaluation results with quantitative and qualitative comparisons demonstrated that our proposed training method has significant advantages over common VR training and conventional training methods. The proposed system has addressed the underlying training issues for hybrid medical device assembly by providing trainees with effective, efficient, risk-free and low cost training.


Interactive training environment Virtual reality Virtual assembly workcell Hybrid medical device Assembly training 



We wish to acknowledge the contributions from Mr. Rahul Singh Chauhan, Mr. Jayendra Laxman Zambre and Mr. Ritesh Kumar Agrahari for their assistance in the development of the VR systems. This project is supported in parts by a MOE FRC Tier 1 Grant from National University of Singapore (WBS: R-265-000-614-114).


  1. 1.
    Alamri A, Eid M, Iglesias R, Shirmohammadi S, Saddik AE (2008) Haptic virtual rehabilitation exercises for post-stroke diagnosis. IEEE Trans Instrum Meas 57(9):1876–1884CrossRefGoogle Scholar
  2. 2.
    Albert DE (2015) Methods for verifying medical device cleanliness. In: Developments in surface contamination and cleaning, volume 7: cleanliness validation and verification. Elsevier, Amsterdam, pp 109–128CrossRefGoogle Scholar
  3. 3.
    Boian RF, Sharma AK, Han C, Merians AN, Burdea GC, Adamovich S, Recce ML, Tremaine M, Poizner H (2002) Virtual reality-based post-stroke hand rehabilitation. Stud Health Technol Inform 85(1):64–70Google Scholar
  4. 4.
    Borst CW, Indugula AP (2006) A spring model for whole-hand virtual grasping. Presence-Teleop VIRT 15(1):47–61CrossRefGoogle Scholar
  5. 5.
    Brough JE, Schwartz M, Gupta SK, Anand DK, Kavetsky R, Pettersen R (2007) Towards the development of a virtual environment-based training system for mechanical assembly operations. Virtual Reality 11(4):189–206CrossRefGoogle Scholar
  6. 6.
    Chua M, Chui CK, Chng CB, Lau D (2013) Carbon nanotube-based artificial tracheal prosthesis: carbon nanocomposite implants for patient-specific ENT care. IEEE Nanotechnol Mag 7(4):27–31CrossRefGoogle Scholar
  7. 7.
    Chua M, Chui CK, Chng CB, Lau D (2013) Experiments of carbon nanocomposite implants for patient specific ENT care. In: The 7th IEEE international conference on nano/molecular medicine and engineering (NANOMED), PhuketGoogle Scholar
  8. 8.
    Chua M, Chui CK, Teo C, Lau D (2015) Patient-specific carbon nanocomposite tracheal prosthesis. Int J Artif Organs 38(1):31–38CrossRefGoogle Scholar
  9. 9.
    Ciurana J (2014) Designing, prototyping and manufacturing medical devices: an overview. Int J Comput Integr Manuf 27(10):901–918CrossRefGoogle Scholar
  10. 10.
    Ciuti G, Ricotti L, Menciassi A, Dario P (2015) MEMS sensor technologies for human centred applications in healthcare, physical activities, safety and environmental sensing: a review on research activities in italy. Sensors (Basel) 15(3):6441–6468CrossRefGoogle Scholar
  11. 11.
    Connolly J, Condell J, O’Flynn B, Sanchez JT, Gardiner P (2018) IMU sensor-based electronic goniometric glove for clinical finger movement analysis. IEEE Sensors J 18(3):1273–1281Google Scholar
  12. 12.
    Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low- and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst Hum 43(4):996–1002CrossRefGoogle Scholar
  13. 13.
    Ding M, Fan G (2016) Articulated and generalized gaussian kernel correlation for human pose estimation. IEEE Trans Image Process 25(2):776–789MathSciNetCrossRefGoogle Scholar
  14. 14.
    Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Surg Endosc 16(12):1746–1752CrossRefGoogle Scholar
  15. 15.
    Gislén L (2016) Alternative design of robot cell concepts for flexible production. (Master’s thesis, University West), Trollhättan, SwedenGoogle Scholar
  16. 16.
    Grantcharov TP, Kristiansen VB, Bendi J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91(2):146–150CrossRefGoogle Scholar
  17. 17.
    He Y, Zhang Z, Nan X, Zhang N, Guo F, Rosales E, Guan L (2017) vConnect: perceive and interact with real world from CAVE. Multimed Tools Appl 76(1):1479–1508CrossRefGoogle Scholar
  18. 18.
    Ho N, Ang W, Chui CK (2017) Workcell for hybrid medical device fabrication. In: 3rd CIRP conference on biomanufacturing, ChicagoGoogle Scholar
  19. 19.
    Hoedt S, Claeys A, Landeghem HV, Cottyn J (2016) Evaluation framework for virtual training within mixed-model manual assembly. IFAC-PapersOnLine 49(12):261–266CrossRefGoogle Scholar
  20. 20.
    Hořejší P (2015) Augmented reality system for virtual training of parts assembly. In: 25th DAAAM international symposium on intelligent manufacturing & automation, ViennaGoogle Scholar
  21. 21.
    Hyppölä J, Martínez H, Laukkanen S (2014) Experiential learning theory and virtual and augmented reality applications. In: 4th global conference on experiential learning in virtual worlds, Prague, Czech RepublicGoogle Scholar
  22. 22.
    Jayaram S, Jayaram U, Kim YJ, DeChenne C, Lyons KW, Palmer C, Mitsui T (2007) Industry case studies in the use of immersive virtual assembly. Virtual Reality 11(4):217–228CrossRefGoogle Scholar
  23. 23.
    Kılıboz NÇ, Güdükbay U (2015) A hand gesture recognition technique for human–computer interaction. J Vis Commun Image Represent 28(C):97–104CrossRefGoogle Scholar
  24. 24.
    Kizony R, Raz L, Katz N, Weingarden H, Weiss PLT (2005) Video-capture virtual reality system for patients with paraplegic spinal cord injury. J Rehabil Res Dev 42(5):595–608CrossRefGoogle Scholar
  25. 25.
    Konig J, Koskela E (2013) The role of profit sharing in dual labour markets with flexible outsourcing. Labour 27(4):357–370CrossRefGoogle Scholar
  26. 26.
    Kusuda Y (2010) IDEC's robot-based cellular production system: a challenge to automate high-mix low-volume production. Assem Autom 30(4):306–312CrossRefGoogle Scholar
  27. 27.
    Langley A, Lawson G, Hermawati S, D'Cruz M, Apold J, Arlt F, Mura K (2016) Establishing the usability of a virtual training system for assembly operations within the automotive industry. Hum Factors Ergon Manuf Serv Ind 26(6):667–679CrossRefGoogle Scholar
  28. 28.
    Lawson MV (2003) Finite automata. Chapman and Hall/CRC, Boca RatonCrossRefGoogle Scholar
  29. 29.
    Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181(1):108–115CrossRefGoogle Scholar
  30. 30.
    Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2016) Action2Activity: recognizing complex activities from sensor data. In: Proceedings of the twenty-fourth international joint conference on artificial intelligence, Buenos AiresGoogle Scholar
  31. 31.
    Lucke L, Anderson D, Smith D (2009) Harnessing experience for efficient medical device product development. J Med Devices 3(2):027551CrossRefGoogle Scholar
  32. 32.
    Manjiyani ZAA, Jacob RT, R KK, Varghese B (2014) Development of MEMS based 3-axis accelerometer for hand movement monitoring. International Journal of Scientific and Research Publ 4(2):1–4Google Scholar
  33. 33.
    Müller BC, Reisea C, Duc BM, Seliger G (2016) Simulation-games for learning conducive workplaces: a case study for manual assembly. In: 13th global conference on sustainable manufacturing - decoupling growth from resource use, Ho Chi MinhGoogle Scholar
  34. 34.
    Nee AYC, Ong SK (2013) Virtual and augmented reality applications in manufacturing. In: 7th IFAC conference on manufacturing modelling, management, and control, Saint Petersburg, RussiaCrossRefGoogle Scholar
  35. 35.
    Nerem RM, Sambanis A (2007) Tissue engineering: from biology to biological substitutes. Tissue Eng 1(1):3–13CrossRefGoogle Scholar
  36. 36.
    Neto P, Pires JN, Moreira AP (2013) 3-D position estimation from inertial sensing: minimizing the error from the process of double integration of accelerations. In: IECON 2013 - 39th annual conference of the IEEE industrial electronics society, ViennaGoogle Scholar
  37. 37.
    Pilia A, Brun C, Doceul L, Gargiulo L, Hatchressian JC, Keller D, Le R, Poli S, Zago B (2015) Application of virtual reality tools for assembly of WEST components: comparison between simulations and physical mockups. Fusion Eng Des 98–99:1589–1592CrossRefGoogle Scholar
  38. 38.
    Premkumar T, Geckeler KE (2012) Graphene–DNA hybrid materials: assembly, applications, and prospects. Prog Polym Sci 37(4):515–529CrossRefGoogle Scholar
  39. 39.
    Rahni AAA, Yahya I (2008) Obtaining translation from a 6-DOF MEMS IMU — an overview. In: 2007 Asia-Pacific conference on applied electromagnetics, MelakaGoogle Scholar
  40. 40.
    Rodriguez L, Quint F, Gorecky D, Romero D, Siller HR (2015) Developing a mixed reality assistance system based on projection mapping technology for manual operations at assembly workstations. In: International conference on virtual and augmented reality in education, MonterreyCrossRefGoogle Scholar
  41. 41.
    Rose P (2014) Whitepaper: contamination control of medical device cleanroom. Maetrics, IndianapolisGoogle Scholar
  42. 42.
    Sahithia K, Swethaa M, Ramasamya K, Srinivasanb N, Selvamurugan N (2010) Polymeric composites containing carbon nanotubes for bone tissue engineering. Int J Biol Macromol 46(3):281–283CrossRefGoogle Scholar
  43. 43.
    Saposnik G, Levin M (2011) Virtual reality in stroke rehabilitation: a meta-analysis and implications for clinicians. Stroke 42(5):1380–1386CrossRefGoogle Scholar
  44. 44.
    Schwaitzberg SD, Dorozhkin D, Sankaranarayanan G, Matthes K, Jones DB, De S (2016) Natural orifice translumenal endoscopic surgery (NOTES): emerging trends and specifications for a virtual simulator. Surg Endosc 30(1):190–198CrossRefGoogle Scholar
  45. 45.
    Seth A, Vance JM, Oliver JH (2011) Virtual reality for assembly methods prototyping: a review. Virtual Reality 15(1):5–20CrossRefGoogle Scholar
  46. 46.
    Shirke S, Chilaveri PG (2013) Traffic light control using accelerometer sensor on ARM platform. Int J Eng Res Technol 2(9):3124–3126Google Scholar
  47. 47.
    Stančin S, Tomažič S (2011) Angle estimation of simultaneous orthogonal rotations from 3D gyroscope measurements. Sensors (Basel) 11(9):8536–8549CrossRefGoogle Scholar
  48. 48.
    Stratos A, Loukas R, Dimitris M, Konstantinos G, Dimitris M, George C (2016) A virtual reality application to attract young talents to manufacturing. In: 49th CIRP Conference on MANUFACTURING SYSTEMS - CIRP CMS 2016, Stuttgart, GermanyGoogle Scholar
  49. 49.
    Sturman DJ, Zeltzer D (1994) A survey of glove-based input. IEEE Comput Graph Appl 14(1):30–39CrossRefGoogle Scholar
  50. 50.
    Suzuki N, Hattori A, Ieiri S, Tomikawa M, Kenmotsu H, Hashizume M (2012) VR training system for endoscopic surgery robot. In: Augmented environments for computer-assisted interventions. AE-CAI 2011. Lecture notes in computer science, vol 7264. Springer, Berlin, pp 117–129Google Scholar
  51. 51.
    Tang Z, Sekine M, Tamura T, Tanaka N, Yoshida M, Chen W (2015) Measurement and estimation of 3D orientation using magnetic and inertial sensors. Adv Biomed Eng 4(1):135–143CrossRefGoogle Scholar
  52. 52.
    Vaughan N, Gabrys B, Dubey VN (2016) An overview of self-adaptive technologies within virtual reality training. Comput Sci Rev 22:65–87MathSciNetCrossRefGoogle Scholar
  53. 53.
    Wang Y, Guo S, Li Y, Tamiya T, Song Y (2018) Design and evaluation of safety operation VR training system for robotic catheter surgery. Med Biol Eng Comput 56(1):25–35CrossRefGoogle Scholar
  54. 54.
    Wasfy A, Wasfy T, Noor A (2004) Intelligent virtual environment for process training. Adv Eng Softw 35(6):337–355CrossRefGoogle Scholar
  55. 55.
    Xia P, Lopes AM, Restivo MT, Yao Y (2012) A new type haptics-based virtual environment system for assembly training of complex products. Int J Adv Manuf Technol 58(1–4):379–396CrossRefGoogle Scholar
  56. 56.
    Yang Q, Wu DL, Zhu HM, Bao JS, Wei ZH (2013) Assembly operation process planning by mapping a virtual assembly simulation to real operation. Comput Ind 64(7):869–879CrossRefGoogle Scholar
  57. 57.
    Yu Y, Duan M, Sun C, Zhong Z, Liu H (2017) A virtual reality simulation for coordination and interaction based on dynamics calculation. Ships Offshore Struc 12(6):873–884CrossRefGoogle Scholar
  58. 58.
    Yuviler-Gavish N, Krupenia S, Gopher D (2013) Task analysis for developing maintenance and assembly VR training simulators. Ergon Des 21(1):12–19Google Scholar
  59. 59.
    Zhou J, Malric F, Shirmohammadi S (2010) A new hand-measurement method to simplify calibration in cyber glove-based virtual rehabilitation. IEEE Trans Instrum Meas 59(10):2496–2504CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Mechanical EngineeringNational University of SingaporeSingaporeSingapore
  2. 2.Institute of Systems ScienceNational University of SingaporeSingaporeSingapore

Personalised recommendations