Real-time microrobot posture recognition via biplane X-ray imaging system for external electromagnetic actuation

  • Phu Bao Nguyen
  • Byungjeon Kang
  • D. M. Bappy
  • Eunpyo Choi
  • Sukho Park
  • Seong Young Ko
  • Jong-Oh ParkEmail author
  • Chang-Sei KimEmail author
Original Article



As a promising intravascular therapeutic approach for autonomous catheterization, especially for thrombosis treatment, a microrobot or robotic catheter driven by an external electromagnetic actuation system has been recently investigated. However, the three-dimensional (3D) real-time position and orientation tracking of the microrobot remains a challenge for precise feedback control in clinical applications owing to the micro-size of the microrobot geometry in vessels, along with bifurcation and vulnerability. Therefore, in this paper, we propose a 3D posture recognition method for the unmanned microrobotic surgery driven by an external electromagnetic actuator system.


We propose a real-time position and spatial orientation tracking method for a millimeter-sized intravascular object or microrobot using a principal component analysis algorithm and an X-ray reconstruction. The suggested algorithm was implemented to an actual controllable wireless microrobot system composed of a bullet-shaped object, a biplane X-ray imaging device, and an electromagnetic actuation system. Numerical computations and experiments were conducted for the performance verification.


The experimental results showed a good performance of the implemented system with tracking errors less than 0.4 mm in position and 2° in orientation. The proposed tracking technique accomplished a fast processing time, ~ 0.125 ms/frame, and high-precision recognition of the micro-sized object.


Since the suggested method does not require pre-information of the object geometry in the human body for its 3D shape and position recognition, it could be applied to various elliptical shapes of the microrobot system with computation time efficacy and recognition accuracy. Hence, the method can be used for therapeutic millimeter- or micron-sized manipulator recognition in vascular, as well as implanted objects in the human body.


Real-time 3D posture recognition Intravascular microrobot Electromagnetic actuation system Principal component analysis X-ray reconstruction 



This research was supported by the Bio & Medical Technology Development Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. 2015M3D5A1065682). Seong Young Ko, for this work, was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1D1A1B03933079).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical approval

This article does not contain any studies, performed by any of the authors, with human participants or animals.

Informed consent

This article does not contain patient data.


  1. 1.
    World Health Organization (2015) Noncommunicable diseases progress monitor 2015Google Scholar
  2. 2.
    Dharmarajah B (2015) European Congress of Radiology, p 1Google Scholar
  3. 3.
    Bashir R, Zack CJ, Zhao H, Comerota AJ, Bove AA (2014) Comparative outcomes of catheter-directed thrombolysis plus anticoagulation vs anticoagulation alone to treat lower-extremity proximal deep vein thrombosis. JAMA Intern Med 174:1494–1501. CrossRefPubMedGoogle Scholar
  4. 4.
    Bowdle A (2014) Vascular complications of central venous catheter placement: evidence-based methods for prevention and treatment. J Cardiothorac Vasc Anesth 28:358–368CrossRefPubMedGoogle Scholar
  5. 5.
    Nelson BJ, Kaliakatsos IK, Abbott JJ (2010) Microrobots for minimally invasive medicine. Annu Rev Biomed Eng 12:55–85. CrossRefPubMedGoogle Scholar
  6. 6.
    Belharet K, Folio D, Ferreira A (2010) 3D MRI-based predictive control of a ferromagnetic microrobot navigating in blood vessels. In: 2010 3rd IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics, BioRob 2010, pp 808–813Google Scholar
  7. 7.
    Climent J, Hexsel RA (2012) Particle filtering in the Hough space for instrument tracking. Comput Biol Med 42:614–623. CrossRefPubMedGoogle Scholar
  8. 8.
    Lugez E, Sadjadi H, Joshi CP, Akl SG, Fichtinger G (2017) Improved electromagnetic tracking for catheter path reconstruction with application in high-dose-rate brachytherapy. Int J Comput Assist Radiol Surg 12:681–689CrossRefPubMedGoogle Scholar
  9. 9.
    Oh J, Park JO, Park S, Ko SY (2015) Image-based guidance system for intravascular microrobot: Fiducial marker-based registration using biplanar fluoroscopic images and CTA images. In: 2015 15th international conference on control, automation and systems (ICCAS), pp 919–922Google Scholar
  10. 10.
    Brattain LJ, Loschak PM, Tschabrunn CM, Anter E, Howe RD (2014) Instrument tracking and visualization for ultrasound catheter guided procedures. In: Augmented environments for computer-assisted interventions, pp 41–50Google Scholar
  11. 11.
    Saxena A, Driemeyer J, Ng AY (2009) Learning 3-d object orientation from images. In Robotics and Automation. In: ICRA’09. 2009 IEEE international conference on robotics and automation, pp 794–800Google Scholar
  12. 12.
    Nakabo Y, Ishi I, Ishikawa M (2002) 3D tracking using two high-speed vision systems. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, pp 360–365Google Scholar
  13. 13.
    Tognarelli S, Castelli V, Ciuti G, Di Natali C, Sinibaldi E, Dario P, Menciassi A (2012) Magnetic propulsion and ultrasound tracking of endovascular devices. J Robot Surg 6:5–12. CrossRefPubMedGoogle Scholar
  14. 14.
    Iaquinto JM, Kindig MW, Haynor DR, Vu QB, Pepin N, Tsai R, Sangeorzan BJ, Ledoux WR (2018) Model-based tracking of the bones of the foot: a biplane fluoroscopy validation study. Comput Biol Med 92:118–127. CrossRefPubMedGoogle Scholar
  15. 15.
    Fitzgibbon A, Pilu M, Fisher RB (1999) Direct least square fitting of ellipses. IEEE Trans Pattern Anal Mach Intell 21:476–480. CrossRefGoogle Scholar
  16. 16.
    Benjamin JR, Jayasree T (2018) Improved medical image fusion based on cascaded PCA and shift invariant wavelet transforms. Int J Comput Assist Radiol Surg 13:229–240CrossRefGoogle Scholar
  17. 17.
    Wijewickrema SNR, Paplinski AP (2005) Principal component analysis for the approximation of a fruit as an ellipse. In: Proceedings of 13th international conference central Europe on computer graphics, visualization and computer vision, pp 1–6Google Scholar
  18. 18.
    Baur C, Milletari F, Belagiannis V, Navab N, Fallavollita P (2016) Automatic 3D reconstruction of electrophysiology catheters from two-view monoplane C-arm image sequences. Int J Comput Assist Radiol Surg 11:1319–1328CrossRefPubMedGoogle Scholar
  19. 19.
    Nguyen PB, Park JO, Park S, Ko SY (2016) Medical micro-robot navigation using image processing-blood vessel extraction and X-ray calibration. In Proceedings of the IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics, vol 2016, pp 365–370Google Scholar
  20. 20.
    Richard Harltey AZ (2004) Multiple view geometry in computer vision. Cambridge University Press, CambridgeGoogle Scholar
  21. 21.
    Jeong S, Choi H, Go G, Lee C, Lim KS, Sim DS, Jeong MH, Ko SY, Park JO, Park S (2016) Penetration of an artificial arterial thromboembolism in a live animal using an intravascular therapeutic microrobot system. Med Eng Phys 38:403–410CrossRefPubMedGoogle Scholar
  22. 22.
    Mönnich H, Stein D, Raczkowsky J, Wörn H (2010) An automatic and complete self-calibration method for robotic guided laser ablation. In: Proceedings of IEEE international conference on robotics and automation, pp 1086–1087Google Scholar
  23. 23.
    Horn BKP (1987) Closed-form solution of absolute orientation using unit quaternions. J Opt Soc Am A 4:629. CrossRefGoogle Scholar
  24. 24.
    Stein D, Moennich H, Raczkowsky J, Woern H (2009) IEEE automatic and hand guided self-registration between a robot and an optical tracking system. In: ICAR 2009 14th international conference on advanced robotics, vol 1–2, pp 13–17Google Scholar

Copyright information

© CARS 2018

Authors and Affiliations

  1. 1.School of Mechanical EngineeringChonnam National UniversityGwangjuSouth Korea
  2. 2.Medical Microrobot Center, Robot Research InitiativeChonnam National UniversityGwangjuSouth Korea
  3. 3.Department of Robotics EngineeringDaegu Gyeongbuk Institute of Science and TechnologyDaeguSouth Korea

Personalised recommendations