Advertisement

Virtual reality visualization of patient specific heart model

  • Matthew Bramlet
  • Kuocheng Wang
  • Alexander Clemons
  • Nathaniel Christopher Speidel
  • Steven M Lavalle
  • Thenkurussi Kesavadas
Open Access
Technologist presentation

Keywords

Virtual Reality Augmented Reality Virtual Reality Technology Immersive Virtual Reality Congenital Cardiac Anomaly 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Background

The diagnostic dilemma that arises with congenital cardiac anomalies is visualizing the 3-D complexity of a child's heart with 2-D tools. The complicated intracardiac relationships and extracardiac connections make it challenging to diagnose the defect and plan treatment for the child. It has been demonstrated that it is difficult to reconstruct an accurate mental 3D model from 2D images [Guillot 2007], yet we rely on this imperfect modality to make critical surgical decisions. 3D printed models are proving to be highly valuable in improving the understanding of these complex defects, but require many resources. In this project we have explored 3D immersive virtual reality technology to provide deeper understanding and visualization of cardiac models and supplant the need for a physical printed model. Using stereoscopic 3D in the form of 3D displays on head mounted displays (HMDs) excellent intuitive model was created. Models were generated from patient specific MRI which was converted to format that was compatible with virtual reality rendering.

Methods

The source images were obtained from a 3D respiratory-navigated balanced SSFP (steady state free precession) multi-slab sequence with T2 preparation. The 3D data set was imported into a segmentation software (Mimics) where a mask was applied over the myocardium and vessel walls and converted into a stereolithography file (STL) which was then converted to either filmbox format (FBX) or 3DS format using Blender. The FBX file is loaded in a 3D environment supported by the Oculus Development Kit 2 (DK2) HMD. In this case, the Unreal 4 engine is used to show and manipulate the FBX file. Three manipulation input devices including a keyboard and mouse, the Nintendo Wii Remote and the Microsoft Kinect were tested. Further an Augmented Reality image based tracking system was also tested.

Results

The Virtual Reality based visualization of the heart was implemented successfully. The system provided intuitive control to allow the user to manipulate the heart into relatable positions allowing for diagnostic assessment of the complex congenital defects. In terms of implementation several challenges were faced. With the Unreal 4 engine, gimbal lock posed an issue controlling visualization with three degrees of freedom in rotation. To provide a temporary fix to this, only two degrees, yaw and roll, were allowed. The keyboard and mouse controls were preferred over other forms of manipulation in the Unreal 4 engine. The Wii Remote posed problems because, while the tracking using the sensor bar is fairly accurate, the data from the Wii Motion Plus is very difficult to access and acceleration data led to drift error.

Conclusions

The virtual reality based visualization of the heart model shows a great deal of promise to deepen understanding of a patient specific anatomy and has the potential of becoming an important diagnostic asset in the future.
Figure 1

Immersive visualization of a patient specific heart model using Occulus head mounted display.

Copyright information

© Bramlet et al. 2016

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors and Affiliations

  • Matthew Bramlet
    • 2
  • Kuocheng Wang
    • 1
  • Alexander Clemons
    • 1
  • Nathaniel Christopher Speidel
    • 1
  • Steven M Lavalle
    • 3
  • Thenkurussi Kesavadas
    • 1
  1. 1.Health Care Engineering SystemsUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.Pediatric Cardiology, Children's Hospital of IllinoisUniversity of Illinois College of Medicine at PeoriaPeoriaUSA
  3. 3.Computer ScienceUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations