Intelligent and Immersive Visual Analytics of Health Data

  • Zhonglin Qu
  • Chng Wei Lau
  • Daniel R. Catchpoole
  • Simeon Simoff
  • Quang Vinh NguyenEmail author
Part of the Studies in Computational Intelligence book series (SCI, volume 891)


Massive amounts of health data have been created together with the advent of computer technologies and next generation sequencing technologies. Analytical techniques can significantly aid in the processing, integration and interpretation of the complex data. Visual analytics field has been rapidly evolving together with the advancement in automated analysis methods such as data mining, machine learning and statistics, visualization, and immersive technologies. Although automated analysis processes greatly support the decision making, conservative domains such as medicine, banking, and insurance need trusts on machine learning models. Explainable artificial intelligence could open the black boxes of the machine learning models to improve the trusts for decision makers. Immersive technologies allow the users to engage naturally with the blended reality in where they can look the information in different angles in addition to traditional screens. This chapter reviews and discusses the intelligent visualization, artificial intelligence and immersive technologies in health domain. We also illustrate the ideas with various case studies in genomic data visual analytics.


  1. 1.
    An, J., Lai, J., Wood, D.L., Sajjanhar, A., Wang, C., Tevz, G., Lehman, M.L., Nelson, C.C.: RNASeqBrowser: a genome browser for simultaneous visualization of raw strand specific RNAseq reads and UCSC genome browser custom tracks. BMC Genom. 16, 145 (2015). Scholar
  2. 2.
    Arya, A., Nowlan, N., Sauriol, N.: Data-driven framework for an online 3D immersive environment for educational applications. In: Proceedings of the International Conference on Education and New Learning Technologies, pp. 4726–4736 (2010)Google Scholar
  3. 3.
    Bhavnani, S., Ganesan, A., Hall, T., Maslowski, E., Eichinger, F., Martini, S., Saxman, P., Bellala, G., Kretzler, M.: Discovering hidden relationships between renal diseases and regulated genes through 3D network visualizations. BMC Res. Notes 3(1), 296 (2010). Scholar
  4. 4.
    Borgo, R., Kehrer, J., Chung, D.H., Maguire, E., Laramee, R.S., Hauser, H., Ward, M., Chen, M.: Glyph-based visualization: foundations, design guidelines, techniques and applications. In: Eurographics (STARs), pp. 39–63 (2013)Google Scholar
  5. 5.
    Boudreaux, E.D., Waring, M.E., Hayes, R.B., Sadasivam, R.S., Mullen, S., Pagoto, S.: Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl. Behav. Med. 4(4), 363–371 (2014). Scholar
  6. 6.
    Breiman, L.: Radom forests. Mach. Learn. 45, 5–32 (2001)CrossRefGoogle Scholar
  7. 7.
    Calì, C., Baghabra, J., Boges, D.J., Holst, G.R., Kreshuk, A., Hamprecht, F.A., Srinivasan, M., Lehväslaiho, H., Magistretti, P.J.: Three-dimensional immersive virtual reality for studying cellular compartments in 3D models from EM preparations of neural tissues. J. Comp. Neurol. 524(1), 23–38 (2016). Scholar
  8. 8.
    Camilleri, V., de Freitas, S., Montebello, M., McDonagh-Smith, P.: A case study inside virtual worlds: use of analytics for immersive spaces (2013).
  9. 9.
    Card, S.K., Mackinlay, J.D., Shneiderman, B.: Readings in information visualization: using vision to think. The Morgan Kaufmann series in interactive technologies. Morgan Kaufmann Publishers, an Francisco, California (1999)Google Scholar
  10. 10.
    Chang, Y., Peng Xu, W., Wang, L.: Research on 3D Visualization of Underground Antique Tomb Based on Augmented Reality, vol. 336–338 (2013).
  11. 11.
    Chelaru, F., Smith, L., Goldstein, N., Bravo, H.C.: Epiviz: interactive visual analytics for functional genomics data. Nat. Methods 11(9), 938–940 (2014). Scholar
  12. 12.
    Claudia, E., Peter, E., Bernd, E., Katrin, E., Torsten, E.: Interactive 3D visualization of structural changes in the brain of a person with corticobasal syndrome. Front. Neuroinformatics 8 (2014).
  13. 13.
    David, B.D., Clifford, A.W., Gibson, J.D., John, M.B., Max, W.: Augmented reality: advances in diagnostic imaging. Multimodal Technol. Interact. 1(4), 29 (2017). Scholar
  14. 14.
    Dockx, K., Bekkers, E.M.J., Van den Bergh, V., Ginis, P., Rochester, L., Hausdorff, J.M., Mirelman, A., Nieuwboer, A.: Virtual reality for rehabilitation in Parkinson’s disease. Cochrane Database Syst. Rev. 12 (2016).
  15. 15.
    Fuchs, R., Waser, J., Groller, M.E.: Visual human + machine learning. IEEE Trans. Vis. Comput. Graph 15(6), 1327–1334 (2009). Scholar
  16. 16.
    García-Hernández, R.J., Anthes, C., Wiedemann, M., Kranzlmüller, D.: Perspectives for using virtual reality to extend visual data mining in information visualization. In: 2016 IEEE Aerospace Conference, 5–12, pp. 1–11 (2016).
  17. 17.
    Gold, J.I., Belmont, K.A., Thomas, D.A.: The neurobiology of virtual reality pain attenuation. Cyberpsychology Behav. Impact Internet, Multimed. Virtual Real. Behav. Soc. 10(4), 536 (2007). Scholar
  18. 18.
    Goldman, M., Craft, B., Swatloski, T., Cline, M., Morozova, O., Diekhans, M., Haussler, D., Zhu, J.: The UCSC cancer genomics browser: update 2015. Nucleic. Acids Res. 43, D812–817 (2015).
  19. 19.
    Golestan Hashemi, F.S., Razi Ismail, M., Rafii Yusop, M., Golestan Hashemi, M.S., Nadimi Shahraki, M.H., Rastegari, H., Miah, G., Aslani, F.: Intelligent mining of large-scale bio-data: bioinformatics applications. Biotechnol. Biotechnol. Equip. 32(1), 10–29 (2017). Scholar
  20. 20.
    Green, T.M., Ribarsky, W., Fisher, B.: Visual analytics for complex concepts using a human cognition model. In: 2008 IEEE Symposium on Visual Analytics Science and Technology, vol. 19–24, pp. 91–98 (2008).
  21. 21.
    Joseph, A.C., David, S.W.: Applications of machine learning in cancer prediction and prognosis. Cancer Inform. 2, 59–78 (2006)Google Scholar
  22. 22.
    Keahey, T.A.: Using visualization to understand big data. Adv. Vis. (2013)Google Scholar
  23. 23.
    Keefe, J.F., Huling, A.D., Coggins, J.M., Keefe, F.D., Rosenthal, M.Z., Herr, R.N., Hoffman, G.H.: Virtual reality for persistent pain: a new direction for behavioral pain management. Pain 153(11), 2163–2166 (2012). Scholar
  24. 24.
    Kiper, P., Szczudlik, A., Agostini, M., Opara, J., Nowobilski, R., Ventura, L., Tonin, P., Turolla, A.: Virtual reality for upper limb rehabilitation in subacute and chronic stroke: a randomized controlled trial. Arch. Phys. Med. Rehabil. 99(5), 834–842.e834 (2018). Scholar
  25. 25.
    Krisa, D., Tailor, S.I.: Data visualization in health care: optimizing the utility of claims data through visual analysis (2014)Google Scholar
  26. 26.
    Lau, C.W., Nguyen, Q.V., Qu, Z., Simoff, S., Catchpoole, D.: Immersive intelligence genomic data visualisation. Paper Presented at the ACM (2019)Google Scholar
  27. 27.
    Laver, K.E., Lange, B., George, S., Deutsch, J.E., Saposnik, G., Crotty, M.: Virtual reality for stroke rehabilitation. Cochrane Database of Syst. Rev. 11 (2017).
  28. 28.
    Leung, M.K.K., Delong, A., Alipanahi, B., Frey, B.J.: Machine learning in genomic medicine: a review of computational problems and data sets. Proc. IEEE 104(1), 176–197 (2016). Scholar
  29. 29.
    Lex, A., Streit, M., Kruijff, E., Schmalstieg, D.: Caleydo: design and evaluation of a visual analysis framework for gene expression data in its biological context. In: 2010 IEEE Pacific Visualization Symposium (PacificVis), vol. 2–5, pp. 57–64 (2010).
  30. 30.
    Lin, Q., Xu, Z., Li, B., Baucom, R., Poulose, B., Landman, B.A., Bodenheimer, R.E.: Immersive virtual reality for visualization of abdominal CT. In: Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment. International Society for Optics and Photonics, p. 867317 (2013)Google Scholar
  31. 31.
    Llobera, J., González-Franco, M., Perez-Marcos, D., Valls-Solé, J., Slater, M., Sanchez-Vives, M.: Virtual reality for assessment of patients suffering chronic pain: a case study. Exp. Brain Res. 225(1), 105–117 (2013). Scholar
  32. 32.
    Luboschik, M., Berger, P., Staadt, O.: On Spatial Perception Issues in Augmented Reality Based Immersive Analytics (2016).
  33. 33.
    Maani, C.V., Hoffman, H.G., Morrow, M., Maiers, A., Gaylord, K., McGhee, L.L., Desocio, P.A.: Virtual reality pain control during burn wound debridement of combat-related burn injuries using robot-like arm mounted VR goggles. J. Trauma: Inj. Infect. Crit. Care 71(1 supplement), S125–S130 (2011). Scholar
  34. 34.
    Matte-Tailliez, O., Toffano-Nioche, C., Ferey, N., Kepes, F., Gherbi, R.: Immersive visualization for genome exploration and analysis. In: 2006 2nd International Conference on Information and Communication Technologies, vol. 24–28, pp. 3510–3515 (2006).
  35. 35.
    Mills, M.: Artificial Intelligence in Law: The State of Play 2016 Thomson Reuters S031401/3–16 (2016)Google Scholar
  36. 36.
    Moran, A., Gadepally, V., Hubbell, M., Kepner, J.: Improving big data visual analytics with interactive virtual reality (2015). Scholar
  37. 37.
    Müller, C., Krone, M., Huber, M., Biener, V., Herr, D., Koch, S., Reina, G., Weiskopf, D., Ertl, T.: Interactive molecular graphics for augmented reality using Hololens. J. Integr. Bioinform. 15(2).
  38. 38.
    Natalia Andrienko, G.A.: Intelligent visualisation and information presentation for civil crisis management. Trans. GIS 11(6), 11 (2007). Scholar
  39. 39.
    Nguyen, H., Marendy, P., Engelke, U.: Collaborative Framework Design for Immersive Analytics (2016). Scholar
  40. 40.
    Nguyen, Q.V., Alzamora, P., Ho, N., Huang, M.L., Simoff, S., Catchpoole, D.: Unlocking the complexity of genomic data of RMS patients through visual analytics. In: Paper presented at the 2012 International Conference on Computerized Healthcare, pp. 17–18. Hong Kong (2012)Google Scholar
  41. 41.
    Nguyen, Q.V., Gleeson, A., Ho, N., Huang, M.L., Simoff, S., Catchpoole, D.: Visual analytics of clinical and genetic datasets of acute lymphoblastic leukaemia. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) Neural Information Processing: 18th International Conference, ICONIP 2011, pp. 13–17. Shanghai, China. Proceedings, Part I. Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 113–120 (2011).
  42. 42.
    Nguyen, Q.V., Khalifa, N.H., Alzamora, P., Gleeson, A., Catchpoole, D., Kennedy, P.J., Simoff, S.: Visual analytics of complex genomics data to guide effective treatment decisions. J. Imaging 2(4), 29 (2016). UNSP 2910.3390/jimaging2040029CrossRefGoogle Scholar
  43. 43.
    Nguyen, Q.V., Qian, Y., Huang, M.L., Zhang, J.W.: TabuVis: a tool for visual analytics multidimensional datasets. Sci. China-Infr. Sci. 56(5), 1–12 (2013). ARTN 05210510.1007/s11432-013-4870-1CrossRefGoogle Scholar
  44. 44.
    Nguyen, Q.V., Nelmes, G., Huang, M.L., Simoff, S., Catchpoole, D.: Interactive visualization for patient-to-patient comparison. Genomics Inf. 12(1), 21–34 (2014). Scholar
  45. 45.
    Nilsson, N.J.: The Quest for Artifical Intelligence: A History of Ideas and Achievements (2009)Google Scholar
  46. 46.
    Olshannikova, E., Ometov, A., Koucheryavy, Y., Olsson, T.: Visualizing big data with augmented and virtual reality: challenges and research agenda. J. Big Data 2(1) (2015).
  47. 47.
    Patrick, H., Wen, P., SriSatish, A.: Ideas on interpreting machine learning. O’Reilly (2017)Google Scholar
  48. 48.
    Pavlopoulos, G.A., Malliarakis, D., Papanikolaou, N., Theodosiou, T., Enright, A.J., Iliopoulos, I.: Visualizing genome and systems biology: technologies, tools, implementation techniques and trends, past, present and future. Gigascience 4, 38 (2015). Scholar
  49. 49.
    Perez-Llamas, C., Lopez-Bigas, N.: Gitools: analysis and visualisation of genomic data using interactive heat-maps. PLoS ONE 6(5), e19541 (2011). Scholar
  50. 50.
    Polys, N., Mohammed, A., Iyer, J., Radics, P., Abidi, F., Arsenault, L., Rajamohan, S.: Immersive analytics: crossing the gulfs with high-performance visualization (2016).
  51. 51.
    Qu, Z., Lau, C.W., Nguyen, Q.V., Zhou, Y., Catchpoole, D.R.: visual analytics of genomic and cancer data: a systematic review. Cancer Inform. 18, 1176935119835546 (2019)CrossRefGoogle Scholar
  52. 52.
    Qu, Z., Zhou, Y., Nguyen, Q.V., Catchpoole, D.R.: Using visualization to illustrate machine learning models for genomic data. Paper Presented at the ACM (2019)Google Scholar
  53. 53.
    Ribeiro, M., Singh, S., Guestrin, C.: Why Should I Trust You? Explaining the Predictions of Any Classifier. arXivorg (2016)Google Scholar
  54. 54.
    Robinson, J.T., Thorvaldsdottir, H., Winckler, W., Guttman, M., Lander, E.S., Getz, G., Mesirov, J.P.: Integrative genomics viewer. Nat. Biotechnol. 29(1), 24–26 (2011). Scholar
  55. 55.
    Sennaar, K.: Machine Learning in Genomics—Current Efforts and Future Applications (2018).
  56. 56.
    Shan, Q., Doyle, T.E., Samavi, R., Al-Rei, M.: Augmented reality based brain tumor 3D visualization. In: 8th International Conference on Emerging Ubiquitous Systems and Pervasive Networks (EUSPN 2017), vol. 113, pp. 400–407 (2017).
  57. 57.
    Shilling, C.: How Augmented Reality will Change Data Visualization (2017).
  58. 58.
    Simpson, R.M., LaViola, J.J., Laidlaw, D.H., Forsberg, A.S., van Dam, A.: Immersive VR for scientific visualization: a progress report. IEEE Comput. Graphics Appl. 20(6), 26–52 (2000). Scholar
  59. 59.
    Slater, M., Sanchez-Vives, M.V.: Enhancing Our Lives with Immersive Virtual Reality 3(74) (2016).
  60. 60.
    Stevens, E.A., Rodriguez, C.P.: Genomic medicine and targeted therapy for solid tumors 111 (2015).
  61. 61.
    Tang, J., Liu, R., Zhang, Y.L., Liu, M.Z., Hu, Y.F., Shao, M.J., Zhu, L.J., Xin, H.W., Feng, G.W., Shang, W.J., Meng, X.G., Zhang, L.R., Ming, Y.Z., Zhang, W.: Application of machine-learning models to predict tacrolimus stable dose in renal transplant recipients. Sci. Rep. 7, 42192 (2017). Scholar
  62. 62.
    Venson, J., Berni, J., Maia, C., Da Silva, A., D’Ornelas, M., Maciel, A.: Medical imaging VR: Can Immersive 3D Aid in Diagnosis? 02–04 (2016).
  63. 63.
    Wachtel, M., Runge, T., Leuschner, I., Stegmaier, S., Koscielniak, E., Treuner, J., Odermatt, B., Behnke, S., Niggli, F., Schafer, B.: Subtype and prognostic classification of rhabdomyosarcoma by immunohistochemistry. J. Clin. Oncol. 24(5), 816–822 (2006). Scholar
  64. 64.
    Ware, C.: Information Visualization Perception for Design (2013)Google Scholar
  65. 65.
    Wei, L., Huang, X., Huang, M.L., Nguyen, Q.V.: Applying graph layout techniques to web information visualization and navigation. In: Paper Presented at the IEEE Int’l Conference on Computer Graphics, Imaging and Vision (CGIV07). Bangkok, Thailand, 13 Aug 2007Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2020

Authors and Affiliations

  • Zhonglin Qu
    • 1
  • Chng Wei Lau
    • 1
  • Daniel R. Catchpoole
    • 2
    • 3
    • 4
  • Simeon Simoff
    • 5
  • Quang Vinh Nguyen
    • 5
    Email author
  1. 1.School of Computer, Data and Mathematical SciencesWestern Sydney UniversitySydneyAustralia
  2. 2.Tumour Bank, Children’s Cancer Research Unit, Kids ResearchChildren’s Hospital at WestmeadWestmeadAustralia
  3. 3.Discipline of Paediatrics and Child Health, Faculty of MedicineUniversity of SydneySydneyAustralia
  4. 4.Faculty of Information TechnologyUniversity of Technology SydneySydneyAustralia
  5. 5.MARCS Institute for Brain, Behaviour and Development, School of Computer, Data and Mathematical SciencesWestern Sydney UniversitySydneyAustralia

Personalised recommendations