Advertisement

Intelligent and Immersive Visual Analytics of Health Data

  • Zhonglin Qu
  • Chng Wei Lau
  • Daniel R. Catchpoole
  • Simeon Simoff
  • Quang Vinh NguyenEmail author
Chapter
  • 5 Downloads
Part of the Studies in Computational Intelligence book series (SCI, volume 891)

Abstract

Massive amounts of health data have been created together with the advent of computer technologies and next generation sequencing technologies. Analytical techniques can significantly aid in the processing, integration and interpretation of the complex data. Visual analytics field has been rapidly evolving together with the advancement in automated analysis methods such as data mining, machine learning and statistics, visualization, and immersive technologies. Although automated analysis processes greatly support the decision making, conservative domains such as medicine, banking, and insurance need trusts on machine learning models. Explainable artificial intelligence could open the black boxes of the machine learning models to improve the trusts for decision makers. Immersive technologies allow the users to engage naturally with the blended reality in where they can look the information in different angles in addition to traditional screens. This chapter reviews and discusses the intelligent visualization, artificial intelligence and immersive technologies in health domain. We also illustrate the ideas with various case studies in genomic data visual analytics.

References

  1. 1.
    An, J., Lai, J., Wood, D.L., Sajjanhar, A., Wang, C., Tevz, G., Lehman, M.L., Nelson, C.C.: RNASeqBrowser: a genome browser for simultaneous visualization of raw strand specific RNAseq reads and UCSC genome browser custom tracks. BMC Genom. 16, 145 (2015).  https://doi.org/10.1186/s12864-015-1346-2CrossRefGoogle Scholar
  2. 2.
    Arya, A., Nowlan, N., Sauriol, N.: Data-driven framework for an online 3D immersive environment for educational applications. In: Proceedings of the International Conference on Education and New Learning Technologies, pp. 4726–4736 (2010)Google Scholar
  3. 3.
    Bhavnani, S., Ganesan, A., Hall, T., Maslowski, E., Eichinger, F., Martini, S., Saxman, P., Bellala, G., Kretzler, M.: Discovering hidden relationships between renal diseases and regulated genes through 3D network visualizations. BMC Res. Notes 3(1), 296 (2010).  https://doi.org/10.1186/1756-0500-3-296CrossRefGoogle Scholar
  4. 4.
    Borgo, R., Kehrer, J., Chung, D.H., Maguire, E., Laramee, R.S., Hauser, H., Ward, M., Chen, M.: Glyph-based visualization: foundations, design guidelines, techniques and applications. In: Eurographics (STARs), pp. 39–63 (2013)Google Scholar
  5. 5.
    Boudreaux, E.D., Waring, M.E., Hayes, R.B., Sadasivam, R.S., Mullen, S., Pagoto, S.: Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl. Behav. Med. 4(4), 363–371 (2014).  https://doi.org/10.1007/s13142-014-0293-9CrossRefGoogle Scholar
  6. 6.
    Breiman, L.: Radom forests. Mach. Learn. 45, 5–32 (2001)CrossRefGoogle Scholar
  7. 7.
    Calì, C., Baghabra, J., Boges, D.J., Holst, G.R., Kreshuk, A., Hamprecht, F.A., Srinivasan, M., Lehväslaiho, H., Magistretti, P.J.: Three-dimensional immersive virtual reality for studying cellular compartments in 3D models from EM preparations of neural tissues. J. Comp. Neurol. 524(1), 23–38 (2016).  https://doi.org/10.1002/cne.23852CrossRefGoogle Scholar
  8. 8.
    Camilleri, V., de Freitas, S., Montebello, M., McDonagh-Smith, P.: A case study inside virtual worlds: use of analytics for immersive spaces (2013).  https://doi.org/10.1145/2460296.2460341
  9. 9.
    Card, S.K., Mackinlay, J.D., Shneiderman, B.: Readings in information visualization: using vision to think. The Morgan Kaufmann series in interactive technologies. Morgan Kaufmann Publishers, an Francisco, California (1999)Google Scholar
  10. 10.
    Chang, Y., Peng Xu, W., Wang, L.: Research on 3D Visualization of Underground Antique Tomb Based on Augmented Reality, vol. 336–338 (2013).  https://doi.org/10.4028/www.scientific.net/AMM.336-338.1434
  11. 11.
    Chelaru, F., Smith, L., Goldstein, N., Bravo, H.C.: Epiviz: interactive visual analytics for functional genomics data. Nat. Methods 11(9), 938–940 (2014).  https://doi.org/10.1038/nmeth.3038CrossRefGoogle Scholar
  12. 12.
    Claudia, E., Peter, E., Bernd, E., Katrin, E., Torsten, E.: Interactive 3D visualization of structural changes in the brain of a person with corticobasal syndrome. Front. Neuroinformatics 8 (2014).  https://doi.org/10.3389/fninf.2014.00042
  13. 13.
    David, B.D., Clifford, A.W., Gibson, J.D., John, M.B., Max, W.: Augmented reality: advances in diagnostic imaging. Multimodal Technol. Interact. 1(4), 29 (2017).  https://doi.org/10.3390/mti1040029CrossRefGoogle Scholar
  14. 14.
    Dockx, K., Bekkers, E.M.J., Van den Bergh, V., Ginis, P., Rochester, L., Hausdorff, J.M., Mirelman, A., Nieuwboer, A.: Virtual reality for rehabilitation in Parkinson’s disease. Cochrane Database Syst. Rev. 12 (2016).  https://doi.org/10.1002/14651858.cd010760.pub2
  15. 15.
    Fuchs, R., Waser, J., Groller, M.E.: Visual human + machine learning. IEEE Trans. Vis. Comput. Graph 15(6), 1327–1334 (2009).  https://doi.org/10.1109/TVCG.2009.199CrossRefGoogle Scholar
  16. 16.
    García-Hernández, R.J., Anthes, C., Wiedemann, M., Kranzlmüller, D.: Perspectives for using virtual reality to extend visual data mining in information visualization. In: 2016 IEEE Aerospace Conference, 5–12, pp. 1–11 (2016).  https://doi.org/10.1109/aero.2016.7500608
  17. 17.
    Gold, J.I., Belmont, K.A., Thomas, D.A.: The neurobiology of virtual reality pain attenuation. Cyberpsychology Behav. Impact Internet, Multimed. Virtual Real. Behav. Soc. 10(4), 536 (2007).  https://doi.org/10.1089/cpb.2007.9993CrossRefGoogle Scholar
  18. 18.
    Goldman, M., Craft, B., Swatloski, T., Cline, M., Morozova, O., Diekhans, M., Haussler, D., Zhu, J.: The UCSC cancer genomics browser: update 2015. Nucleic. Acids Res. 43, D812–817 (2015).  https://doi.org/10.1093/nar/gku1073
  19. 19.
    Golestan Hashemi, F.S., Razi Ismail, M., Rafii Yusop, M., Golestan Hashemi, M.S., Nadimi Shahraki, M.H., Rastegari, H., Miah, G., Aslani, F.: Intelligent mining of large-scale bio-data: bioinformatics applications. Biotechnol. Biotechnol. Equip. 32(1), 10–29 (2017).  https://doi.org/10.1080/13102818.2017.1364977CrossRefGoogle Scholar
  20. 20.
    Green, T.M., Ribarsky, W., Fisher, B.: Visual analytics for complex concepts using a human cognition model. In: 2008 IEEE Symposium on Visual Analytics Science and Technology, vol. 19–24, pp. 91–98 (2008).  https://doi.org/10.1109/vast.2008.4677361
  21. 21.
    Joseph, A.C., David, S.W.: Applications of machine learning in cancer prediction and prognosis. Cancer Inform. 2, 59–78 (2006)Google Scholar
  22. 22.
    Keahey, T.A.: Using visualization to understand big data. Adv. Vis. (2013)Google Scholar
  23. 23.
    Keefe, J.F., Huling, A.D., Coggins, J.M., Keefe, F.D., Rosenthal, M.Z., Herr, R.N., Hoffman, G.H.: Virtual reality for persistent pain: a new direction for behavioral pain management. Pain 153(11), 2163–2166 (2012).  https://doi.org/10.1016/j.pain.2012.05.030CrossRefGoogle Scholar
  24. 24.
    Kiper, P., Szczudlik, A., Agostini, M., Opara, J., Nowobilski, R., Ventura, L., Tonin, P., Turolla, A.: Virtual reality for upper limb rehabilitation in subacute and chronic stroke: a randomized controlled trial. Arch. Phys. Med. Rehabil. 99(5), 834–842.e834 (2018).  https://doi.org/10.1016/j.apmr.2018.01.023CrossRefGoogle Scholar
  25. 25.
    Krisa, D., Tailor, S.I.: Data visualization in health care: optimizing the utility of claims data through visual analysis (2014)Google Scholar
  26. 26.
    Lau, C.W., Nguyen, Q.V., Qu, Z., Simoff, S., Catchpoole, D.: Immersive intelligence genomic data visualisation. Paper Presented at the ACM (2019)Google Scholar
  27. 27.
    Laver, K.E., Lange, B., George, S., Deutsch, J.E., Saposnik, G., Crotty, M.: Virtual reality for stroke rehabilitation. Cochrane Database of Syst. Rev. 11 (2017).  https://doi.org/10.1002/14651858.cd008349.pub4
  28. 28.
    Leung, M.K.K., Delong, A., Alipanahi, B., Frey, B.J.: Machine learning in genomic medicine: a review of computational problems and data sets. Proc. IEEE 104(1), 176–197 (2016).  https://doi.org/10.1109/jproc.2015.2494198CrossRefGoogle Scholar
  29. 29.
    Lex, A., Streit, M., Kruijff, E., Schmalstieg, D.: Caleydo: design and evaluation of a visual analysis framework for gene expression data in its biological context. In: 2010 IEEE Pacific Visualization Symposium (PacificVis), vol. 2–5, pp. 57–64 (2010).  https://doi.org/10.1109/pacificvis.2010.5429609
  30. 30.
    Lin, Q., Xu, Z., Li, B., Baucom, R., Poulose, B., Landman, B.A., Bodenheimer, R.E.: Immersive virtual reality for visualization of abdominal CT. In: Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment. International Society for Optics and Photonics, p. 867317 (2013)Google Scholar
  31. 31.
    Llobera, J., González-Franco, M., Perez-Marcos, D., Valls-Solé, J., Slater, M., Sanchez-Vives, M.: Virtual reality for assessment of patients suffering chronic pain: a case study. Exp. Brain Res. 225(1), 105–117 (2013).  https://doi.org/10.1007/s00221-012-3352-9CrossRefGoogle Scholar
  32. 32.
    Luboschik, M., Berger, P., Staadt, O.: On Spatial Perception Issues in Augmented Reality Based Immersive Analytics (2016).  https://doi.org/10.1145/3009939.3009947
  33. 33.
    Maani, C.V., Hoffman, H.G., Morrow, M., Maiers, A., Gaylord, K., McGhee, L.L., Desocio, P.A.: Virtual reality pain control during burn wound debridement of combat-related burn injuries using robot-like arm mounted VR goggles. J. Trauma: Inj. Infect. Crit. Care 71(1 supplement), S125–S130 (2011).  https://doi.org/10.1097/TA.0b013e31822192e2CrossRefGoogle Scholar
  34. 34.
    Matte-Tailliez, O., Toffano-Nioche, C., Ferey, N., Kepes, F., Gherbi, R.: Immersive visualization for genome exploration and analysis. In: 2006 2nd International Conference on Information and Communication Technologies, vol. 24–28, pp. 3510–3515 (2006).  https://doi.org/10.1109/ictta.2006.1684982
  35. 35.
    Mills, M.: Artificial Intelligence in Law: The State of Play 2016 Thomson Reuters S031401/3–16 (2016)Google Scholar
  36. 36.
    Moran, A., Gadepally, V., Hubbell, M., Kepner, J.: Improving big data visual analytics with interactive virtual reality (2015).  https://doi.org/10.1109/HPEC.2015.7322473CrossRefGoogle Scholar
  37. 37.
    Müller, C., Krone, M., Huber, M., Biener, V., Herr, D., Koch, S., Reina, G., Weiskopf, D., Ertl, T.: Interactive molecular graphics for augmented reality using Hololens. J. Integr. Bioinform. 15(2).  https://doi.org/10.1515/jib-2018-0005
  38. 38.
    Natalia Andrienko, G.A.: Intelligent visualisation and information presentation for civil crisis management. Trans. GIS 11(6), 11 (2007).  https://doi.org/10.1111/j.1467-9671.2007.01078.xCrossRefGoogle Scholar
  39. 39.
    Nguyen, H., Marendy, P., Engelke, U.: Collaborative Framework Design for Immersive Analytics (2016).  https://doi.org/10.1109/BDVA.2016.7787044CrossRefGoogle Scholar
  40. 40.
    Nguyen, Q.V., Alzamora, P., Ho, N., Huang, M.L., Simoff, S., Catchpoole, D.: Unlocking the complexity of genomic data of RMS patients through visual analytics. In: Paper presented at the 2012 International Conference on Computerized Healthcare, pp. 17–18. Hong Kong (2012)Google Scholar
  41. 41.
    Nguyen, Q.V., Gleeson, A., Ho, N., Huang, M.L., Simoff, S., Catchpoole, D.: Visual analytics of clinical and genetic datasets of acute lymphoblastic leukaemia. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) Neural Information Processing: 18th International Conference, ICONIP 2011, pp. 13–17. Shanghai, China. Proceedings, Part I. Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 113–120 (2011).  https://doi.org/10.1007/978-3-642-24955-6_14
  42. 42.
    Nguyen, Q.V., Khalifa, N.H., Alzamora, P., Gleeson, A., Catchpoole, D., Kennedy, P.J., Simoff, S.: Visual analytics of complex genomics data to guide effective treatment decisions. J. Imaging 2(4), 29 (2016). UNSP 2910.3390/jimaging2040029CrossRefGoogle Scholar
  43. 43.
    Nguyen, Q.V., Qian, Y., Huang, M.L., Zhang, J.W.: TabuVis: a tool for visual analytics multidimensional datasets. Sci. China-Infr. Sci. 56(5), 1–12 (2013). ARTN 05210510.1007/s11432-013-4870-1CrossRefGoogle Scholar
  44. 44.
    Nguyen, Q.V., Nelmes, G., Huang, M.L., Simoff, S., Catchpoole, D.: Interactive visualization for patient-to-patient comparison. Genomics Inf. 12(1), 21–34 (2014).  https://doi.org/10.5808/GI.2014.12.1.21CrossRefGoogle Scholar
  45. 45.
    Nilsson, N.J.: The Quest for Artifical Intelligence: A History of Ideas and Achievements (2009)Google Scholar
  46. 46.
    Olshannikova, E., Ometov, A., Koucheryavy, Y., Olsson, T.: Visualizing big data with augmented and virtual reality: challenges and research agenda. J. Big Data 2(1) (2015).  https://doi.org/10.1186/s40537-015-0031-2
  47. 47.
    Patrick, H., Wen, P., SriSatish, A.: Ideas on interpreting machine learning. O’Reilly (2017)Google Scholar
  48. 48.
    Pavlopoulos, G.A., Malliarakis, D., Papanikolaou, N., Theodosiou, T., Enright, A.J., Iliopoulos, I.: Visualizing genome and systems biology: technologies, tools, implementation techniques and trends, past, present and future. Gigascience 4, 38 (2015).  https://doi.org/10.1186/s13742-015-0077-2CrossRefGoogle Scholar
  49. 49.
    Perez-Llamas, C., Lopez-Bigas, N.: Gitools: analysis and visualisation of genomic data using interactive heat-maps. PLoS ONE 6(5), e19541 (2011).  https://doi.org/10.1371/journal.pone.0019541CrossRefGoogle Scholar
  50. 50.
    Polys, N., Mohammed, A., Iyer, J., Radics, P., Abidi, F., Arsenault, L., Rajamohan, S.: Immersive analytics: crossing the gulfs with high-performance visualization (2016).  https://doi.org/10.1109/IMMERSIVE.2016.7932376
  51. 51.
    Qu, Z., Lau, C.W., Nguyen, Q.V., Zhou, Y., Catchpoole, D.R.: visual analytics of genomic and cancer data: a systematic review. Cancer Inform. 18, 1176935119835546 (2019)CrossRefGoogle Scholar
  52. 52.
    Qu, Z., Zhou, Y., Nguyen, Q.V., Catchpoole, D.R.: Using visualization to illustrate machine learning models for genomic data. Paper Presented at the ACM (2019)Google Scholar
  53. 53.
    Ribeiro, M., Singh, S., Guestrin, C.: Why Should I Trust You? Explaining the Predictions of Any Classifier. arXivorg (2016)Google Scholar
  54. 54.
    Robinson, J.T., Thorvaldsdottir, H., Winckler, W., Guttman, M., Lander, E.S., Getz, G., Mesirov, J.P.: Integrative genomics viewer. Nat. Biotechnol. 29(1), 24–26 (2011).  https://doi.org/10.1038/nbt.1754CrossRefGoogle Scholar
  55. 55.
    Sennaar, K.: Machine Learning in Genomics—Current Efforts and Future Applications (2018). https://www.techemergence.com/machine-learning-in-genomics-applications/
  56. 56.
    Shan, Q., Doyle, T.E., Samavi, R., Al-Rei, M.: Augmented reality based brain tumor 3D visualization. In: 8th International Conference on Emerging Ubiquitous Systems and Pervasive Networks (EUSPN 2017), vol. 113, pp. 400–407 (2017).  https://doi.org/10.1016/j.procs.2017.08.356
  57. 57.
    Shilling, C.: How Augmented Reality will Change Data Visualization (2017). http://blog.i2econsulting.com/how-augmented-reality-will-change-data-visualization/
  58. 58.
    Simpson, R.M., LaViola, J.J., Laidlaw, D.H., Forsberg, A.S., van Dam, A.: Immersive VR for scientific visualization: a progress report. IEEE Comput. Graphics Appl. 20(6), 26–52 (2000).  https://doi.org/10.1109/38.888006CrossRefGoogle Scholar
  59. 59.
    Slater, M., Sanchez-Vives, M.V.: Enhancing Our Lives with Immersive Virtual Reality 3(74) (2016).  https://doi.org/10.3389/frobt.2016.00074
  60. 60.
    Stevens, E.A., Rodriguez, C.P.: Genomic medicine and targeted therapy for solid tumors 111 (2015).  https://doi.org/10.1002/jso.23699
  61. 61.
    Tang, J., Liu, R., Zhang, Y.L., Liu, M.Z., Hu, Y.F., Shao, M.J., Zhu, L.J., Xin, H.W., Feng, G.W., Shang, W.J., Meng, X.G., Zhang, L.R., Ming, Y.Z., Zhang, W.: Application of machine-learning models to predict tacrolimus stable dose in renal transplant recipients. Sci. Rep. 7, 42192 (2017).  https://doi.org/10.1038/srep42192CrossRefGoogle Scholar
  62. 62.
    Venson, J., Berni, J., Maia, C., Da Silva, A., D’Ornelas, M., Maciel, A.: Medical imaging VR: Can Immersive 3D Aid in Diagnosis? 02–04 (2016).  https://doi.org/10.1145/2993369.2996333
  63. 63.
    Wachtel, M., Runge, T., Leuschner, I., Stegmaier, S., Koscielniak, E., Treuner, J., Odermatt, B., Behnke, S., Niggli, F., Schafer, B.: Subtype and prognostic classification of rhabdomyosarcoma by immunohistochemistry. J. Clin. Oncol. 24(5), 816–822 (2006).  https://doi.org/10.1200/JCO.2005.03.4934CrossRefGoogle Scholar
  64. 64.
    Ware, C.: Information Visualization Perception for Design (2013)Google Scholar
  65. 65.
    Wei, L., Huang, X., Huang, M.L., Nguyen, Q.V.: Applying graph layout techniques to web information visualization and navigation. In: Paper Presented at the IEEE Int’l Conference on Computer Graphics, Imaging and Vision (CGIV07). Bangkok, Thailand, 13 Aug 2007Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2020

Authors and Affiliations

  • Zhonglin Qu
    • 1
  • Chng Wei Lau
    • 1
  • Daniel R. Catchpoole
    • 2
    • 3
    • 4
  • Simeon Simoff
    • 5
  • Quang Vinh Nguyen
    • 5
    Email author
  1. 1.School of Computer, Data and Mathematical SciencesWestern Sydney UniversitySydneyAustralia
  2. 2.Tumour Bank, Children’s Cancer Research Unit, Kids ResearchChildren’s Hospital at WestmeadWestmeadAustralia
  3. 3.Discipline of Paediatrics and Child Health, Faculty of MedicineUniversity of SydneySydneyAustralia
  4. 4.Faculty of Information TechnologyUniversity of Technology SydneySydneyAustralia
  5. 5.MARCS Institute for Brain, Behaviour and Development, School of Computer, Data and Mathematical SciencesWestern Sydney UniversitySydneyAustralia

Personalised recommendations