Physiological Parameter Estimation from Multispectral Images Unleashed
Multispectral imaging in laparoscopy can provide tissue reflectance measurements for each point in the image at multiple wavelengths of light. These reflectances encode information on important physiological parameters not visible to the naked eye. Fast decoding of the data during surgery, however, remains challenging. While model-based methods suffer from inaccurate base assumptions, a major bottleneck related to competing machine learning-based solutions is the lack of labelled training data. In this paper, we address this issue with the first transfer learning-based method to physiological parameter estimation from multispectral images. It relies on a highly generic tissue model that aims to capture the full range of optical tissue parameters that can potentially be observed in vivo. Adaptation of the model to a specific clinical application based on unlabelled in vivo data is achieved using a new concept of domain adaptation that explicitly addresses the high variance often introduced by conventional covariance-shift correction methods. According to comprehensive in silico and in vivo experiments our approach enables accurate parameter estimation for various tissue types without the need for incorporating specific prior knowledge on optical properties and could thus pave the way for many exciting applications in multispectral laparoscopy.
Funding for this work was provided by the European Research Council (ERC) starting grant COMBIOSCOPY (637960).
- 1.Claridge, E., Hidovic-Rowe, D.: Model based inversion for deriving maps of histological parameters characteristic of cancer from ex-vivo multispectral images of the colon. IEEE TMI 33, 822–835 (2013)Google Scholar
- 3.Wirkert, S.J.: others: Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression. IJCARS 11(6), 909–917 (2016)Google Scholar
- 7.Huang, J., et al.: Correcting sample selection bias by unlabeled data. In: Advances in Neural Information Processing Systems, vol. 19 (2007)Google Scholar
- 9.Goetz, M., et al.: DALSA: domain adaptation for supervised learning from sparsely annotated MR images. IEEE TMI 35(1), 184–196 (2016)Google Scholar
- 10.Rahimi, A., et al.: Random features for large-scale kernel machines. In: NIPS, vol. 3 (2007)Google Scholar
- 11.Reddi, S.J., et al.: Doubly robust covariate shift correction. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)Google Scholar