Advertisement

Guided Unsupervised Desmoking of Laparoscopic Images Using Cycle-Desmoke

  • V. Vishal
  • Neeraj Sharma
  • Munendra SinghEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11796)

Abstract

The generation of smoke in laparoscopic surgery due to laser ablation and cauterization causes deterioration in the visual quality of the operative field. In order to reduce the effect of smoke, the present paper proposes an end-to-end network, called Cycle-Desmoke. The network enhances the CycleGAN framework by adoption of a new generator architecture and addition of new Guided-Unsharp Upsample loss in combination to adversarial and cycle-consistency loss. The Atrous Convolution Feature Extraction Module present in the encoder blocks of the generator helps distinguishing smoke by capturing features at multiple scales by the use of kernels with different receptive fields. Further, the use of Guided-Unsharp Upsample loss supervises the upsampling process of the feature maps and helps improve the contrast of the desmoked image. The network performs robust unsupervised Image-to-Image Translation from smoke domain to smoke-free domain. The public Cholec80 dataset is used to evaluate the performance of the proposed method. Quantitative and qualitative comparative analysis of the proposed method over the state-of-the-methods reveals the effectiveness of the method at the task of smoke removal and enhancement of the image.

Keywords

Smoke removal Image enhancement Laparoscopic surgery 

References

  1. 1.
    Takahashi, H., et al.: Automatic smoke evacuation in laparoscopic surgery: a simplified method for objective evaluation. Surg. Endosc. 27(8), 2980–2987 (2013)CrossRefGoogle Scholar
  2. 2.
    He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)Google Scholar
  3. 3.
    Wang, C., Cheikh, F.A., Kaaniche, M., Beghdadi, A., Elle, O.J.: Variational based smoke removal in laparoscopic images. Biomed. Eng. Online 17(1), 139 (2018)CrossRefGoogle Scholar
  4. 4.
    Kotwal, A., Bhalodia, R., Awate, S.P.: Joint desmoking and denoising of laparoscopy images. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1050–1054. IEEE (2016)Google Scholar
  5. 5.
    Bolkar, S., Wang, C., Cheikh, F.A., Yildirim, S.: Deep smoke removal from minimally invasive surgery videos. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3403–3407. IEEE (2018)Google Scholar
  6. 6.
    Sidorov, O., Wang, C., Cheikh, F.A.: Generative Smoke Removal. arXiv preprint arXiv:1902.00311 (2019)
  7. 7.
    Wang, C., Mohammed, A.K., Cheikh, F.A., Beghdadi, A., Elle, O.J.: Multiscale deep desmoking for laparoscopic surgery. In: Medical Imaging 2019: Image Processing, vol. 10949, p. 109491Y. International Society for Optics and Photonics (2019)Google Scholar
  8. 8.
    Chen, L., Tang, W., John, W.N.: Unsupervised learning of surgical smoke removal from simulation (2018)Google Scholar
  9. 9.
    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)Google Scholar
  10. 10.
    Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01234-2_49CrossRefGoogle Scholar
  11. 11.
    Guan, T., Zhu, H.: Atrous faster R-CNN for small scale object detection. In: 2017 2nd International Conference on Multimedia and Image Processing (ICMIP), pp. 16–21. IEEE (2017)Google Scholar
  12. 12.
    Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)Google Scholar
  13. 13.
    Leibetseder, A., Primus, M.J., Petscharnig, S., Schoeffmann, K.: Real-time image-based smoke detection in endoscopic videos. In: Proceedings of the on Thematic Workshops of ACM Multimedia 2017, pp. 296–304. ACM (2017)Google Scholar
  14. 14.
    Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., De Mathelin, M., Padoy, N.: EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36(1), 86–97 (2016)CrossRefGoogle Scholar
  15. 15.
    Berman, D., Avidan, S.: Non-local image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1674–1682 (2016)Google Scholar
  16. 16.
    Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: DehazeNet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Mittal, A., Moorthy, A.K., Bovik, C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Venkatanath, N., Praneeth, D., Bh, M.C., Channappayya, S.S., Medasani, S.S.: Blind image quality evaluation using perception based features. In: 2015 Twenty First National Conference on Communications (NCC), pp. 1–6. IEEE (2015)Google Scholar
  19. 19.
    Yan, J., Li, J., Fu, X.: No-reference quality assessment of contrast-distorted images using contrast enhancement. arXiv preprint arXiv:1904.08879 (2019)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Mechatronics Engineering, Manipal Institute of TechnologyManipal Academy of Higher EducationManipalIndia
  2. 2.School of Biomedical EngineeringIndian Institute of Technology (Banaras Hindu University)VaranasiIndia

Personalised recommendations