Abstract
Deep learning has brought the most profound contribution towards biomedical image segmentation to automate the process of delineation in medical imaging. To accomplish such task, the models are required to be trained using huge amount of annotated or labelled data that highlights the region of interest with a binary mask. However, efficient generation of the annotations for such huge data requires expert biomedical analysts and extensive manual effort. It is a tedious and expensive task, while also being vulnerable to human error. To address this problem, a self-supervised learning framework, BT-Unet is proposed that uses the Barlow Twins approach to pre-train the encoder of a U-Net model via redundancy reduction in an unsupervised manner to learn data representation. Later, complete network is fine-tuned to perform actual segmentation. The BT-Unet framework can be trained with a limited number of annotated samples while having high number of unannotated samples, which is mostly the case in real-world problems. This framework is validated over multiple U-Net models over diverse datasets by generating scenarios of a limited number of labelled samples using standard evaluation metrics. With exhaustive experiment trials, it is observed that the BT-Unet framework enhances the performance of the U-Net models with significant margin under such circumstances.
This is a preview of subscription content, access via your institution.
Availability of data
All datasets are publicly accessible.
Code availability
Code for using BT-Unet framework is available at https://github.com/nspunn1993/BT-Unet.
References
Alzubaidi, L., Fadhel, M. A., Al-Shamma, O., et al. (2020). Towards a better understanding of transfer learning for medical imaging: A case study. Applied Sciences, 10(13), 4523.
Asano, YM., Rupprecht, C., Vedaldi, A. (2019). Self-labelling via simultaneous clustering and representation learning. arXiv preprint arXiv:1911.05371
Caron, M., Bojanowski, P., Joulin, A., et al. (2018). Deep clustering for unsupervised learning of visual features. In: Proceedings of the european conference on computer vision (ECCV), pp 132–149
Caron, M., Misra, I., Mairal, J., et al. (2020). Unsupervised learning of visual features by contrasting cluster assignments. arXiv preprint arXiv:2006.09882
Chaitanya, K., Erdil, E., Karani, N., et al. (2020). Contrastive learning of global and local features for medical image segmentation with limited annotations. arXiv preprint arXiv:2006.10511
Chen, T., Kornblith, S., Norouzi, M., et al. (2020). A simple framework for contrastive learning of visual representations. In: International conference on machine learning, PMLR, pp 1597–1607
Chen, X., He, K. (2021). Exploring simple siamese representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 15,750–15,758
Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258
Dhere, A., Sivaswamy, J. (2021). Self-supervised learning for segmentation. arXiv preprint arXiv:2101.05456
Doersch, C., Gupta, A., Efros, AA. (2015). Unsupervised visual representation learning by context prediction. In: Proceedings of the IEEE international conference on computer vision, pp 1422–1430
Gidaris, S., Singh, P., Komodakis, N. (2018). Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728
Grill, JB., Strub, F., Altché, F., et al. (2020). Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733
Haque, I. R. I., & Neubert, J. (2020). Deep learning approaches to biomedical image segmentation. Informatics in Medicine Unlocked, 18(100), 297.
He, K., Fan, H., Wu, Y., et al. (2020). Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9729–9738
Isensee, F., Jaeger, P. F., Kohl, S. A., et al. (2021). Nnu-net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2), 203–211.
ISIC (2018) Isic. (2018). Skin lesion analysis towards melanoma detection. https://challenge2018.isic-archive.com/
Jing, L., & Tian, Y. (2020). Self-supervised visual feature learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 4037–4058.
Kaggle. (2018). Kaggle data science bowl challenge 2018. https://www.kaggle.com/c/data-science-bowl-2018
Ker, J., Wang, L., Rao, J., et al. (2017). Deep learning applications in medical image analysis. IEEE Access, 6, 9375–9389.
Lei, T., Wang, R., Wan, Y., et al. (2020). Medical image segmentation using deep learning: a survey. arXiv preprint arXiv:2009.13120
Li, H., Xue, F. F., Chaitanya, K., et al. (2021). Imbalance-aware self-supervised learning for 3d radiomic representations. International conference on medical image computing and computer-assisted intervention (pp. 36–46). Cham: Springer.
MICCAI, (2018). Brats 2018: Multimodal brain tumor segmentation challenge. https://www.med.upenn.edu/sbia/brats2018.html
Misra, I., Maaten, Lvd. (2020). Self-supervised learning of pretext-invariant representations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6707–6717
Noroozi, M., & Favaro, P. (2016). Unsupervised learning of visual representations by solving jigsaw puzzles. European conference on computer vision (pp. 69–84). Cham: Springer.
Oktay, O., Schlemper, J., Folgoc, LL., et al. (2018). Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999
Pathak, D., Krahenbuhl, P., Donahue, J., et al. (2016). Context encoders: Feature learning by inpainting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2536–2544
Punn, N. S., and Agarwal, S. (2020). Inception u-net architecture for semantic segmentation to identify nuclei in microscopy cell images. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 16(1), 1–15.
Punn, NS., Agarwal, S. (2021a). Modality specific u-net variants for biomedical image segmentation: A survey. arXiv preprint arXiv:2107.04537
Punn, NS., Agarwal, S. (2021b). Rca-iunet: A residual cross-spatial attention guided inception u-net model for tumor segmentation in breast ultrasound imaging. arXiv preprint arXiv:2108.02508
Raghu, M., Zhang, C., Kleinberg, J., et al. (2019). Transfusion: Understanding transfer learning for medical imaging. Advances in Neural Information Processing Systems, 32, 2019.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International conference on medical image computing and computer-assisted intervention (pp. 234–241). Cham: Springer.
Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 1–48.
Xian, M., Zhang, Y., Cheng, HD., et al. (2018). A benchmark for breast ultrasound image segmentation (BUSIS). Infinite Study
Zbontar, J., Jing, L., Misra, I., et al. (2021). Barlow twins: Self-supervised learning via redundancy reduction. arXiv preprint arXiv:2103.03230
Zeng, D., Wu, Y., Hu, X., et al. (2021). Positional contrastive learning for volumetric medical image segmentation. International conference on medical image computing and computer-assisted intervention (pp. 221–230). Cham: Springer.
Zheng, H., Han, J., Wang, H., et al. (2021). Hierarchical self-supervised learning for medical image segmentation based on multi-domain data aggregation. International conference on medical image computing and computer-assisted intervention (pp. 622–632). Cham: Springer.
Acknowledgements
We thank our institute, Indian Institute of Information Technology Allahabad (IIITA), India and Big Data Analytics (BDA) lab for allocating necessary resources to perform this research. We extend our thanks to our colleagues for their valuable guidance and suggestions.
Author information
Authors and Affiliations
Contributions
All authors contributed equally in conceptualizing the research problem and preparation of the manuscript.
Corresponding author
Ethics declarations
Ethics approval
Not Applicable.
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Consent to participate
Not Applicable.
Consent for publication
Not Applicable.
Additional information
Editors: Krzysztof Dembczynski and Emilie Devijver.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Punn, N.S., Agarwal, S. BT-Unet: A self-supervised learning framework for biomedical image segmentation using barlow twins with U-net models. Mach Learn 111, 4585–4600 (2022). https://doi.org/10.1007/s10994-022-06219-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-022-06219-3
Keywords
- Barlow twins
- Biomedical image segmentation
- Self-supervised learning
- U-Net