Due to the influence of color, boundaries, and shapes of melanoma, the segmentation of the lesion area is still a challenging problem. In this paper, we propose a method to connect two attention modules (channel attention and spatial attention) serially and embed them into the skip connection of the encoder–decoder network. Different from previous work on image segmentation through attention mechanism, we have made a different combination of the existing popular spatial attention module and channel attention module. The experimental results show that the sequence combinations with channel attention in front and spatial attention in the rear are more likely to aggregate global and local information as well as information between channels than other combinations. Our method achieved an average Jaccard Index of 0.7692 on the ISIC2017 dataset. At the same time, we also compared with some advanced methods of image segmentation, the experimental results show our proposed method has a competitive performance.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615
Bi L, Kim J, Ahn E, Kumar A, Fulham M, Feng D (2017) Dermoscopic image segmentation via multistage fully convolutional networks. IEEE Trans Biomed Eng 64:2065–2074. https://doi.org/10.1109/TBME.2017.2712771
Chaki J, Dey N, Rajinikanth V, Ashour AS, Shi F (2021) Recognition of skin diseases using curvelet transforms and law’s texture energy measures. Soft computing applications. Springer International Publishing, Cham, pp 51–61
Chen L-C, Papandreou G, Schroff F, Adam HJA (2017) Rethinking atrous convolution for semantic image segmentation abs/1706.05587
Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018a) Encoder-decoder with atrous separable convolution for semantic image segmentation. Computer vision—ECCV 2018. Springer International Publishing, Cham, pp 833–851
Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018b) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40:834–848. https://doi.org/10.1109/TPAMI.2017.2699184
Codella NCF et al. (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), 4–7 April 2018, pp 168–172. https://doi.org/10.1109/ISBI.2018.8363547
Dey N, Rajinikanth V, Ashour AS, Tavares JMRS (2018) Social group optimization supported segmentation and evaluation of skin melanoma images. Symmetry. https://doi.org/10.3390/sym10020051
Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 15–20 June 2019, pp 3141–3149. https://doi.org/10.1109/CVPR.2019.00326
Garcia-Arroyo JL, Garcia-Zapirain B (2019) Segmentation of skin lesions in dermoscopy images using fuzzy classification of pixels and histogram thresholding. Comput Methods Programs Biomed 168:11–19. https://doi.org/10.1016/j.cmpb.2018.11.001
Gu Z et al (2019) CE-Net: context encoder network for 2D medical image segmentation. IEEE Trans Med Imaging 38:2281–2292. https://doi.org/10.1109/TMI.2019.2903562
He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37:1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), 27–30 June 2016, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 18–23 June 2018, pp 7132–7141. https://doi.org/10.1109/CVPR.2018.00745
Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372
Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD (2020) DoubleU-Net: a deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd International symposium on computer-based medical systems (CBMS), 28–30 July 2020, pp 558–564. https://doi.org/10.1109/CBMS49503.2020.00111
Kaul C, Manandhar S, Pears N (2019) Focusnet: an attention-based fully convolutional network for medical image segmentation. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019), 8–11 April 2019, pp 455–458. https://doi.org/10.1109/ISBI.2019.8759477
Li Y, Shen L (2018) Skin lesion analysis towards melanoma detection using deep learning network. Sensors. https://doi.org/10.3390/s18020556
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), 7–12 June 2015, pp 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965
Md Zahangir A, Chris Y, Mahmudul H, Tarek MT, Vijayan KA (2019) Recurrent residual U-Net for medical image segmentation. J Med Imaging 6:1–16. https://doi.org/10.1117/1.JMI.6.1.014006
Peng C, Zhang X, Yu G, Luo G, Sun J (2017) Large Kernel matters—improve semantic segmentation by global convolutional network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), 21–26 July 2017, pp 1743–1751. https://doi.org/10.1109/CVPR.2017.189
Rahman M, Alpaslan N, Bhattacharya P (2016) Developing a retrieval based diagnostic aid for automated melanoma recognition of dermoscopic images. In: 2016 IEEE applied imagery pattern recognition workshop (AIPR), 18–20 Oct. 2016, pp 1–7. https://doi.org/10.1109/AIPR.2016.8010594
Rajinikanth V, Satapathy SC, Dey N, Fernandes SL, Manic KS (2019) Skin melanoma assessment using Kapur’s entropy and level set—a study with bat algorithm. Smart intelligent computing and applications. Springer Singapore, Singapore, pp 193–202
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Medical image computing and computer-assisted intervention—MICCAI 2015. Springer International Publishing, Cham, pp 234–241
Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. Medical image computing and computer assisted intervention—MICCAI 2018. Springer International Publishing, Cham, pp 421–429
Silveira M et al (2009) Comparison of segmentation methods for melanoma diagnosis in dermoscopy images. IEEE J Sel Top Signal Process 3:35–45. https://doi.org/10.1109/JSTSP.2008.2011119
Society AC (2020) Key statistics for melanoma skin cancer. https://www.cancer.org/cancer/melanoma-skin-cancer/about/keystatistics.html. Accessed 4 Jul 2020
Sun K, Xiao B, Liu D, Wang J (2019) Deep high-resolution representation learning for human pose estimation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 15–20 June 2019, pp 5686–5696. https://doi.org/10.1109/CVPR.2019.00584
Thao LT, Quang NH (2017) Automatic skin lesion analysis towards melanoma detection. In: 2017 21st Asia Pacific symposium on intelligent and evolutionary systems (IES), 15–17 Nov. 2017, pp 106–111. https://doi.org/10.1109/IESYS.2017.8233570
Vaswani A et al. (2017) Attention is all you need. In: Paper presented at the proceedings of the 31st international conference on neural information processing systems, Long Beach, California, USA
Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 18–23 June 2018, pp 7794–7803. https://doi.org/10.1109/CVPR.2018.00813
Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) ECA-Net: efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 13–19 June 2020, pp 11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155
Wen HJA (2017) II-FCN for skin lesion analysis towards melanoma detection. abs/1702.08699
Wong A, Scharcanski J, Fieguth P (2011) Automatic skin lesion segmentation via iterative stochastic region merging. IEEE Trans Inf Technol Biomed 15:929–936. https://doi.org/10.1109/TITB.2011.2157829
Yang M, Yu K, Zhang C, Li Z, Yang K (2018) DenseASPP for semantic segmentation in street scenes. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 18–23 June 2018. pp 3684–3692. https://doi.org/10.1109/CVPR.2018.00388
Yu L, Chen H, Dou Q, Qin J, Heng P (2017b) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36:994–1004. https://doi.org/10.1109/TMI.2016.2642839
Yu F, Koltun V, Funkhouser T (2017a) Dilated residual networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), 21–26 July 2017, pp 636–644. https://doi.org/10.1109/CVPR.2017.75
Yuan Y, Lo YC (2019) Improving dermoscopic image segmentation with enhanced convolutional-deconvolutional networks. IEEE J Biomed Health Inform 23:519–526. https://doi.org/10.1109/JBHI.2017.2787487
Yuan Y, Chao M, Lo Y (2017) Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance. IEEE Trans Med Imaging 36:1876–1886. https://doi.org/10.1109/TMI.2017.2695227
Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: Paper presented at the proceedings of the 36th international conference on machine learning, proceedings of machine learning research
Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), 21–26 July 2017, pp 6230–6239. https://doi.org/10.1109/CVPR.2017.660
Zhou H, Li X, Schaefer G, Celebi ME, Miller P (2013) Mean shift based gradient vector flow for image segmentation. Comput Vis Image Underst 117:1004–1016. https://doi.org/10.1016/j.cviu.2012.11.015
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39:1856–1867. https://doi.org/10.1109/TMI.2019.2959609
This research is partially supported by Science and Technology Department of Xinjiang Uyghur Autonomous Region Fund Project (2020E0234), and Department of Education, Xinjiang Uygur Autonomous Region (CN) Postgraduate Research and Innovation Project (XJ2020G072). We would also like to thank our tutor for the careful guidance and all the participants for their insightful comments.
Conflict of interest
The authors declare no conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Ren, Y., Yu, L., Tian, S. et al. Serial attention network for skin lesion segmentation. J Ambient Intell Human Comput (2021). https://doi.org/10.1007/s12652-021-02933-3
- Spatial attention
- Channel attention
- Skin lesion segmentation
- Deep learning