Fish Detection Using Convolutional Neural Networks with Limited Training Data

  • Shih-Lun Tseng
  • Huei-Yung LinEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12046)


Due to the effect of global climate changes to marine biology and aquaculture, researchers start to investigate the deep ocean environment and living circumstances of rare fish species. One major issue of the related research is the difficulty of sufficient image data acquisition. This paper presents a method for underwater fish detection using limited training data. Current convolutional neural network based techniques have good performance on object detection and segmentation but require a large collection of image data. The proposed network structure is based on the U-Net model, modified with various encoders, convolutional layers, and residual blocks, to achieve high accuracy detection and segmentation results. It is able to provide better mIoU compared to other improved U-Net variants with a small amount of training data. Experiments carried out on fish tank scenes and the underwater environment have demonstrated the effectiveness of the proposed technique compared to other state-of-the-art detection networks.


CNN Semantic segmentation Fish detection U-Net 



The support of this work in part by the Ministry of Science and Technology of Taiwan under Grant MOST 106-2221-E-194-004 is gratefully acknowledged.


  1. 1.
    Abraham, N., Khan, N.M.: A novel focal Tversky loss function with improved attention U-Net for lesion segmentation. arXiv preprint arXiv:1810.07842 (2018)
  2. 2.
    Aneesh Kumar, K., Nikki, R., Oxona, K., Hashim, M., Sudhakar, M.: Relationships between fish and otolith size of nine deep-sea fishes from the Andaman and Nicobar waters, North Indian Ocean. J. Appl. Ichthyol. 33(6), 1187–1195 (2017)CrossRefGoogle Scholar
  3. 3.
    Aneesh Kumar, K., Thomy, R., Manjebrayakath, H., Sudhakar, M.: Length-weight relationships of 11 deep-sea fishes from the western Bay of Bengal and Andaman waters, India. J. Appl. Ichthyol. 34(4), 1048–1051 (2018)CrossRefGoogle Scholar
  4. 4.
    Beauxis-Aussalet, E., He, J., Spampinato, C., Boom, B., van Ossenbruggen, J., Hardman, L.: Fish4knowledge deliverable d2. 3 component-based prototypes and evaluation criteriaGoogle Scholar
  5. 5.
    Boom, B.J., Huang, P.X., He, J., Fisher, R.B.: Supporting ground-truth annotation of image datasets using clustering. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR 2012), pp. 1542–1545, November 2012Google Scholar
  6. 6.
    Brown, C.J., Broadley, A., Adame, M.F., Branch, T.A., Turschwell, M.P., Connolly, R.M.: The assessment of fishery status depends on fish habitats. Fish Fish. 20(1), 1–14 (2019)CrossRefGoogle Scholar
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  8. 8.
    Jäger, J., Rodner, E., Denzler, J., Wolff, V., Fricke-Neuderth, K.: SeaCLEF 2016: object proposal classification for fish detection in underwater videos. In: CLEF (Working Notes), pp. 481–489 (2016)Google Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  10. 10.
    Li, X., Shang, M., Qin, H., Chen, L.: Fast accurate fish detection and recognition of underwater images with fast R-CNN. In: OCEANS 2015-MTS/IEEE Washington, pp. 1–5. IEEE (2015)Google Scholar
  11. 11.
    Li, X., Tang, Y., Gao, T.: Deep but lightweight neural networks for fish detection. In: OCEANS 2017-Aberdeen, pp. 1–5. IEEE (2017)Google Scholar
  12. 12.
    Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)Google Scholar
  13. 13.
    Liu, S., et al.: Embedded online fish detection and tracking system via YOLOv3 and parallel correlation filter. In: OCEANS 2018 MTS/IEEE Charleston, pp. 1–6. IEEE (2018)Google Scholar
  14. 14.
    Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Heidelberg (2016). Scholar
  15. 15.
    Malik, S., Kumar, T., Sahoo, A.: Image processing techniques for identification of fish disease. In: 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), pp. 55–59. IEEE (2017)Google Scholar
  16. 16.
    McConville, M.M., et al.: The sensitivity of a deep-sea fish species (anoplopoma fimbria) to oil-associated aromatic compounds, dispersant, and Alaskan North Slope crude oil. Environ. Toxicol. Chem. 37(8), 2210–2221 (2018)CrossRefGoogle Scholar
  17. 17.
    Muhling, B., Lindegren, M., Clausen, L.W., Hobday, A., Lehodey, P.: Impacts of climate change on pelagic fish and fisheries. Climate Change Impacts Fish. Aquac.: Glob. Anal. 2, 771–814 (2017)CrossRefGoogle Scholar
  18. 18.
    Olsvik, E., et al.: Biometric fish classification of temperate species using convolutional neural network with squeeze-and-excitation. arXiv preprint arXiv:1904.02768 (2019)
  19. 19.
    Qin, H., Li, X., Liang, J., Peng, Y., Zhang, C.: Deepfish: accurate underwater live fish recognition with a deep architecture. Neurocomputing 187, 49–58 (2016)CrossRefGoogle Scholar
  20. 20.
    Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. CoRR abs/1710.05941 (2017).
  21. 21.
    Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. arXiv preprint arXiv:1710.05941 (2017)
  22. 22.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)Google Scholar
  23. 23.
    Shevchenko, V., Eerola, T., Kaarna, A.: Fish detection from low visibility underwater videos. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 1971–1976. IEEE (2018)Google Scholar
  24. 24.
    Shi, C., Jia, C., Chen, Z.: FFDet: a fully convolutional network for coral reef fish detection by layer fusion. In: 2018 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4. IEEE (2019)Google Scholar
  25. 25.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  26. 26.
    Sun, T., Chen, Z., Yang, W., Wang, Y.: Stacked U-Nets with multi-output for road extraction. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 187–1874. IEEE (2018)Google Scholar
  27. 27.
    Sung, M., Yu, S.C., Girdhar, Y.: Vision based real-time fish detection using convolutional neural network. In: OCEANS 2017, Aberdeen, pp. 1–6. IEEE (2017)Google Scholar
  28. 28.
    Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)Google Scholar
  29. 29.
    Verschae, R., Kawashima, H., Nobuhara, S.: A multi-camera system for underwater real-time 3D fish detection and tracking. In: OCEANS 2017, Anchorage, pp. 1–5. IEEE (2017)Google Scholar
  30. 30.
    Wang, M., Liu, M., Zhang, F., Lei, G., Guo, J., Wang, L.: Fast classification and detection of fish images with YOLOv2. In: 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO), pp. 1–4. IEEE (2018)Google Scholar
  31. 31.
    Xiao, X., Lian, S., Luo, Z., Li, S.: Weighted Res-UNet for high-quality retina vessel segmentation. In: 2018 9th International Conference on Information Technology in Medicine and Education (ITME), pp. 327–331. IEEE (2018)Google Scholar
  32. 32.
    Zhang, X., Vincent, A.: Predicting distributions, habitat preferences and associated conservation implications for a genus of rare fishes, seahorses (Hippocampus spp.). Diversity and Distributions (2018). Scholar
  33. 33.
    Zhao, X., Yan, S., Gao, Q.: An algorithm for tracking multiple fish based on biological water quality monitoring. IEEE Access 7, 15018–15026 (2019)CrossRefGoogle Scholar
  34. 34.
    Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: a nested u-net architecture for medical image segmentation. In: Stoyanov, D. et al. (eds.) DLMIA 2018, ML-CDS 2018. LNCS, vol. 11045, pp. 3–11. Springer, Heidelberg (2018). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Electrical EngineeringNational Chung Cheng UniversityChiayiTaiwan

Personalised recommendations