Skip to main content
SpringerLink
Log in

Marine Vertebrate Predator Detection and Recognition in Underwater Videos by Region Convolutional Neural Network

  • Conference paper
  • First Online:
Knowledge Management and Acquisition for Intelligent Systems (PKAW 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11669))

Included in the following conference series:

Abstract

In this paper, we present R-CNN, Fast R-CNN and Faster R-CNN methods to automatically detect and recognise the predators in underwater videos. We compare the results of these methods on real data and discuss their strengths and weaknesses. We build a dataset using footage captured from representative environment of the wild and devise a data model with three classes (seal, dolphin, background). Following this, we train R-CNN, Fast R-CNN and Faster R-CNN, then evaluate them on a test dataset compose of challenging objects that had not been seen during training. We perform evaluation on GPU, acquiring information about the AP and IOU for each model and network based on various proposal numbers as well as runtime speeds. Based on the results, we found that the best model of predator detection using visual deep learning models is Faster R-CNN with 2000 proposals.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Pikitch, E.K., et al.: The global contribution of forage fish to marine fisheries and ecosystems. Fish Fish. 15(1), 43–64 (2014)

    Article  Google Scholar 

  2. Engelhard, G.H., et al.: Forage fish, their fisheries, and their predators: who drives whom? ICSE J. Mar. Sci. 71(1), 90–104 (2013)

    Article  Google Scholar 

  3. Kane, E.A., Marshall, C.D.: Comparative feeding kinematics and performance of odontocetes: belugas, Pacific white-sided dolphins and long-finned pilot whales. J. Exp. Biol. 212(24), 3939–3950 (2009)

    Article  Google Scholar 

  4. Austin, D., et al.: Linking movement, diving, and habitat to foraging success in a large marine predator. Ecology 87(12), 3095–3108 (2006)

    Article  Google Scholar 

  5. Hume, F., et al.: Spatial and temporal variation in the diet of a high trophic level predator, the Australian fur seal (Arctocephalus pusillus doriferus). Mar. Biol. 144(3), 407–415 (2004)

    Article  Google Scholar 

  6. Kirkwooe, R., Hume, F., Hindell, M.: Sea temperature variations mediate annual changes in the diet of Australian fur seals in Bass Strait. Mar. Ecol. Prog. Ser. 369, 297–309 (2008)

    Article  Google Scholar 

  7. Young, J.W., et al.: Feeding ecology and interannual variatons in diet of southern bluefin tuna, Thunnus maccoyii, in relation to coastal and oceanic waters off eastern Tasmania. Aust. Environ. Biol. Fishes 50(3), 275 (1997)

    Article  Google Scholar 

  8. Gales, R., et al.: Stomach contents of long-finned pilot whales (Globicephala melas) and bottlenose dolphins (Tursiops truncatus) in Tasmania. Mar. Mamm. Sci. 8(4), 405–413 (1992)

    Article  Google Scholar 

  9. LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989)

    Article  Google Scholar 

  10. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 2012, pp. 1097–1105 (2012)

    Google Scholar 

  11. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recoginiton. arXiv, arXiv:1409.1556 (2015)

  12. Szegedy, C., et al.: Going deeper with convolutions. In: IEEE Conference on computer vision and pattern recognition, Boston, MA, USA (2015)

    Google Scholar 

  13. Girshick, R.: Fast R-CNN. In: IEEE International Conference on Computer Vision, Los Alamitos, CA, USA (2015)

    Google Scholar 

  14. Ren, S., et al.: Faster R-CNN: towards real-time object detection with region proposal networkss. IEEE Trans. Patt. Anal. 39, 1137–1149 (2017)

    Article  Google Scholar 

  15. Dai, J., et al.: R-FCN: object detection via region-based fully convolutional networks. In: Advance in Neural Information 2016, pp. 379–387 (2016)

    Google Scholar 

  16. Girshick, R., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA (2014)

    Google Scholar 

  17. Zhong, J., Lei, T., Yao, G.: Robust vehicle detection in aerial images based on cascaded convolutional neural networks. Sensors 17, 2720 (2017)

    Article  Google Scholar 

  18. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  19. Oh, S.I., Kang, H.B.: Object detection and classification by decision-level fusion for intelligent vehicle systems. Sensors 17, 207 (2017)

    Article  Google Scholar 

  20. Huang, P.X., Boom, B.J., Fisher, R.B.: Hierarchical classification with reject option for live fish recognition. Mach. Vis. Appl. 26, 89–102 (2015)

    Article  Google Scholar 

  21. Chuang, M.C., et al.: Tracking live fish from low-contrast and low-frame-rate stereo videos. IEEE Trans. Circ. Syst. Video Technol. 25, 167–179 (2015)

    Article  Google Scholar 

  22. Jones, D.T., et al.: Evaluation of rockfish abundance in untrawlable habitat: combining acoustic and complementary sampling tools. Fish. Bull. 110, 332–343 (2012)

    Google Scholar 

  23. Pelletier, D., et al.: Comparison of visual census and high definition video transects for monitoring coral reef fish assemblages. Fish. Res. 107, 84–93 (2011)

    Article  Google Scholar 

  24. Struthers, D.P., et al.: Action cameras: Bringing aquatic and fisheries research into view. Fisheries 40, 502–512 (2015)

    Article  Google Scholar 

  25. Cappo, M., Harvey, E., Shortis, M.: Counting and measuring fish with baited video techniques - an overview. In: Australian Society for Fish Biology Workshop Proceedings, pp. 101–114 (2006)

    Google Scholar 

  26. Oquab, M., et al.: Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  27. Donahue, J., et al.: DeCAF: a deep convolutional activation feature for generic visual recognition. arXiv, arXiv:1310.1531 (2013)

  28. Zhang, N., Donahue, J., Girshick, R., Darrell, T.: Part-based R-CNNs for fine-grained category detection. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 834–849. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_54

    Chapter  Google Scholar 

  29. Kang, K., et al.: Object detection from video tubelets with convolutional neural networks. In: IEEE Converence on Computer Vision and Pattern Recognition, pp. 817–825 (2016)

    Google Scholar 

  30. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  31. Qin, H., et al.: DeepFish: accurate underwater live fish recognition with a deep architecture. Neurocomputing 187, 1–10 (2015). https://doi.org/10.1016/j.neucom.2015.10.122. (0925-2312)

    Article  Google Scholar 

  32. Uijlings, J.R., et al.: Selective search for object recognition. Int. J. Comput. Vis. 104, 154–171 (2013)

    Article  Google Scholar 

  33. Shou, Z., Wang, D. Chang, S.F.: Temporal action localization in untrimmed videos via multi-stage CNNs. In: CVPR, pp. 1–10 (2016)

    Google Scholar 

  34. Ren, S., et al.: Faster R-CNN: Towards real-time object detection with region proposal networks. arXiv:1506.01497 (2015)

  35. Everingham, M., et al.: The PASCAL visual object classes challenge: A retrospective. IJCV 111(1), 98–136 (2015)

    Article  Google Scholar 

  36. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  37. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  38. Hosang, J., Nenenson, R., Schiele, R.: How good are detection proposals, really?. arXiv, arXiv:1406.6962 (2014)

  39. Okuyama, J., et al.: Application of a computer vision technique to animal-borne video data: extraction of head movement to understand sea turtles’ visual assessment of surroundings. Anim. Biotelemetry 3, 35 (2015)

    Article  Google Scholar 

  40. Fang, Y., et al.: Motion based animal detection in aerial videos. Procedia Comput. Sci. 92, 13–17 (2016)

    Article  Google Scholar 

  41. Villon, S., Chaumont, M., Subsol, G., Villéger, S., Claverie, T., Mouillot, D.: Coral reef fish detection and recognition in underwater videos by supervised machine learning: comparison between deep learning and HOG+SVM methods. In: Blanc-Talon, J., Distante, C., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2016. LNCS, vol. 10016, pp. 160–171. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48680-2_15

    Chapter  Google Scholar 

  42. Xu, W., Matzner, S.: Underwater fish detection using deep learning for water power applications. In: 5th Annual Conference on Computational Science and Computational Intelligence, Las Vegas, NV (2018)

    Google Scholar 

  43. Li, X., et al.: Fast accurate fish detection and recognition of underwater images with R-CNN. In: OCEANS 2015 MTS/IEEE, Washington, pp. 1–5 (2015)

    Google Scholar 

  44. Rathi, D., Jain, S., Indu, D.S.: Underwater fish species classification using convolutional neural network and deep learning. arXiv, arXiv:1805.10106 (2018)

  45. Mandal, R., et al.: Assessing fish abundance from underwater video using deep neural networks. arXiv, arXiv:1807.05838 (2018)

  46. Zhou, H., et al.: Faster R-CNN for marine organism detection and recognition using data augmentation. In: ICVIP, Singapore (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information and Communication Technology, University of Tasmania, Sandy Bay, Hobart, TAS, 7000, Australia

    Mira Park, Wenli Yang, Zehong Cao & Byeong Kang

  2. Wild Ocean Tasmania, Eaglehawk Neck, TAS, 7179, Australia

    Damian Connor

  3. Ecology and Biodiversity Centre, Institute for Marine and Antarctic Studies, College of Science and Engineering, University of Tasmania, Hobart, TAS, 7001, Australia

    Mary-Anne Lea

Authors
  1. Mira Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenli Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zehong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Byeong Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Damian Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mary-Anne Lea
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mira Park .

Editor information

Editors and Affiliations

  1. Aoyama Gakuin University, Tokyo, Japan

    Kouzou Ohara

  2. University of Tasmania, Tasmania, Australia

    Quan Bai

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Park, M., Yang, W., Cao, Z., Kang, B., Connor, D., Lea, MA. (2019). Marine Vertebrate Predator Detection and Recognition in Underwater Videos by Region Convolutional Neural Network. In: Ohara, K., Bai, Q. (eds) Knowledge Management and Acquisition for Intelligent Systems. PKAW 2019. Lecture Notes in Computer Science(), vol 11669. Springer, Cham. https://doi.org/10.1007/978-3-030-30639-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-30639-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-30638-0

  • Online ISBN: 978-3-030-30639-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation