Skip to main content
SpringerLink
Log in

Analysis of Motion Patterns in Video Streams for Automatic Health Monitoring in Koi Ponds

  • Conference paper
  • First Online:
Image and Video Technology (PSIVT 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11854))

Included in the following conference series:

  • 1276 Accesses

Abstract

We present a motion analysis framework for anomaly detection in the context of health monitoring in carp and koi ponds. We study recent advances in computer vision and deep learning for an automated motion assessment based on video streams and propose a specifically designed image acquisition system.

It turned out that the accurate detection and recognition of individual fish objects remains a difficult topic for scenarios with dense homogeneous groups and frequently occurring occlusions. We thus tackled this challenging field of aquatic scene understanding by applying deep state-of-the-art architectures from the areas of object detection, semantic segmentation and instance segmentation as a first step for further extraction of motion information. We used dense optical flow as an estimation of collective fish movements and restricted the motion extraction according to the resulting masks from the previous image segmentation step.

We introduce a heatmap visualization as an intermediate representation of the spatio-temporal distribution of fish locations. We derived several metrics to quantify changes in motion patterns and apparent location hotspots as indicators of anomalous behavior. Based on this representation, we were able to identify different classes of behavior like feeding times, shoaling or night’s rest as well as anomalous group behavior like mobbing or hiding in an experimental setup.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The recording of the fish video dataset, as well as the manual fish observations, were done by co-author Arndt Christian Hofmann as part of the data acquisition process for his unpublished PhD thesis at Friedrich-Loeffler-Institut.

References

  1. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online, realtime tracking. In: IEEE International Conference on Image Processing, pp. 3464–3468 (2016)

    Google Scholar 

  2. Girshick, R.: Fast R-CNN. In: IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

    Google Scholar 

  3. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection, semantic segmentation. In: IEEE Conference on Computer Vision, Pattern Recognition, pp. 580–587 (2014)

    Google Scholar 

  4. Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. SMC 3(6), 610–621 (1973)

    Article  Google Scholar 

  5. He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

    Google Scholar 

  6. He, X., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision, Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  7. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)

    Article  Google Scholar 

  8. Jiang, Y., Fang, J., Li, Z., Yue, J., Wang, Z., Li, D.: Automatic tracking of swimming koi using a particle filter with a center-surrounding cue. Mathe. Comput. Model. 58(3), 859–867 (2013)

    Article  Google Scholar 

  9. Kuznetsova, A., et al.: The open images dataset V4: unified image classification, object detection, and visual relationship detection at scale. arXiv:1811.00982 (2018)

  10. Li, X., Wei, Z., Huang, L., Nie, J., Zhang, W., Wang, L.: Real-time underwater fish tracking based on adaptive multi-appearance model. In: 25th IEEE International Conference on Image Processing, pp. 2710–2714 (2018)

    Google Scholar 

  11. Li, X., Shang, M., Qin, H., Chen, L.: Fast accurate fish detection, recognition of underwater images with Fast R-CNN. In: OCEANS 2015 MTS, pp. 1–5. IEEE, Washington (2015)

    Google Scholar 

  12. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: IEEE Conference on Computer Vision, Pattern Recognition, pp. 936–944 (2017)

    Google Scholar 

  13. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  14. Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  15. Niu, B., Li, G., Peng, F., Wu, J., Zhang, L., Li, Z.: Survey of fish behavior analysis by computer vision. J. Aquac. Res. Dev. 9(5) (2018)

    Google Scholar 

  16. Peng, C., Zhang, X., Yu, G., Luo, G., Sun, J.: Large kernel matters - improve semantic segmentation by global convolutional network. In: IEEE Conference on Computer Vision, Pattern Recognition, pp. 1743–1751 (2017)

    Google Scholar 

  17. Raman, R., Prakash, C., Marappan, M., Pawar, N.A.: Environmental stress mediated diseases of fish: an overview. Adv. Fish Res. 5, 141–158 (2013)

    Google Scholar 

  18. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: IEEE Conference on Computer Vision, Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  19. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: IEEE Conference on Computer Vision, Pattern Recognition, pp. 6517–6525 (2017)

    Google Scholar 

  20. Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv:1804.02767 (2018)

  21. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  22. Saberioon, M., Gholizadeh, A., Cisar, P., Pautsina, A., Urban, J.: Application of machine vision systems in aquaculture with emphasis on fish: state-of-the-art, key issues. Rev. Aquac. 9(4), 369–387 (2017)

    Article  Google Scholar 

  23. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE Conference on Computer Vision, Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  24. Shevchenko, V., Eerola, T., Kaarna, A.: Fish detection from low visibility underwater videos. In: 24th International Conference on Pattern Recognition, pp. 1971–1976 (2018)

    Google Scholar 

  25. Snieszko, S.F.: The effects of environmental stress on outbreaks of infectious diseases of fishes. J. Fish Biol. 6(2), 197–208 (1974)

    Article  Google Scholar 

  26. Sung, M., Yu, S., Girdhar, Y.: Vision based real-time fish detection using convolutional neural network. In: OCEANS 2017, Aberdeen, pp. 1–6 (2017)

    Google Scholar 

  27. Terayama, K., Habe, H., Sakagami, M.: Multiple fish tracking with an NACA airfoil model for collective behavior analysis. IPSJ Trans. Comput. Vis. Appl. 8(1), 1–7 (2016)

    Article  Google Scholar 

  28. Terayama, K., Hioki, H., Sakagami, M.: A measurement method for speed distribution of collective motion with optical flow, its application to estimation of rotation curve. In: IEEE International Symposium on Multimedia, pp. 32–39 (2014)

    Google Scholar 

  29. Terayama, K., Hongo, K., Habe, H., Sakagami, M.: Appearance-based multiple fish tracking for collective motion analysis. In: 3rd IAPR Asian Conference on Pattern Recognition, pp. 361–365 (2015)

    Google Scholar 

  30. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: DeepFlow: large displacement optical flow with deep matching. In: IEEE International Conference on Computer Vision, pp. 1385–1392 (2013)

    Google Scholar 

  31. Yu, X., Hou, X., Lu, H., Yu, X., Fan, L., Liu, Y.: Anomaly detection of fish school behavior based on features statistical, optical flow methods. Trans. Chin. Soc. Agric. Eng. 30(2), 162–168 (2014)

    Google Scholar 

  32. Zhao, X., Yan, S., Gao, Q.: An algorithm for tracking multiple fish based on biological water quality monitoring. IEEE Access 7, 15018–15026 (2019)

    Article  Google Scholar 

  33. Zion, B.: The use of computer vision technologies in aquaculture - a review. Comput. Electron. Agric. 88, 125–132 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Informatik, Humboldt-Universität zu Berlin, Unter den Linden 6, 10117, Berlin, Germany

    Christian Hümmer, Dominik Rueß, Jochen Rueß, Niklas Deckers & Ralf Reulke

  2. Friedrich-Loeffler-Institut, Südufer 10, 17493, Greifswald - Insel Riems, Germany

    Arndt Christian Hofmann & Sven M. Bergmann

Authors
  1. Christian Hümmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dominik Rueß
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jochen Rueß
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Niklas Deckers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arndt Christian Hofmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sven M. Bergmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ralf Reulke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Hümmer .

Editor information

Editors and Affiliations

  1. Chonnam National University, Gwangju, Korea (Republic of)

    Chilwoo Lee

  2. Dalian University of Technology, Dalian, China

    Zhixun Su

  3. National Institute of Informatics, Tokyo, Japan

    Akihiro Sugimoto

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hümmer, C. et al. (2019). Analysis of Motion Patterns in Video Streams for Automatic Health Monitoring in Koi Ponds. In: Lee, C., Su, Z., Sugimoto, A. (eds) Image and Video Technology. PSIVT 2019. Lecture Notes in Computer Science(), vol 11854. Springer, Cham. https://doi.org/10.1007/978-3-030-34879-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-34879-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-34878-6

  • Online ISBN: 978-3-030-34879-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation