Automated recognition and discrimination of human–animal interactions using Fisher vector and hidden Markov model
- 7 Downloads
Human–animal interactions may affect the animal welfare and productivity in rearing environments. Previously proposed human–animal-related techniques focus on the manual discrimination of single animal behaviors or simple human–animal interactions. To address the automatic detection and classification of complex animal behaviors and the animals reactions to human, we propose an approach built upon both the visual representation with Fisher vectors and the end-to-end generative hidden Markov model to facilitate the discrimination of both coarse- and fine-grained animal–human interactions. To satisfy the requirement for abundant data samples of the generative approach, we recorded and annotated more than 480 hours of videos featuring eight persons and 210 laying hens during the process of feeding and cleaning. The experimental results show that the proposed method outperforms state-of-the-art approaches. According to the experimental performance of our method on practical videos, our approach can be used to monitor the human–animal interactions or animal behaviors in modern poultry farms.
KeywordsClassification Machine vision Animal behavior Artificial intelligence
The authors sincerely thank the editors and reviewers for their work.
Y. Zheng conceived of the study and designed the experiments. JL, WJ and Y. Zhao performed the experiments, MF, DW and SS analyzed the data. JL wrote the paper. All authors helped revise and approved the manuscript.
This work was made possible through support from the Natural Science Foundation of China (61572300), Taishan Scholar Program of Shandong Province in China (TSHW201502038) and SDUST Excellent Teaching Team Construction Plan JXTD20160512.
Compliance with ethical standards
Conflict of interest
The authors declare no conflict of interest. The funding sponsors had no role in the design of the study; in the collection, analyses or interpretation of the data; in the writing of the manuscript, or in the decision to publish the results.
This study was approved by the Animal Care and Use Committee of Qingdao Agricultural University (Qingdao, China).
Availability of data and material
The datasets analyzed during the current study are available from the corresponding authors upon reasonable request.
- 4.Cinbis, R.G., Verbeek, J.J., Schmid, C.: Segmentation driven object detection with fisher vectors. 2968–2975 (2013)Google Scholar
- 7.Forman, G, Scholz, M.B., Rajaram, S.S.,: Feature shaping for linear svm classifiers. 299–308 (2009)Google Scholar
- 10.Ke, Y., Sukthankar, R.: Pca-sift: a more distinctive representation for local image descriptors. 2, 506–513 (2004)Google Scholar
- 12.Linden, D.V.D., Zamansky. A.: Agile with animals: towards a development method. In: IEEE International Requirements Engineering Conference Workshops, 423–426 (2017)Google Scholar
- 13.Mancini, C.: Animal-computer interaction: a manifesto. interactions 18(4), 69–73 (2011)Google Scholar
- 15.Nakarmi, A.D., Tang, L., Xin, H.: Automated tracking and behavior quantification of laying hens using 3d computer vision and radio frequency identification technologies. Trans. ASABE. 57(5), 1455–1472 (2014)Google Scholar
- 19.Perronnin, F., Larlus, D.: Fisher vectors meet neural networks: A hybrid classification architecture, 3743–3752 (2015)Google Scholar
- 23.Wang, C., Chen, H., Zhang, X., Meng, C.: Evaluation of a laying-hen tracking algorithm based on a hybrid support vector machine. J. Anim. Sci. Biotechnol. 8(1), 226–235 (2017)Google Scholar
- 24.Welch, Lloyd R.: Hidden markov models and the Baum-Welch algorithm. IEEE Inf. Theory Soc. Newsl. 53(2), 194–211 (2003)Google Scholar
- 27.Young, S., Evermann, G., Gales, M., Kershaw, D., Moore, G., Odell, J., Ollason, D., Povey, D., Valtchev, V., Woodland, P.: The htk book version 3.4. Cambridge University Engineering Department, 2006Google Scholar
- 28.Zamansky, A., Roshier, A., Mancini, C., Collins, E.C., Hall, C., Grillaert, K., Morrison, A., North, S., Wirman, H.: A report on the first international workshop on research methods in animal-computer interaction. In: CHI Conference Extended Abstracts on Human Factors in Computing Systems, 806–815 (2017)Google Scholar