Stereo and Motion Based 3D High Density Object Tracking

  • Junli Tao
  • Benjamin Risse
  • Xiaoyi Jiang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8333)


In order to understand the behavior of adult Drosophila melanogaster (fruit flies), vision-based 3D trajectory reconstruction methods are adopted. To improve the statistical strength of subsequent analysis, high-throughput measurements are necessary. However, ambiguities in both stereo matching and temporal tracking appear more frequently in high density situations, aggravating the complexity of the 3D tracking situation. In this paper we propose a high density object tracking algorithm. Instead of approximating trajectories for all frames in a direct manner, in ambiguous situations, tracking is terminated to generate robust tracklets based on the modified tracking-by-matching method. The terminated tracklets are linked to ongoing (unterminated) tracklets with minimum linking cost in an on-line fashion. Furthermore, we introduce a set of new evaluation metrics to analyze the tracking results. These metrics are used to analyse the effect of detection noise and compare our tracking algorithm with two state-of-the-art 3D tracking methods based on simulated data with hundreds of flies. The results indicate that our proposed algorithm outperforms both, the tracking-by-matching algorithm and a global correspondence selection approach.


Drosophila melanogaster fruit flies 3D tracking tracklets stereo matching Kalman filter evaluation metrics 


  1. 1.
    Ardekani, R., Biyani, A., Dalton, J.E., Saltz, J.B., Arbeitman, M.N., Tower, J., Nuzhdin, S., Tavaré, S.: Three-dimensional Tracking and Behaviour Monitoring of Multiple Fruit Flies. J. Royal Society Interface 10, 1–13 (2013)Google Scholar
  2. 2.
    Burkard, R.E., Dell’Amico, M., Martello, S.: Assignment Problems, 1st edn. Society of Industrial Mathematics (2009)Google Scholar
  3. 3.
    Colomb, J., Reiter, L., Blaszkiewicz, J., Wessnitzer, J.: Open Source Tracking and Analysis of Adult Drosophila Locomotion in Buridan’s Paradigm with and without Visual Targets. PloS One 7(8), e41642 (2012)Google Scholar
  4. 4.
    Du, H., Zou, D., Chen, Y.Q.: Relative Epipolar Motion of Tracked Features for Correspondence in Binocular Stereo. In: 11th Internitional Conference Computer Vision, pp. 1–8. IEEE Press, Rio de Janeiro (2007)Google Scholar
  5. 5.
    Fry, S.N., Bichsel, M., Müller, P., Robert, D.: Tracking of Flying Insects Using Pan-tilt Cameras. J. Neurosci Methods 101(1), 59–67 (2000)CrossRefGoogle Scholar
  6. 6.
    Grover, D., Tower, J., Tavaré, S.: O fly, Where Art Thou? J. Royal Society 5, 1181–1191 (2008)Google Scholar
  7. 7.
    Kohlhoff, K.J., Jahn, T.R., Lomas, D.A., Dobson, C.M., Crowther, D.C., Vendruscolo, C.M.: The iFly tracking system for an automated locomotor and behavioural analysis of Drosophila melanogaster. Integrative Biology (2011)Google Scholar
  8. 8.
    Kuo, C., Nevatia, R.: How Does Person Identity Recognition Help Multi-person Tracking? In: Computer Vision and Pattern Recognition, pp. 1217–1224. IEEE Press, Colorado Springs (2011)Google Scholar
  9. 9.
    Liu, Y., Li, H., Chen, Y.Q.: Automatic Tracking of a Large Number of Moving Targets in 3D. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 730–742. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Marden, J.H., Wolf, K.E.: Areal performance of Drosophila melanogaster from populations selected for upwind flight ability. Journal of Experimental Biology 200, 2747–2755 (1997)Google Scholar
  11. 11.
    Pereira, F., Stuer, H., Graff, E.C., Gharib, M.: Two-frame 3d Particle Tracking. J. Measurement Science Technology 17, 1–8 (2006)CrossRefGoogle Scholar
  12. 12.
    Risse, B., Thomas, S., Otto, N., Löpmeier, T., Valkov, D., Jiang, X., Klämbt, C.: FIM, a Novel FTIR-Based Imaging Method for High Throughput Locomotion Analysis. PloS One 8(1), e53963 (2013)Google Scholar
  13. 13.
    Sokolowski, M.A.: Drosophila: Genetics Meets Behavior. J. Nature Reviews Genetics 2(11), 879–890 (2001)CrossRefGoogle Scholar
  14. 14.
    Straw, A.D., Branson, K., Neumann, T.R., Dickinson, M.H.: Multi-camera Real-time Three-dimensional Tracking of Multiple Flying Aanimals. J. Royal Society 8, 395–409 (2011)Google Scholar
  15. 15.
    Tao, J., Klette, R.: Tracking of 2D or 3D Irregular Movement by a Family of Unscented Kalman Filters. J. Information Communication Convergence Engineering 1, 307–314 (2012)CrossRefGoogle Scholar
  16. 16.
    Tao, J., Risse, B., Jiang, X., Klette, R.: 3D Trajectory Estimation of Simulated Fruit Flies. In: 27th Image Vision Computing New Zealand, pp. 31–36. ACM, Dunedin (2012)Google Scholar
  17. 17.
    Valente, D., Golani, I., Mitra, P.P.: Analysis of the Trajectory of Drosophila Melanogaster in a Circular Open Field Arena. PloS One 2(10), e1083 (2007)Google Scholar
  18. 18.
    Wu, H.S., Zhao, Q., Zou, D., Chen, Y.Q.: Acquiring 3d Motion Trajectories of Large Numbers of Swarming Animals. In: 12th Internitional Conference Computer Vision Workshop, pp. 593–600. IEEE Press, Kyoto (2009)Google Scholar
  19. 19.
    Yang, B., Nevatia, R.: An Online Learned CRF Model for Multi-target Tracking. In: Computer Vision and Pattern Recognition, pp. 2034–2041. IEEE Press, Oregon (2012)Google Scholar
  20. 20.
    Zou, D., Zhao, Q., Wu, H.S., Chen, Y.Q.: Reconstructing 3d Motion Trajectories of Particle Swarms by Global Correspondence Selection. In: 12th Internitional Conference Computer Vision, pp. 1578–1585. IEEE Press, Kyoto (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Junli Tao
    • 1
  • Benjamin Risse
    • 2
  • Xiaoyi Jiang
    • 2
  1. 1.Computer ScienceUniversity of AucklandNew Zealand
  2. 2.Computer Science, NeurobiologyUniversity of MünsterGermany

Personalised recommendations