Advertisement

Group LSTM: Group Trajectory Prediction in Crowded Scenarios

  • Niccoló BisagnoEmail author
  • Bo Zhang
  • Nicola Conci
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11131)

Abstract

The analysis of crowded scenes is one of the most challenging scenarios in visual surveillance, and a variety of factors need to be taken into account, such as the structure of the environments, and the presence of mutual occlusions and obstacles. Traditional prediction methods (such as RNN, LSTM, VAE, etc.) focus on anticipating individual’s future path based on the precise motion history of a pedestrian. However, since tracking algorithms are generally not reliable in highly dense scenes, these methods are not easily applicable in real environments. Nevertheless, it is very common that people (friends, couples, family members, etc.) tend to exhibit coherent motion patterns. Motivated by this phenomenon, we propose a novel approach to predict future trajectories in crowded scenes, at the group level. First, by exploiting the motion coherency, we cluster trajectories that have similar motion trends. In this way, pedestrians within the same group can be well segmented. Then, an improved social-LSTM is adopted for future path prediction. We evaluate our approach on standard crowd benchmarks (the UCY dataset and the ETH dataset), demonstrating its efficacy and applicability.

Keywords

Group prediction Crowd analysis Trajectory clustering Social-LSTM 

Notes

Acknowledgement

This work is partly supported by the National Natural Science Foundation of China (Grant No. 61702073), and the Fundamental Research Funds for the Central Universities (Grant No. 3132018190).

References

  1. 1.
    Context-aware trajectory prediction in crowded spaces. In: Proceedings of theGoogle Scholar
  2. 2.
    Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: human trajectory prediction in crowded spaces. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 961–971. IEEE (2016)Google Scholar
  3. 3.
    Alahi, A., Ramanathan, V., Li, F.F.: Socially-aware large-scale crowd forecasting. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 2203–2210. IEEE (2014)Google Scholar
  4. 4.
    Ali, S., Shah, M.: A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1–6. IEEE (2007)Google Scholar
  5. 5.
    Allain, P., Courty, N., Corpetti, T.: Crowd flow characterization with optimal control theory. In: Zha, H., Taniguchi, R., Maybank, S. (eds.) ACCV 2009. LNCS, vol. 5995, pp. 279–290. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-12304-7_27CrossRefGoogle Scholar
  6. 6.
    Vemula, A., Muelling, K., Oh, J.: Social attention: modeling attention in human crowds. https://arxiv.org/abs/1710.04689 (2017)
  7. 7.
    Bagautdinov, T., Alahi, A., Fleuret, F., Fua, P., Savarese, S.: Social scene understanding: end-to-end multi-person action localization and collective activity recognition. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 3425–3434. IEEE (2016)Google Scholar
  8. 8.
    Ballan, L., Castaldo, F., Alahi, A., Palmieri, F., Savarese, S.: Knowledge transfer for scene-specific motion prediction. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 697–713. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_42CrossRefGoogle Scholar
  9. 9.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
  10. 10.
    Conci, N., Bisagno, N., Cavallaro, A.: On modeling and analyzing crowds from videos. In: Computer Vision for Assistive Healthcare, pp. 319–336. Elsevier (2018)Google Scholar
  11. 11.
    Dauphin, Y., de Vries, H., Bengio, Y.: Equilibrated adaptive learning rates for non-convex optimization. In: Advances in Neural Information Processing Systems (NIPS), pp. 1504–1512 (2015)Google Scholar
  12. 12.
    Ge, W., Collins, R.T., Ruback, R.B.: Vision-based analysis of small groups in pedestrian crowds. IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 1003–1016 (2012)CrossRefGoogle Scholar
  13. 13.
    Grant, J., Flynn, P.: Crowd scene understanding from video: a survey. ACM Trans. Multimedia Comput. Commun. Appl. 13(2), 19 (2017)CrossRefGoogle Scholar
  14. 14.
    Gupta, A., Savarese, S., Alahi, A., et al.: Social gan: Socially acceptable trajectories with generative adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), No. CONF (2018)Google Scholar
  15. 15.
    Hall, E.T., et al.: Proxemics [and comments and replies]. Curr. Anthropol. 9(2/3), 83–108 (1968)Google Scholar
  16. 16.
    Hassner, T., Itcher, Y., Kliper-Gross, O.: Violent flows: Real-time detection of violent crowd behavior. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–6. IEEE (2012)Google Scholar
  17. 17.
    Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51(5), 4282 (1995)CrossRefGoogle Scholar
  18. 18.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  19. 19.
    Jain, A., Zamir, A., Savarese, S., Saxena, A.: Structural-RNN: deep learning on spatio-temporal graphs. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 5308–5317. IEEE (2015)Google Scholar
  20. 20.
    Kok, V., Mei, K., Chan, C.: Crowd behavior analysis: a review where physics meets biology. Neurocomputing 177, 342–362 (2016)CrossRefGoogle Scholar
  21. 21.
    Lawal, I., Poiesi, F., Aguita, D., Cavallaro, A.: Support vector motion clustering. IEEE Trans. Circuits Syst. Video Technol. 27(11), 2395–2408 (2017)CrossRefGoogle Scholar
  22. 22.
    Lee, N., Choi, W., Vernaza, P., Choy, C., Torr, P., Chandraker, M.: Desire: Distant future prediction in dynamic scenes with interacting agents. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 2165–2174. IEEE (2017)Google Scholar
  23. 23.
    Lerner, A., Chrysanthou, Y., Lischinski, D.: Crowds by example, vol. 26, pp. 655–664. Wiley Online Library (2007)Google Scholar
  24. 24.
    Li, T., Chang, H., Wang, M., Ni, B., Hong, R., Yan, S.: Crowded scene analysis: a survey. IEEE Trans. Circuits Syst. Video Technol. 25(3), 367–386 (2015)CrossRefGoogle Scholar
  25. 25.
    Li, W., Mahadevan, V., Vasconcelos, N.: Anomaly detection and localization in crowded scenes. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 18–32 (2013)Google Scholar
  26. 26.
    Ma, W., Huang, D., Lee, N., Kitani, K.M.: A game-theoretic approach to multi-pedestrian activity forecasting (2016)Google Scholar
  27. 27.
    Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: modeling social behavior for multi-target tracking. In: Proceedings of the International Conference on Computer Vision, pp. 261–268. IEEE (2009)Google Scholar
  28. 28.
    Robicquet, A., Sadeghian, A., Alahi, A., Savarese, S.: Learning social etiquette: human trajectory understanding in crowded scenes. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 549–565. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46484-8_33CrossRefGoogle Scholar
  29. 29.
    Robicquet, A., et al.: Forecasting social navigation in crowded complex scenes (2016)Google Scholar
  30. 30.
    Stewart, R., Andriluka, M., Ng, A.: End-to-end people detection in crowded scenes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 2325–2333. IEEE (2016)Google Scholar
  31. 31.
    Ryoo, M., Aggarwal, J.: Stochastic representation and recognition of high-level group activities. Int. J. Comput. Vis. 93(2), 183–200 (2011)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Shao, J., Loy, C.C., Wang, X.: Learning scene-independent group descriptors for crowd understanding. IEEE Trans. Circuits Syst. Video Technol. 27(6), 1290–1303 (2017)CrossRefGoogle Scholar
  33. 33.
    Solmaz, B., Moore, B.E., Shah, M.: Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 2064–2070 (2012)CrossRefGoogle Scholar
  34. 34.
    Sutskever, I., Vinyals, O., Le, Q.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)Google Scholar
  35. 35.
    Tomasi, C., Kanade, T.: Detection and tracking of point features (1991)Google Scholar
  36. 36.
    Treuille, A., Cooper, S., Popović, Z.: Continuum crowds. ACM Trans. Graphics (TOG) 25, 1160–1168 (2006)CrossRefGoogle Scholar
  37. 37.
    Van Den Berg, J., Guy, S.J., Lin, M., Manocha, D.: Reciprocal n-body collision avoidance. In: Pradalier, C., Siegwart, R., Hirzinger, G. (eds.) Robot. Res., pp. 3–19. Springer, Berlin (2011)CrossRefGoogle Scholar
  38. 38.
    Varshneya, D., Srinivasaraghavan, G.: Human trajectory prediction using spatially aware deep attention models. https://arxiv.org/abs/1705.09436 (2017)
  39. 39.
    Vemula, A., Muelling, K., Oh, J.: Social attention: Modeling attention in human crowds. arXiv:1710.04689 (2017)
  40. 40.
    Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: Proceedings of the International Conference on Computer Vision and Computer Vision, pp. 1345–1352. IEEE (2011)Google Scholar
  41. 41.
    Yi, S., Li, H., Wang, X.: Understanding pedestrian behaviors from stationary crowd groups. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 3488–3496. IEEE (2015)Google Scholar
  42. 42.
    Zhong, J., Cai, W., Luo, L., Yin, H.: Learning behavior patterns from video: a data-driven framework for agent-based crowd modeling. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, pp. 801–809 (2015)Google Scholar
  43. 43.
    Zhou, B., Tang, X., Wang, X.: Learning collective crowd behaviors with dynamic agent. Int. J. Comput. Vis. 111(1), 50–68 (2015)CrossRefGoogle Scholar
  44. 44.
    Zhou, B., Tang, X., Wang, X.: Coherent filtering: detecting coherent motions from crowd clutters. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 857–871. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33709-3_61CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of TrentoTrentoItaly
  2. 2.Dalian Maritime UniversityDalianChina

Personalised recommendations