Virtual Reality Rendered Video Precognition with Deep Learning for Crowd Management

  • Howard MeadowsEmail author
  • George Frangou
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1069)


We describe an AI driven model based on the Nethra Video Analytic platform, optimised for overhead detection and designed specifically for CCTV and which can detect and categorize people from any angle. The model runs in real time in crowded and noisy environments and can be in- stalled on devices as in edge analytics or applied directly to existing video feeds. By mapping an entire space, we link together individual camera feeds and data points to calculate the total number of people to assist with capacity planning and to pin point bottlenecks in people flows. Solving the problem of aggregating multiple 360-degree video camera feeds into a single combined rendering, we further describe a novel use of interactive Virtual Reality. This model renders St Pancras International Station in VR and can track people movement in real time. Real people are rep- resented by avatars in real-time in the model. Users are able to change their viewpoint to look at any angle. The movement of the avatars exactly mirrors what can be seen in the cameras.


Video Virtual Reality Mixed Reality Deep learning Crowd management CCTV City Railway station Airport Shopping Precinct Metro webGL Massive Analytic Nethra Precognition Path analysis People detection People counting GPU 


  1. 1.
  2. 2.
    Frangou, G.J.: Apparatus for Controlling a Land Vehicle which Is Self-Driving or Partially Self-Driving. U.S. Patent No. 9,645,576 Chinese Patent 105189237, Japanese Patent 2016520464, Israel Patent 241688, European Patent Application 2976240, Korean Patent Application 20150138257, PCT Patent Application 2014147361 (2013)Google Scholar
  3. 3.
    Cao, Z., Simon, T., Wei, S.-E., Sheikh, Y.: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. The Robotics Institute, Carnegie Mellon University. arXiv:1611.08050v2 [cs.CV]. Accessed 14 Apr 2017
  4. 4.
    Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale Video Classification with Convolutional Neural Networks (2014)Google Scholar
  5. 5.
    Marcenaro, L.: Access to Data Sets, ERNCIP Thematic Group on Video Surveillance for Security of Critical Infrastructure. European Commission, JRC Science Hub, University of Genoa, Italy (2016)Google Scholar
  6. 6.
    Arroyo, R., Javier Yebes, J., Bergasa, L.M., Daza, I.G., Almazan, J.: Expert video surveillance system for real-time detection of suspicious behaviors in shopping malls, expert systems with applications (2015)CrossRefGoogle Scholar
  7. 7.
    Bleser, G., Stricker, D.: Computer Vision: Object and People Tracking, Proceedings of Seminar and Project, Winter semester 2013/14, University of Kaiserslautern and DFKI GmbHGoogle Scholar
  8. 8.
    Romera, E., Bergasa, L.M., Arroyo, R.: A real-time multi-scale vehicle detection and tracking approach for smartphones. In: 2015 IEEE 18th International Conference on Intelligent Transportation Systems (2015)Google Scholar
  9. 9.
    Romera, E., Alvarez, J.M.A., Bergasa, L.M., Arroyo, R.: Efficient ConvNet for real-time semantic segmentation. In: 2017 IEEE Intelligent Vehicles Symposium IV (2017)Google Scholar
  10. 10.
    Maddalena, L., Petrosino, A.: Background subtraction for moving object detection in RGBD data: a survey. J. Imaging 4, 71 (2018)CrossRefGoogle Scholar
  11. 11.
    Laaraiedh, M.: Implementation of Kalman Filter with Python Language. The Python Papers (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Massive Analytic Limited, IDEALondonLondonUK

Personalised recommendations