# The Multi-strand Graph for a PTZ Tracker

- 233 Downloads

## Abstract

High-resolution images can be used to resolve matching ambiguities between trajectory fragments (tracklets), which is a key challenge in multiple-target tracking. A pan–tilt–zoom (PTZ) camera, which can pan, tilt and zoom, is a powerful and efficient tool that offers both close-up views and wide area coverage on demand. The wide area enables tracking of many targets, while the close-up view allows individuals to be identified from high-resolution images of their faces. A central component of a PTZ tracking system is a scheduling algorithm that determines which target to zoom in on, particularly when the high-resolution images are also used for tracklet matching. In this paper, we study this scheduling problem from a theoretical perspective. We propose a novel data structure, the Multi-strand Tracking Graph (MSG), which represents the set of tracklets computed by a tracker and the possible associations between them. The MSG allows efficient scheduling as well as resolving of matching ambiguities between tracklets. The main feature of the MSG is the auxiliary data saved in each vertex, which allows efficient computation while avoiding time-consuming graph traversal. Synthetic data simulations are used to evaluate our scheduling algorithm and to demonstrate its superiority over a naïve one.

## Keywords

Tracking Scheduling Data structure PTZ camera## Notes

### Acknowledgements

This research was supported by the Israeli Ministry of Science, Grant No. 3-8700 and by Award No. 2011-IJ-CX-K054, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.

## Supplementary material

Supplementary material 1 (avi 49205 KB)

## References

- 1.Bagdanov, A., del Bimbo, A., Pernici, F.: Acquisition of high-resolution images through on-line saccade sequence planning. In: International Workshop on Video Surveillance and Sensor Networks (VSSN) (2005)Google Scholar
- 2.Cai, Y., Medioni, G.: Persistent people tracking and face capture using a PTZ camera. Mach. Vis. Appl.
**27**(3), 397–413 (2016)CrossRefGoogle Scholar - 3.Cai, Y., Medioni, G., Dinh, T.: Towards a practical PTZ face detection and tracking system. In: IEEE Winter Conference on Applications of Computer Vision (WACV) (2013)Google Scholar
- 4.Costello, C., Diehl, C., Banerjee, A., Fisher, H.: Scheduling an active camera to observe people. In: International Workshop on Video Surveillance and Sensor Networks (VSSN) (2004)Google Scholar
- 5.Costello, C., Wang, I.: Surveillance camera coordination through distributed scheduling. In: IEEE Conference on Decision and Control and European Control Conference (CDC-ECC) (2005)Google Scholar
- 6.Del Bimbo, A., Pernici, F.: Towards on-line saccade planning for high-resolution image sensing. Pattern Recognit. Lett.
**27**(15), 1826–1834 (2006)CrossRefGoogle Scholar - 7.Henriques, J., Caseiro, R., Batista, J.: Globally optimal solution to multi-object tracking with merged measurements. In: International Conference on Computer Vision (ICCV) (2011)Google Scholar
- 8.Krahnstoever, N., Yu, T., Lim, S., Patwardhan, K., Tu, P.: Collaborative real-time control of active cameras in large scale surveillance systems. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (2008)Google Scholar
- 9.Lim, S., Davis, L., Mittal, A.: Constructing task visibility intervals for video surveillance. Multimed. Syst.
**12**(3), 211–226 (2006)CrossRefGoogle Scholar - 10.Lim, S., Davis, L., Mittal, A.: Task scheduling in large camera networks. In: Asian Conference on Computer Vision (2007)Google Scholar
- 11.Morye, A., Ding, C., Roy-Chowdhury, A., Farrell, J.: Distributed constrained optimization for bayesian opportunistic visual sensing. IEEE Trans. Control Syst. Technol.
**22**(6), 2302–2318 (2014)CrossRefGoogle Scholar - 12.Natarajan, P., Hoang, T., Low, K., Kankanhalli, M.: Decision-theoretic approach to maximizing observation of multiple targets in multi-camera surveillance. In: International Conference on Autonomous Agents and Multiagent Systems (2012)Google Scholar
- 13.Neves, J.C., Proença, H.: Dynamic camera scheduling for visual surveillance in crowded scenes using Markov random fields. In: IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS) (2015)Google Scholar
- 14.Nillius, P., Sullivan, J., Carlsson, S.: Multi-target tracking-linking identities using Bayesian network inference. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2006)Google Scholar
- 15.Prokaj, J., Duchaineau, M., Medioni, G.: Inferring tracklets for multi-object tracking. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2011)Google Scholar
- 16.Qureshi, F., Terzopoulos, D.: Surveillance camera scheduling: a virtual vision approach. Multimed. Syst.
**12**(3), 269–283 (2006)CrossRefGoogle Scholar - 17.Qureshi, F., Terzopoulos, D.: Surveillance in virtual reality: system design and multi-camera control. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007)Google Scholar
- 18.Qureshi, F., Terzopoulos, D.: Planning ahead for ptz camera assignment and handoff. In: ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC) (2009)Google Scholar
- 19.Reta, C., Altamirano, L., Gonzalez, J.A., Medina-Carnicer, R.: Three hypothesis algorithm with occlusion reasoning for multiple people tracking. J. Electron. Imaging
**24**(1), 013015–013015 (2015)CrossRefGoogle Scholar - 20.Salvagnini, P., Pernici, F., Cristani, M., Lisanti, G., Del Bimbo, A., Murino, V.: Non-myopic information theoretic sensor management of a single pan-tilt-zoom camera for multiple object detection and tracking. Comput. Vis. Image Underst.
**134**, 74–88 (2015)CrossRefGoogle Scholar - 21.Salvagnini, P., Pernici, F., Cristani, M., Lisanti, G., Masi, I., Del Bimbo, A., Murino, V.: Information theoretic sensor management for multi-target tracking with a single pan-tilt-zoom camera. In: IEEE Winter Conference on Applications of Computer Vision (WACV) (2014)Google Scholar
- 22.Sekii, T.: Robust, real-time 3D tracking of multiple objects with similar appearances. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
- 23.Sommerlade, E., Reid, I.: Information-theoretic active scene exploration. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
- 24.Strat, T., Arambel, P., Antone, M., Rago, C., Landan, H.: A multiple-hypothesis tracking of multiple ground targets from aerial video with dynamic sensor control. In: Signal Processing, Sensor Fusion, and Target Recognition (SPIE) (2004)Google Scholar
- 25.Sullivan, J., Carlsson, S.: Tracking and labelling of interacting multiple targets. In: European Conference on Computer Vision (ECCV) (2006)Google Scholar
- 26.Wang, X., Türetken, E., Fleuret, F., Fua, P.: Tracking interacting objects optimally using integer programming. In: European Conference on Computer Vision (ECCV) (2014)Google Scholar
- 27.Ward, C., Naish, M.: Scheduling active camera resources for multiple moving targets. In: Canadian Conference on Electrical and Computer Engineering (CCECE) (2009)Google Scholar
- 28.Wheeler, F.W., Weiss, R.L., Tu, P.H.: Face recognition at a distance system for surveillance applications. In: IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS) (2010)Google Scholar
- 29.Wu, Z., Kunz, T., Betke, M.: Efficient track linking methods for track graphs using network-flow and set-cover techniques. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
- 30.Yang, B., Nevatia, R.: An online learned CRF model for multi-target tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar