Tracking in Urban Environments Using Sensor Networks Based on Audio-Video Fusion

  • Manish Kushwaha
  • Songhwai Oh
  • Isaac Amundson
  • Xenofon Koutsoukos
  • Akos Ledeczi


Heterogeneous sensor networks (HSNs) are gaining popularity in diverse fields, such as military surveillance, equipment monitoring, and target tracking yarvis:2005:infocom. They are natural steps in the evolution of wireless sensor networks wireless sensor network (WSNs) driven by several factors. Increasingly, WSNs will need to support multiple, although not necessarily concurrent, applications. Different applications may require different resources. Some applications can make use of nodes with different capabilities. As the technology matures, new types of nodes will become available and existing deployments will be refreshed. Diverse nodes will need to coexist and support old and new applications.


Sensor Network Wireless Sensor Network Target Tracking Time Synchronization Sensor Fusion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. M. Ali, K. Yao, T. C. Collier, C. E. Taylor, D. T. Blumstein, and L. Girod. An empirical study of collaborative acoustic source localization. In IPSN ’07: Proceedings of the 6th international conference on Information processing in sensor networks, pages 41–50, 2007.Google Scholar
  2. [2]
    I. Amundson, B. Kusy, P. Volgyesi, X. Koutsoukos, and A. Ledeczi. Time synchronization in heterogeneous sensor networks. In International Conference on Distributed Computing in Sensor Networks (DCOSS 2008), 2008.Google Scholar
  3. [3]
    M. J. Beal, N. Jojic, and H. Attias. A graphical model for audiovisual object tracking. volume 25, pages 828–836, 2003.Google Scholar
  4. [4]
    P. Bergamo, S. Asgari, H. Wang, D. Maniezzo, L. Yip, R. E. Hudson, K. Yao, and D. Estrin. Collaborative sensor networking towards real-time acoustical beamforming in free-space and limited reverberence. In IEEE Transactions On Mobile Computing, volume 3, pages 211–224, 2004.CrossRefGoogle Scholar
  5. [5]
    S. T. Birchfield. A unifying framework for acoustic localization. In Proceedings of the 12th European Signal Processing Conference (EUSIPCO), Vienna, Austria, September 2004.Google Scholar
  6. [6]
    N. Checka, K. Wilson, V. Rangarajan, and T. Darrell. A probabilistic framework for multi-modal multi-person tracking. In IEEE Workshop on Multi-Object Tracking, 2003.Google Scholar
  7. [7]
    J. C. Chen, K. Yao, and R. E. Hudson. Acoustic source localization and beamforming: theory and practice. In EURASIP Journal on Applied Signal Processing, pages 359–370, April 2003.Google Scholar
  8. [8]
    T. Darrell, D. Demirdjian, N. Checka, and P. Felzenszwalb. Plan-view trajectory estimation with dense stereo background models. In Proceedings. Eighth IEEE International Conference on Computer Vision (ICCV 2001), 2001.Google Scholar
  9. [9]
    M. Ding, A. Terzis, I.-J. Wang, and D. Lucarelli. Multi-modal calibration of surveillance sensor networks. In Military Communications Conference, MILCOM 2006, 2006.Google Scholar
  10. [10]
    E. Duarte-Melo and M. Liu. Analysis of energy consumption and lifetime of heterogeneous wireless sensor networks. In IEEE Globecom, 2002.Google Scholar
  11. [11]
    A. Elgammal, D. Harwood, and L. Davis. Non-parametric model for background subtraction. In IEEE ICCV’99 Frame-Rate workshop, 1999.Google Scholar
  12. [12]
    J. Elson, L. Girod, and D. Estrin. Fine-grained network time synchronization using reference broadcasts. In Operating Systems Design and Implementation (OSDI), 2002.Google Scholar
  13. [13]
    N. Friedman and S. Russell. Image segmentation in video sequences: A probabilistic approach. In Conference on Uncertainty in Artificial Intelligence, 1997.Google Scholar
  14. [14]
    S. Ganeriwal, R. Kumar, and M. B. Srivastava. Timing-sync protocol for sensor networks. In ACM SenSys, 2003.Google Scholar
  15. [15]
    L. Girod, V. Bychkovsky, J. Elson, and D. Estrin. Locating tiny sensors in time and space: A case study. In ICCD, 2002.Google Scholar
  16. [16]
    L. Girod, M. Lukac, V. Trifa, and D. Estrin. The design and implementation of a self-calibrating distributed acoustic sensing platform. In SenSys ’06, 2006.Google Scholar
  17. [17]
    D. Hall and J. Llinas. An introduction to multisensor data fusion. In Proceedings of the IEEE, volume 85, pages 6–23, 1997.CrossRefGoogle Scholar
  18. [18]
    H. Y. Hau and R. L. Kashyap. On the robustness of Dempster’s rule of combination. In IEEE International Workshop on Tools for Artificial Intelligence, 1989.Google Scholar
  19. [19]
    P. KaewTraKulPong and R. B. Jeremy. An improved adaptive background mixture model for realtime tracking with shadow detection. In Workshop on Advanced Video Based Surveillance Systems (AVBS), 2001.Google Scholar
  20. [20]
    Z. Khan, T. Balch, and F. Dellaert. MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(11):1805–1918, Nov. 2005.CrossRefGoogle Scholar
  21. [21]
    D. Koller, J. Weber, T. Huang, J. Malik, G. Ogasawara, B. Rao, and S. Russell. Towards robust automatic traffic scene analysis in realtime. In IEEE Conference on Decision and Control, 1994.Google Scholar
  22. [22]
    R. Kumar, V. Tsiatsis, and M. B. Srivastava. Computation hierarchy for in-network processing. 2003.Google Scholar
  23. [23]
    B. Kusy, P. Dutta, P. Levis, M. Maroti, A. Ledeczi, and D. Culler. Elapsed time on arrival: A simple and versatile primitive for time synchronization services. International Journal of Ad hoc and Ubiquitous Computing, 2(1), January 2006.Google Scholar
  24. [24]
    A. Ledeczi, A. Nadas, P. Volgyesi, G. Balogh, B. Kusy, J. Sallai, G. Pap, S. Dora, K. Molnar, M. Maroti, and G. Simon. Countersniper system for urban warfare. ACM Trans. Sensor Networks, 1(2), 2005.Google Scholar
  25. [25]
    P. Levis, S. Madden, D. Gay, J. Polastre, R. Szewczyk, A. Woo, E. Brewer, and D. Culler. The emergence of networking abstractions and techniques in TinyOS. In NSDI, 2004.Google Scholar
  26. [26]
    J. Liu, J. Reich, and F. Zhao. Collaborative in-network processing for target tracking. In EURASIP, Journal on Applied Signal Processing, 2002.Google Scholar
  27. [27]
    M. Maroti, B. Kusy, G. Simon, and A. Ledeczi. The flooding time synchronization protocol. In ACM SenSys, 2004.Google Scholar
  28. [28]
    M. Meingast, M. Kushwaha, S. Oh, X. Koutsoukos, and S. S. Akos Ledeczi. Heterogeneous camera network localization using data fusion. In In ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC-08), 2008.Google Scholar
  29. [29]
    S. Oh, S. Russell, and S. Sastry. Markov chain Monte Carlo data association for general multiple-target tracking. In IEEE Transactions on Automatic Control, 54(3):481–497, Mar. 2009.CrossRefMathSciNetGoogle Scholar
  30. [30]
    K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe. A boosted particle filter: Multitarget detection and tracking. In European Conference on Computer Vision, 2004.Google Scholar
  31. [31]
    J. Piater and J. Crowley. Multi-modal tracking of interacting targets using Gaussian approximations. In IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, 2001.Google Scholar
  32. [32]
    A. Poore. Multidimensional assignment and multitarget tracking. In I. J. Cox, P. Hansen, and B. Julesz, editors, Partitioning Data Sets, pages 169–196. American Mathematical Society, 1995.Google Scholar
  33. [33]
    D. Reid. An algorithm for tracking multiple targets. IEEE Trans. Automatic Control, 24(6):843–854, December 1979.CrossRefGoogle Scholar
  34. [34]
    S. Rhee, D. Seetharam, and S. Liu. Techniques for minimizing power consumption in low data-rate wireless sensor networks. 2004.Google Scholar
  35. [35]
    J. Sallai, B. Kusy, A. Ledeczi, and P. Dutta. On the scalability of routing integrated time synchronization. In Workshop on Wireless Sensor Networks (EWSN), 2006.Google Scholar
  36. [36]
    C. Stauffer and W. Grimson. Learning patterns of activity using real-time tracking. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000.Google Scholar
  37. [37]
    N. Strobel, S. Spors, and R. Rabenstein. Joint audio video object localization and tracking. In IEEE Signal Processing Magazine, 2001.Google Scholar
  38. [38]
    K. Toyama, J. Krumm, B. Brumitt, and B. Meyers. Wallflower: Principles and practice of background maintenance. In IEEE International Conference on Computer Vision, 1999.Google Scholar
  39. [39]
    J. Valin, F. Michaud, and J. Rouat. Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering. In Robot. Auton. Syst., volume 55, pages 216–228, March 2007.CrossRefGoogle Scholar
  40. [40]
    P. Volgyesi, G. Balogh, A. Nadas, C. Nash, and A. Ledeczi. Shooter localization and weapon classification with soldier-wearable networked sensors. In Mobisys, 2007.Google Scholar
  41. [41]
    M. Yarvis, N. Kushalnagar, H. Singh, A. Rangarajan, Y. Liu, and S. Singh. Exploiting heterogeneity in sensor networks. In IEEE INFOCOM, 2005.Google Scholar
  42. [42]
    B. H. Yoshimi and G. S. Pingali. A multimodal speaker detection and tracking system for teleconferencing. In ACM Multimedia ’02, 2002.Google Scholar
  43. [43]
    T. Zhao and R. Nevatia. Tracking multiple humans in crowded environment. In IEEE Conference on Computer Vision and Pattern Recognition, 2004.Google Scholar
  44. [44]
    D. Zotkin, R. Duraiswami, H. Nanda, and L. S. Davis. Multimodal tracking for smart videoconferencing. In In Proc. 2nd Int. Conference on Multimedia and Expo (ICME’01), 2001.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Institute for Software Integrated SystemsVanderbilt UniversityNashvilleUSA
  2. 2.Electrical Engineering and Computer ScienceUniversity of California at MercedMercedUSA

Personalised recommendations