Skip to main content
SpringerLink
Log in

Particle Flow SMC-PHD Filter for Audio-Visual Multi-speaker Tracking

  • Conference paper
  • First Online:
Latent Variable Analysis and Signal Separation (LVA/ICA 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10169))

Abstract

Sequential Monte Carlo probability hypothesis density (SMC-PHD) filtering has been recently exploited for audio-visual (AV) based tracking of multiple speakers, where audio data are used to inform the particle distribution and propagation in the visual SMC-PHD filter. However, the performance of the AV-SMC-PHD filter can be affected by the mismatch between the proposal and the posterior distribution. In this paper, we present a new method to improve the particle distribution where audio information (i.e. DOA angles derived from microphone array measurements) is used to detect new born particles and visual information (i.e. histograms) is used to modify the particles with particle flow (PF). Using particle flow has the benefit of migrating particles smoothly from the prior to the posterior distribution. We compare the proposed algorithm with the baseline AV-SMC-PHD algorithm using experiments on the AV16.3 dataset with multi-speaker sequences.

Y. Liu—This work was supported by the EPSRC Programme Grant S3A: Future Spatial Audio for an Immersive Listener Experience at Home (EP/L000539/1), the BBC as part of the BBC Audio Research Partnership, the China Scholarship Council (CSC), and the EPSRC grant EP/K014307/1 and the MOD University Defence Research Collaboration in Signal Processing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baser, E., Efe, M.: A novel auxiliary particle PHD filter. In: 15th International Conference on Information Fusion, pp. 165–172. IEEE (2012)

    Google Scholar 

  2. Bernardin, K., Gehrig, T., Stiefelhagen, R.: Multi-level particle filter fusion of features and cues for audio-visual person tracking. In: Stiefelhagen, R., Bowers, R., Fiscus, J. (eds.) CLEAR/RT -2007. LNCS, vol. 4625, pp. 70–81. Springer, Heidelberg (2008). doi:10.1007/978-3-540-68585-2_5

    Chapter  Google Scholar 

  3. Cevher, V., Sankaranarayanan, A.C., McClellan, J.H., Chellappa, R.: Target tracking using a joint acoustic video system. IEEE Trans. Multimedia 9(4), 715–727 (2007)

    Article  Google Scholar 

  4. Cui, P., Sun, L.F., Wang, F., Yang, S.Q.: Contextual mixture tracking. IEEE Trans. Multimedia 11(2), 333–341 (2009)

    Article  Google Scholar 

  5. Daum, F., Huang, J.: Particle flow for nonlinear filters with log-homotopy. In: Proceedings of SPIE, pp. 696918-1–696918-12 (2008)

    Google Scholar 

  6. Daum, F., Huang, J.: Nonlinear filters with log-homotopy. In: Drummond, O.E., Teichgraeber, R.D. (eds.) Optical Engineering + Applications. pp. 669918–669918-15. International Society for Optics and Photonics (2007)

    Google Scholar 

  7. Daum, F., Huang, J.: Nonlinear filters with particle flow. In: SPIE Optical Engineering + Applications, pp. 74450R-1–74450R-9. International Society for Optics and Photonics (2009)

    Google Scholar 

  8. Daum, F., Huang, J.: Particle flow for nonlinear filters. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5920–5923 (2011)

    Google Scholar 

  9. Daum, F., Huang, J.: Renormalization group flow and other ideas inspired by physics for nonlinear filters, Bayesian decisions, and transport. In: SPIE Defense + Security, p. 90910I. International Society for Optics and Photonics (2014)

    Google Scholar 

  10. Daum, F., Huang, J.: Renormalization group flow in k-space for nonlinear filters, Bayesian decisions and transport. In: 18th International Conference on Information Fusion, pp. 1617–1624. IEEE (2015)

    Google Scholar 

  11. Daum, F., Huang, J., Noushin, A.: Exact particle flow for nonlinear filters. In: SPIE Defense, Security, and Sensing, p. 769704 (2010)

    Google Scholar 

  12. Daum, F., Huang, J., Noushin, A.: Small curvature particle flow for nonlinear filters. Signal Processing, Sensor Fusion, and Target Recognition XIX 7697(1), 769704 (2010)

    Article  Google Scholar 

  13. Daum, F., Huang, J., Noushin, A.: Coulomb’s law particle flow for nonlinear filters. In: Drummond, O.E. (ed.) SPIE Optical Engineering + Applications, pp. 1–15. International Society for Optics and Photonics (2011)

    Google Scholar 

  14. Fortmann, T., Bar-Shalom, Y., Scheffe, M.: Sonar tracking of multiple targets using joint probabilistic data association. IEEE J. Oceanic Eng. 8(3), 173–184 (1983)

    Article  Google Scholar 

  15. Gatica-Perez, D., Lathoud, G., Odobez, J.M., McCowan, I.: Audiovisual probabilistic tracking of multiple speakers in meetings. IEEE Trans. Audio Speech Lang. Process. 15(2), 601–616 (2007)

    Article  Google Scholar 

  16. Hafner, J., Sawhney, H.S., Equitz, W., Flickner, M., Niblack, W.: Efficient color histogram indexing for quadratic form distance functions. IEEE Trans. Pattern Anal. Mach. Intell. 17(7), 729–736 (1995)

    Article  Google Scholar 

  17. Khan, M.A., Ulmke, M.: Non-linear and non-Gaussian state estimation using log-homotopy based particle flow filters. In: 2014 Workshop on Sensor Data Fusion: Trends, Solutions, Applications, SDF (2014)

    Google Scholar 

  18. Khan, M.A., Ulmke, M.: Non-linear and non-Gaussian state estimation using log-homotopy based particle flow filters. In: Sensor Data Fusion: Trends, Solutions, Applications (SDF), pp. 1–6. IEEE (2014)

    Google Scholar 

  19. Kidron, E., Schechner, Y.Y., Elad, M.: Cross-modal localization via sparsity. IEEE Trans. Sig. Process. 55(4), 1390–1404 (2007)

    Article  MathSciNet  Google Scholar 

  20. Kilic, V., Barnard, M., Wang, W., Hilton, A., Kittler, J.: Mean-shift and sparse sampling based SMC-PHD filtering for audio informed visual speaker tracking. IEEE Trans. Multimedia 18(12), 2417–2431 (2016)

    Article  Google Scholar 

  21. Kılıç, V., Barnard, M., Wang, W., Kittler, J.: Audio assisted robust visual tracking with adaptive particle filtering. IEEE Trans. Multimedia 17(2), 186–200 (2015)

    Article  Google Scholar 

  22. Kong, A., Liu, J.S., Wong, W.H.: Sequential imputations and Bayesian missing data problems. J. Am. Stat. Assoc. 89(425), 278–288 (1994)

    Article  MATH  Google Scholar 

  23. Lathoud, G., Odobez, J.-M., Gatica-Perez, D.: AV16.3: an audio-visual corpus for speaker localization and tracking. In: Bengio, S., Bourlard, H. (eds.) MLMI 2004. LNCS, vol. 3361, pp. 182–195. Springer, Heidelberg (2005). doi:10.1007/978-3-540-30568-2_16

    Chapter  Google Scholar 

  24. Li, Y., Zhao, L., Coates, M.: Particle flow for particle filtering. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3979–3983. IEEE (2016)

    Google Scholar 

  25. Liu, Q., Rui, Y., Gupta, A., Cadiz, J.J.: Automating camera management for lecture room environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 442–449. ACM (2001)

    Google Scholar 

  26. Maggio, E., Piccardo, E., Regazzoni, C., Cavallaro, A.: Particle PHD filtering for multi-target visual tracking. In: 2007 IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 1, pp. I–1101. IEEE (2007)

    Google Scholar 

  27. Polat, E., Ozden, M.: A nonparametric adaptive tracking algorithm based on multiple feature distributions. IEEE Trans. Multimedia 8(6), 1156–1163 (2006)

    Article  Google Scholar 

  28. Ristic, B., Vo, B.N., Clark, D., Vo, B.T.: A metric for performance evaluation of multi-target tracking algorithms. IEEE Trans. Sig. Process. 59(7), 3452–3457 (2011)

    Article  MathSciNet  Google Scholar 

  29. Talantzis, F., Constantinides, A.G., Polymenakos, L.C.: Estimation of direction of arrival using information theory. IEEE Sig. Process. Lett. 12(8), 561–564 (2005)

    Article  Google Scholar 

  30. Zhao, L., Wang, J., Li, Y., Coates, M.J.: Gaussian particle flow implementation of PHD filter. In: SPIE Defense + Security, p. 98420D (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, University of Surrey, Guildford, GU2 7XH, UK

    Yang Liu, Wenwu Wang & Adrian Hilton

  2. School of Electrical and Electronic Engineering, Newcastle University, Newcastle upon Tyne, NE1 7RU, UK

    Jonathon Chambers

  3. Department of Electrical and Electronics Engineering, Izmir Katip Celebi University, 35620, Cigli-Izmir, Turkey

    Volkan Kilic

Authors
  1. Yang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenwu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathon Chambers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Volkan Kilic
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adrian Hilton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Liu .

Editor information

Editors and Affiliations

  1. Institute of Information Theory and Automation, Prague, Czech Republic

    Petr Tichavský

  2. Sharif University of Technology, Tehran, Iran

    Massoud Babaie-Zadeh

  3. Grenoble-Alpes University, Grenoble, France

    Olivier J.J. Michel

  4. Toulon University, Toulon, France

    Nadège Thirion-Moreau

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Liu, Y., Wang, W., Chambers, J., Kilic, V., Hilton, A. (2017). Particle Flow SMC-PHD Filter for Audio-Visual Multi-speaker Tracking. In: Tichavský, P., Babaie-Zadeh, M., Michel, O., Thirion-Moreau, N. (eds) Latent Variable Analysis and Signal Separation. LVA/ICA 2017. Lecture Notes in Computer Science(), vol 10169. Springer, Cham. https://doi.org/10.1007/978-3-319-53547-0_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-53547-0_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-53546-3

  • Online ISBN: 978-3-319-53547-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation