Who is Really Talking? A Visual-Based Speaker Diarization Strategy

  • Pedro A. Marín-ReyesEmail author
  • Javier Lorenzo-Navarro
  • Modesto Castrillón-Santana
  • Elena Sánchez-Nielsen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10672)


The speaker activity at the Canary Islands Parliament is recorded, and later manually annotated. This task can be modelled as a diarization problem, that is a way to automatically annotated who and when is speaking. In this paper, we propose the use of the visual cue to solve the diarization task. To perform this approach, it is mandatory to detect individuals, determine the one speaking, and extract features for matching. In order to test the performance of our proposal, we evaluate four different strategies based on the visual shot features.


Visual diarization strategies Local descriptors Histogram distances F-reid 



This work is partially supported by Government of Spain through TIN2015-64395-R and by the Ministerio de Economía y Competitividad, Government of Spain and FEDER funds of the European Union through TIN2016-78919-R (MINECO/FEDER).


  1. 1.
    Miró, X.A., Bozonnet, S., Evans, N.W.D., Fredouille, C., Friedland, G., Vinyals, O.: Speaker diarization: a review of recent research. IEEE Trans. Audio Speech Lang. Process. 20(2), 356–370 (2012)CrossRefGoogle Scholar
  2. 2.
    Barra-Chicote, R., Pardo, J.M., Ferreiros, J., Montero, J.M.: Speaker diarization based on intensity channel contribution. IEEE Trans. Audio Speech Lang. Process. 19(4), 754–761 (2011)CrossRefGoogle Scholar
  3. 3.
    Tranter, S.E., Reynolds, D.A.: An overview of automatic speaker diarization systems. IEEE Trans. Audio Speech Lang. Process. 14(5), 1557–1565 (2006)CrossRefGoogle Scholar
  4. 4.
    Ning, H., Liu, M., Tang, H., Huang, T.: A spectral clustering approach to speaker diarization. In: Proceedings of ICSLP (2006)Google Scholar
  5. 5.
    Lupu, E., Apatean, A., Arsinte, R.: Speaker diarization experiments for Romanian parliamentary speech. In: 2015 International Symposium on Signals, Circuits and Systems (ISSCS), pp. 1–4, July 2015Google Scholar
  6. 6.
    Meignier, S., Merlin, T.: Lium spkdiarization: an open source toolkit for diarization. In: CMU SPUD Workshop, Dallas (Texas, USA), mars 2010Google Scholar
  7. 7.
    Campr, P., Kunešová, M., Vaněk, J., Čech, J., Psutka, J.: Audio-video speaker diarization for unsupervised speaker and face model creation. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2014. LNCS (LNAI), vol. 8655, pp. 465–472. Springer, Cham (2014). Google Scholar
  8. 8.
    Everingham, M., Sivic, J., Zisserman, A.: Taking the bite out of automated naming of characters in TV video. Image Vis. Comput. 27(5), 545–559 (2009)CrossRefGoogle Scholar
  9. 9.
    Sang, J., Xu, C.: Robust face-name graph matching for movie character identification. IEEE Trans. Multimed. 14(3), 586–596 (2012)CrossRefGoogle Scholar
  10. 10.
    Marín-Reyes, P.A., Lorenzo-Navarro, J., Castrillón-Santana, M., Sánchez-Nielsen, E.: Shot classification and keyframe detection for vision based speakers diarization in parliamentary debates. In: Luaces, O., Gámez, J.A., Barrenechea, E., Troncoso, A., Galar, M., Quintián, H., Corchado, E. (eds.) CAEPIA 2016. LNCS (LNAI), vol. 9868, pp. 48–57. Springer, Cham (2016). CrossRefGoogle Scholar
  11. 11.
    Castrillón-Santana, M., Lorenzo-Navarro, J., Ramón-Balmaseda, E.: Multi-scale score level fusion of local descriptors for gender classification in the wild. Multimed. Tools Appl. (2016, in press)Google Scholar
  12. 12.
    Cong, D.N.T., Khoudour, L., Achard, C., Meurie, C., Lezoray, O.: People re-identification by spectral classification of silhouettes. Sig. Process. 90(8), 2362–2374 (2010). Special Section on Processing and Analysis of High-Dimensional Masses of Image and Signal DataCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Pedro A. Marín-Reyes
    • 1
    Email author
  • Javier Lorenzo-Navarro
    • 1
  • Modesto Castrillón-Santana
    • 1
  • Elena Sánchez-Nielsen
    • 2
  1. 1.Instituto Universitario SIANIUniversidad de las Palmas de Gran CanariaLas PalmasSpain
  2. 2.Departamento de Ingeniería Informática y de SistemasUniversidad de la LagunaSanta Cruz de TenerifeSpain

Personalised recommendations