Skip to main content
SpringerLink
Log in

Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment

  • Conference paper
  • First Online:
Research and Advanced Technology for Digital Libraries (TPDL 2015)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9316))

Included in the following conference series:

  • 1255 Accesses

Abstract

In this paper we report on an evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion are sufficiently high. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and also assess the effect of the transformation from manual to automatic from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This data is sometimes also referred to as ‘context data’ but as for example newspaper data can also be regarded as ‘context’ we prefer the term ‘collateral data’.

  2. 2.

    http://datahub.io/dataset/gemeenschappelijke-thesaurus-audiovisuele-archieven.

  3. 3.

    http://www.openarchives.org/pmh/.

  4. 4.

    http://xtas.net/. Specifically, the FROG module was used using default settings.

  5. 5.

    http://www.cltl.nl/. Here the OpenNER web service was used in combination with the CLTL POS tagger.

  6. 6.

    http://www.elastic.co/products/elasticsearch.

  7. 7.

    For this non-optimized variant, recall was 21 %.

  8. 8.

    This prioritization is done by archivists independently of this work. It is in use throughout the archive and mostly determined by potential (re)use by archive clients.

References

  1. Gazendam, L., Wartena, C., Malaisé, V., Schreiber, G., de Jong, A., Brugman, H.: Automatic annotation suggestions for audiovisual archives: evaluation aspects. Interdisc. Sci. Rev. 34(2–3), 172–188 (2009)

    Article  Google Scholar 

  2. Ordelman, R., Heeren, W., Huijbregts, M., de Jong, F., Hiemstra, D.: Towards affordable disclosure of spoken heritage archives. J. Digital Inf. 10(6), 17 (2009)

    Google Scholar 

  3. Declerck, T., Kuper, J., Saggion, H., Samiotou, A., Wittenburg, J.P., Contreras, J.: Contribution of NLP to the content indexing of multimedia documents. In: Enser, P.G.B., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A.F., Smeulders, A.W.M. (eds.) CIVR 2004. LNCS, vol. 3115, pp. 610–618. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  4. Iivonen, M.: Consistency in the selection of search concepts and search terms. Inf. Process. Manage. 31(2), 173–190 (1995)

    Article  Google Scholar 

  5. Bizer, C., Heath, T., Berners-Lee, T.: Linked data - the story so far. Int. J. Semantic Web Inf. Syst. 5(3), 1–22 (2009)

    Article  Google Scholar 

  6. Likert, R.: A technique for the measurement of attitudes. Arch. Psychol. 22, 1–55 (1932)

    Google Scholar 

  7. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)

    Google Scholar 

  8. Bontcheva, K., Tablan, V., Maynard, D., Cunningham, H.: Evolving gate to meet new challenges in language engineering. Nat. Lang. Eng. 10, 349–373 (9 2004)

    Google Scholar 

  9. Tommasi, T., Aly, R., McGuinness, K., Chatfield, K., Arandjelovic, R., Parkhi, O., Ordelman, R., Zisserman, A., Tuytelaars, T.: Beyond metadata: searching your archive based on its audio-visual content. In: IBC 2014, Amsterdam, The Netherlands (2014)

    Google Scholar 

Download references

Acknowledgments

This research was funded by the MediaManagement Programme at the Netherlands Institute for Sound and Vision, the Dutch National Research Programme COMMIT/ and supported by NWO CATCH program (http://www.nwo.nl/catch) and the Dutch Ministry of Culture.

Author information

Authors and Affiliations

  1. Netherlands Institute for Sound and Vision, Hilversum, The Netherlands

    Victor de Boer, Roeland J. F. Ordelman & Josefien Schuurman

  2. The Network Institute, VU University Amsterdam, Amsterdam, The Netherlands

    Victor de Boer

  3. University of Twente, Enschede, The Netherlands

    Roeland J. F. Ordelman

Authors
  1. Victor de Boer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roeland J. F. Ordelman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Josefien Schuurman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Victor de Boer .

Editor information

Editors and Affiliations

  1. Ionian University, Corfu, Greece

    Sarantos Kapidakis

  2. Poznań Supercomputing and Networking Center, Poznań, Poland

    Cezary Mazurek

  3. Networking Center, Poznań Supercomputing and, Poznań, Poland

    Marcin Werla

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

de Boer, V., Ordelman, R.J.F., Schuurman, J. (2015). Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment. In: Kapidakis, S., Mazurek, C., Werla, M. (eds) Research and Advanced Technology for Digital Libraries. TPDL 2015. Lecture Notes in Computer Science(), vol 9316. Springer, Cham. https://doi.org/10.1007/978-3-319-24592-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24592-8_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24591-1

  • Online ISBN: 978-3-319-24592-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation