Skip to main content
SpringerLink
Log in

Expressive Performance Rendering with Probabilistic Models

  • Chapter
  • First Online:
Guide to Computing for Expressive Music Performance

Abstract

We present YQX, a probabilistic performance rendering system based on Bayesian network theory. It models dependencies between score and performance and predicts performance characteristics using information extracted from the score. We discuss the basic system that won the Rendering Contest RENCON 2008 and then present several extensions, two of which aim to incorporate the current performance context into the prediction, resulting in more stable and consistent predictions. Furthermore, we describe the first steps towards a multilevel prediction model: Segmentation of the work, decomposition of tempo trajectories, and combination of different prediction models form the basis for a hierarchical prediction system. The algorithms are evaluated and compared using two very large data sets of human piano performances: 13 complete Mozart sonatas and the complete works for solo piano by Chopin.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    see http://www.visiv.co.uk

  2. 2.

    Some of the posthumously published works were played as encores but have not yet been included in the dataset.

  3. 3.

    The unit of the duration does not matter in this case, as it cancels out with the unit of the complete duration of the performance.

  4. 4.

    Computer-controlled pianos measure loudness by measuring the velocity at which a hammer strikes a string.

  5. 5.

    In the case of two equally long durations, we only discriminate between long and neutral. Hence, there are no situations labelled lsl, sls, ssl, etc., only lnl, nln, nnl, etc., which reduces the number of combinations used.

  6. 6.

    The category Pieces comprises Rondos (op. 1, op. 5, op. 16), Variations op. 12, Bolero op. 19, Impromptus (op. 36, op. 51), Tarantelle op. 43, Allegro de Concert op. 46, Fantaisie op. 49, Berceuse op. 57, and Barcarolle op. 61.

  7. 7.

    The performed piece “My Nocturne,” a piano piece in a Chopin-like style, was composed by Prof. Tadahiro Murao specifically for the competition.

  8. 8.

    The construct \((\overrightarrow{x},{y}_{t-1})\) is a concatenation of the vector \(\overrightarrow{x}\) and the value y t − 1 leading to a new vector of dimension \(dim(\overrightarrow{x}) + 1\).

  9. 9.

    We use α(y t ) and p(y t ) as abbreviations of α(Y t  = y t ) and p(Y t  = y t ), respectively.

References

  1. Arcos J, de Mántaras R (2001) An interactive CBR approach for generating expressive music. J Appl Intell 27(1):115–129

    Article  Google Scholar 

  2. Dorard L, Hardoon D, Shawe-Taylor J (2007) Can style be learned? A machine learning approach towards “performing” as famous pianists. In: Proceedings of music, brain & cognition workshop – the neural information processing systems 2007 (NIPS 2007), Whistler

    Google Scholar 

  3. Flossmann S, Grachten M, Widmer G (2008) Experimentally investigating the use of score features for computational models of expressive timing. In: Proceedings of the 10th international conference on music perception and cognition 2008 (ICMPC ’08), Sapporo

    Google Scholar 

  4. Flossmann S, Grachten M, Widmer G (2009) Expressive performance rendering: introducing performance context. In: Proceedings of the 6th sound and music computing conference 2009 (SMC ’09), Porto, pp 155–160

    Google Scholar 

  5. Flossmann S, Goebl W, Grachten M, Niedermayer B, Widmer G (2010) The magaloff project: an interim report. J New Music Res 39(4):363–377

    Article  Google Scholar 

  6. Friberg A, Sundberg J (1999) Does music performance allude to locomotion? A model of final ritardandi derived from measurements of stopping runners. J Acoust Soc Am 105(3):1469–1484

    Article  Google Scholar 

  7. Friberg A, Bresin R, Sundberg J (2006) Overview of the KTH rule system for musical performance. Adv Cognit Psychol 2(2–3):145–161

    Article  Google Scholar 

  8. Grachten M (2006) Expressivity-aware tempo transformations of music performances using case based reasoning. Ph.D. thesis, Pompeu Fabra University, Barcelona

    Google Scholar 

  9. Grindlay GC (2005) Modeling expressive musical performance with Hidden Markov models. Master’s thesis, University of California, Santa Cruz

    Google Scholar 

  10. Grindlay G, Helmbold D (2006) Modeling, analyzing, and synthesizing expressive piano performance with graphical models. Mach Learn 65(2–3):361–387

    Article  Google Scholar 

  11. Hashida M (2008) RENCON – Performance Rendering Contest for computer systems. http://www.renconmusic.org/. Accessed Sep 2008

  12. Juang BH, Rabiner LR (1991) Hidden Markov Models for speech recognition. Technometrics 33(3):251–272

    Article  MathSciNet  MATH  Google Scholar 

  13. Kim TH, Fukayama S, Nishimoto T, Sagayama S (2010) Performance rendering for polyphonic piano music with a combination of probabilistic models for melody and harmony. In: Proceedings of the 7th sound and music computing conference 2010 (SMC ’10), Barcelona

    Google Scholar 

  14. Krumhansl CL, Kessler EJ (1982) Tracing the dynamic changes in perceived tonal organization in a spatioal representation of musical keys. Psychol Rev 89:334–368

    Article  Google Scholar 

  15. Lerdahl F, Jackendoff R (1983) A generative theory of tonal music. The MIT Press, Cambridge

    Google Scholar 

  16. Mazzola G (2002) The topos of music – geometric logic of concepts, theory, and performance. Birkhäuser Verlag, Basel

    MATH  Google Scholar 

  17. Mazzola G (2006) Rubato software. http://www.rubato.org

  18. Meyer L (1956) Emotion and meaning in music. University of Chicago Press, Chicago

    Google Scholar 

  19. Milmeister G (2006) The Rubato composer music software: component-based implementation of a functorial concept architecture. Ph.D. thesis, Universität Zürich, Zürich

    Google Scholar 

  20. Moog RA, Rhea TL (1990) Evolution of the keyboard interface: the Boesendorfer 290 SE recording piano and the moog multiply-touch-sensitive keyboards. Comput Music J 14(2):52–60

    Article  Google Scholar 

  21. Murphy K (2002) Dynamic Bayesian networks: presentation, inference and learning. Ph.D. thesis, University of California, Berkeley

    Google Scholar 

  22. Narmour E (1990) The analysis and cognition of basic melodic structures: the implication–realization model. University of Chicago Press, Chicago

    Google Scholar 

  23. Narmour E (1992) The analysis and cognition of melodic complexity: the implication–realization model. University of Chicago Press, Chicago

    Google Scholar 

  24. Perez A, Maestre E, Ramirez R, Kersten S (2008) Expressive irish fiddle performance model informed with bowing. In: Proceedings of the international computer music conference 2008 (ICMC ’08), Belfast

    Google Scholar 

  25. Ramirez R, Hazan A, Gòmez E, Maestre E (2004) Understanding expressive transformations in saxophone Jazz performances using inductive machine learning in saxophone jazz performances using inductive machine learning. In: Proceedings of the sound and music computing international conference 2004 (SMC ’04), Paris

    Google Scholar 

  26. Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press, Cambridge. www.GaussianProcess.org/gpml

  27. Recordare (2003) MusicXML definition. http://www.recordare.com/xml.html

  28. Sundberg J, Askenfelt A, Frydén L (1983) Musical performance: a synthesis-by-rule approach. Comput Music J 7:37–43

    Article  Google Scholar 

  29. Suzuki T (2003) The second phase development of case based performance rendering system “Kagurame”. In: Working notes of the IJCAI-03 rencon workshop, Acapulco, pp 23–31

    Google Scholar 

  30. Temperley D (2007) Music and probability. MIT Press, Cambridge

    MATH  Google Scholar 

  31. Teramura K, Okuma H, et al (2008) Gaussian process regression for rendering music performance. In: Proceedings of the 10th international conference on music perception and cognition 2008 (ICMPC ’08), Sapporo

    Google Scholar 

  32. Tobudic A, Widmer G (2006) Relational IBL in classical music. Mach Learn 64(1–3):5–24

    Article  Google Scholar 

  33. Todd NPM (1992) The dynamics of dynamics: a model of musical expression. J Acoust Soc Am 91:3450–3550

    Google Scholar 

  34. Widmer G (2002) Machine discoveries: a few simple, robust local expression principles. J New Music Res 31(1):37–50

    Article  MathSciNet  Google Scholar 

  35. Widmer G (2003) Discovering simple rules in complex data: a meta-learning algorithm and some surprising musical discoveries. Artif Intell 146(2):129–148

    Article  MathSciNet  MATH  Google Scholar 

  36. Widmer G, Goebl W (2004) Computational models of expressive music performance: the state of the art. J New Music Res 33(3):203–216

    Article  Google Scholar 

  37. Widmer G, Tobudic A (2003) Playing Mozart by analogy: learning multi-level timing and dynamics strategies. J New Music Res 32(3):259–268

    Article  Google Scholar 

  38. Widmer G, Flossmann S, Grachten M (2009) YQX plays Chopin. AI Mag 30(3):35–48

    Google Scholar 

Download references

Acknowledgements

We express our gratitude to Mme Irène Magaloff for her generous permission to use the unique resource that is the Magaloff Corpus for our research. This work is funded by the Austrian National Research Fund FWF via grants TRP 109-N23 and Z159 (“Wittgenstein Award”). The Austrian Research Institute for Artificial Intelligence acknowledges financial support from the Austrian Federal Ministries BMWF and BMVIT.

Author information

Authors and Affiliations

  1. Department of Computational Perception, Johannes Kepler Universitat, Altenbergerstr 69, 4040, Linz, AT, Austria

    Sebastian Flossmann

  2. Post Doctoral Researcher at the Department of Computational Perception, Johannes Kepler University, Linz, Austria

    Maarten Grachten

  3. Department of Computational Perception, Johannes Kepler University (JKU), Linz, Austria

    Gerhard Widmer

  4. The Austrian Research Institute for Artificial Intelligence (OFAI), Freyung 6/6, A-1010, Vienna, Austria

    Gerhard Widmer

Authors
  1. Sebastian Flossmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maarten Grachten
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhard Widmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Flossmann .

Editor information

Editors and Affiliations

  1. ICCMR, University of Plymouth, Smeaton Building 206, Plymouth, PL4 8AA, Devon, United Kingdom

    Alexis Kirke

  2. Computing, Communication & Electronics, University of Plymouth, Plymouth, PL4 8AA, United Kingdom

    Eduardo R. Miranda

Questions

Questions

  1. 1.

    Aside from the central problem of mapping the score to the performance, what are the other main challenges in the process of generating a computer performance?

  2. 2.

    Why is evaluating automatically by measuring the similarity between rendered and real performances of a piece problematic?

  3. 3.

    What are the two methods on which score models (i.e., representations of the music and its structure) may be based?

  4. 4.

    What three different categories can be distinguished regarding the learning and prediction models used in CSEMPs?

  5. 5.

    In probabilistic approaches, how is the performance model regarded?

  6. 6.

    For data used in developing an expressive performance statistical model, the data must provide information on what two elements?

  7. 7.

    What musicological model was selected for the YQX system?

  8. 8.

    In what three dimensions are performances characterized in YQX?

  9. 9.

    What is the difference in implementation between the local and the global maximization approaches in YQX?

  10. 10.

    What is the difference in results between the local and the global maximization approaches in YQX?

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Flossmann, S., Grachten, M., Widmer, G. (2013). Expressive Performance Rendering with Probabilistic Models. In: Kirke, A., Miranda, E. (eds) Guide to Computing for Expressive Music Performance. Springer, London. https://doi.org/10.1007/978-1-4471-4123-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4123-5_3

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4122-8

  • Online ISBN: 978-1-4471-4123-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation