Domain Adaptation for Handwritten Symbol Recognition: A Case of Study in Old Music Manuscripts

  • Tudor N. MateiuEmail author
  • Antonio-Javier Gallego
  • Jorge Calvo-Zaragoza
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11868)


The existence of a large amount of untranscripted music manuscripts has caused initiatives that use Machine Learning (ML) for Optical Music Recognition, in order to efficiently transcribe the music sources into a machine-readable format. Although most music manuscript are similar in nature, they inevitably vary from one another. This fact can negatively influence the complexity of the classification task because most ML models fail to transfer their knowledge from one domain to another, thereby requiring learning from scratch on new domains after manually labeling new data. This work studies the ability of a Domain Adversarial Neural Network for domain adaptation in the context of classifying handwritten music symbols. The main idea is to exploit the knowledge of a specific manuscript to classify symbols from different (unlabeled) manuscripts. The reported results are promising, obtaining a substantial improvement over a conventional Convolutional Neural Network approach, which can be used as a basis for future research.


Optical Music Recognition Domain Adaptation Convolutional Neural Network Handwritten music symbols 



This work is supported by the Spanish Ministry HISPAMUS project TIN2017-86576-R, partially funded by the EU.


  1. 1.
    Bainbridge, D., Bell, T.: The challenge of optical music recognition. Comput. Humanit. 35(2), 95–121 (2001)CrossRefGoogle Scholar
  2. 2.
    Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Vaughan, J.W.: A theory of learning from different domains. Mach. Learn. 79(1–2), 151–175 (2010)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Calvo-Zaragoza, J., Gallego, A.J., Pertusa, A.: Recognition of handwritten music symbols with convolutional neural codes. In: 14th International Conference on Document Analysis and Recognition, Kyoto, Japan, pp. 691–696 (2017)Google Scholar
  4. 4.
    Castellanos, F.J., Calvo-Zaragoza, J., Vigliensoni, G., Fujinaga, I.: Document analysis of music score images with selectional auto-encoders. In: 19th International Society for Music Information Retrieval Conference, Paris, France, pp. 256–263 (2018)Google Scholar
  5. 5.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001)zbMATHGoogle Scholar
  6. 6.
    Fujinaga, I., Hankinson, A., Cumming, J.E.: Introduction to SIMSSA (single interface for music score searching and analysis). In: 1st International Workshop on Digital Libraries for Musicology, pp. 1–3. ACM (2014)Google Scholar
  7. 7.
    Ganin, Y., Lempitsky, V.S.: Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 1180–1189 (2015)Google Scholar
  8. 8.
    Hajič Jr., J., Dorfer, M., Widmer, G., Pecina, P.: Towards full-pipeline handwritten OMR with musical symbol detection by U-Nets. In: 19th International Society for Music Information Retrieval Conference, Paris, France, pp. 225–232 (2018)Google Scholar
  9. 9.
    Pacha, A., Calvo-Zaragoza, J.: Optical music recognition in mensural notation with region-based convolutional neural networks. In: 19th International Society for Music Information Retrieval Conference, Paris, France, pp. 240–247 (2018)Google Scholar
  10. 10.
    Rebelo, A., Fujinaga, I., Paszkiewicz, F., Marcal, A.R., Guedes, C., Cardoso, J.D.S.: Optical music recognition: state-of-the-art and open issues. Int. J. Multimed. Inf. Retr. 1(3), 173–190 (2012)CrossRefGoogle Scholar
  11. 11.
    Rizo, D., Calvo-Zaragoza, J., Iñesta, J.M.: MuRET: a music recognition, encoding, and transcription tool. In: 5th International Conference on Digital Libraries for Musicology, pp. 52–56. ACM, Paris (2018)Google Scholar
  12. 12.
    Zeiler, M.D.: ADADELTA: an adaptive learning rate method. The Computing Research Repository (CoRR) abs/1212.5701 (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Tudor N. Mateiu
    • 1
    Email author
  • Antonio-Javier Gallego
    • 1
  • Jorge Calvo-Zaragoza
    • 1
  1. 1.Department of Software and Computing SystemsUniversity of AlicanteAlicanteSpain

Personalised recommendations