Melody Completion Based on Convolutional Neural Networks and Generative Adversarial Learning

  • Kosuke Nakamura
  • Takashi Nose
  • Yuya Chiba
  • Akinori ItoEmail author
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 110)


In this paper, we deal with melody completion, a technique which smoothly completes melodies that are partially masked. Melody completion can be used to help people compose or arrange pieces of music in several ways, such as editing existing melodies or connecting two other melodies. In recent years, various methods have been proposed for realizing high-quality completion via neural networks. Therefore, in this research, we examine a method of melody completion based on an image completion network. We represent melodies of a certain length as images and train a completion network to complete those images. The completion network consists of convolution layers and is trained in the framework of generative adversarial networks. We also consider chord progression from musical pieces as conditions.


Melody completion Automatic music composition Convolutional neural networks Generative adversarial networks 


  1. 1.
    Fukayama, S., Nakatsuma, K., Sako, S., Yonebayashi, Y., Kim, T.H., Qin, S.W., Nakano, T., Nishimoto, T., Sagayama, S.: Orpheus: automatic composition system considering prosody of Japanese lyrics. In: Natkin, S., Dupire, J. (eds.) Entertainment Computing - ICEC 2009, pp. 309–310. Springer, Heidelberg (2009)Google Scholar
  2. 2.
    Goodfellow, I.: NIPS 2016 tutorial: generative adversarial networks, January 2017Google Scholar
  3. 3.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc. (2014)Google Scholar
  4. 4.
    Huang, C.A., Cooijmans, T., Roberts, A., Courville, A.C., Eck, D.: Counterpoint by convolution. In: Proceedings of the 18th International Society for Music Information Retrieval Conference, pp. 211–218 (2017)Google Scholar
  5. 5.
    Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 107:1–107:14 (2017). Proceedings of SIGGRAPH 2017CrossRefGoogle Scholar
  6. 6.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv prepreint arXiv:1502.03167 (2015)
  7. 7.
    Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., Chen, X.: Improved techniques for training gans. In: Advances in Neural Information Processing Systems , vol. 29, pp. 2234–2242. Curran Associates, Inc. (2016)Google Scholar
  8. 8.
    Simon, I., Oore, S.: Performance RNN: generating music with expressive timing and dynamics (2017).
  9. 9.
    Yang, L.C., Chou, S.Y., Yang, Y.H.: MidiNet: a convolutional generative adversarial network for symbolic-domain music generation using 1D and 2D conditions. arXiv preprint arXiv:1703.10847 (2017)
  10. 10.
    Zeiler, M.D.: ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Kosuke Nakamura
    • 1
  • Takashi Nose
    • 1
  • Yuya Chiba
    • 1
  • Akinori Ito
    • 1
    Email author
  1. 1.Graduate School of EngineeringTohoku UniversitySendai-shiJapan

Personalised recommendations