Making Music Composing Easier for Amateurs: A Hybrid Machine Learning Approach
Creating your own musical pieces is one of the most attractive ways to enjoy music. However, many musically untrained people lack the basic musical skills to do so. In this paper, we seek to explore how machine learning algorithms can enable musically untrained users to create their own music.
To achieve this, we propose a Neural Hidden Markov Model (NHMM). It is a hybrid of a Hidden Markov model (HMM) and Convolution neural network (CNN) with a Long Short-Term Memory (LSTM) neural network. This model takes users’ original musical ideas in an easy intuitive way and automatically modifies the input to generate musically appropriate melodies as output. We further extend the model to allow users to specify the magnitude of revision, duration of music segment to be revised, choice of music genres, popularity of songs, and co-creation of songs in social settings. These extensions enhance user understanding of music theory, enrich their experience of self-learning, and enable social aspects of music creation. The model is trained using MIDI files of existing songs. We also conduct experiments on melody generation.
We also hope to design a mobile application with an intuitive, interactive, and graphical user interface, which is suitable for the elderly and young children. Different from most existing literature focusing on computer music composition itself, our research and application aim at using computers to aid human composition and enriching the music education of musically untrained people.
- 3.Kitahara, T., Tsuchiya, Y.: A Machine Learning Approach to Support Music Creation by Musically Untrained PeopleGoogle Scholar
- 4.Mozer, M.C., Soukup, T.: Connectionist music composition based on melodic and stylistic constraints. In: Advances in Neural Information Processing Systems (1990)Google Scholar
- 6.Eck, D., Schmidhuber, J.: A first look at music composition using LSTM recurrent neural networks. Istituto Dalle Molle Di Studi Sull Intelligenza Artificiale (2007)Google Scholar
- 7.Sturm, B.L., et al.: Music transcription modelling and composition using deep learning (2016)Google Scholar
- 8.Liang, F., et al.: Automatic stylistic composition of Bach Chorales with deep LSTM (2017)Google Scholar
- 9.Coca, A.E., Correa, D.C., Zhao, L.: Computer-aided music composition with LSTM neural network and chaotic inspiration. In: International Joint Conference on Neural Networks (2014)Google Scholar
- 11.Mirghafori, N., Morgan, N., Bourlard, H.: Parallel training of MLP probability estimators for speech recognition: a gender-based approach. In: Neural Networks for Signal Processing (1994)Google Scholar
- 12.Krakovna, V., Doshivelez, F.: Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models (2016)Google Scholar
- 13.Ke, T., et al.: Unsupervised Neural Hidden Markov Models (2016)Google Scholar
- 14.Deng, J., Kwok, Y.K.: Automatic Chord estimation on seventhsbass Chord vocabulary using deep neural network. In: IEEE International Conference on Acoustics, Speech and Signal Processing (2016)Google Scholar
- 17.Rabiner, L., Juang, B.: An introduction to hidden Markov models. Current protocols in bioinformatics/editoral board, Andreas D. Baxevanis …[et al.], 2007. Appendix 3(Appendix 3): p. Appendix 3AGoogle Scholar
- 20.Mozer, M.C.: Induction of multiscale temporal structure. In: International Conference on Neural Information Processing Systems (1991)Google Scholar
- 21.Todd, P.M., Loy, G.: Creation by Refinement and the Problem of Algorithmic Music Composition (1991)Google Scholar
- 22.Hild, H., Feulner, J., Menzel, W.: HARMONET: a neural net for harmonizing chorales in the style of J. S. Bach. In: Advances in Neural Information Processing Systems (1991)Google Scholar
- 23.Bellgard, M.I., Tsang, C.P.: Harmonising music using a network of Boltzmann machines (1992)Google Scholar
- 24.Johnson, M.: Why doesn’t EM find good HMM POS-taggers? In: EMNLP-CoNLL 2007, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Prague, 28–30 June 2007Google Scholar
- 25.Sutskever, I.: Training Recurrent Neural Networks. Doctoral (2013)Google Scholar
- 26.Liu, I., Ramakrishnan, B.: Bach in 2014: Music Composition with Recurrent Neural Network. Eprint Arxiv (2014)Google Scholar
- 28.Welch, L.R.: Hidden Markov models and the Baum-Welch algorithm. IEEE Inf. Theory Soc. Newsl. 53(2), 194–211 (2003)Google Scholar