Advertisement

Musical Query-by-Semantic-Description Based on Convolutional Neural Network

  • Jing Qin
  • Hongfei LinEmail author
  • Dongyu Zhang
  • Shaowu Zhang
  • Xiaocong Wei
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10390)

Abstract

We present a new music retrieval system based on query by semantic description (QBSD) system, by which a novel song can be used as query and transformed into semantic vector by a convolutional neural network. This method based on Supervised Multi-class labeling (SML), which a song can be annotated by some semantically meaningful tags and retrieved relevant song in semantically annotated database. CAL500 data set is used in experiment, we can learn a deep learning model for each tag in semantic space. To improve the annotation effect, loss function adjustment algorithm and SMOTE algorithm are employed. The experiment results show that this model can get songs with high semantically similarity, and provide a more nature way to music retrieval.

Keywords

Query by semantic description Convolutional neural network Supervised multi-class labeling Semantically retrieval 

Notes

Acknowledgment

Supported by the National Natural Science Foundation of China (Grant No. 61632011); the National Natural Science Foundation of China (Grant No. 61562080); the National Natural Science Foundation of China (Grant No. 61602079)

References

  1. 1.
  2. 2.
    Casey, M., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., Slaney, M.: Content-based music information retrieval: current directions and future challenges. Proc. IEEE 96(4), 668–696 (2008)CrossRefGoogle Scholar
  3. 3.
    Wang, J., Deng, H., Yan, Q.: A collaborative model of low-level and high-level descriptors for semantics-based music information retrieval. In: International Conference on Web Intelligence and Intelligent Agent Technology, pp. 532–535. IEEE, New York (2008)Google Scholar
  4. 4.
    Buccoli, M., Gallo, A., Zanoni, M., Sarti, A., Tubaro, S.: A dimensional contextual semantic model for music description and retrieval. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 673–677. IEEE, New York (2015)Google Scholar
  5. 5.
    Buccoli, M., Zanoni, M., Sarti, A., Tubaro, S.: A music search engine based on semantic text-based query. In: IEEE International Workshop on Multimedia Signal Processing, pp. 254–259. IEEE, New York (2013)Google Scholar
  6. 6.
    Miotto, R., Lanckriet, G.: A generative context model for semantic music annotation and retrieval. IEEE Trans. Audio Speech Lang. Process. 20(4), 1096–1108 (2012)CrossRefGoogle Scholar
  7. 7.
    Su, J.H., Wang, C.Y., Chiu, T.W., Ying, J.C., Tseng, V.S.: Semantic content-based music retrieval using audio and fuzzy-music-sense features. In: IEEE International Conference on Granular Computing, pp. 259–264. IEEE, New York (2014)Google Scholar
  8. 8.
    Foster, P., Mauch, M., Dixon, S.: Sequential complexity as a descriptor for musical similarity. IEEE Press 22(12), 1965–1977 (2014)Google Scholar
  9. 9.
    Turnbull, D., Barrington, L., Torres, D., Lanckriet, G.: Towards musical query- by- semantic description using the CAL500 data set. In: International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 439–446. ACM, New York (2007)Google Scholar
  10. 10.
    Turnbull, D., Barrington, L., Torres, D., Lanckriet, G.: Semantic annotation and retrieval of music and sound effects. IEEE Trans. Audio Speech Lang. Process. 16(2), 467–476 (2008)CrossRefGoogle Scholar
  11. 11.
    Turnbull, D.R., Barrington, L., Lanckriet, G., Yazdani, M.: Combining audio content and social context for semantic music discovery. In: International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 387–394. ACM, New York (2009)Google Scholar
  12. 12.
    Lee, H., Yan, L., Pham, P., Ng, A.Y.: Unsupervised feature learning for audio classification using convolutional deep belief networks. In: International Conference on Neural Information Processing Systems, pp. 1096–1104. Springer, Heidelberg (2009)Google Scholar
  13. 13.
    Dieleman, S., Brakel, P., Schrauwen, B.: Audio-based music classification with a pretrained convolutional network. In: Proceedings of the ISMIR (2011)Google Scholar
  14. 14.
    Hu, Z., Fu, K., Zhang, C.: Audio classical composer identification by deep neural network. J. Comput. Res. Dev. 51(9), 1945–1954 (2014)Google Scholar
  15. 15.
    Humphrey, E.J., Cho, T., Bello, J.P.: Learning a robust Tonnetz-space transform for automatic chord recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 453–456. IEEE, New York (2012)Google Scholar
  16. 16.
    Hamel, P., Eck, D.: Learning features from music audio with deep belief networks. In: Proceedings of the ISMIR, pp. 339–344 (2010)Google Scholar
  17. 17.
    Hinton, G., Deng, L., Yu, D., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Sig. Process. Mag. 29(6), 82–97 (2012)CrossRefGoogle Scholar
  18. 18.
    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16(1), 321–357 (2002)zbMATHGoogle Scholar
  19. 19.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105. ACM, New York (2012)Google Scholar
  20. 20.
    Lemaitre, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18(17), 1–5 (2017)zbMATHMathSciNetGoogle Scholar
  21. 21.
    Chollet, F.: Keras, GitHub repository (2015). https://github.com/fchollet/keras
  22. 22.
    Coviello, E., Chan, A.B., Lanckriet, G.: Time series models for semantic music annotation. IEEE Trans. Audio Speech Lang. Process. 19(5), 1343–1359 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jing Qin
    • 1
    • 2
  • Hongfei Lin
    • 1
    Email author
  • Dongyu Zhang
    • 1
  • Shaowu Zhang
    • 1
  • Xiaocong Wei
    • 3
  1. 1.School of Computer Science and TechnologyDalian University of TechnologyDalianChina
  2. 2.College of Information EngineeringDalian UniversityDalianChina
  3. 3.School of Software EngineeringDalian University of Foreign LanguageDalianChina

Personalised recommendations