Advertisement

Real-Time Audio-to-Score Alignment Using Particle Filter for Coplayer Music Robots

  • Takuma Otsuka
  • Kazuhiro Nakadai
  • Toru Takahashi
  • Tetsuya Ogata
  • HiroshiG Okuno
Open Access
Research Article
Part of the following topical collections:
  1. Musical Applications of Real-Time Signal Processing

Abstract

Our goal is to develop a coplayer music robot capable of presenting a musical expression together with humans. Although many instrument-performing robots exist, they may have difficulty playing with human performers due to the lack of the synchronization function. The robot has to follow differences in humans' performance such as temporal fluctuations to play with human performers. We classify synchronization and musical expression into two levels: (1) melody level and (2) rhythm level to cope with erroneous synchronizations. The idea is as follows: When the synchronization with the melody is reliable, respond to the pitch the robot hears, when the synchronization is uncertain, try to follow the rhythm of the music. Our method estimates the score position for the melody level and the tempo for the rhythm level. The reliability of the score position estimation is extracted from the probability distribution of the score position. The experimental results demonstrate that our method outperforms the existing score following system in 16 songs out of 20 polyphonic songs. The error in the prediction of the score position is reduced by 69% on average. The results also revealed that the switching mechanism alleviates the error in the estimation of the score position.

Keywords

Probability Distribution Information Technology Tempo Quantum Information Temporal Fluctuation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publisher note

To access the full article, please see PDF.

Copyright information

© Takuma Otsuka et al. 2011

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Authors and Affiliations

  • Takuma Otsuka
    • 1
  • Kazuhiro Nakadai
    • 2
    • 3
  • Toru Takahashi
    • 1
  • Tetsuya Ogata
    • 1
  • HiroshiG Okuno
    • 1
  1. 1.Graduate School of InformaticsKyoto UniversityKyotoJapan
  2. 2.Honda Research Institute Japan, Co., Ltd.Wako, SaitamaJapan
  3. 3.Graduate School of Information Science and EngineeringTokyo Institute of TechnologyTokyoJapan

Personalised recommendations