Voice Communication in Performing a Cooperative Task with a Robot

  • Koliya Pulasinghe
  • Keigo Watanabe
  • Kazuo Kiguchi
  • Kiyotaka Izumi
Conference paper


This paper investigates the credibility of voice (especially natural language commands) as a communication medium in sharing advanced sensory capacity and knowledge of the human with a robot to perform a cooperative task. Identification of the machine sensitive words in the unconstrained speech signal and interpretation of the imprecise natural language commands for the machine has been considered. The system constituents include a hidden Markov model (HMM) based continuous automatic speech recognizer (ASR) to identify the lexical content of the user’s speech signal, a fuzzy neural network (FNN) to comprehend the natural language (NL) contained in identified lexical content, an artificial neural network (ANN) to activate the desired functional ability, and control modules to generate output signals to the actuators of the machine. The characteristic features have been tested experimentally by utilizing them to navigate a Khepera® in real time using the user’s visual information transferred by speech signals.


Hide Markov Model Speech Signal Fuzzy Neural Network Linguistic Label Speech Recognizer 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mazo M., Rodrìguez F. J. et al. (1995) Electronic control of a wheelchair guided by voice commands. Control Eng. Practice 3(5):665–674CrossRefGoogle Scholar
  2. 2.
    Sugisaka M., Fan X. (2001) Control of a welfare life robot guided by voice commands. In: Proc. of the ICCAS 2001, Cheju Korea, 390–393Google Scholar
  3. 3.
    Lin C. T., Kan M. C. (1998) Adaptive fuzzy command acquisition with reinforcement learning. IEEE Transaction on Fuzzy Systems 6(1): 102–121CrossRefGoogle Scholar
  4. 4.
    Komiya K., Morita K. et al. (2000) Guidence of a wheelchair by voice. In: IECON 2000, Nagoya Japan, 102–107Google Scholar
  5. 5.
    Pulasinghe K., Watanabe K. et al. (2001) Modular fuzzy neural controller driven by voice commands. In: Proc. of the ICCAS 2001, Cheju Korea, 194–197Google Scholar
  6. 6.
    Rose R. C, Paul D. B. (1990) A hidden Markov model based keyword recognition system. In: Proc. of the IEEE ICASSP ’90, Albuquerque New Mexico, 129–132Google Scholar
  7. 7.
    Jeanrenaud P., Ng K. et al. (1993) Phonetic-based word spotter: Various configurations and application to event spotting. In: Proc. of the EUROSPEECH ’93, Berlin Germany, 1057–1060Google Scholar
  8. 8.
    Bazzi I., Glass J. R. (2000) Modeling out-of-vocabulary words for robust speech recognition. In: Proc. of the ICSLP 2000, Beijing China.Google Scholar
  9. 9.
    Leeuwen D. A. V., Kraaij W. et al. (1999) Prediction of keywords spotting performance based on phonemic contents. In: Proc. of the ESCA/ETRW, Cambridge UK, 73–77Google Scholar
  10. 10.
    Young S. J. (1993) The HTK hidden Markov model toolkit: Design and philosophy. Technical Report TR.153, Department of Engineering, Cambridge University UKGoogle Scholar
  11. 11.
    Lippmann R. P. (1987) An introduction to computing with neural nets. IEEE Magazine on Acoustics, Signal, and Speech Processing 4:4–22Google Scholar
  12. 12.
    Jang J. S. R., Sun C. T. (1995) Neuro-fuzzy modeling and control. In: Proc. of the IEEE 83(3):378–406Google Scholar

Copyright information

© Springer-Verlag Tokyo 2002

Authors and Affiliations

  • Koliya Pulasinghe
    • 1
  • Keigo Watanabe
    • 2
  • Kazuo Kiguchi
    • 2
  • Kiyotaka Izumi
    • 2
  1. 1.Faculty of Engineering Systems and Technology, Graduate School of Science and EngineeringSaga UniversitySagaJapan
  2. 2.Department of Advanced Systems Control Engineering, Graduate School of Science and EngineeringSaga UniversitySagaJapan

Personalised recommendations