Artificial Intelligence Review

, Volume 52, Issue 3, pp 1781–1804 | Cite as

Comparison of algorithms to divide noisy phone sequences into syllables for automatic unconstrained English speaking proficiency scoring

  • David O. JohnsonEmail author
  • Okim Kang


Four algorithms for syllabifying phones are compared in automatically scoring English oral proficiency. The first algorithm clusters consonants into groups with the vowel nearer to them temporally, taking into account the maximal onset principle. A Hidden Markov Model (HMM) predicts the syllable boundaries based on their sonority value in the second algorithm. The third one employs three HMMs which are tuned to specific categories of utterances. The final algorithm uses a genetic algorithm to identify a set of rules for syllabifying the phones. They were evaluated by: (1) how well they syllabified utterances from the Boston University Radio News Corpus (BURNC) and (2) how well they worked as part of a process to automatically score English speaking proficiency. A measure of the temporal alignment of the syllables was utilized to judge how satisfactorily they syllabified utterances. Their suitability in the proficiency process was assessed with the Pearson correlation between the computer’s predicted proficiency scores and the scores determined by human examiners. We found that syllabification-by-genetic-algorithm performed the best in syllabifying the BURNC, but that syllabification-by-grouping (i.e., syllables are made by grouping non-syllabic consonant phones with the vowel or syllabic consonant phone nearest to them with respect to time) performed the best in the English oral proficiency rating application.


Automatic syllabification Automatic speaking proficiency scoring Sonority sequencing principle Maximal onset principle ASR phone recognition 



The authors would like to thank Michael Albanese, Tory Bottiglieriy, Trent Coopery, Drew McDaniely, and Adam Thomas for developing the initial prototypes of the HMM, k-means, and genetic algorithm methods of syllabification as part of their senior Computer Science Capstone project at Northern Arizona University.


  1. Ananthakrishnan S (2004) Statistical syllabification of english phoneme sequences using supervised unsupervised algorithms. CS562 term project reportGoogle Scholar
  2. Baayen RH, Piepenbrock R, Van Rijn H (1993) The CELEX lexical database. Linguistic data consortium, University of Pennsylvania, PhiladelphiaGoogle Scholar
  3. Bartlett S, Kondrak G, Cherry C (2009) On the syllabification of phonemes. In: Proceedings of human language technologies: the 2009 annual conference of the North American chapter of the association for computational linguistics, pp 308–316Google Scholar
  4. Bernstein J, Van Moere A, Cheng J (2010) Validating automated speaking tests. Lang Test 27:355–377CrossRefGoogle Scholar
  5. Boersma P, Weenink D (2014) Praat: doing phonetics by computer (version 5.3.83) [Computer program]. Retrieved 19 Aug 2014Google Scholar
  6. Brazil D (1997) The communicative value of intonation in english book. Cambridge University Press, CambridgeGoogle Scholar
  7. Cambridge English Language Assessment (2015) [Online]. Accessed 24 Sept 2017
  8. Clements GN (1990) The role of the sonority cycle in core syllabification. Pap Lab Phonol 1:283–333CrossRefGoogle Scholar
  9. Daelemans W, van den Bosch A (1992) Generalization performance of backpropagation learning on a syllabification task. In: Proceedings of the 3rd twente workshop on language technology, pp 27–38Google Scholar
  10. Daelemans W, van den Bosch A, Weijters T (1997) IGTree: using trees for compression classification in lazy learning algorithms. Artif Intell Rev 11(1–5):407–423CrossRefGoogle Scholar
  11. Dehak N, Dumouchel P, Kenny P (2007) Modeling prosodic features with joint factor analysis for speaker verification. IEEE Trans Audio Speech Lang Process 15(7):2095–2103CrossRefGoogle Scholar
  12. Demberg V (2006) Letter-to-phoneme conversion for a German text-to-speech system. Master’s thesis, University of StuttgartGoogle Scholar
  13. Evanini K, Wang X (2013) Automated speech scoring for non-native middle school students with multiple task types. In: INTERSPEECH, pp 2435–2439Google Scholar
  14. Fine S, Singer Y, Tishby N (1998) The hierarchical hidden Markov model: analysis and applications. Mach Learn 32(1):41–62CrossRefzbMATHGoogle Scholar
  15. Fisher W (1996) The tsylb2 program: algorithm description. NIST, 1996b, Part of the tsylb2-11 software packageGoogle Scholar
  16. Garofolo JS, Lamel LF, Fisher WM, Fiscus JG, Pallett DS (1993) DARPA TIMIT acoustic-phonetic continous speech corpus CD-ROM NIST speech disc 1–11. NASA STI/Recon Technical Report N, vol 93, p 27403Google Scholar
  17. Goldwater S, Johnson M (2005) Representational bias in unsupervised learning of syllable structure. In: Proceedings of the ninth conference on computational natural language learning, pp 112–119Google Scholar
  18. Hammond M (1997) Parsing syllables: modeling OT computationally. Preprint arXiv:cmp-lg/9710004
  19. Hooper JB (1972) The syllable in phonological theory. Language 48:525–540CrossRefGoogle Scholar
  20. Johnson DO, Kang O (2015) Automatic prominent syllable detection with machine learning classifiers. Int J Speech Technol 18(4):583–592CrossRefGoogle Scholar
  21. Johnson DO, Kang O (2016) Automatic detection of Brazil’s prosodic tone unit. In: Proceedings of speech prosody 8, Boston, MA, USA, May 31–June 3, 2016Google Scholar
  22. Johnson DO, Kang O, Ghanem R (2016a) Improved automatic english proficiency rating of unconstrained speech with multiple corpora. Int J Speech Technol 19(4):755–768. CrossRefGoogle Scholar
  23. Johnson DO, Kang O, Ghanem R (2016b) Language proficiency ratings: human versus machine. In: Proceedings of the 7th pronunciation in second language learning and teaching conference, pp 119–129Google Scholar
  24. Kahn D (1976) Syllable-based generalizations in English phonology, vol 156. Indiana University Linguistics Club, BloomingtonGoogle Scholar
  25. Kibre N, Pearson S, Kuhn R, Fincke S (2000) Automatic methods for lexical stress assignment syllabification. In: The proceedings of the 6th international conference on spoken language processing, vol 2Google Scholar
  26. Kiraz GA, Möbius B (1998) Multilingual syllabification using weighted finite-state transducers. In: The third ESCA/COCOSDA workshop (ETRW) on speech synthesisGoogle Scholar
  27. Kockmann M, Burget L (2008) Contour modeling of prosodic acoustic features for speaker recognition. In: 2008 IEEE spoken language technology workshop—SLT, pp 45–48Google Scholar
  28. Kockmann M, Burget L, Černocky JH (2010) Investigations into prosodic syllable contour features for speaker recognition. In: 2010 IEEE international conference on acoustics speech signal processing (ICASSP), pp 4418–4421Google Scholar
  29. Krenn B (1997) Tagging syllables. In: Fifth European conference on speech communication and technology (EUROSPEECH’97)Google Scholar
  30. Lin CY, Wang HC (2005) Language identification using pitch contour information. In: ICASSP, vol 1, pp 601–604Google Scholar
  31. Longman P (2009) Official guide to Pearson test of English academic (with CD-ROM). Pearson Education, IndiaGoogle Scholar
  32. Marchand Y, Adsett CR, Damper RI (2007) Evaluating automatic syllabification algorithms for English. 316–321Google Scholar
  33. Marchand Y, Adsett CR, Damper RI (2009) Automatic syllabification in English: a comparison of different algorithms. Lang Speech 52(1):1–27CrossRefGoogle Scholar
  34. Mary L, Yegnanarayana B (2008) Extraction representation of prosodic features for language speaker recognition. Speech Commun 50(10):782–796CrossRefGoogle Scholar
  35. MathWorks, Inc (2013) MATLAB release 2013a. [Computer program]Google Scholar
  36. Mayer T (2010) Toward a totally unsupervised, language-independent method for the syllabification of written texts. In: Proceedings of the 11th meeting of the ACL special interest group on computational morphology phonology, pp 63–71Google Scholar
  37. Müller K (2001) Automatic detection of syllable boundaries combining the advantages of treebank bracketed corpora training. In: Proceedings of the 39th annual meeting on association for computational linguistics, pp 410–417Google Scholar
  38. Müller K (2006) Improving syllabification models with phonotactic knowledge. In: Proceedings of the eighth meeting of the ACL special interest group on computational phonology morphology, pp 11–20Google Scholar
  39. Oller DK, Niyogi P, Gray S, Richards JA, Gilkerson J, Xu D, Yapaneld U, Warren SF (2010) Automated vocal analysis of naturalistic recordings from children with autism, language delay, typical development. Proc Natl Acad Sci 107(30):13354–13359CrossRefGoogle Scholar
  40. Ostendorf M, Price PJ, Shattuck-Hufnagel S (1995) The Boston University radio news corpus. Linguist Data Consort 323:1–19Google Scholar
  41. Ouellet P, Dumouchel P (2001) Heuristic syllabification and statistical syllable-based modeling for speech-input topic identification. In: Workshop on grammar and NLP, pp 13–14Google Scholar
  42. Pearson Education, Inc (2015) Versant Spanish test. Accessed 24 Sept 2017
  43. Povey D, Ghoshal A, Boulianne G, Burget L, Glembek O, Goel N, Hannemann M, Motlíček P, Qian Y, Schwarz P, Silovsky J, Stemmer G, Vesel K (2011) The Kaldi speech recognition toolkit. In: IEEE 2011 workshop on automatic speech recognition and understanding (No. EPFL-CONF-192584). IEEE Signal Processing SocietyGoogle Scholar
  44. Pulgram E (1970) Syllable, word, nexus, cursus. No. 81–85. Mouton, The HagueGoogle Scholar
  45. Rogova K, Demuynck K, Van Compernolle D (2013) Automatic syllabification using segmental conditional random fields. In: Book of abstracts of the 23rd meeting of computational linguistics in the Netherlands: CLIN, p 41Google Scholar
  46. Schmid H, Möbius B, Weidenkaff J (2007) Tagging syllable boundaries with joint n-gram models. In: INTERSPEECH, pp 2857–2860Google Scholar
  47. Selkirk EO (1984) On the major class features and syllable theory. 107–136Google Scholar
  48. Shriberg E, Ferrer L, Kajarekar S, Venkataraman A, Stolcke A (2005) Modeling prosodic feature sequences for speaker recognition. Speech Commun 46(3):455–472CrossRefGoogle Scholar
  49. Taherdangkoo M, Paziresh M, Yazdi M, Bagheri M (2013) An efficient algorithm for function optimization: modified stem cells algorithm. Open Eng 3(1):36–50CrossRefGoogle Scholar
  50. Tuzlukov V (2002) Signal processing noise, vol 8. CRC Press, Boca RotanCrossRefzbMATHGoogle Scholar
  51. Vennemann T (1987) Preference laws for syllable structure: and the explanation of sound change with special reference to German, Germanic, Italian, and Latin. Walter de Gruyter, BerlinCrossRefGoogle Scholar
  52. Zechner K, Higgins D, Xi X, Williamson DM (2009) Automatic scoring of non-native spontaneous speech in tests of spoken English. Speech Commun 51(10):883–895CrossRefGoogle Scholar
  53. Zhang J, Hamilton HJ (1997) Learning English syllabification for words. Foundations of intelligent systems. Springer, Berlin, pp 177–186CrossRefGoogle Scholar
  54. Ziaei A, Kaushik L, Sangwan A, Hansen JH, Oard D (2014) Speech activity detection for nasa apollo space missions: challenges solutions. In: INTERSPEECHGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2017

Authors and Affiliations

  1. 1.Northern Arizona UniversityFlagstaffUSA

Personalised recommendations