Advertisement

Speech Coding pp 219-235 | Cite as

Quality Evaluation

  • Tom BäckströmEmail author
Chapter
Part of the Signals and Communication Technology book series (SCT)

Abstract

For evaluating the performance of speech codecs or their algorithms, we need metrics for quality. Since humans are the ultimate users of most speech codecs, subjective testing is the gold standard in quality measurement. In this chapter we present a range of typical subjective evaluation methods and discuss their strengths and weaknesses. Subjective evaluation requires in general plenty of work and time, whereby there exists a selection of automated objective evaluation methods, which simulate subjective measurements. Objective quality estimation methods therefore provide fast and reproducible measures of quality, even if their reliability can never reach that of extensive subjective measurements.

References

  1. 1.
    3GPP. TS 26.190, Adaptive Multi-Rate (AMR-WB) Speech Codec (2007)Google Scholar
  2. 2.
    3GPP. TS 26.445, EVS Codec Detailed Algorithmic Description; 3GPP Technical Specification (Release 12) (2014)Google Scholar
  3. 3.
    CCIR Document 11/17. Subjective Assessment of the Quality of Television Pictures (EBU) (1978–1982)Google Scholar
  4. 4.
    ISO/IEC 23003–3:2012. MPEG-D (MPEG Audio Technologies), Part 3: Unified Speech and Audio Coding (2012)Google Scholar
  5. 5.
    ITU-R Recommendation BS.1284. General Methods for the Subjective Assessment of Sound Quality (2003)Google Scholar
  6. 6.
    ITU-R Recommendation BS.1387. Method for Objective Measurements of Perceived Audio Quality (PEAQ), 2001Google Scholar
  7. 7.
    ITU-R Recommendation BS.562. Subjective Assessment of Sound Quality (1990)Google Scholar
  8. 8.
    ITU-T Recommendation P.800. Methods for Subjective Determination of Transmission Quality (1996)Google Scholar
  9. 9.
    ITU-T Recommendation P.835. Subjective Test Methodology for Evaluating Speech Communication Systems that Include Noise Suppression Algorithm (2003)Google Scholar
  10. 10.
    ITU-T Recommendation P.861. Objective Quality Measurement of Telephoneband (300-3400 Hz) Speech Codecs (1996)Google Scholar
  11. 11.
    ITU-T Recommendation P.862. Perceptual Evaluation of Speech Quality (PESQ): An Objective Method for End-to-end Speech Quality Assessment of Narrow-band Telephone Networks and Speech Codecs (2001)Google Scholar
  12. 12.
    ITU-T Recommendation P.863. Perceptual Objective Listening Quality Assessment (POLQA) (2011)Google Scholar
  13. 13.
    Recommendation BS.1534. Method for the Subjective Assessment of Intermediate Quality Levels of Coding Systems. ITU-R (2003)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.International Audio Laboratories Erlangen (AudioLabs)Friedrich-Alexander University Erlangen-Nürnberg (FAU)ErlangenGermany

Personalised recommendations