Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 22, pp 31987–32006 | Cite as

Video telephony - quality of experience: a simple QoE model to assess video calls using subjective approach

  • Phisit Pornpongtechavanich
  • Therdpong DaengsiEmail author
Article
  • 58 Downloads

Abstract

One problem about using video calls or video telephony applications is a quality issue. Therefore, this paper contributes the simple QoE model, a mathematical model, to assess quality of video calls provided by video telephony applications or services. This mathematical model has been created based on the results from 200 subjects who joined the subjective test using short conversation scenario via two applications, Skype and LINE, two popular applications/services in Thailand. Based on the subjective results, the weighted coefficients for audio quality and video quality have been found, while the coefficient for audio quality is weighted higher than the one for video quality. Nevertheless, after using ANOVA and t-test, statistical analysis tools, based on the first set of the subjective data, it has been found that there are significant differences (p-values < 0.05) when compared to most previous QoE models from other research works. However, after performance evaluating using Mean Absolute Percent Error (MAPE), it has been found that its performances are better than previous models with average MAPE values of 10.4 and 5.65% in the first and the second round evaluation with the other 48 subjects and 32 subjects respectively. Therefore, these are evident that this model can be applied to evaluation quality of video calls with confidence.

Keywords

Video telephony QoE model Audiovisual Subjective tests Mean opinion score 

Notes

Acknowledgments

Thank you to the Faculty of Engineering, Rajamangala University of Technology Phra Nakhon and the Faculty of Industry and Technology, Rajamangala University of Technology Rattanakosin (Wang Klai Kangwon Campus) for supporting. Especially, thank you to all students, staff and lecturers who were participants or involved.

References

  1. 1.
    Alsmirat MA, Jararweh Y, Obaidat I, Gupta BB (2017) Automated wireless video surveillance: an evaluation framework. J Real-Time Image Proc 13:527–546CrossRefGoogle Scholar
  2. 2.
    Belmudez B, Möller S (2013) Audiovisual quality integration for interactive communications. EURASIP J Audio Spee.  https://doi.org/10.1186/1687-4722-2013-24
  3. 3.
    Belmudez B, Möller S, Lewcio B, Raake A, Mehmood A (2009) Audio and video channel impact on perceived audio-visual quality in different interactive contexts. In: Proceedings of IEEE Int. Wkshp MMSP’09, Rio De Janeiro, pp 1–5Google Scholar
  4. 4.
    Chen Y, Wu K, Zhang Q (2015) From QoS to QoE: a tutorial on video quality assessment. IEEE Commun Surv Tut 17(2):1126–1165CrossRefGoogle Scholar
  5. 5.
  6. 6.
  7. 7.
    Daengsi T, Wuttidittachotti P (2017) Subjective MOS model and simplified E-model enhancement for Skype associated with packet loss effects: a case using conversation-like tests with Thai users. Multimed Tools Appl 76(15):16163–16187CrossRefGoogle Scholar
  8. 8.
    Daengsi T, Wutiwiwatchai C, Preechayasomboon A, Sukparungsee S (2014) IP telephony: comparison of subjective assessment methods for voice quality evaluation. Walailak J Sci Technol 11(2):87–92Google Scholar
  9. 9.
    Daengsi T, Khitmoh N, Wattidittachotti P (2017) VoIP quality measurement: subjective VoIP quality estimation model for G.711 and G.729 based on native Thai users. Multimedia Systems 22(5):575–586CrossRefGoogle Scholar
  10. 10.
    De Pessemier T, Stevens I, De Marez L, Martens L, Joseph W (2016) Quality assessment and usage behavior of a mobile voice-over-IP service. Telecommun Syst 61:417–432CrossRefGoogle Scholar
  11. 11.
    De Rango F, Tropea M, Fazio P, Marano S (2006) Overview on VoIP: subjective and objective measurement methods. Int J Comput Sci Network Security 6(1B):140–153Google Scholar
  12. 12.
    Farid F, Shahrestani S, Ruan C (2013) Quality of service concerns in wireless and cellular networks. Communications of the IBIMA 2013(794626)Google Scholar
  13. 13.
    Hands DS (2004) A basic multimedia quality model. IEEE Trans Multimedia 6(6):806–816CrossRefGoogle Scholar
  14. 14.
    Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: Proceedings of the 20th ICPR 2010, Istanbul, pp 2366–2369Google Scholar
  15. 15.
    Hossain MS, Muhammad G, Abdul W, Song B, Gupta BB (2018) Cloud-assisted secure video transmission and sharing framework for smart cities. Futur Gener Comput Syst 83:596–606CrossRefGoogle Scholar
  16. 16.
  17. 17.
    ITU-T (1996) ITU-T recommendation P.800 methods for subjective determination of transmission qualityGoogle Scholar
  18. 18.
    ITU-T (1998) ITU-R recommendation BT.710 subjective assessment methods for image quality in high-definition televisionGoogle Scholar
  19. 19.
    ITU-T (1998) ITU-T recommendation P.911 subjective audiovisual quality assessment methods for multimedia applicationsGoogle Scholar
  20. 20.
    ITU-T (2000) ITU-T recommendation P.920 interactive test methods for audiovisual communicationsGoogle Scholar
  21. 21.
    ITU-T (2000) ITU-T recommendation G.107 the E-model: a computational model for use in transmissionGoogle Scholar
  22. 22.
    ITU-T (2001) ITU-T recommendation P.862 perceptual evaluation of speech quality (PESQ): an objective method for end-to-end speech quality assessment of narrow-band telephoneGoogle Scholar
  23. 23.
    ITU-T (2003) ITU-T recommendation J.148 requirements for an objective perceptual multimedia quality modelGoogle Scholar
  24. 24.
    ITU-T (2007) ITU-T recommendation P.805 subjective evaluation of conversational qualityGoogle Scholar
  25. 25.
    ITU-T (2008) ITU-T recommendation P.910 subjective video quality assessment methods for multimedia applicationsGoogle Scholar
  26. 26.
    ITU-T (2012) ITU-T recommendation G.1070 opinion model for video-telephony applicationsGoogle Scholar
  27. 27.
    ITU-T (2016) ITU-T recommendation P.913 methods for the subjective assessment of video quality, audio quality and audiovisual quality of internet video and distribution quality television in any environmentGoogle Scholar
  28. 28.
    ITU-T (2017) ITU-T recommendation P.1301 subjective quality evaluation of audio and audiovisual multiparty telemeetingsGoogle Scholar
  29. 29.
    Jana S, Chan A, Pande A, Mohapatra P (2016) QoE prediction model for mobile video telephony. Multimed Tools Appl 75(13):7957–7980CrossRefGoogle Scholar
  30. 30.
    Khalifeh, A, Gholamhosseinian, A, Hajibagher, NZ: QOS For Multimedia Applications with Emphasize on Video Conferencing. http://www.diva-portal.org/smash/get/diva2:504299/FULLTEXT01.pdf
  31. 31.
    Le H, Behboodi A, Wolisz A (2015) Quality driven resource allocation for adaptive video streaming in ofdma uplink. In: Proceedings of PIMRC 2015, Hong Kong, pp 1277–1282Google Scholar
  32. 32.
    Li J, Yu C, Gupta BB, Ren X (2018) Color image watermarking scheme based on quaternion Hadamard transform and Schur decomposition. Multimed Tools Appl 77:4545–4561CrossRefGoogle Scholar
  33. 33.
    Liu H, Guo Q, Wang G, Gupta BB, Zhang C (2019) Medical image resolution enhancement for healthcare using nonlocal self-similarity and low-rank prior. Multimed Tools Appl.  https://doi.org/10.1007/s11042-017-5277-6
  34. 34.
    Martinez HB, Farias MCQ (2014) An objective model for audio-visual quality. In: Proceedings of SPIE 9016, image quality and system performance XI, 90160P.  https://doi.org/10.1117/12.2042425
  35. 35.
    Martinez HB, Farias MCQ (2014) A no-reference audio-visual video quality metric. In: Proceedings of the 22nd EUSIPCO, Lisbon, pp 2125–2129Google Scholar
  36. 36.
    Martinez-Rach, M, López, O, Piñol, P, Malumbres, MP, Oliver, J: PSNR vs. quality assessment metrics for image and video codec performance evaluation. https://pdflegend.com/downloadFile/downloadFile/59fea151d64ab29c64dd3d6f
  37. 37.
    Mohammadi P, Ebrahimi-Moghadam A, Shirani S (2014) Subjective and objective quality assessment of image: a survey. https://arxiv.org/ftp/arxiv/papers/1406/1406.7799.pdf
  38. 38.
    Saidi I, Zhang L, Barriac V, Deforges O (2016) Audiovisual quality study for videotelephony on IP networks. In: Proceedings of IEEE Int. Wkshp MMSP’16.  https://doi.org/10.1109/MMSP.2016.7813379
  39. 39.
    Sirawongphatsara P, Wuttidittachotti P, Daengsi T (2015) Comparison of video telephony: a case study of LINE and tango over 3G in Bangkok. In: Proceedings of ICOIN 2015, Siem Reap, pp 205–209Google Scholar
  40. 40.
    Streijl RC, Winkler S, Hands DS (2016) Mean opinion score (MOS) revisited: methods and applications limitations and alternatives. Multimedia Systems 22(2):213–227CrossRefGoogle Scholar
  41. 41.
    Sun L, Alfayly A (2015) QoE-driven management schemes for multimedia services. IEEE COMSOC MMTC E-Letter 10(3):14–17Google Scholar
  42. 42.
    Takahashi A (2009) Framework and standardization of quality of experience (QoE) design and management for audiovisual communication services. NTT Tech Review 7(4):1–5Google Scholar
  43. 43.
    Telchemy. Telchemy video quality metrics. http://www.iptvtroubleshooter.com/tvqm.html
  44. 44.
    Timmerer C, Maiero M, Rainer B, Petscharnig S, Weinberger D, Mueller C, Lederer S (2015) Quality of experience of adaptive HTTP streaming in real-world environments. IEEE COMSOC MMTC E-Letter 10(3):6–9Google Scholar
  45. 45.
    Tsolkas D, Liotou E, Passas N, Merakos L (2017) A survey on parametric QoE estimation for popular services. J Netw Comput Appl 77:1–17CrossRefGoogle Scholar
  46. 46.
    Wamser F, Deschner S, Zinner T, Tran-Gia P (2013) Investigation of different approaches for QoE-oriented scheduling in OFDMA networks. In: Proceedings of MONAMI 2013.  https://doi.org/10.1007/978-3-319-04277-0_14
  47. 47.
    Wang Y Survey of objective video quality measurements. ftp://ftp.cs.wpi.edu/pub/techreports/pdf/06-02.pdf
  48. 48.
    Wang J, Hou YB (2018) Packet loss rate mapped to the quality of experience. Multimed Tools Appl 77(1):387–422CrossRefGoogle Scholar
  49. 49.
  50. 50.
  51. 51.
    Winkler S, Faller C (2006) Perceived audiovisual quality of low-bitrate multimedia content. IEEE Trans Multimedia 8(5):973–980CrossRefGoogle Scholar
  52. 52.
    Wu P-H, Huang C-W, Hwang J-N, Pyun J-Y, Zhang J (2015) Video-quality-driven resource allocation for real-time surveillance video Uplinking over OFDMA-based wireless networks. IEEE Trans Veh Technol 64(7):3233–3246Google Scholar
  53. 53.
    Wuttidittachotti P, Daengsi T (2015) Quality evaluation of mobile networks using VoIP applications: a case study with Skype and LINE based-on stationary tests in Bangkok. Int J Comput Network Inform Security 7(12):28–41CrossRefGoogle Scholar
  54. 54.
    Xu Y, Yu C, Li J, Liu Y (2014) Video telephony for end-consumers: measurement study of Google+, iChat, and Skype. IEEE/ACM Trans Networking 22(3):826–839CrossRefGoogle Scholar
  55. 55.
    Yamagishi K, Tominaga T, Hayashi T, Takahashi A (2007) Objective quality evaluation model for videophone services. NTT Tech Review. 5(6):1–5Google Scholar
  56. 56.
    Yu C, Xu Y, Liu B, Liu Y (2014) Can you SEE me now? A study of mobile video calls. In: Proceedings of IEEE INFOCOM 2014, Toronto, pp 1456–1464Google Scholar
  57. 57.
    Yue T, Wang H, Cheng H (2018) Learning from users: a data-driven method of QoE evaluation for internet video. Multimed Tools Appl 77(20):27269–27300CrossRefGoogle Scholar
  58. 58.
    Zhang W, Chang Y, Liu Y, Tian Y (2014) Performance analyze of QoE-based speech quality evaluation model. In: Proceedings of IEEE ICMEW 2014.  https://doi.org/10.1109/ICMEW.2014.6890640
  59. 59.
    Zhu Y, Heynderickx I, Redi JA (2015) Understanding the role of social context and user factors in video quality of experience. Comput Hum Behav 49:412–426CrossRefGoogle Scholar
  60. 60.
    Zinner T, Abboud O, Hohlfeld O, Hossfeld T, Tran-Gia P (2010) Towards QoE management for scalable video streaming. In: Proceedings of the 21th ITC specialist seminar on multimedia applications – traffic, performance and QoE, Miyazaki, pp 64–69Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Faculty of Industry and TechnologyRajamangala University of Technology Rattanakosin (Wang Klai Kangwon Campus)Hua HinThailand
  2. 2.Department of Sustainable Industrial Management Engineering, Faculty of EngineeringRajamangala University of Technology Phra Nakhon (North Bangkok Center)BangkokThailand

Personalised recommendations