Advertisement

Optimal inter-view rate allocation for multi-view video plus depth over MPEG-DASH using QoE measures and paired comparison

  • Nükhet ÖzbekEmail author
  • Engin Şenol
Original Paper
  • 24 Downloads

Abstract

Rapid advances in compression and transmission technologies enabled multi-view video streaming over the Internet through state-of-the-art MPEG standards such as dynamic adaptive streaming over HTTP (DASH) and high-efficiency video coding. Yet, standards for subjective and objective measurement of quality of experience (QoE) are lagging in the area. The MPEG-DASH system requires constant bitrate (CBR) coding and an efficient rate control strategy in order to increase QoE. Multi-view video plus depth format is preferable due to high compression gain but quality of intermediate view is crucial, strictly depending on performance of inter/intra-view rate allocation as well as rendering algorithms. We propose to find optimal ratio for inter-view rate allocation in CBR using our previously developed objective QoE measures using depth maps and structural similarities. To assess small differences and discriminate similar quality levels, a new subjective assessment methodology is also proposed as a multiple stimuli plus simultaneous presentation for paired comparison. The results show that the unequal rate allocation is superior to the equal rate allocation in terms of objective QoE as 0.012–0.04 (which translates into a bit rate saving of 15–24%) and subjective QoE as 0.3–0.5. Furthermore, a very high correlation is achieved between the new QoE measures and paired comparison results.

Keywords

QoE measurement Multi-view video plus depth Rate allocation Paired comparison MPEG-DASH SSIM 

Notes

Supplementary material

11760_2019_1464_MOESM1_ESM.docx (2.5 mb)
Supplementary material 1 (DOCX 2518 kb)

References

  1. 1.
    Merkle, P., Smolic, A., Müller, K., Wiegand, T.: Multi-view video plus depth representation and coding. In: Proceedings of IEEE International Conference on Image Processing (ICIP’07), San Antonio, TX, USA, pp. 201–204 (Sept. 2007)Google Scholar
  2. 2.
    Fehn, C.: Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV. In: Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems XI, vol. 5291, p. 93104 (2004)Google Scholar
  3. 3.
    Zhu, C., Zhao, Y., Yu, L., Tanimoto, M.: 3D-TV System with Depth-Image Based Rendering: Architectures Techniques and Challenges. Springer, New York (2013)CrossRefGoogle Scholar
  4. 4.
    Müller, K., Smolic, A., Dix, K., Merkle, P., Wiegand, T.: Coding and intermediate view synthesis of multi-view video plus depth. In: Proceedings of IEEE International Conference on Image Processing (ICIP’09) (2009)Google Scholar
  5. 5.
    Hamza, A., Hefeeda, M.: A DASH-based free viewpoint video streaming system. In: Proceedings of Network and Operating System Support on Digital Audio and Video Workshop, p. 55 (2013)Google Scholar
  6. 6.
    Oztas, B., Pourazad, M.T., Nasiopoulos, P., Sodagar, I., Leung, V.C.M.: A rate adaptation approach for streaming multiview plus depth content. In: International Conference on Computing, Networking and Communications (ICNC), no. Mvd, pp. 1006–1010 (2014)Google Scholar
  7. 7.
    Su, T., Sobhani, A., Yassine, A., Shirmohammadi, S., Javadtalab, A.: A DASH-based HEVC multi-view video streaming system. J. Real Time Image Process. 12(2), 329–342 (2016)CrossRefGoogle Scholar
  8. 8.
    Ozcinar, C., Anbarjafari, G.: Dynamic bitrate allocation of interactive real-time streamed multi-view video with view-switch prediction. Signal Image Video Process. 11, 1279–1285 (2017)CrossRefGoogle Scholar
  9. 9.
    Sodagar, I., Vetro, A.: The MPEG-DASH Standard for Multimedia Streaming over the Internet. IEEE Computer Society Industry and Standards (2011)Google Scholar
  10. 10.
    Duanmu, Z., Rehman, A., Zeng, K., Wang, Z.: Quality-of-experience prediction for streaming video. In: Proceedings of IEEE International Conference on Multimedia and Expo (ICME’16), Seattle, WA, USA (July 2016)Google Scholar
  11. 11.
    Qi, F., Zhao, D., Fan, X., Jiang, T.: Stereoscopic video quality assessment based on visual attention and just-noticeable difference models. Signal Image Video Process. 10, 737–744 (2016)CrossRefGoogle Scholar
  12. 12.
    Şenol, E., Özbek, N.: Quality of experience measurement of compressed multi-view video. Signal Process. Image Commun. 57, 147–156 (2017)CrossRefGoogle Scholar
  13. 13.
    Recommendation ITU-R P.910: Subjective Video Quality Assessment Methods for Multimedia Applications. ITU-R Recommendations (2008)Google Scholar
  14. 14.
    Bosc, E., Riou, P., Pressigout, M., Morin, L.: Bit-rate allocation between texture and depth: influence of data sequence characteristics. In: 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), 2012 (2012)Google Scholar
  15. 15.
    Bosc, E., Racapé, F., Jantet, V., Riou, P., Pressigout, M., Morin, L.: A study of depth/texture bit-rate allocation in multi-view video plus depth compression. Ann. Telecommun. 68(11–12), 615–625 (2013)CrossRefGoogle Scholar
  16. 16.
    Vetro, A., Pandit, P., Kimata, H., Smolic, A., Wang, Y.-K.: Joint Draft 8.0 on Multiview Video Coding. JVT-AB204 (2008)Google Scholar
  17. 17.
    Sullivan, G.J., Ohm, J.-R., Han, J.-R., Wiegand, T.: Overview of highly efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(7), 1649–1668 (2012)CrossRefGoogle Scholar
  18. 18.
    Liu, Y., Huang, Q., Ma, S., Zhao, D., Gao, W., Ci, S., Tang, H.: A novel rate control technique for multiview video plus depth based 3D video coding. IEEE Trans. Broadcast. 57(2), 562–571 (2011)CrossRefGoogle Scholar
  19. 19.
    Sullivan, G.J., Boyce, J.M., Chen, Y., Ohm, J.-R., Segall, C.A., Vetro, A.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Top. Signal Process. 7(6), 1001–1016 (2013)CrossRefGoogle Scholar
  20. 20.
    Zhang, L., Tech, G., Wegner, K., Yea, S.: 3D-HEVC Test Model 5. JCT-3V (2013)Google Scholar
  21. 21.
    Rusanovskyy, D., Müller, K., Vetro, A.: Common Test Condition of 3DV Core Experiments. JCT-3V, Stockholm (2012)Google Scholar
  22. 22.
    Li, B., Li, H., Li, L., Zhang, J.: Rate control by R-lambda model for HEVC. In: ITU-T SG16 Contribution, JCTVC-K0103, pp. 1–5, Shanghai (2012)Google Scholar
  23. 23.
    Tanimoto Laboratory: FTV Test Sequences, Nagoya University, JP. http://www.tanimoto.nuee.nagoya-u.ac.jp/. Accessed 1 Apr 2018
  24. 24.
    Fraunhofer Heinrich Hertz Institute: 3D-HEVC Reference Software [Online] (2015). https://hevc.hhi.fraunhofer.de/3dhevc. Accessed 1 Apr 2018
  25. 25.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  26. 26.
    Li, J., Kaller, O., Simone, F.D., Hakala, J., Juzska, D., Callet, P.L.: Cross-lab study on preference of experience in 3DTV: influence from display technology and test environment. 5th IEEE QoMEX Workshop: Quality of Multimedia Experience (2013)Google Scholar
  27. 27.
    Recommendation ITU-R BT.2021: Subjective Methods for the Assessment of Stereoscopic 3DTV Systems. ITU-R Recommendations (2012)Google Scholar
  28. 28.
    Li, J., Barkowsky, M., Le Callet, P.: Analysis and improvement of a paired comparison method in the application of 3DTV subjective experiment. In: IEEE International Conference on Image Processing (ICIP 2012), pp. 1–4 (2012)Google Scholar
  29. 29.
    Lee, J., De Simone, F., Ebrahimi, T.: Subjective quality evaluation via paired comparison: application to scalable video coding. IEEE Trans. Multimed. 13(5), 882–893 (2011)CrossRefGoogle Scholar
  30. 30.
    Lee, J., Goldmann, L., Ebrahimi, T.: Paired comparison-based subjective quality assessment of stereoscopic images. Multimed. Tools Appl. 67, 31–48 (2012)CrossRefGoogle Scholar
  31. 31.
    Li, J., Barkowsky, M., Callet, P.L.: Subjective assessment methodology for preference of experience in 3dtv. 11th IEEE IVMSP Workshop: 3D Image/Video Technologies and Applications, Jun 2013, Seoul, South Korea, pp. 1–4 (2013)Google Scholar
  32. 32.
    Bradley, R., Terry, M.: Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39(3/4), 324–345 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Şenol, E., Özbek, N.: Time-efficient subjective testing methodology for 3D video quality assessment. In: Proceedings of IEEE International Conference on Image Processing (ICIP’16), Phoenix, Arizona, USA (Sept. 2016)Google Scholar
  34. 34.
    Wickelmaier, F., Schmid, C.: A matlab function to estimate choice model parameters from paired comparison data. Behav. Res. Methods 36(1), 29–40 (2004)CrossRefGoogle Scholar
  35. 35.
    De Silva, V., Arachchi, H.K., Ekmekcioglu, E., Kondoz, A.: Toward an impairment metric for stereoscopic video: a full-reference video quality metric to assess compressed stereoscopic video. IEEE Trans. Image Process. 22(9), 3392–3404 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Bjøntegaard, G.: Calculation of average PSNR differences between RD curves, document VCEG-M33. Presented at the VCEG Meeting, Austin, TX, USA (Apr. 2001)Google Scholar
  37. 37.
    Aflaki, P., Hannuksela, M.M., Gabbouj, M.: Subjective quality assessment of asymmetric stereoscopic 3D video. Signal Image Video Process. 9, 331–345 (2015)CrossRefGoogle Scholar
  38. 38.
    Su, T., Javadtalab, A., Yassine, A., Shirmohammadi, S.: A DASH-based 3d multi-view video rate control system. In: International Conference on Signal Processing and Communication Systems (ICSPCS), no. 8, pp. 14–19 (2014)Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electrical & Electronics EngineeringEge UniversityIzmirTurkey

Personalised recommendations