Spherical image QoE approximations for vision augmentation scenarios

  • B. Bauman
  • P. SeelingEmail author


Augmented Reality (AR) devices are commonly head-worn to overlay context-dependent information into the field of view of the device operators. One particular scenario is the overlay of still images, for which we evaluate the interplay of user ratings as Quality of Experience (QoE) with (i) the non-referential BRISQUE objective image quality metric as Quality of Service (QoS) and (ii) human subject dry electrode EEG signals gathered with a commercial off-the-shelf device. We employ basic machine learning approaches to perform QoE and QoS predictions based on this data. We find strong correlations for QoS inputs with aggregated user ratings as Mean Opinion Scores with spherical images. For subject-specific EEG portfolios, overall predictability of the QoE for both media types can be attained. Our overall results can be employed in practical scenarios by content and network service providers to optimize the user experience in augmented reality scenarios with a passive human in-the-loop in the future.


Augmented reality Quality of experience Image quality Quality of service Electroencephalography 



This material is based upon work supported by the Faculty Research and Creative Endeavors (FRCE) program at Central Michigan University under grant #48146.


  1. 1.
    Acqualagna L, Bosse S, Porbadnigk AK, Curio G, Müller KR, Wiegand T, Blankertz B (2015) EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs). J Neural Eng 12(2):026012CrossRefGoogle Scholar
  2. 2.
    Antons JN, Arndt S, Schleicher R, Möller S (2014) Brain activity correlates of quality of experience. In: Möller S, Raake M (eds) Quality of experience advanced concepts, applications and methods. Springer International Publishing., pp 109–119
  3. 3.
    Arampatzis A, Kamps J (2009) A signal-to-noise approach to score normalization. In: Proceedings of the 18th ACM conference on information and knowledge management, CIKM ’09. ACM, New York
  4. 4.
    Arnau-Gonzalez P, Althobaiti T, Katsigiannis S, Ramzan N (2017) Perceptual video quality evaluation by means of physiological signals. In: 2017 9th international conference on quality of multimedia experience (qoMEX), pp 1–6.
  5. 5.
    Bauman B, Seeling P (2017) Towards still image experience predictions in augmented vision settings. In: Proceedings of the IEEE consumer communications and networking conference (CCNC). Las Vegas, NV, USA, pp 1–6Google Scholar
  6. 6.
    Bauman B, Seeling P (2017) Visual interface evaluation for wearables datasets: predicting the subjective augmented vision image qoe and qos. Future Internet 9(3):40CrossRefGoogle Scholar
  7. 7.
    Blankertz B, Acqualagna L, Dähne S, Haufe S, Schultze-Kraft M, Sturm I, Ušćumlic M, Wenzel MA, Curio G, Müller KR (2016) The Berlin brain-computer interface: progress beyond communication and control. Frontiers in Neuroscience, p 10Google Scholar
  8. 8.
    Bosse S, Müller KR, Wiegand T, Samek W (2016) Brain-computer interfacing for multimedia quality assessment. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC)., pp 002834–002839
  9. 9.
    Brunnström K, Beker SA, De Moor K, Dooms A, Egger S, Garcia MN, Hossfeld T, Jumisko-Pyykkö S, Keimel C, Larabi C et al (2013) Qualinet white paper on definitions of quality of experienceGoogle Scholar
  10. 10.
    Chen Y, Wu K, Zhang Q (2015) From qos to qoe: a tutorial on video quality assessment. IEEE Commun Surv Tutorials 17(2):1126–1165CrossRefGoogle Scholar
  11. 11.
    Chikkerur S, Sundaram V, Reisslein M, Karam LJ (2011) Objective video quality assessment methods: a classification, review, and performance comparison. IEEE Trans Broadcast 57(2):165–182. CrossRefGoogle Scholar
  12. 12.
    Davis P, Creusere CD, Kroger J (2016) The effect of perceptual video quality on EEG power distribution. In: 2016 IEEE international conference on image processing (ICIP). Phoenix, AZ, USA, pp 2420–2424Google Scholar
  13. 13.
    Eiris Pereira R, Moore HF, Gheisari M, Esmaeili B (2019) Development and usability testing of a panoramic augmented reality environment for fall hazard safety training. In: Mutis I, Hartmann T (eds) Advances in informatics and computing in civil and construction engineering. Springer International Publishing, Cham, pp 271–279Google Scholar
  14. 14.
    Engelke U, Nguyen H, Ketchell S (2017) Quality of augmented reality experience: a correlation analysis. In: 2017 9th international conference on quality of multimedia experience (qoMEX), pp 1–3, DOI, (to appear in print)
  15. 15.
    Fiedler M, Hossfeld T, Tran-Gia P (2010) A generic quantitative relationship between quality of experience and quality of service. IEEE Netw 24(2):36–41CrossRefGoogle Scholar
  16. 16.
    Han Y, Yuan Z, Muntean GM (2016) An innovative no-reference metric for real-time 3d stereoscopic video quality assessment. IEEE Trans Broadcast 62(3):654–663. CrossRefGoogle Scholar
  17. 17.
    He L, Gao F, Hou W, Hao L (2014) Objective image quality assessment: a survey. Int J Comput Math 91(11):2374–2388. MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Holm A, Lukander K, Korpela J, Sallinen M, Müller K (2009) Estimating brain load from the eeg. Sci World J 9:639–651CrossRefGoogle Scholar
  19. 19.
    Irshad S, Awang Rambli DR, Muhamad Nazri NIA, Mohd Shukri SRb, Omar Y (2018) Measuring user experience of mobile augmented reality systems through non-instrumental quality attributes. In: Abdullah N, Wan Adnan WA, Foth M (eds) User science and engineering. Springer Singapore, Singapore, pp 349–357Google Scholar
  20. 20.
    ITU-T (2008) Recommendation ITU-T P.910: subjective video quality assessment methods for multimedia applicationsGoogle Scholar
  21. 21.
    ITU-T (2012) Recommendation ITU-R BT.500-13: methodology for the subjective assessment of the quality of television pictureGoogle Scholar
  22. 22.
    Jain R (1991) The art of computer systems performance analysis: techniques for experimental design, measurement, simulation, and modeling. WileyGoogle Scholar
  23. 23.
    Kroupi E, Hanhart P, Lee JS, Rerabek M, Ebrahimi T (2014) Eeg correlates during video quality perception. In: 2014 22nd European signal processing conference (EUSIPCO), pp 2135–2139Google Scholar
  24. 24.
    Kumar N, Kumar J (2016) Measurement of cognitive load in hci systems using eeg power spectrum: an experimental study. Procedia Comput Sci 84:70–78. CrossRefGoogle Scholar
  25. 25.
    Lin W, Kuo CCJ (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312. CrossRefGoogle Scholar
  26. 26.
    Lindemann L, Magnor MA (2011) Assessing the quality of compressed images using EEG. In: Proceedings of the IEEE international conference on image processing (ICIP), pp 3109–3112.
  27. 27.
    Mazher M, Aziz AA, Malik AS, Amin HU (2017) An eeg-based cognitive load assessment in multimedia learning using feature extraction and partial directed coherence. IEEE Access 5:14819–14829CrossRefGoogle Scholar
  28. 28.
    Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708. MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Moldovan AN, Ghergulescu I, Weibelzahl S, Muntean CH (2013) User-centered eeg-based multimedia quality assessment. In: 2013 IEEE international symposium on broadband multimedia systems and broadcasting (BMSB)., pp 1–8
  30. 30.
    Moldovan AN, Ghergulescu I, Muntean CH (2016) Vqamap: a novel mechanism for mapping objective video quality metrics to subjective mos scale. IEEE Trans Broadcast 62(3):610–627. CrossRefGoogle Scholar
  31. 31.
    Moon SE, Lee JS (2017) Implicit analysis of perceptual multimedia experience based on physiological response: a review. IEEE Trans Multimed 19(2):340–353. CrossRefGoogle Scholar
  32. 32.
    Nagelkerke N (1991) A note on a general definition of the coefficient of determination. Biometrika 78(3):691MathSciNetCrossRefGoogle Scholar
  33. 33.
    Pan C, Xu Y, Yan Y, Gu K, Yang X (2016) Exploiting neural models for no-reference image quality assessment. In: 2016 visual communications and image processing (VCIP), pp 1–4. IEEEGoogle Scholar
  34. 34.
    Perrin AFNM, Xu H, Kroupi E, Řeřábek M, Ebrahimi T (2015) Multimodal dataset for assessment of quality of experience in immersive multimedia. In: Proceedings of the 23rd ACM international conference on multimedia, MM ’15. ACM, New York, pp 1007–1010
  35. 35.
    Plechawska-Wójcik M, Wawrzyk M, Wesołowska K, Kaczorowska M, Tokovarov M, Dmytruk R, Borys M (2017) Eeg spectral analysis of human cognitive workload study. Studia Informatica 38(2):17–30Google Scholar
  36. 36.
    Reichl P, Tuffin B, Schatz R (2013) Logarithmic laws in service quality perception: where microeconomics meets psychophysics and quality of experience. Telecommun Syst 52(2):587–600Google Scholar
  37. 37.
    Rein S, Fitzek FHP, Reisslein M (2005) Voice quality evaluation in wireless packet communication systems: a tutorial and performance results for rhc. IEEE Wirel Commun 12(1):60–67. CrossRefGoogle Scholar
  38. 38.
    Scholler S, Bosse S, Treder MS, Blankertz B, Curio G, Mueller KR, Wiegand T (2012) Toward a direct measure of video quality perception using EEG. IEEE Trans Image Process 21(5):2619–2629MathSciNetCrossRefGoogle Scholar
  39. 39.
    Seeling P (2015) Augmented vision and quality of experience assessment: Towards a unified evaluation framework. In: Proceedings of IEEE ICC workshop on quality of experience-based management for future internet applications and services (qoe-FI). London, United KingdomGoogle Scholar
  40. 40.
    Seeling P (2015) Towards quality of experience determination for video in augmented binocular vision scenarios. Signal Process Image Commun 33:41–50CrossRefGoogle Scholar
  41. 41.
    Seeling P (2016) Visual user experience difference: image compression impacts on the quality of experience in augmented binocular vision. In: Proceedings of IEEE consumer communications and networking conference (CCNC). Las Vegas, NV, USA, pp 931–936Google Scholar
  42. 42.
    Shahid M, Rossholm A, Lövström B, Zepernick HJ (2014) No-reference image and video quality assessment: a classification and review of recent approaches. EURASIP J Image Video Process 2014(1):40. CrossRefGoogle Scholar
  43. 43.
    Staelens N, Wallendael GV, Crombecq K, Vercammen N, Cock JD, Vermeulen B, de Walle RV, Dhaene T, Demeester P (2012) No-reference bitstream-based visual quality impairment detection for high definition h.264/avc encoded video sequences. IEEE Trans Broadcast 58(2):187–199. CrossRefGoogle Scholar
  44. 44.
    Xiao X, Ni LM (1999) Internet qos: a big picture. IEEE Netw 13(2):8–18. MathSciNetCrossRefGoogle Scholar
  45. 45.
    You J, Reiter U, Hannuksela MM, Gabbouj M, Perkis A (2010) Perceptual-based quality assessment for audio-visual services: a survey. Signal Process Image Commun 25:482–501CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceCentral Michigan UniversityMount PleasantUSA

Personalised recommendations