Skip to main content
SpringerLink
Log in

How Passive Image Viewers Became Active Multimedia Users

New Trends and Recent Advances in Subjective Assessment of Quality of Experience

  • Chapter
  • First Online:
Visual Signal Quality Assessment

Abstract

Subjective assessment of quality of experience (QoE) is key to understanding user preferences with respect to multimedia fruition. As such, it is a necessary step to multimedia delivery optimization, since QoE needs to take into account technology limitations as well as user satisfaction. The study of QoE appreciation dates back to the twentieth century, when it exploded with the advent of CRT first and LCD displays later. For a long time, this branch of research was targeted at determining user sensitivity to impairments induced in the media by suboptimal delivery. The media recipient was considered a passive observer, whose appreciation of the video material was determined primarily by the degree of annoyance due to the impairments affecting it. With the advent of mobile technology and Internet-based media delivery, this impairment-centric concept of QoE has shown to be incomplete. The media recipient became an active user who creates content, interacts with the system, and selects the media he/she wants to have delivered. As a result, elements such as visual semantics, user personality, preferences and intent, social and environmental context of media fruition also concur to the final experience assessment. The role played by these elements in QoE and the cognitive/affective processes that underlie them are still to be understood, although several models of QoE appreciation have already been proposed. In this paper, we review the evolution of subjective QoE assessment and models from the impairment-centric approach to a more user-centric approach. We analyze relevant features and factors influencing QoE, and point out future directions for subjective QoE assessment research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Nicola Adami, Alberto Signoroni, and Riccardo Leonardi. State-of-the-art and trends in scalable video compression with wavelet-based approaches. Circuits and Systems for Video Technology, IEEE Transactions on, 17(9):1238–1255, 2007.

    Google Scholar 

  2. Hani Alers, Judith Redi, Hantao Liu, and Ingrid Heynderickx. Studying the effect of optimizing image quality in salient regions at the expense of background content. Journal of Electronic Imaging, 22(4):043012–043012, 2013.

    Article  Google Scholar 

  3. John Allnatt. Transmitted-picture assessment. Wiley Chichester, UK, 1983.

    Google Scholar 

  4. Munisamy Anandan. Progress of led backlights for lcds. Journal of the Society for Information Display, 16(2):287–310, 2008.

    Article  Google Scholar 

  5. Javed Asghar, Francois Le Faucheur, and Ian Hood. Preserving video quality in iptv networks. Broadcasting, IEEE Transactions on, 55(2):386–395, 2009.

    Google Scholar 

  6. Simon Attfield, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. Towards a science of user engagement (position paper). In WSDM Workshop on User Modelling for Web Applications, 2011.

    Google Scholar 

  7. Roberto Baldwin. Google glasses face serious hurdles, augmented-reality experts say. Wired Magazine, 2012.

    Google Scholar 

  8. Peter GJ Barten. Contrast sensitivity of the human eye and its effects on image quality, volume 72. SPIE press, 1999.

    Google Scholar 

  9. Soren Bech, Roelof Hamberg, Marco Nijenhuis, Kees Teunissen, Henny Looren de Jong, Paul Houben, and Sakti K Pramanik. Rapid perceptual image description (rapid) method. In Electronic Imaging: Science & Technology, pages 317–328. International Society for Optics and Photonics, 1996.

    Google Scholar 

  10. Steve Benford, Chris Greenhalgh, Tom Rodden, and James Pycock. Collaborative virtual environments. Communications of the ACM, 44(7):79–85, 2001.

    Article  Google Scholar 

  11. Daniel E Berlyne. Aesthetics and psychobiology. 1971.

    Google Scholar 

  12. Cheryl Campanella Bracken. Presence and image quality: The case of high-definition television. Media Psychology, 7(2):191–205, 2005.

    Google Scholar 

  13. Margaret M Bradley, Maurizio Codispoti, Dean Sabatinelli, and Peter J Lang. Emotion and motivation ii: sex differences in picture processing. Emotion, 1(3):300, 2001.

    Google Scholar 

  14. Frances Brassington and Stephen Pettitt. Principles of marketing. FT Prentice Hall, 2005.

    Google Scholar 

  15. Joost Broekens and Willem-Paul Brinkman. Affectbutton: A method for reliable and valid affective self-report. International Journal of Human-Computer Studies, 71(6):641–667, 2013.

    Article  Google Scholar 

  16. Barry Brown and Marek Bell. Play and sociability in there: Some lessons from online games for collaborative virtual environments. In Avatars at Work and Play, pages 227–245. Springer, 2006.

    Google Scholar 

  17. ITUR BT2020. Parameter values for ultra-high definition television systems for production and international program exchange, 2012.

    Google Scholar 

  18. Diane Carr, Gareth Schott, Andrew Burn, and David Buckingham. Doing game studies: A multi-method approach to the study of textuality, interactivity and narrative space. Media International Australia, Incorporating Culture & Policy, (110):19, 2004.

    Google Scholar 

  19. Joseph A Castellano. Handbook of display technology. Elsevier, 1992.

    Google Scholar 

  20. Cathleen D Cerosaletti, Alexander C Loui, and Andrew C Gallagher. Investigating two features of aesthetic perception in consumer photographic images: clutter and center. In IS&T/SPIE Electronic Imaging, pages 786507–786507. International Society for Optics and Photonics, 2011.

    Google Scholar 

  21. Damon M Chandler. Seven challenges in image quality assessment: past, present, and future research. ISRN Signal Processing, 2013, 2013.

    Google Scholar 

  22. Kuan-Ta Chen, Chen-Chi Wu, Yu-Chun Chang, and Chin-Laung Lei. A crowdsourceable qoe evaluation framework for multimedia content. In Proceedings of the 17th ACM international conference on Multimedia, pages 491–500. ACM, 2009.

    Google Scholar 

  23. Konstantinos Chorianopoulos and George Lekakos. Introduction to social tv: Enhancing the shared experience with interactive tv. Intl. Journal of Human–Computer Interaction, 24(2):113–120, 2008.

    Google Scholar 

  24. I Cisco. Cisco visual networking index: Forecast and methodology, 2011–2016. CISCO White paper, pages 2011–2016, 2012.

    Google Scholar 

  25. T Coppens, Frie Vanparijs, and K Handekyn. Amigotv: A social tv experience through triple-play convergence. Alcatel Technology white paper, 2005.

    Google Scholar 

  26. Paul T Costa and Robert R McCrae. The revised neo personality inventory (neo-pi-r). The SAGE handbook of personality theory and assessment, 2:179–198, 2008.

    Google Scholar 

  27. Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z Wang. Studying aesthetics in photographic images using a computational approach. In Computer Vision–ECCV 2006, pages 288–301. Springer, 2006.

    Google Scholar 

  28. Herbert Aron David. The method of paired comparisons, volume 12. DTIC Document, 1963.

    Google Scholar 

  29. Huib de Ridder. Subjective evaluation of scale-space image coding. In Electronic Imaging’91, San Jose, CA, pages 31–42. International Society for Optics and Photonics, 1991.

    Google Scholar 

  30. Huib de Ridder. Minkowski-metrics as a combination rule for digital-image-coding impairments. In SPIE/IS&T 1992 Symposium on Electronic Imaging: Science and Technology, pages 16–26. International Society for Optics and Photonics, 1992.

    Google Scholar 

  31. Huib de Ridder. Naturalness and image quality: saturation and lightness variation in color images of natural scenes. Journal of imaging science and technology, 40(6):487–493, 1996.

    Google Scholar 

  32. Huib de Ridder. Cognitive issues in image quality measurement. Journal of Electronic Imaging, 10(1):47–55, 2001.

    Article  MathSciNet  Google Scholar 

  33. Huib de Ridder, Frans JJ Blommaert, and Elena A Fedorovskaya. Naturalness and image quality: chroma and hue variation in color images of natural scenes. In IS&T/SPIE’s Symposium on Electronic Imaging: Science & Technology, pages 51–61. International Society for Optics and Photonics, 1995.

    Google Scholar 

  34. Carlo Demichelis and Philip Chimento. Ip packet delay variation metric for ip performance metrics (ippm). 2002.

    Google Scholar 

  35. Robert Desimone and John Duncan. Neural mechanisms of selective visual attention. Annual review of neuroscience, 18(1):193–222, 1995.

    Article  Google Scholar 

  36. Anhai Doan, Raghu Ramakrishnan, and Alon Y Halevy. Crowdsourcing systems on the world-wide web. Communications of the ACM, 54(4):86–96, 2011.

    Google Scholar 

  37. Florin Dobrian, Vyas Sekar, Asad Awan, Ion Stoica, Dilip Joseph, Aditya Ganjam, Jibin Zhan, and Hui Zhang. Understanding the impact of video quality on user engagement. ACM SIGCOMM Computer Communication Review, 41(4):362–373, 2011.

    Article  Google Scholar 

  38. Alexander Eichhorn, Pengpeng Ni, and Ragnhild Eg. Randomised pair comparison: an economic and robust method for audiovisual quality assessment. In Proceedings of the 20th international workshop on Network and operating systems support for digital audio and video, pages 63–68. ACM, 2010.

    Google Scholar 

  39. S.N. Endrikhovskij. Image quality and colour categorization. Colour image sci-ence: exploiting digital media, pages 363–382, 2002.

    Google Scholar 

  40. Peter G Engeldrum. Psychometric scaling: a toolkit for imaging systems development. Imcotek Press, 2000.

    Google Scholar 

  41. Peter G Engeldrum. A theory of image quality: The image quality circle. Journal of imaging science and technology, 48(5):447–457, 2004.

    Google Scholar 

  42. Ulrich Engelke, Hagen Kaprykowsky, H Zepernick, and Patrick Ndjiki-Nya. Visual attention in quality assessment. Signal Processing Magazine, IEEE, 28(6):50–59, 2011.

    Google Scholar 

  43. Ulrich Engelke, Romuald Pepion, Patrick Le Callet, and Hans-Jürgen Zepernick. Linking distortion perception and visual saliency in h. 264/avc coded video containing packet loss. In Visual Communications and Image Processing 2010, pages 774406–774406. International Society for Optics and Photonics, 2010.

    Google Scholar 

  44. Mylene CQ Farias and Sanjit K Mitra. No-reference video quality metric based on artifact measurements. In Image Processing, 2005. ICIP 2005. IEEE International Conference on, volume 3, pages III–141. IEEE, 2005.

    Google Scholar 

  45. Mylène CQ Farias, Sanjit K Mitra, and John M Foley. Perceptual contributions of blocky, blurry and noisy artifacts to overall annoyance. In Multimedia and Expo, 2003. ICME’03. Proceedings. 2003 International Conference on, volume 1, pages I–529. IEEE, 2003.

    Google Scholar 

  46. Elena Fedorovskaya, Carman Neustaedter, and Wei Hao. Image harmony for consumer images. In Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on, pages 121–124. IEEE, 2008.

    Google Scholar 

  47. Elena A Fedorovskaya and Huib De Ridder. Subjective matters: from image quality to image psychology. In IS&T/SPIE Electronic Imaging, pages 86510O–86510O. International Society for Optics and Photonics, 2013.

    Google Scholar 

  48. Rony Ferzli and Lina J Karam. A no-reference objective image sharpness metric based on the notion of just noticeable blur (jnb). Image Processing, IEEE Transactions on, 18(4):717–728, 2009.

    Google Scholar 

  49. Markus Fiedler, Tobias Hossfeld, and Phuoc Tran-Gia. A generic quantitative relationship between quality of experience and quality of service. Network, IEEE, 24(2):36–41, 2010.

    Google Scholar 

  50. David Gauntlett and Annette Hill. TV living: Television, culture and everyday life. Routledge, 2002.

    Google Scholar 

  51. George Ghinea and JT Thomas. Quality of perception: user quality of service in multimedia presentations. IEEE Transactions on Multimedia, 7(4):786–789, 2005.

    Article  Google Scholar 

  52. WILFRED A Gibson. A least-squares solution for case iv of the law of comparative judgment. Psychometrika, 18(1):15–21, 1953.

    Google Scholar 

  53. Bernd Girod. What’s wrong with mean-squared error? In Digital images and human vision, pages 207–220. MIT press, 1993.

    Google Scholar 

  54. Lutz Goldmann, Francesca De Simone, Frederic Dufaux, Touradj Ebrahimi, Rudolf Tanner, and Mauro Lattuada. Impact of video transcoding artifacts on the subjective quality. In Quality of Multimedia Experience (QoMEX), 2010 Second International Workshop on, pages 52–57. IEEE, 2010.

    Google Scholar 

  55. James J Gross and Robert W Levenson. Emotion elicitation using films. Cognition & Emotion, 9(1):87–108, 1995.

    Google Scholar 

  56. Stephen R Gulliver and George Ghinea. Stars in their eyes: What eye-tracking reveals about multimedia perceptual quality. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 34(4):472–482, 2004.

    Google Scholar 

  57. Stephen R Gulliver and Gheorghita Ghinea. Defining user perception of distributed multimedia quality. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), 2(4):241–257, 2006.

    Google Scholar 

  58. Stephen R Gulliver and Gheorghita Ghinea. The perceptual and attentive impact of delay and jitter in multimedia delivery. Broadcasting, IEEE Transactions on, 53(2):449–458, 2007.

    Google Scholar 

  59. Jukka Häkkinen, Takashi Kawai, Jari Takatalo, Tuomas Leisti, Jenni Radun, Anni Hirsaho, and Göte Nyman. Measuring stereoscopic image quality experience with interpretation based quality methodology. In Electronic Imaging 2008, pages 68081B–68081B. International Society for Optics and Photonics, 2008.

    Google Scholar 

  60. Andrew M Haun and Eli Peli. Is image quality a function of contrast perception? In IS&T/SPIE Electronic Imaging, pages 86510C–86510C. International Society for Optics and Photonics, 2013.

    Google Scholar 

  61. Sheila S Hemami and Amy R Reibman. No-reference image and video quality estimation: Applications and human-motivated design. Signal processing: Image communication, 25(7):469–481, 2010.

    Google Scholar 

  62. T Hobfeld, Raimund Schatz, and Sebastian Egger. Sos: The mos is not enough! In Quality of Multimedia Experience (QoMEX), 2011 Third International Workshop on, pages 131–136. IEEE, 2011.

    Google Scholar 

  63. Donald Hoffman. The interface theory of perception: Natural selection drives true perception to swift extinction. Object categorization: Computer and human vision perspectives, pages 148–165, 2009.

    Google Scholar 

  64. Robyn M Holmes and Anthony D Pellegrini. Children’s social behavior during video game play. Handbook of Computer Game Studies. The MIT Press, Cambridge, Massachusetts, 2005.

    Google Scholar 

  65. T Hoßfeld, C Keimel, M Hirth, B Gardlo, J Habigt, K Diepold, and P Tran-Gia. Crowdtesting: a novel methodology for subjective user studies and qoe evaluation. University of Würzburg, Tech. Rep, 486, 2013.

    Google Scholar 

  66. Jeff Howe. The rise of crowdsourcing. Wired magazine, 14(6):1–4, 2006.

    MathSciNet  Google Scholar 

  67. Ming-Hui Huang. Designing website attributes to induce experiential encounters. computers in Human Behavior, 19(4):425–442, 2003.

    Google Scholar 

  68. Quan Huynh-Thu, M-N Garcia, Filippo Speranza, Philip Corriveau, and Alexander Raake. Study of rating scales for subjective quality assessment of high-definition video. Broadcasting, IEEE Transactions on, 57(1):1–14, 2011.

    Google Scholar 

  69. Mansoor Hyder, Noel Crespi, Michael Haun, Christian Hoene, et al. Are qoe requirements for multimedia services different for men and women? analysis of gender differences in forming qoe in virtual acoustic environments. In Emerging Trends and Applications in Information Communication Technologies, pages 200–209. Springer, 2012.

    Google Scholar 

  70. Selim Ickin, Katarzyna Wac, Markus Fiedler, Lucjan Janowski, Jin-Hyuk Hong, and Anind K Dey. Factors influencing quality of experience of commonly used mobile applications. Communications Magazine, IEEE, 50(4):48–56, 2012.

    Google Scholar 

  71. Wijnand A IJsselsteijn, Huib de Ridder, and Roelof Hamberg. Perceptual factors in stereoscopic displays: the effect of stereoscopic filming parameters on perceived quality and reported eyestrain. In Photonics West’98 Electronic Imaging, pages 282–291. International Society for Optics and Photonics, 1998.

    Google Scholar 

  72. Wijnand A IJsselsteijn, Huib de Ridder, and Joyce Vliegen. Subjective evaluation of stereoscopic images: effects of camera parameters and display duration. Circuits and Systems for Video Technology, IEEE Transactions on, 10(2):225–233, 2000.

    Google Scholar 

  73. P ITU-T RECOMMENDATION. Subjective video quality assessment methods for multimedia applications. 1999.

    Google Scholar 

  74. Ruud Janssen. Computational image quality, volume 101. SPIE press, 2001.

    Google Scholar 

  75. TJWM Janssen and FJJ Blommaert. Image quality semantics. Journal of imaging science and Technology, 41(5):555–560, 1997.

    Google Scholar 

  76. Wei Jiang, Alexander C Loui, and Cathleen Daniels Cerosaletti. Automatic aesthetic value assessment in photographic images. In Multimedia and Expo (ICME), 2010 IEEE International Conference on, pages 920–925. IEEE, 2010.

    Google Scholar 

  77. Dhiraj Joshi, Ritendra Datta, Elena Fedorovskaya, Quang-Tuan Luong, James Z Wang, Jia Li, and Jiebo Luo. Aesthetics and emotions in images. Signal Processing Magazine, IEEE, 28(5):94–115, 2011.

    Google Scholar 

  78. Satu Jumisko-Pyykkö. User-centered quality of experience and its evaluation methods for mobile television. PhD thesis, Doctoral thesis, Tampere University of Technology, Tampere, 2011.

    Google Scholar 

  79. Satu Jumisko-Pyykkö and Miska M Hannuksela. Does context matter in quality evaluation of mobile television? In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services, pages 63–72. ACM, 2008.

    Google Scholar 

  80. Sandeep Kanumuri, Pamela C Cosman, Amy R Reibman, and Vinay A Vaishampayan. Modeling packet-loss visibility in mpeg-2 video. Multimedia, IEEE Transactions on, 8(2):341–355, 2006.

    Google Scholar 

  81. Brian Keelan. Handbook of image quality: characterization and prediction. CRC Press, 2002.

    Google Scholar 

  82. Brian W Keelan and Hitoshi Urabe. Iso 20462: A psychophysical image quality measurement standard. In Electronic Imaging 2004, pages 181–189. International Society for Optics and Photonics, 2003.

    Google Scholar 

  83. Kalevi Kilkki. Quality of experience in communications ecosystem. J. UCS, 14(5):615–624, 2008.

    Google Scholar 

  84. Hyun-Jong Kim, Dong Hyeon Lee, Jong Min Lee, Kyoung-Hee Lee, Won Lyu, and Seong-Gon Choi. The qoe evaluation method through the qos-qoe correlation model. In Networked Computing and Advanced Information Management, 2008. NCM’08. Fourth International Conference on, volume 2, pages 719–725. IEEE, 2008.

    Google Scholar 

  85. Hendrik Knoche and John McCarthy. Mobile users” needs and expectations of future multimedia services. Technical report, 2004.

    Google Scholar 

  86. Hendrik Knoche and M Angela Sasse. Getting the big picture on small screens: Quality of experience in mobile tv. Technical report, Information Science Reference, 2008.

    Google Scholar 

  87. Hendrik Knoche and M Angela Sasse. The big picture on small screens delivering acceptable video quality in mobile tv. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), 5(3):20, 2009.

    Google Scholar 

  88. Jan Koenderink. Vision as a user interface. In IS&T/SPIE Electronic Imaging, pages 786504–786504. International Society for Optics and Photonics, 2011.

    Google Scholar 

  89. Jari Korhonen, Claire Mantel, Nino Burini, and Soren Forchhammer. Searching for the preferred backlight intensity in liquid crystal displays with local backlight dimming. In Quality of Multimedia Experience (QoMEX), 2013 Fifth International Workshop on, pages 118–123. IEEE, 2013.

    Google Scholar 

  90. Philip Kortum and Marc Sullivan. The effect of content desirability on subjective video quality ratings. Human factors: the journal of the human factors and ergonomics society, 52(1):105–118, 2010.

    Article  Google Scholar 

  91. Robert Kubey and Mihalyi Csikszentmihalyi. Television and the quality of life: How viewing shapes everyday experience. Routledge, 2013.

    Google Scholar 

  92. Andre Kuijsters, Wijnand A Ijsselsteijn, Marc TM Lambooij, and Ingrid EJ Heynderickx. Influence of chroma variations on naturalness and image quality of stereoscopic images. In IS&T/SPIE Electronic Imaging, pages 72401E–72401E. International Society for Optics and Photonics, 2009.

    Google Scholar 

  93. Wen-Hung Kuo, Po-Hung Lin, and Sheue-Ling Hwang. A framework of perceptual quality assessment on lcd-tv. Displays, 28(1):35–43, 2007.

    Article  Google Scholar 

  94. Marc Lambooij, Marten Fortuin, Ingrid Heynderickx, and Wijnand IJsselsteijn. Visual discomfort and visual fatigue of stereoscopic displays: a review. Journal of Imaging Science and Technology, 53(3):30201–1, 2009.

    Google Scholar 

  95. Marc Lambooij, Wijnand IJsselsteijn, Don G Bouwhuis, and Ingrid Heynderickx. Evaluation of stereoscopic images: beyond 2d quality. Broadcasting, IEEE Transactions on, 57(2):432–444, 2011.

    Google Scholar 

  96. Peter J Lang. Behavioral treatment and bio-behavioral assessment: Computer applications. 1980.

    Google Scholar 

  97. Peter J Lang, Margaret M Bradley, Bruce N Cuthbert, et al. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. NIMH, Center for the Study of Emotion & Attention, 2005.

    Google Scholar 

  98. Julie Lassalle, Laetitia Gros, Thierry Morineau, and Gilles Coppin. Impact of the content on subjective evaluation of audiovisual quality: What dimensions influence our perception? In Broadband Multimedia Systems and Broadcasting (BMSB), 2012 IEEE International Symposium on, pages 1–6. IEEE, 2012.

    Google Scholar 

  99. Patrick Le Callet, Sebastian Möller, Andrew Perkis, et al. Qualinet white paper on definitions of quality of experience. European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003), 2012.

    Google Scholar 

  100. Olivier Le Meur, Alexandre Ninassi, Patrick Le Callet, and Dominique Barba. Do video coding impairments disturb the visual attention deployment? Signal Processing: Image Communication, 25(8):597–609, 2010.

    Google Scholar 

  101. Barbara Lee and Robert S Lee. How and why people watch tv: Implications for the future of interactive television. Journal of advertising research, 35(6):9–18, 1995.

    Google Scholar 

  102. Jong-Seok Lee, Francesca De Simone, and Touradj Ebrahimi. Subjective quality evaluation via paired comparison: application to scalable video coding. Multimedia, IEEE Transactions on, 13(5):882–893, 2011.

    Google Scholar 

  103. Jong-Seok Lee, Francesca De Simone, Naeem Ramzan, Zhijie Zhao, Engin Kurutepe, Thomas Sikora, Jörn Ostermann, Ebroul Izquierdo, and Touradj Ebrahimi. Subjective evaluation of scalable video coding for content distribution. In Proceedings of the international conference on Multimedia, pages 65–72. ACM, 2010.

    Google Scholar 

  104. Congcong Li and Tsuhan Chen. Aesthetic visual quality assessment of paintings. Selected Topics in Signal Processing, IEEE Journal of, 3(2):236–252, 2009.

    Google Scholar 

  105. Weisi Lin and C-C Jay Kuo. Perceptual visual quality metrics: A survey. Journal of Visual Communication and Image Representation, 22(4):297–312, 2011.

    Google Scholar 

  106. Hantao Liu and Ingrid Heynderickx. A perceptually relevant no-reference blockiness metric based on local image characteristics. EURASIP Journal on Advances in Signal Processing, 2009:2, 2009.

    Google Scholar 

  107. Hantao Liu and Ingrid Heynderickx. Visual attention in objective image quality assessment: based on eye-tracking data. Circuits and Systems for Video Technology, IEEE Transactions on, 21(7):971–982, 2011.

    Google Scholar 

  108. James Lull et al. Inside family viewing: ethnographic research on television’s audiences. Routledge, 1990.

    Google Scholar 

  109. Yiwen Luo and Xiaoou Tang. Photo and video quality evaluation: Focusing on the subject. In Computer Vision–ECCV 2008, pages 386–399. Springer, 2008.

    Google Scholar 

  110. Wendy Ann Mansilla, Andrew Perkis, and Touradj Ebrahimi. Implicit experiences as a determinant of perceptual quality and aesthetic appreciation. In Proceedings of the 19th ACM international conference on Multimedia, pages 153–162. ACM, 2011.

    Google Scholar 

  111. Claire Mantel, Nathalie Guyader, Patricia Ladret, Gelu Ionescu, and Thomas Kunlin. Characterizing eye movements during temporal and global quality assessment of h. 264 compressed video sequences. In IS&T/SPIE Electronic Imaging, pages 82910Y–82910Y. International Society for Optics and Photonics, 2012.

    Google Scholar 

  112. John D McCarthy, M Angela Sasse, and Dimitrios Miras. Sharp or smooth?: comparing the effects of quantization vs. frame rate for streamed video. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 535–542. ACM, 2004.

    Google Scholar 

  113. Ricky KP Mok, Edmond WW Chan, and Rocky KC Chang. Measuring the quality of experience of http video streaming. In Integrated Network Management (IM), 2011 IFIP/IEEE International Symposium on, pages 485–492. IEEE, 2011.

    Google Scholar 

  114. Anush Krishna Moorthy and Alan Conrad Bovik. Visual quality assessment algorithms: what does the future hold? Multimedia Tools and Applications, 51(2):675–696, 2011.

    Google Scholar 

  115. Margaret Morrison and Dean M Krugman. A look at mass and computer mediated technologies: Understanding the roles of television and computers in the home. Journal of Broadcasting & Electronic Media, 45(1):135–161, 2001.

    Google Scholar 

  116. JH Morrissey. New method for the assignment of psychometric scale values from incomplete paired comparisons. JOSA, 45(5):373–378, 1955.

    Article  Google Scholar 

  117. Niall Murray, Yuansong Qiao, Brian Lee, Gabriel-Miro Muntean, and AK Karunakar. Age and gender influence on perceived olfactory & visual media synchronization. In Multimedia and Expo (ICME), 2013 IEEE International Conference on, pages 1–6. IEEE, 2013.

    Google Scholar 

  118. Anja B Naumann, Ina Wechsung, and Jörn Hurtienne. Multimodal interaction: A suitable strategy for including older users? Interacting with Computers, 22(6):465–474, 2010.

    Google Scholar 

  119. A Neil. Autostereoscopic 3d displays. Computer, 8:32–36, 2005.

    Google Scholar 

  120. MRM Nijenhuis. Sampling, interpolation, images: a perceptual view. PhD thesis, Eindhoven University of technology, Eindhoven, 1993.

    Google Scholar 

  121. MRM Nijenhuis and FJJ Blommaert. Perceptual error measure for sampled and interpolated images. Journal of Imaging Science and Technology, 41(3):249–258, 1997.

    Google Scholar 

  122. Alexandre Ninassi, Olivier Le Meur, Patrick Le Callet, and Dominique Barba. Considering temporal variations of spatial visual distortions in video quality assessment. Selected Topics in Signal Processing, IEEE Journal of, 3(2):253–265, 2009.

    Google Scholar 

  123. Alexandre Ninassi, Olivier Le Meur, Patrick Le Callet, Dominique Barba, Arnaud Tirel, et al. Task impact on the visual attention in subjective image quality assessment. In Proceedings of European Signal Processing Conference, 2006.

    Google Scholar 

  124. Heather L O’Brien and Elaine G Toms. What is user engagement? a conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology, 59(6):938–955, 2008.

    Google Scholar 

  125. Lora Oehlberg, Nicolas Ducheneaut, James D Thornton, Robert J Moore, and Eric Nickell. Social tv: Designing for distributed, sociable television viewing. In Proc. EuroITV, volume 2006, pages 25–26, 2006.

    Google Scholar 

  126. Joana Palhais, Rui S Cruz, and Mário S Nunes. Quality of experience assessment in internet tv. In Mobile Networks and Management, pages 261–274. Springer, 2012.

    Google Scholar 

  127. Thrasyvoulos N Pappas, Robert J Safranek, and Junqing Chen. Perceptual criteria for image quality evaluation. Handbook of image and video processing, pages 669–684, 2000.

    Google Scholar 

  128. Fernando Pereira. Sensations, perceptions and emotions towards quality of experience evaluation for consumer electronics video adaptations. In Int. Workshop on Video Processing and Quality Metrics for Consumer Electronics, 2005.

    Google Scholar 

  129. Pablo Pérez, Jesús Macías, Jaime J Ruiz, and Narciso García. Effect of packet loss in video quality of experience. Bell Labs Technical Journal, 16(1):91–104, 2011.

    Google Scholar 

  130. Lawrence A Pervin and Oliver P John. Handbook of personality: Theory and research. Elsevier, 1999.

    Google Scholar 

  131. Kandaraj Piamrat, Cesar Viho, J Bonnin, and Adlen Ksentini. Quality of experience measurements for video streaming over wireless networks. In Information Technology: New Generations, 2009. ITNG’09. Sixth International Conference on, pages 1184–1189. IEEE, 2009.

    Google Scholar 

  132. Scott Plous. The psychology of judgment and decision making. Mcgraw-Hill Book Company, 1993.

    Google Scholar 

  133. Jordi Puig, Andrew Perkis, Frank Lindseth, and Touradj Ebrahimi. Towards an efficient methodology for evaluation of quality of experience in augmented reality. In Quality of Multimedia Experience (QoMEX), 2012 Fourth International Workshop on, pages 188–193. IEEE, 2012.

    Google Scholar 

  134. Jenni Radun, Tuomas Leisti, Jukka Häkkinen, Harri Ojanen, Jean-Luc Olives, Tero Vuori, and Göte Nyman. Content and quality: Interpretation-based estimation of image quality. ACM Transactions on Applied Perception (TAP), 4(4):2, 2008.

    Google Scholar 

  135. Benjamin Rainer, Markus Waltl, Eva Cheng, Muawiyath Shujau, Christian Timmerer, Stephen Davis, Ian Burnett, Christian Ritz, and Hermann Hellwagner. Investigating the impact of sensory effects on the quality of experience and emotional response in web videos. In Quality of Multimedia Experience (QoMEX), 2012 Fourth International Workshop on, pages 278–283. IEEE, 2012.

    Google Scholar 

  136. JC Read, SJ MacFarlane, and Chris Casey. Endurability, engagement and expectations: Measuring children’s fun. In Interaction design and children, volume 2, pages 1–23. Shaker Publishing Eindhoven, 2002.

    Google Scholar 

  137. ITU-R BT Recommendation. 2021, subjective methods for the assessment stereoscopic 3dtv systems. International Telecommunication Union, Geneva, Switzerland, 2012.

    Google Scholar 

  138. ITURBT Recommendation. 500-11, methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union, Geneva, Switzerland, 4:2, 2002.

    Google Scholar 

  139. ITUT Recommendation. E. 800: Terms and definitions related to quality of service and network performance including dependability. ITU-T 2008, 2008.

    Google Scholar 

  140. Judith Redi, Ingrid Heynderickx, Bruno Macchiavello, and Mylene Farias. On the impact of packet-loss impairments on visual attention mechanisms. In Circuits and Systems (ISCAS), 2013 IEEE International Symposium on, pages 1107–1110. IEEE, 2013.

    Google Scholar 

  141. Judith Redi, Hantao Liu, Hani Alers, Rodolfo Zunino, and Ingrid Heynderickx. Comparing subjective image quality measurement methods for the creation of public databases. In IS&T/SPIE Electronic Imaging, pages 752903–752903. International Society for Optics and Photonics, 2010.

    Google Scholar 

  142. Judith Redi, Hantao Liu, Paolo Gastaldo, Rodolfo Zunino, and Ingrid Heynderickx. How to apply spatial saliency into objective metrics for jpeg compressed images? In Image Processing (ICIP), 2009 16th IEEE International Conference on, pages 961–964. IEEE, 2009.

    Google Scholar 

  143. Judith Redi, Hantao Liu, Rodolfo Zunino, and Ingrid Heynderickx. Interactions of visual attention and quality perception. In IS&T/SPIE Electronic Imaging, pages 78650S–78650S. International Society for Optics and Photonics, 2011.

    Google Scholar 

  144. Judith A Redi. Visual quality beyond artifact visibility. In IS&T/SPIE Electronic Imaging, pages 86510N–86510N. International Society for Optics and Photonics, 2013.

    Google Scholar 

  145. Judith A Redi and Ingrid Heynderickx. Image integrity and aesthetics: towards a more encompassing definition of visual quality. In IS&T/SPIE Electronic Imaging, pages 829115–829115. International Society for Optics and Photonics, 2012.

    Google Scholar 

  146. Judith A Redi and Isabel Povoa. The role of visual attention in the aesthetic appeal of consumer images: A preliminary study. In Visual Communications and Image Processing (VCIP), 2013, pages 1–6. IEEE, 2013.

    Google Scholar 

  147. Judith Alice Redi, Tobias Hoßfeld, Pavel Korshunov, Filippo Mazza, Isabel Povoa, and Christian Keimel. Crowdsourcing-based multimedia subjective evaluations: a case study on image recognizability and aesthetic appeal. In Proceedings of the 2nd ACM international workshop on Crowdsourcing for multimedia, pages 29–34. ACM, 2013.

    Google Scholar 

  148. Ulrich Reiter and Katrien De Moor. Content categorization based on implicit and explicit user feedback: combining self-reports with eeg emotional state analysis. In Quality of multimedia experience (QoMEX), 2012 fourth international workshop on, pages 266–271. IEEE, 2012.

    Google Scholar 

  149. Huib Ridder and Serguei Endrikhovski. 33.1: Invited paper: image quality is fun: reflections on fidelity, usefulness and naturalness. In SID Symposium Digest of Technical Papers, volume 33, pages 986–989. Wiley Online Library, 2002.

    Google Scholar 

  150. Raimund Schatz, Siegfried Wagner, Sebastian Egger, and Norbert Jordan. Mobile tv becomes social-integrating content with communications. In Information Technology Interfaces, 2007. ITI 2007. 29th International Conference on, pages 263–270. IEEE, 2007.

    Google Scholar 

  151. Jose A Scheinkman. Social interactions. The New Palgrave Dictionary of Economics, 2, 2008.

    Google Scholar 

  152. Dimitri Schuurman, Katrien De Moor, Lieven De Marez, and Tom Evens. A living lab research approach for mobile tv. Telematics and Informatics, 28(4):271–282, 2011.

    Article  Google Scholar 

  153. Eric WK See-To, Savvas Papagiannidis, and Vincent Cho. User experience on mobile video appreciation: How to engross users and to enhance their enjoyment in watching mobile video clips. Technological Forecasting and Social Change, 79(8):1484–1494, 2012.

    Google Scholar 

  154. Helge Seetzen, Wolfgang Heidrich, Wolfgang Stuerzlinger, Greg Ward, Lorne Whitehead, Matthew Trentacoste, Abhijeet Ghosh, and Andrejs Vorozcovs. High dynamic range display systems. In ACM Transactions on Graphics (TOG), volume 23, pages 760–768. ACM, 2004.

    Google Scholar 

  155. Pieter Seuntiens, Lydia Meesters, and Wijnand Ijsselsteijn. Perceived quality of compressed stereoscopic images: Effects of symmetric and asymmetric jpeg coding and camera separation. ACM Transactions on Applied Perception (TAP), 3(2):95–109, 2006.

    Article  Google Scholar 

  156. Pieter Seuntiens, Ingrid Vogels, and Arnold van Keersop. Visual experience of 3d-tv with pixelated ambilight. Proceedings of PRESENCE, 2007, 2007.

    Google Scholar 

  157. Pieter J Seuntiëns, Ingrid E Heynderickx, Wijnand A IJsselsteijn, Paul MJ van den Avoort, Jelle Berentsen, Iwan J Dalm, Marc T Lambooij, and Willem Oosting. Viewing experience and naturalness of 3d images. In Optics East 2005, pages 601605–601605. International Society for Optics and Photonics, 2005.

    Google Scholar 

  158. Mario Siller and John Woods. Improving quality of experience for multimedia services by qos arbitration on a qoe framework. In in Proc. of the 13th Packed Video Workshop 2003. Citeseer, 2003.

    Google Scholar 

  159. Paul J Silvia. Interest – the curious emotion. Current Directions in Psychological Science, 17(1):57–60, 2008.

    Google Scholar 

  160. Joshua A Solomon, Andrew B Watson, and Albert Ahumada. Visibility of dct basis functions: Effects of contrast masking. In Data Compression Conference, 1994. DCC’94. Proceedings, pages 361–370. IEEE, 1994.

    Google Scholar 

  161. Nicolas Staelens, Stefaan Moens, Wendy Van den Broeck, Ilse Marien, Brecht Vermeulen, Peter Lambert, Rik Van de Walle, and Piet Demeester. Assessing quality of experience of iptv and video on demand services in real-life environments. Broadcasting, IEEE Transactions on, 56(4):458–466, 2010.

    Google Scholar 

  162. Nicolas Staelens, Glenn Van Wallendael, Karel Crombecq, Nick Vercammen, Jan De Cock, Brecht Vermeulen, Rik Van de Walle, Tom Dhaene, and Piet Demeester. No-reference bitstream-based visual quality impairment detection for high definition h. 264/avc encoded video sequences. Broadcasting, IEEE Transactions on, 58(2):187–199, 2012.

    Google Scholar 

  163. Wa James Tam, Lew B Stelmach, and Philip J Corriveau. Psychovisual aspects of viewing stereoscopic video sequences. In Photonics West’98 Electronic Imaging, pages 226–235. International Society for Optics and Photonics, 1998.

    Google Scholar 

  164. KT Tan, Mohammed Ghanbari, and Donald E Pearson. An objective measurement tool for mpeg video quality. Signal Processing, 70(3):279–294, 1998.

    Google Scholar 

  165. Cornelis Teunissen. Flat panel display characterization: a perceptual approach. 2009.

    Google Scholar 

  166. Louis L Thurstone. A law of comparative judgment. Psychological review, 34(4):273, 1927.

    Google Scholar 

  167. International Telecommunication Union. Itu-t rec. 109: Definition of quality of experience (qoe). Liaison Statement, Ref.: TD 109rev2 (PLEN/12), 2007.

    Google Scholar 

  168. Vanessa Vakili, Willem-Paul Brinkman, and Mark A Neerincx. Lessons learned from the development of technological support for ptsd prevention: a review. Stud Health Technol Inform, 181:22–6, 2012.

    Google Scholar 

  169. Patricia Valdez and Albert Mehrabian. Effects of color on emotions. Journal of Experimental Psychology: General, 123(4):394, 1994.

    Google Scholar 

  170. Martin Varela, Toni Maki, Lea Skorin-Kapov, and Tobias Hossfeld. Towards an understanding of visual appeal in website design. In Quality of Multimedia Experience (QoMEX), 2013 Fifth International Workshop on, pages 70–75. IEEE, 2013.

    Google Scholar 

  171. Elena Vicario, Ingrid Heynderickx, Giulio Ferretti, and Paola Carrai. 17.1: Design of a tool to benchmark scaling algorithms on lcd monitors. In SID Symposium Digest of Technical Papers, volume 33, pages 704–707. Wiley Online Library, 2002.

    Google Scholar 

  172. Ingrid MLC Vogels. How to make life more colorful: from image quality to atmosphere experience. In Color and Imaging Conference, volume 2009, pages 123–128. Society for Imaging Science and Technology, 2009.

    Google Scholar 

  173. ECL Vu and DM Chandler. Visual fixation patterns when judging image quality: Effects of distortion type, amount, and subject experience. In Image Analysis and Interpretation, 2008. SSIAI 2008. IEEE Southwest Symposium on, pages 73–76. IEEE, 2008.

    Google Scholar 

  174. Tero Vuori, Maria Olkkonen, Monika Pölönen, Ari Siren, and Jukka Häkkinen. Can eye movements be quantitatively applied to image quality studies? In Proceedings of the third Nordic conference on Human-computer interaction, pages 335–338. ACM, 2004.

    Google Scholar 

  175. Demin Wang, Filippo Speranza, Andre Vincent, Taali Martin, and Phil Blanchfield. Toward optimal rate control: a study of the impact of spatial resolution, frame rate, and quantization on subjective video quality and bit rate. In Visual Communications and Image Processing 2003, pages 198–209. International Society for Optics and Photonics, 2003.

    Google Scholar 

  176. Junle Wang, Damon M Chandler, and Patrick Le Callet. Quantifying the relationship between visual salience and visual importance. In IS&T/SPIE Electronic Imaging, pages 75270K–75270K. International Society for Optics and Photonics, 2010.

    Google Scholar 

  177. Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. Image Processing, IEEE Transactions on, 13(4):600–612, 2004.

    Google Scholar 

  178. Zhou Wang and Xinli Shang. Spatial pooling strategies for perceptual image quality assessment. In Image Processing, 2006 IEEE International Conference on, pages 2945–2948. IEEE, 2006.

    Google Scholar 

  179. Andrew B Watson. Efficiency of a model human image code. JOSA A, 4(12):2401–2417, 1987.

    Google Scholar 

  180. Ina Wechsung, Matthias Schulz, Klaus-Peter Engelbrecht, Julia Niemann, and Sebastian Möller. All users are (not) equal-the influence of user characteristics on perceived quality, modality choice and performance. In Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop, pages 175–186. Springer, 2011.

    Google Scholar 

  181. Amaya Becvar Weddle and Hua Yu. How does audio-haptic enhancement influence emotional response to mobile media? In Quality of Multimedia Experience (QoMEX), 2013 Fifth International Workshop on, pages 158–163. IEEE, 2013.

    Google Scholar 

  182. J.H.D.M. Westerink. Influences of subject expertise in quality assessment of digitally coded images. In SID International Symposium Digest of Technical Papers, volume 20, pages 124–127. SID, 1989.

    Google Scholar 

  183. Joyce HDM Westerink and Jacques AJ Roufs. Subjective image quality as a function of viewing distance, resolution, and picture size. SMPTE journal, 98(2):113–119, 1989.

    Google Scholar 

  184. Stefan Winkler and Subramanian Ramanathan. Overview of eye tracking datasets. In QoMEX, pages 212–217, 2013.

    Google Scholar 

  185. K Maria Wolters, Klaus-Peter Engelbrecht, Florian Gödde, Sebastian Möller, Anja Naumann, and Robert Schleicher. Making it easier for older people to talk to smart homes: the effect of early help prompts. Universal Access in the Information Society, 9(4):311–325, 2010.

    Google Scholar 

  186. Chen-Chi Wu, Kuan-Ta Chen, Yu-Chun Chang, and Chin-Laung Lei. Crowdsourcing multimedia qoe evaluation: A trusted framework. IEEE transactions on multimedia, 15(5):1121–1137, 2013.

    Article  Google Scholar 

  187. Qianqian Xu, Qingming Huang, Tingting Jiang, Bowei Yan, Weisi Lin, and Yuan Yao. Hodgerank on random graphs for subjective video quality assessment. Multimedia, IEEE Transactions on, 14(3):844–857, 2012.

    Google Scholar 

  188. Qianqian Xu, Qingming Huang, and Yuan Yao. Online crowdsourcing subjective image quality assessment. In Proceedings of the 20th ACM international conference on Multimedia, pages 359–368. ACM, 2012.

    Google Scholar 

  189. Qianqian Xu, Jiechao Xiong, Qingming Huang, and Yuan Yao. Robust evaluation for quality of experience in crowdsourcing. In Proceedings of the 21st ACM international conference on Multimedia, pages 43–52. ACM, 2013.

    Google Scholar 

  190. Kyoko Yamori and Yoshiaki Tanaka. Relation between willingness to pay and guaranteed minimum bandwidth in multiple-priority service. In Communications, 2004 and the 5th International Symposium on Multi-Dimensional Mobile Communications Proceedings. The 2004 Joint Conference of the 10th Asia-Pacific Conference on, volume 1, pages 113–117. IEEE, 2004.

    Google Scholar 

  191. Akiko Yoshida, Volker Blanz, Karol Myszkowski, and Hans-Peter Seidel. Perceptual evaluation of tone mapping operators with real-world scenes. In Electronic Imaging 2005, pages 192–203. International Society for Optics and Photonics, 2005.

    Google Scholar 

  192. Ai Yoto, Tetsuo Katsuura, Koichi Iwanaga, and Yoshihiro Shimomura. Effects of object color stimuli on human brain activities in perception and attention referred to eeg alpha band response. Journal of Physiological Anthropology, 26(3):373–379, 2007.

    Article  Google Scholar 

  193. H van Zee and DW Kaandorp. Kwaliteit en degradatie. ipo rapport no. 853. Technical report, Institute for Perception Research, Eindhoven, 1992.

    Google Scholar 

  194. Thomas Zinner, Oliver Hohlfeld, Osama Abboud, and Tobias Hoßfeld. Impact of frame rate and resolution on objective qoe metrics. In Quality of Multimedia Experience (QoMEX), 2010 Second International Workshop on, pages 29–34. IEEE, 2010.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Intelligent Systems, Delft University of Technology, Delft, The Netherlands

    Judith A. Redi & Yi Zhu

  2. Department of Industrial Design, Delft University of Technology, Delft, The Netherlands

    Huib de Ridder

  3. Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology, Eindhoven, The Netherlands

    Ingrid Heynderickx

  4. Philips Research Laboratories, Eindhoven, The Netherlands

    Ingrid Heynderickx

Authors
  1. Judith A. Redi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huib de Ridder
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ingrid Heynderickx
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Judith A. Redi .

Editor information

Editors and Affiliations

  1. School of Information and Electronics, Beijing Institute of Technology, Beijing, China

    Chenwei Deng

  2. Huawei Noah’s Ark Lab, Quarry Bay, Hong Kong SAR

    Lin Ma

  3. School of Computer Engineering, Nanyang Technological University, Singapore

    Weisi Lin

  4. Department of Electronic Engineering, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong SAR

    King Ngi Ngan

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Redi, J.A., Zhu, Y., de Ridder, H., Heynderickx, I. (2015). How Passive Image Viewers Became Active Multimedia Users. In: Deng, C., Ma, L., Lin, W., Ngan, K. (eds) Visual Signal Quality Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-10368-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-10368-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-10367-9

  • Online ISBN: 978-3-319-10368-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation