Vision and Video: Models and Applications

  • Stefan Winkler
  • Murat Kunt
  • Christian J. van den Branden Lambrecht


While traditional analog systems still form the vast majority of television sets today, production studios, broadcasters and network providers have been installing digital video equipment at an ever-increasing rate. The border line between analog and digital video is moving closer and closer to the consumer. Digital satellite and cable service have been available for a while, and recently terrestrial digital television broadcast has been introduced in a number of locations around the world.


Video Quality Contrast Sensitivity Video Code Human Visual System Perceptual Distortion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. J. Ahumada, Jr., B. L. Beard, R. Eriksson: “Spatio-temporal discrimination model predicts temporal masking function.” in Proc. SPIE, vol. 3299, pp. 120–127, San Jose, CA, 1998.Google Scholar
  2. [2]
    D. G. Albrecht, W. S. Geisler: “Motion selectivity and the contrast-response function of simple cells in the visual cortex.” Vis. Neurosci. 7: 531–546, 1991.CrossRefGoogle Scholar
  3. [3]
    V. Baroncini, A. Pierotti: “Single-ended objective quality assessment of DTV” in Proc. SPIE, vol. 3845, Boston, MA, 1999.Google Scholar
  4. [4]
    V. Bhaskaran, K. Konstantinides: Image and Video Compression Standards. Algorithms and Architectures. Kluwer Academic Publishers, 2nd edn., 1997.Google Scholar
  5. [5]
    P. Bretillon, J. Baina: “Method for image quality monitoring on digital television networks.” in Proc. SPIE, vol. 3845, Boston, MA, 1999.Google Scholar
  6. [6]
    C. A. Burbeck, D. H. Kelly: “Spatiotemporal characteristics of visual mechanisms: Excitatory-inhibitory model.” J. Opt. Soc. Am. 70(9):11211126, 1980.Google Scholar
  7. [7]
    T. Carney, S. A. Klein, Q. Hu: “Visual masking near spatiotemporal edges.” in Proc. SPIE, vol. 2657, pp. 393–402, San Jose, CA, 1996.Google Scholar
  8. [8]
    G. R. Cole, C. F. Stromeyer, III., R. E. Kronauer: “Visual interactions with luminance and chromatic stimuli.” J. Opt. Soc. Am. A 7 (1): 128–140, 1990.Google Scholar
  9. [9]
    S. Comes, B. Macq, M. Mattavelli: “Postprocessing of images by filtering the unmasked coding noise.” IEEE Trans. Image Processing 8(8):10501062, 1999.Google Scholar
  10. [10]
    D. Costantini et al.: “Motion rendition quality metric for MPEG coded video.” in Proc. ICIP, vol. 1, pp. 889–892, Lausanne, Switzerland, 1996.Google Scholar
  11. [11]
    I. Dalgiç, H. Fang: “Comparison of H.323 and SIP for internet telephony signaling.” in Proc. SPIE, vol. 3845, Boston, MA, 1999.Google Scholar
  12. [12]
    S. Daly: “The visible differences predictor: An algorithm for the assessment of image fidelity.” in Digital Images and Human Vision, ed. A. B. Watson, pp. 179–206, MIT Press, 1993.Google Scholar
  13. [13]
    S. Daly: “Engineering observations from spatiovelocity and spatiotemporal visual models.” in Proc. SPIE, vol. 3299, pp. 180–191, San Jose, CA, 1998.Google Scholar
  14. [14]
    J. G. Daugman: “Two-dimensional spectral analysis of cortical receptive field profiles.” Vision Res. 20 (10): 847–856, 1980.CrossRefGoogle Scholar
  15. [15]
    G. de Haan, E. B. Betters: “Deinterlacing — an overview.” Proc. IEEE 86 (9): 1839–1857, 1998.CrossRefGoogle Scholar
  16. [16]
    J. M. Foley: “Human luminance pattern-vision mechanisms: Masking experiments require a new model.” J. Opt. Soc. Am. A 11 (6): 1710–1719, 1994.CrossRefGoogle Scholar
  17. [17]
    K. H. Foster et al.: “Spatial and temporal frequency selectivity of neurons in visual cortical areas V1 and V2 of the macaque monkey.” J. Physiol. 365: 331–363, 1985.Google Scholar
  18. [18]
    R. E. Fredericksen, R. F. Hess: “Temporal detection in human vision: Dependence on stimulus energy.” J. Opt. Soc. Am. A 14 (10): 2557–2569, 1997.Google Scholar
  19. [19]
    R. E. Fredericksen, R. F. Hess: “Estimating multiple temporal mechanisms in human vision.” Vision Res. 38 (7): 1023–1040, 1998.CrossRefGoogle Scholar
  20. [20]
    P. Frossard, O. Verscheure: Joint Source/FEC Rate Selection for Quality-Optimal MPEG-2 Video Delivery. Tech. Rep. 1999–04, Signal Processing Lab, Swiss Federal Institute of Technology, Lausanne, 1999.Google Scholar
  21. [21]
    B. Girod: “The information theoretical significance of spatial and temporal masking in video signals.” in Proc. SPIE, vol. 1077, pp. 178–187, Los Angeles, CA, 1989.Google Scholar
  22. [22]
    M. W. Greenlee, J. P. Thomas: “Effect of pattern adaptation on spatial frequency discrimination.” J. Opt. Soc. Am. A 9 (6): 857–862, 1992.Google Scholar
  23. [23]
    T. Hamada, S. Miyaji, S. Matsumoto: “Picture quality assessment system by three-layered bottom-up noise weighting considering human visual perception.” SMPTE J. 108(1):20–26 1999.Google Scholar
  24. [24]
    S. T. Hammett, A. T. Smith: “Two temporal channels or three? A reevaluation.” Vision Res. 32 (2): 285–291, 1992.CrossRefGoogle Scholar
  25. [25]
    D. J. Heeger: “Half-squaring in responses of cat striate cells.” Vis. Neurosci. 9: 427–443, 1992.CrossRefGoogle Scholar
  26. [26]
    D. J. Heeger: “Normalization of cell responses in cat striate cortex” Vis. Neurosci. 9: 181–197, 1992.CrossRefGoogle Scholar
  27. [27]
    R. F. Hess, R. J. Snowden: “Temporal properties of human visual filters: Number, shapes and spatial covariation.” Vision Res. 32 (1): 47–59, 1992.CrossRefGoogle Scholar
  28. [28]
    ITU-R Recommendation BT.500–10: “Methodology for the subjective assessment of the quality of television pictures.” ITU, Geneva, Switzerland, 2000.Google Scholar
  29. [29]
    ITU-R Recommendation BT.601–5: “Studio encoding parameters of digital television for standard 4:3 and wide-screen 16:9 aspect ratios.” ITU, Geneva, Switzerland, 1995.Google Scholar
  30. [30]
    ITU-T Recommendation H.323: “Visual telephone systems and equipment for local area networks which provide a non-guaranteed quality of service.” ITU, Geneva, Switzerland, 1998.Google Scholar
  31. [31]
    D. H. Kelly: “Motion and vision. I. Stabilized images of stationary gratings.” J. Opt. Soc. Am. 69 (9): 1266–1274, 1979.Google Scholar
  32. [32]
    D. H. Kelly: “Motion and vision. II. Stabilized spatio-temporal threshold surface.” J. Opt. Soc. Am. 69 (10): 1340–1349, 1979.CrossRefGoogle Scholar
  33. [33]
    D. H. Kelly: “Spatiotemporal variation of chromatic and achromatic contrast thresholds.” J. Opt. Soc. Am. 73 (6): 742–750, 1983.Google Scholar
  34. [34]
    J. J. Koenderink, A. J. van Doorn: “Spatiotemporal contrast detection threshold surface is bimodal.” Opt. Letters 4 (1): 32–34, 1979.CrossRefGoogle Scholar
  35. [35]
    P. Lindh, C. J. van den Branden Lambrecht: “Efficient spatio-temporal decomposition for perceptual processing of video sequences.” in Proc. ICIP, vol. 3, pp. 331–334, Lausanne, Switzerland, 1996.Google Scholar
  36. [36]
    M. A. Losada, K. T. Mullen: “The spatial tuning of chromatic mechanisms identified by simultaneous masking.” Vision Res. 34 (3): 331–341, 1994.CrossRefGoogle Scholar
  37. [37]
    J. Lubin, D. Fibush: “Sarnoff JND vision model.” T1A1.5 Working Group Document #97–612, ANSI T1 Standards Committee, 1997.Google Scholar
  38. [38]
    F. X. J. Lukas, Z. L. Budrikis: “Picture quality prediction based on a visual model.” IEEE Trans. Comm. 30 (7): 1679–1692, 1982.CrossRefGoogle Scholar
  39. [39]
    M. B. Mandler, W. Makous: “A three-channel model of temporal frequency perception.” Vision Res. 24 (12): 1881–1887, 1984.CrossRefGoogle Scholar
  40. [40]
    MOSAIC: A New Single Stimulus Quality Assessment Methodology. RACE R2111, 1996.Google Scholar
  41. [41]
    E. Peli: “Contrast in complex images.” J. Opt. Soc. Am. A 7(10):20322040, 1990.Google Scholar
  42. [42]
    E. Peli: “In search of a contrast metric: Matching the perceived contrast of Gabor patches at different phases and bandwidths.” Vision Res. 37 (23): 3217–3224, 1997.CrossRefGoogle Scholar
  43. [43]
    A. B. Poirson, B. A. Wandell: “Appearance of colored patterns: Pattern-color separability.” J. Opt. Soc. Am. A 10 (12): 2458–2470, 1993.Google Scholar
  44. [44]
    A. B. Poirson, B. A. Wandell: “Pattern-color separable pathways predict sensitivity to simple colored patterns” Vision Res. 36 (4): 515–526, 1996.CrossRefGoogle Scholar
  45. [45]
    C. Poynton: “The rehabilitation of gamma” in Proc. SPIE, vol. 3299, pp. 232–249, San Jose, CA, 1998.Google Scholar
  46. [46]
    R. F. Quick, Jr.: “A vector-magnitude model of contrast detection.” Kybernetik 16: 65–67, 1974.CrossRefGoogle Scholar
  47. [47]
    J. G. Robson: “Spatial and temporal contrast-sensitivity functions of the visual system.” J. Opt. Soc. Am. 56: 1141–1142, 1966.CrossRefGoogle Scholar
  48. [48]
    A. M. Rohaly et al.: “Video Quality Experts Group: Current results and future directions.” in Proc. SPIE, vol. 4067, Perth, Australia, 2000.Google Scholar
  49. [49]
    J. Ross, H. D. Speed: “Contrast adaptation and contrast masking in human vision.” Proc. R. Soc. Lond. B 246: 61–70, 1991.CrossRefGoogle Scholar
  50. [50]
    H. Schulzrinne, J. Rosenberg: “Internet telephony: Architecture and protocols — an IETF perspective.” Computer Networks and ISDN Systems 31: 237–255, 1999.Google Scholar
  51. [51]
    H. Schulzrinne et al.: RTP: A Transport Protocol for Real-Time Applications. Tech. Rep. RFC 1889, IETF, 1996.Google Scholar
  52. [52]
    A. J. Seyler, Z. L. Budrikis: “Measurements of temporal adaptation to spatial detail vision.” Nature 184: 1215–1217, 1959.CrossRefGoogle Scholar
  53. [53]
    A. J. Seyler, Z. L. Budrikis: “Detail perception after scene changes in television image presentations.” IEEE Trans. Inform. Theory 11 (1): 3143, 1965.CrossRefGoogle Scholar
  54. [54]
    E. P. Simoncelli et al.: “Shiftable multi-scale transforms.” IEEE Trans. Inform. Theory 38 (2): 587–607, 1992.MathSciNetCrossRefGoogle Scholar
  55. [55]
    R. J. Snowden, S. T. Hammett: “Spatial frequency adaptation: Threshold elevation and perceived contrast” Vision Res. 36 (12): 1797–1809, 1996.CrossRefGoogle Scholar
  56. [56]
    E. Switkes, A. Bradley, K. K. De Valois: “Contrast dependence and mechanisms of masking interactions among chromatic and luminance gratings.” J. Opt. Soc. Am. A 5(7):1149–1162,1988.Google Scholar
  57. [57]
    W. J. Tam et al.: “Visual masking at video scene cuts.” in Proc. SPIE, vol. 2411, pp. 111–119, San Jose, CA, 1995.Google Scholar
  58. [58]
    P. C. Teo, D. J. Heeger: “Perceptual image distortion.” in Proc. SPIE, vol. 2179, pp. 127–141, San Jose, CA, 1994.Google Scholar
  59. [59]
    G. Thomas: “A comparison of motion-compensated interlace-toprogressive conversion methods.” Signal Processing: Image Communication 12 (3): 209–229, 1998.CrossRefGoogle Scholar
  60. [60]
    C. J. van den Branden Lambrecht: Perceptual Models and Architectures for Video Coding Applications. Ph.D. thesis, École Polytechnique Fédérale de Lausanne, Switzerland, 1996.Google Scholar
  61. [61]
    C. J. van den Branden Lambrecht et al.: “Automatically assessing MPEG coding fidelity.” IEEE Design and Test Magazine 12 (4): 28–33, 1995.CrossRefGoogle Scholar
  62. [62]
    O. Verscheure: User-Oriented QoS in MPEG-2 Video Delivery. Ph.D. thesis, École Polytechnique Fédérale de Lausanne, Switzerland, 1999.Google Scholar
  63. [63]
    VQEG: “Final report from the Video Quality Experts Group on the validation of objective models of video quality assessment.” 2000, available at Scholar
  64. [64]
    A. B. Watson: “Temporal sensitivity.” in Handbook of Perception and Human Performance, eds. K. R. Boff, L. Kaufman, J. P. Thomas, vol. 1, chap. 6, John Wiley Sons, 1986.Google Scholar
  65. [65]
    A. B. Watson: “DCTune: A technique for visual optimization of DCT quantization matrices for individual images.” in SID Symposium Digest, vol. 24, pp. 946–949, 1993.Google Scholar
  66. [66]
    A. B. Watson: “Toward a perceptual video quality metric.” in Proc. SPIE, vol. 3299, pp. 139–147, San Jose, CA, 1998.Google Scholar
  67. [67]
    A. B. Watson, J. A. Solomon: “Model of visual contrast gain control and pattern masking.” J. Opt. Soc. Am. A 14 (9): 2379–2391, 1997.Google Scholar
  68. [68]
    M. A. Webster, E. Miyahara: “Contrast adaptation and the spatial structure of natural images.” J. Opt. Soc. Am. A 14 (9): 2355–2366, 1997.Google Scholar
  69. [69]
    M. A. Webster, J. D. Mollon: “Adaptation and the color statistics of natural images.” Vision Res. 37 (23): 3283–3298, 1997.CrossRefGoogle Scholar
  70. [70]
    S. J. P. Westen, R. L. Lagendijk, J. Biemond: “Spatio-temporal model of human vision for digital video compression” in Proc. SPIE, vol. 3016, pp. 260–268, San Jose, CA, 1997.Google Scholar
  71. [71]
    H. R. Wilson, R. Humanski: “Spatial frequency adaptation and contrast gain control.” Vision Res. 33 (8): 1133–1149, 1993.CrossRefGoogle Scholar
  72. [72]
    S. Winkler: “A perceptual distortion metric for digital color images.” in Proc. ICIP, vol. 3, pp. 399–403, Chicago, IL, 1998.Google Scholar
  73. [73]
    S. Winkler: “Issues in vision modeling for perceptual video quality assessment.” Signal Processing 78 (2): 231–252, 1999.zbMATHCrossRefGoogle Scholar
  74. [74]
    S. Winkler: “A perceptual distortion metric for digital color video.” in Proc. SPIE, vol. 3644, pp. 175–184, San Jose, CA, 1999.Google Scholar
  75. [75]
    S. Winkler: “Quality metric design: A closer look.” in Proc. SPIE, vol. 3959, pp. 37–44, San Jose, CA, 2000.Google Scholar
  76. [76]
    S. Winkler, P. Vandergheynst: “Computing isotropic local contrast from oriented pyramid decompositions.” in Proc. ICIP, vol. 4, pp. 420–424, Kyoto, Japan, 1999.Google Scholar
  77. [77]
    J. Yang, W. Makous: “Spatiotemporal separability in contrast sensitivity.” Vision Res. 34 (19): 2569–2576, 1994.CrossRefGoogle Scholar
  78. [78]
    J. Yang, W. Makous: “Implicit masking constrained by spatial inhomogeneities.” Vision Res. 37 (14): 1917–1927, 1997.CrossRefGoogle Scholar
  79. [79]
    M. Yuen, H. R. Wu: “A survey of hybrid MC/DPCM/DCT video coding distortions.” Signal Processing 70 (3): 247–278, 1998.zbMATHCrossRefGoogle Scholar
  80. [80]
    W. Y. Zou: “Performance evaluation: From NTSC to digitally compressed video.” SMPTE J. 103 (12): 795–800, 1994.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2001

Authors and Affiliations

  • Stefan Winkler
    • 1
  • Murat Kunt
    • 1
  • Christian J. van den Branden Lambrecht
    • 2
  1. 1.Signal Processing LaboratorySwiss Federal Institute of Technology - EPFLLausanneSwitzerland
  2. 2.EMC Media Solutions GroupHopkintonUSA

Personalised recommendations