Human Perception of Audio-Visual Skew

  • Ralf Steinmetz
Chapter

Abstract

Multimedia synchronization comprises the definition and the establishment of temporal relationships among audio, video, and other data. The presentation of ‘in sync’ data streams by computers is essential to achieve a natural impression. If data is ‘out of sync’, the human perception tends to identify the presentation as artificial, strange, or annoying. Therefore, the goal of any multimedia system is to present all data without perceptible synchronization errors. The achievement of this goal requires a detailed knowledge of the synchronization requirements at the user interface. This paper presents the results of a series of experiments about human media perception of skew mainly related to audio and video data. It leads to a first guideline for the definition of a synchronization quality of service. The results show that a skew between related data streams may still let data appear ‘in sync’. It also turned out that the notion of a synchronization error highly depends on the types of media.

Keywords

Human Perception Synchronization Error Multimedia System Audio Stream Test Candidate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Ralf Steinmetz, Ralf Guido Herrtwich: “Integrated Distributed Multimedia-Systems,” Informatik Spektrum, Springer Verlag, vol. 14, no. 5, October 1991, pp. 280–282.Google Scholar
  2. [2]
    Ralf Steinmetz, “Multimedia-Technology: Fundamentals (in German: ”Multimedia Technologie: Einführung und Grundlagen“),” Springer-Verlag, September 1993.Google Scholar
  3. [3]
    David P. Anderson, George Homsy: “Synchronization Policies and Mechanisms in a Continuous Media I/O Server,” International Computer Science Institute, Technical Report no. 91–003, Berkeley, 1991.Google Scholar
  4. [4]
    Gerold Blakowski, Jens Huebel, Ulrike Langrehr, Max Muhlhaeuser: “Tools Support for the Synchronization and Presentation of Distributed Multimedia,” computer communications, vol. 15, no. 10, December 1992.Google Scholar
  5. [5]
    L.Li, A. Karmouch, N.D. Georganas, “Synchonization in Real Time Multimedia Data Delivery,” IEEE ICC’92, Chicago, USA, June 1992. pp. 322. 1.Google Scholar
  6. [6]
    L.Li, L. Lamont, A. Karmouch, N.D. Georganas: “A Distributed Synchronization Control Scheme in A Group-oriented Conferencing Systems,” Proceedings of the second international conference, Broadband Islands, Athens, Greece, June 15–16, 1993Google Scholar
  7. [7]
    Doug Shepherd, Michael Salmony: “Extending OSI to Support Synchronisation Required by Multimedia Applications,” Computer Communications, vol. 13, no. 7, September 1990, pp. 399–406.CrossRefGoogle Scholar
  8. [8]
    Ralf Steinmetz: “Multimedia Synchronization Techniques: Experiences Based on Different System Structures,” IEEE Multimedia Workshop `92, Monterey, April 1992.Google Scholar
  9. [9]
    Thomas D.C. Little, Arif Ghafoor: “Network Considerations for Distributed Multimedia Objects Composition and Communication, ” IEEE Network Magazine, vol. 4 no. 6, November 1990, pp. 32–49.CrossRefGoogle Scholar
  10. [10]
    Thomas D.C. Little, A. Ghafoor: “Synchronization and Storage Models for Multimedia Objects,” IEEE Journal on Selected Areas in Communication, vol. 8, no. 3, Apr. 1990, pp. 413–427.CrossRefGoogle Scholar
  11. [11]
    Cosmos Nicolaou: “An Architecture for Real-Time Multimedia Communication Systems,” IEEE Journal on Selected Areas in Communication, vol. 8, no. 3, April 1990, pp. 391–400.CrossRefGoogle Scholar
  12. [12]
    Kaliappa Ravindran: “Real-time Synchronization of Multimedia Datastreams in High Speed Packet switching Networks,” in Workshop on Multimedia Information Systems (MMIS `92), IEEE Communications Society, Tempe, AZ, February 1992.Google Scholar
  13. [13]
    Ralf Steinmetz: “Synchronization Properties in Multimedia Systems,” IEEE Journal on Selected Areas in Communication, vol. 8, no. 3, April 1990, pp. 401–412.CrossRefGoogle Scholar
  14. [14]
    Alan Murphy: “Lip Synchronization,” personal communication on a concerning set of experiments, 1990.Google Scholar
  15. [15]
    Roger Dannenberg: Personal communication on sound effects and video synchronization and on music play back and visualization of the corresponding strokes, 1993.Google Scholar
  16. [16]
    Harald Rau, personal communication, 1993.Google Scholar
  17. [17]
    Barry Blesser: “Digitization of Audio: A Comprehensive Examination of Theory, Implementation, and Current Practice, Journal of the Audio Engineering Society, JAES 26(10), October 1978, pp. 739–771.Google Scholar
  18. [18]
    T. Stockham: “A/D and D/A Converters: Their Effect on Digital Audio Fidelity,” in Digital Signal Processing, L. Rabiner and C. Rader, (Eds.), IEEE Press, NY 1972.Google Scholar
  19. [19]
    J.C.R. Licklider: “Basic correlates of the auditory stimulus,” in S. S. Stevens, ed. Handbook of Experimental Psychology, Wiley, 1951.Google Scholar
  20. [20]
    H. Woodrow: “Time Perception,” in S. S. Stevens, (Ed.) Handbook of Experimental Psychology, Wiley, 1951.Google Scholar
  21. [21]
    Dean Rubine, Paul McAvinney: “Programmable Finger-tracking Instrument Controllers,” Computer Music Journal, vol. 14, no. 1, Spring 1980, pp. 2641.Google Scholar
  22. [22]
    Roger Dannenberg, Richard Stern: Experiments Concerning the Allowable Skew of Two Audio Channels Operating in the Stereo Mode, personal communication, 1993.Google Scholar
  23. [23]
    M. Clynes: “Secrets of Life in Music: Musicality Realized by Computer,” in Proceedings of the 1984 International Computer Music Conference, San Francisco, International Computer Music Association, 1985.Google Scholar
  24. [24]
    M. Stewart: “The Feel Factor: Music with Soul,” Electronic Musician, vol. 3, no. 10, pp. 55–66, 1987.Google Scholar
  25. [25]
    Ralf Steinmetz, Clemens Engler: “Human Perception of Media Synchronization,” IBM Technical Report 43. 9310, IBM European Networking Center, Heidelberg, 1993.Google Scholar
  26. [26]
    Carsten Vogt, Ralf Guido Herrtwich, Ramesh Nagarajan: “HeiRat: The Heidelberg Resource Administration Technique Design Philosophy and Goals,” IBM Technical Report 43. 9307, IBM European Networking Center, Heidelberg 1992.Google Scholar
  27. [27]
    Luca Delgrossi, Ralf Guido Herrtwich, Frank Oliver Hoffmann: “An Implementation of ST-II for the Heidelberg Transport System,” IBM Technical Report 43. 9303, IBM European Networking Center, Heidelberg, 1993Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1994

Authors and Affiliations

  • Ralf Steinmetz
    • 1
  1. 1.IBM European Networking CenterHeidelbergGermany

Personalised recommendations