The Future of Media Computing

From Ontology-Based Semiosis to Communal Intelligence
  • Frank Nack
Part of the The Springer International Series in Video Computing book series (VICO, volume 4)


In this chapter we investigate the underlying structural requirements for media-aware knowledge spaces. We discuss the merging of media generation and annotation to facilitate the use of media-based information for diverse purposes. In particular, we pay attention to the description of various tools in a distributed digital production environment supporting many distinct phases of media production. We then outline the accessibility of annotated media material for the purpose of discourse in encyclopaedic spaces. The examples throughout this chapter are taken from two domains: Media production with an emphasis on news and film production, and encyclopaedic spaces as provided by domains such as theory, history, and anthropology of film.


Knowledge exploration semantic networks adaptive discourse media infrastructure media production media-aware knowledge spaces encyclopaedic spaces information space editing dynamic presentation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Aguierre Smith, T.G. and Davenport, G. The Stratification System. A Design Environment for Random Access Video. In ACM workshop on Networking and Operating System Support for Digital Audio and Video (San Diego, California, 1992).Google Scholar
  2. [2]
    Aigrain, P., Joly, P., and Longueville, V. Medium Knowledge-Based Macro-Segmentation of Video into Sequences. In IJCAI 95 — Workshop on Intelligent Multimedia Information Retrieval M. Maybury (Ed.) (Montreal, August 19, 1995), pages 5–16.Google Scholar
  3. [3]
    André, E., Müller, J., and Rist. T. WIP/PPP: Knowledge-Based Methods for Fully Automated Multimedia Authoring. In EUROMEDIA′96, (London, UK, 1996), pages 95–102.Google Scholar
  4. [4]
    André, E., Müller, J., and Rist, T. Presenting Through Performing: On the Use of Multiple Lifelike Characters in Knowledge-Based Presentation Systems. In Proc. of the Second International Conference on Intelligent User Interfaces (IUI 2000) (2000), pages 1–8.CrossRefGoogle Scholar
  5. [5]
    Andrew, D. Concepts in Film Theory. Oxford: Oxford University Press, 1984.Google Scholar
  6. [6]
    Arnheim, R. Art and Visual Perception: A Psychology of the Creative Eye. London: Faber and Faber, 1956.Google Scholar
  7. [7]
    Auffret, G., Carrive, J., Chevet, O., Dechilly, T., Ronfard, R., and Bachimont, B. Audiovisual-based Hypermedia Authoring: Using structured representations for efficient access to AV documents. In Proceedings of the 10th ACM conference on Hypertext and Hypermedia (1999), pages 169–178.Google Scholar
  8. [8]
    Bailey, B.P., Konstan, J.A., and Carlis, J.V. DEMAINS: Designing Multimedia Applications with Interactive Storyboards. In Proceedings of the 9th ACM International Conference on Multimedia (Ottawa, Canada, Sept. 30 – Oct 5, 2001), pages 241–250.Google Scholar
  9. [9]
    Bettetini, G. The Language and Technique of the Film. The Hague: Mouton Publishers, 1973.Google Scholar
  10. [10]
    Bloch, G.R. Elements d’une Machine de Montage Pour l’Audio-Visuel. Ph.D. Thesis, Ecole Nationale Supérieure Des Télécommunications, 1986.Google Scholar
  11. [11]
    Bloom, P.J. High-quality Digital Audio in the Entertainment Industry: An Overview of Achievements and Challenges. IEEE Acoust. Speech Signal Process. Mag.. 2 (1985), 2–25.Google Scholar
  12. [12]
    Boland, R.J.J. and Tenkasi, R.V. Perspective Making and Perspective Taking in Communities of Knowing. Organization Science. 6 (4) (1995), 350–372.CrossRefGoogle Scholar
  13. [13]
    Boll, S., Klas, W., and Wandel, J. A Cross-Media Adaptation Strategy for Multimedia Presentations. In ACM Multimedia 1999 Proceedings (Orlando, Florida, October 30 – November 5, 1999), pages 37–46.Google Scholar
  14. [14]
    Bordwell, D. Making Meaning — Inference and Rhetoric in the Interpretation of Cinema. Cambridge, Massachusetts: Harvard University Press, 1989.Google Scholar
  15. [15]
    Brachman, R.J. and Levesque, H.J. Readings in Knowledge Representation. San Mateo, California: Morgan Kaufmann Publishers, 1983.Google Scholar
  16. [16]
    Brooks, K.M.. Metalinear Cinematic Narrative: Theory, Process, and Tool. Ph.D. Thesis, M.I.T., 1999.Google Scholar
  17. [17]
    Buckingham Shum, S. and Selvin, A. Structuring Discourse for Collective Interpretation. In Electronic Proceedings of Open Conference on Collective Cognition and Memory Practices (September 19–20, 2000). Google Scholar
  18. [18]
    Brusilovsky P., Kobsa A., and Vassileva J. (ed.). Adaptive Hypertext and Hypermedia. Kluwer Academic Publishers, Dordrecht, The Netherlands, 1998.Google Scholar
  19. [19]
    Carroll, J.M. Toward a Structural Psychology of Cinema. The Hague: Mouton Publishers, 1980.CrossRefGoogle Scholar
  20. [20]
    Campell, I. Interactive Evaluation of the Ostensive Model Using a New test Collection of Images with Multiple relevance Assessment. Information Retrieval. Vol 2, 1, (2000), pages 87–114, Kluwer Academic Publishers, Boston.Google Scholar
  21. [21]
    Collier G.H. Thoth-II: Hypertext with Explicit Semantics. In Hypertext 1987 Proceedings (Chapel Hill, North Carolina ACM Press, November 13–15, 1987), pages 269–289.CrossRefGoogle Scholar
  22. [22]
    Crestani, F., Lalmas, M., Van Rijsbergen, C.J., and Campbell, I. Is this document relevant?…Probably: A survey of probabilistic models in information retrieval. ACM Computing Surveys. Volume 30, No. 4 (Dec. 1998), pages 528–552.CrossRefGoogle Scholar
  23. [23]
    DAML+OIL., 2001.
  24. [24]
    Davis, M. Media Streams: Representing Video for Retrieval and Repurposing. Ph.D. Thesis, MIT, 1995.Google Scholar
  25. [25]
    De Bra, P. and Calvi. L. AHA: A Generic Adaptive Hypermedia System. In Proceedings of the 2nd Workshop on Adaptive Hypertext and Hypermedia (Pittsburgh, June 1998), pages 5–17.Google Scholar
  26. [26]
    Del Bimbo, A. Visual Information Retrieval. Morgan Kaufmann Ed, San Francisco, USA, 1999.Google Scholar
  27. [27]
    DeRose, S. and Durand, D. Making Hypermedia Work-A User’s Guide to HyTime. Kluwer Academic Publishers, Boston, 1994.CrossRefGoogle Scholar
  28. [28]
    Digital Life Consortium — MIT., 2001.
  29. [29]
    Dorai, C. and Venkatesh, S. Bridging the Semantic Gap in Content Management Systems: Computational Media Aesthetics. In Proceedings of the First Conference on Computational Semiotics for Games and New Media — COSIGN 2001 (Amsterdam, 10 – 12 September, 2001), pages 94–99.Google Scholar
  30. [30]
    Dramatica., 2001.
  31. [31]
    Eco, U. Articulations of the Cinematic Code. In B. Nichols (Eds.), Movies and Methods (Berkeley: University of, 1976), pages 590–607.Google Scholar
  32. [32]
    Eisenstein, S.M. Selected Works: Towards a Theory of Montage. London: BFI Publishing, 1991.Google Scholar
  33. [33]
    Fehlis, H. Hybrides Trackingsystem für virtuelle Studios. Fernseh- + Kinotechnik; Bd. 53, Nr. 5, 1999.Google Scholar
  34. [34]
    Final Draft., 2001.
  35. [35]
    Gibson, W. Neuromancer. Phantasia Press, 1st Phantasia Press ed. West Bloomfield, 1986.Google Scholar
  36. [36]
    Gregory, J.R. Some Psychological Aspects of Motion Picture Montage. Ph.D. Thesis, University of Illinois, 1961.Google Scholar
  37. [37]
    Greimas, J. Structural Semantics: An Attempt at a Method. Lincoln: University of Nebraska Press, 1983.Google Scholar
  38. [38]
    Grosso, W.E., Eriksson, H., Fergerson, R.W., Gennari, J. H., Tu, S.W., and Musen M.A. Knowledge Modeling at the Millennium (The Design and Evolution of Protege-2000). Stanford Medical Informatics (SMI), 1999, SMI Report Number: SMI-1999-0801. Stanford, USA.Google Scholar
  39. [39]
    Gupta A. and Jain R. Visual information retrieval. Communications of the ACM. (40, 1997), 71–79.Google Scholar
  40. [40]
    Halasz, F.G. Reflection On Notecards: Seven Issues For The Next Generation Of Hypermedia Systems. Communications of the ACM. July 1988, 31 (7).Google Scholar
  41. [41]
    Hardman, L. Modelling and Authoring Hypermedia Documents. Ph.D. Thesis, University of Amsterdam, 1998.Google Scholar
  42. [42]
    Hirata, K. Towards Formalizing Jazz Piano Knowledge with a Deductive Object-Oriented Approach. In Proceedings of Artificial Intelligence and Music, IJCAI (Montreal 1995), pages 77–80.Google Scholar
  43. [43]
    HMS — Center of Human Modeling and Simulation., 2001.
  44. [44]
    ISO HyTime. ISO/IEC JTC1/SC18 WG8, W1920 rev., Hypermedia/Time-Based Structuring Language (HyTime), 2d ed., International Organization for Standardization, Geneva, May 1997.Google Scholar
  45. [45]
    ISO MHEG. International Standard ISO/IEC 13522-5:1997 (MHEG-5)Google Scholar
  46. [46]
    ISO MPEG-4. MPEG-4 Overview — (V.15 — Beijing Version), ISO/IEC JTC1/SC29/WG11 N3536, Beijing, July 2000.Google Scholar
  47. [47]
    ISO MPEG-7 DDL. Text of ISO/IEC FCD 15938-2 Information Technology — Multimedia Content Description Interface — Part 2: Description Definition Language, ISO/IEC JTC 1/SC 29/WG 11 N4288, 19/09/2001.Google Scholar
  48. [48]
    ISO MPEG-7 MDS. Text of ISO/IEC 15938-5/FCD Information Technology — Multimedia Content Description Interface — Part 5: Multimedia Description Schemes, ISO/IEC JTC 1/SC 29/WG 11 N4242, 23/10/2001.Google Scholar
  49. [49]
    Janowitz, M., and Street, D. The Social Organization of Education. In P.H. Rossi and B.J. Biddle (Eds.), The New Media and Education. Chicago: Aldine, 1966.Google Scholar
  50. [50]
    Johnson, S.E., Jourlin, P., Spärk Jones, K., and Woodland P.C. Audio Indexing and retrieval of Complete Broadcast News Shows. In RIAO 2000 Conference Proceedings (Collége de France, Paris, France, April 12–14 2000), Vol 2, pages 1163–1177.Google Scholar
  51. [51]
    Kamps, T. Diagram Design: A Constructive Theory. Springer Verlag, 1999.Google Scholar
  52. [52]
    Kuleshov, L. Kuleshov on Film — Writing of Lev Kuleshov. Berkeley: University of California Press, 1974.Google Scholar
  53. [53]
    Lemström, K. and Tarhio, J. Searching Monophonic Patterns within Polyphonic Sources. In RIAO 2000 Conference Proceedings (Collége de France, Paris, France, April 12–14 2000), Vol 2, pages 1261–1279.Google Scholar
  54. [54]
    Lévy, P. L’intelligence collective. Pour une anthropologie du cyberspace. Édition la Découverte, Paris, 1994.Google Scholar
  55. [55]
    Lindley, C. A Video Annotation Methodology for Interactive Video Sequence Generation, In BCS Computer Graphics and Displays Group Conference on Digital Content Creation (Bradford, UK, 12–13 April 2000).Google Scholar
  56. [56]
    Lindley, C. and Nack, F. Hybrid narrative and categorical strategies for interactive and dynamic video presentation generation. The New Review of Hypermedia and Multimedia, (vol 6, 2000), pages 111–146, Taylor Graham.Google Scholar
  57. [57]
    Melucci, M. and Orio, N. SMILE: a System for Content-based Musical Information Retrieval Environments. In RIAO 2000 Conference Proceedings (Collége de France, Paris, France, April 12–14 2000), Vol 2, pages 1246–1260.Google Scholar
  58. [58]
    Metz, C. Film Language: A Semiotic Of The Cinema. New York: Oxford University Press, 1974.Google Scholar
  59. [59]
    Mills, T.J., Pye, D., Hollinghurst, N.J., and Wood, K.R. At&TV: Broadcast Television and Radio Retrieval. In RIAO 2000 Conference Proceedings (Collége de France, Paris, France, April 12–14 2000), Vol 2, pages 1135–1144.Google Scholar
  60. [60]
    Movie Magic Screenwriter., 2001.
  61. [61]
    Movie Master. By Comprehensive Cinema Software, 148 Veterans Drive Northvale, NJ 07647, 2001.Google Scholar
  62. [62]
    Nack, F. AUTEUR: The Application of Video Semantics and Theme Representation in Automated Video Editing. Ph.D. Thesis, Lancaster University, 1996.Google Scholar
  63. [63]
    Nack, F. and Putz, W. Designing Annotation Before It’s Needed. In Proceedings of the 9th ACM International Conference on Multimedia (Ottawa, Canada, Sept. 30 – Oct. 5, 2001), pages 251–260.Google Scholar
  64. [64]
    Nack, F., Windhouwer, M., Pauwels, E., Huijberts, M., and Hardman, L. The Role of Highlevel and Lowlevel Features in Semiautomated Retrieval and Generation of Multimedia Presentations. CWI-technical report, INS-R0108, 2001.Google Scholar
  65. [65]
    Ossenbruggen, v.J., Cornelissen, F., Geurts, J., Rutledge, L., and Hardman, L. Towards second and third generation Web-based multimedia. In The Tenth International World Wide Web Conference (Hong Kong, May 1 – 5, 2001), pages 479–488.Google Scholar
  66. [66]
    Parkes, A.P. An Artificial Intelligence Approach to the Conceptual Description of Videodisc Images. Ph.D. Thesis, Lancaster University, 1989.Google Scholar
  67. [67]
    E. Pauwels and G. Frederix. Image Segmentation by Nonparametric Clustering Based on the Kolmogorov-Smirnov Distance. In Proc. of the 6th European Conference on Computer Vision, ECCV 2000 (Dublin, June 2000), pages 85–99.Google Scholar
  68. [68]
    Peirce, C.S. The Collected Papers of Charles Sanders Peirce — 1 Principles of Philosophy and 2 Elements of Logic. Edited by Charles Hartshone and Paul Weiss. Cambridge, MA: The Belknap Press of Harvard University Press, 1960.Google Scholar
  69. [69]
  70. [70]
  71. [71]
    Robertson, J., De Quincey, A., Stapleford T., and Wiggins, G. Real-Time Music Generation for a Virtual Environment. In Proceedings of ECAI-98 Workshop on AI/A life and Entertainment (Brighton, August 24, 1998).Google Scholar
  72. [72]
    Rutledge L., van Ossenbruggen J., Hardman L., and Bulterman D.C.A. Mix’n’Match: Exchangeable Modules of Hypermedia Style. In Proceedings of the 10th ACM conference on Hypertext and Hypermedia (February 21–25, Darmstadt, Germany, 1999), pages 179–188.Google Scholar
  73. [73]
    Rutledge, L., Davis, J., van Ossenbruggen, J., and Hardman L. Interdimensional Hypermedia Communicative Devices for Rhetorical Structure In Proceedings of International Conference on Multimedia Modeling 2000 (MMMOO) (November 13–15, 2000, Nagano, Japan).Google Scholar
  74. [74]
    Sack, W. Coding News And Popular Culture. In The International Joint Conference on Artificial Intelligence (IJCA93) Workshop on Models of Teaching and Models of Learning (Chambery, Savoie, France, 1993).Google Scholar
  75. [75]
    Santini, S. and Ramesh J. Integrated Browsing and Querying for Image Databases. IEEE Multimedia. July–September 2000, pages 26–39, IEEE Computer Society.Google Scholar
  76. [76]
    Semantic Web., 2001.
  77. [77]
  78. [78]
    SMPTE. Dynamic Data Dictionary Structure, 6. Draft, September 1999.Google Scholar
  79. [79]
    Schreiber, A.T.G., Dubbeldam, B., Wielemaker, J., and Wielinga, B. Ontology-based Photo Annotation, IEEE Intelligent Systems, pages 66–74, May/June 2001 (Vol. 16, No. 3). CrossRefGoogle Scholar
  80. [80]
    Sowa, J.F. Conceptual Structures: Information Processing in Mind and Machine. Reading, MA: Addison-Wesley Publishing Company, 1984.zbMATHGoogle Scholar
  81. [81]
    Steinmetz, A. and Hemmje, M. Konzeption eines digitalen Videoeditiersystems auf Basis des internationalen Standards ISO/IEC 11172 (MPEG-1) (1994), Sankt Augustin, GMD: 134.Google Scholar
  82. [82]
    Streitz, N.A., Tandler, P., Müller-Tomfelde, C., and Konomi, S. Roomware: Towards the Next Generation of Human-Computer Interaction based on an Integrated Design of Real and Virtual Worlds. In J.A. Carroll (Ed.), Human-Computer Interaction in the New Millennium (2001), pages 553–578, Addison Wesley.Google Scholar
  83. [83]
  84. [84]
    Tonomura, Y., Akutsu, A., Taniguchi, Y., and Suzuki, G. Structured Video Computing. IEEE Multimedia. 1(3), 1994, 34–43.CrossRefGoogle Scholar
  85. [85]
    Toepper, H. Sergei Eisenstein — a documentation about his work and life. 1. Workshop on Digital Storytelling (Darmstadt, Germany, 15–16/6/2000), pages 59–72. Scholar
  86. [86]
    Vries, A.P. de, Windhouwer, M.A., Apers, P.M.G., and Kersten, M.L. Information Access in Multimedia Databases based on Feature Models. New Generation Computing. 18(4):323–339, October 2000.CrossRefGoogle Scholar
  87. [87]
    Windhouwer, M.A., Schmidt, R.A., and Kersten, M.L. Acoi: A System for Indexing Multimedia Objects. In International Workshop on Information Integration and Web-based Applications and Services (Yogyakarta, Indonesia, November 1999).Google Scholar
  88. [88]
    Wold, E., Blum, T., Keislar, D. and Wheaton, J. Content-Based Classification, Search, and Retrieval of Audio. IEEE Multimedia Magazine. (Vol. 3, No. 3, Fall 1996), pages 27–36.Google Scholar
  89. [89]
  90. [90]
    XML Schema. XML Schema Part 0: Primer, W3C Recommendation, 2 May 2001 XML Schema Part 1: Structures, W3C Recommendation, 2 May 2001 XML Schema Part 2: Datatypes, W3C Recommendation, 2 May 2001 Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2002

Authors and Affiliations

  • Frank Nack
    • 1
  1. 1.Centrum voor Wiskunde en Informatica (CWI)AmsterdamThe Netherlands

Personalised recommendations