Skip to main content
SpringerLink
Log in
Book cover

Integration of Natural Language and Vision Processing pp 151–165Cite as

From Chinese Rooms to Irish Rooms: New Words on Visions for Language

  • Chapter

  • 60 Accesses

Abstract

Natural languages like English have been constrained in expressing perceptions like vision, sound and touch for years despite the efforts of Joyce (1922, 1939) and others. In situ, lexicons have been limited in their form and content. They have typically been structured in the form of sequences of natural language words with their content defined using flat symbolic descriptions in natural languages. In particular, we believe that today’s dictionaries in general, and with respect to Artificial Intelligence (AI) systems in particular, are unnatural in the sense that they do not encode pictures for words just like we do in our heads. There is now a move towards integrated systems in Artificial Intelligence (AI) (see Dennett 1991; Mc Kevitt 1994a, 1994b) and that will cause a need for dire actions on lexical research in the form of integrated lexicons. We believe that lexicons must move towards a situation where natural language words are also defined in terms of spatial and visual structures. These spatial and visual structures will solve what have been two of the most prominent problems in the field of Natural Language Processing (NLP) for years: (1) Where are symbolic semantic primitive meanings in computer programs grounded? and (2) how come some words, typically in the defining vocabulary, in dictionaries have circular definitions so that words end up defining each other? We believe integrated lexicons will cause these two problems to go away and hence help solve Searle’s Chinese Room Problem and move more towards Irish Rooms of people like James Joyce.

Paul Mc Kevitt is currently funded for five years on a British Engineering and Physical Sciences Research Council (EPSRC) Advanced Fellowship under grant B/941AF/1833 for the Integration of Natural Language, Speech and Vision Processing.

An abridged version of this paper has appeared in the Proceedings of the Post-COLING94 International Workshop on Directions of Lexical Research, Beijing, China, August, 1994.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Ballim, Afzal & Wilks, Yorick (1990). Stereotypical Belief and Dynamic Agent Modeling. In Proceedings of he Second international Conference on User Modelling, University of Hawaii at Manoa, Honolulu, Hawaii, U.S.A.

    Google Scholar 

  • Ballim, Afzal & Wilks, Yorick (1991). Artificial Believers. Lawrence Erlbaum Associates: Hillsdale, New Jersey.

    Google Scholar 

  • Barnden, John A. (1990). Naive Metaphysics: A Metaphor-Based Approach to Propositional Attitude Representation (unabridged version). Memorandum in Computer and Cognitive Science, MCCS-90–174, Computing Research Laboratory, Dept. 3CRL, Box 30001, New Mexico State University, Las Cruces, NM 88003–0001, U.S.A.

    Google Scholar 

  • Beardon, Colin (1995). Discourse Structures in Iconic Communication. In McKevitt, Paul (ed.), Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 9(2–3), 189–203.

    Google Scholar 

  • Beckwith, R., Fellbaum, C, Gross, D. & Miller, G. A. (1991). WordNet: A Lexical Database Organized on Psycholinguistic Primitives. In Lexicons: Using Online Resources to Build a Lexicon. Lawrence Erlbaum: Hillsdale, N.J.

    Google Scholar 

  • Brown, John Seely, Burton, Richard R. & Bell, Alan G. (1975). SOPHIE: A Step Towards Creating a Reactive Learning Environment. International Journal of Man-Machine Studies 7, 675–696.

    Article  MATH  Google Scholar 

  • Burton, R. (1976). Semantic Grammar: An Engineering Technique for Constructing Natural-Language Understanding Systems. BBN Report No. 3453, Bolt, Beranek and Newman, Cambridge, MA, U.S.A.

    Google Scholar 

  • Chakravarthy, Anil (1994). Towards a Perceptually-Based Semantics of Action Verbs. In Mc Kevitt, Paul (ed.) Proceedings of The Workshop on Integration of Natural Language and Vision Processing, 161–164. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Denis, M. & Carfantan, M. (eds.) (1993). Images et langages: multimodalité et modelisation cognitive. Actes du Colloque Interdisciplinaire du Comité National de la Recherche Scientifique, Salle des Conférences, Siège du CNRS, Paris, April.

    Google Scholar 

  • Dennett, Daniel (1991). Consciousness Explained. Penguin: Harmondsworth.

    Google Scholar 

  • Dolan, William B. (1994). Exploiting Lexical Information for Visual Processing. In Mc Kevitt, Paul (ed.) Proceedings of The Workshop on Integration of Natural Language and Vision Processing, 185–188. Twelfth American National Conference on Artificial intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Gapp, Klaus-Peter & Maaß, Wolfgang (1994). Spatial Layout Identification and Incremental Descriptions. In Mc Kevitt, Paul (ed.) Proceedings of The Workshop on Integration of Natural Language and Vision Processing, 145–152. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Guo, Chengming (1995). Machine Tractable Dictionaries. Ablex: Norwood, NJ.

    Google Scholar 

  • Guthrie, Louise, McKevitt, Paul & Wilks, Yorick (1989). OSCON: An Operating System Consultant. In Proceedings of The Fourth Annual Rocky Mountain Conference on Artificial Intelligence (RMCAI-89), subtitled, ‘Augmenting Human Intellect By Computer’, 103–113. Registry Hotel, Denver, Colorado, U.S.A., June.

    Google Scholar 

  • Guthrie, Joe, Guthrie, Louise, Wilks, Yorick & Aidinejad H. (1991). Subject-Dependent Cooccurrence and Word Sense Disambiguation. In Proceedings of The 29th Annual Meeting of the Association for Computational Linguistics, 146–152. Berkeley, California. Also Memoranda in Computer and Cognitive Science, MCCS-91–206, CRL, NMSU.

    Google Scholar 

  • Harnad S. (1990). The Symbol Grounding Problem. Physica D., 335–346.

    Google Scholar 

  • Harnad S. (1993). Grounding Symbols in the Analog World with Neural Nets: A Hybrid Model. Think 2, 12–20.

    Google Scholar 

  • Herzog, Gerd & Wazinski, Peter (1994). Visual TRAnslator: Linking Perceptions and Natural Language Descriptions. In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 8(2–3): 175–187.

    Google Scholar 

  • ISX 1991 (1991). ISX Corporation. LOOM Users Guide, version 4.1 edition.

    Google Scholar 

  • Jackson, Stuart & Sharkey, Noel (1995). Grounding Computational Engines. In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 10 (this volume).

    Google Scholar 

  • Joyce, James (1922). Ulysses. Faber and Faber: London.

    Google Scholar 

  • Joyce, James (1939). Finnegans Wake. Faber and Faber: London.

    Google Scholar 

  • Katz, J. (1972). Semantic Theory. Harper & Row: New York.

    Google Scholar 

  • Lakoff, G. (1986). Women, Fire and Dangerous Things. University of Chicago Press: Chicago, Illinois.

    Google Scholar 

  • Lenat, Doug & Guha, R. V. (1989). Building Large Knowledge-Based Systems: Representation and Inference in the Cyc Project. Addison-Wesley: Reading, MA.

    Google Scholar 

  • Maaß, Wolfgang (1994). From Vision to Multimodal Communication: Incremental Route Descriptions. In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 8(2–3): 159–174.

    Google Scholar 

  • Marconi, Diego (1995). On the Referential Competence of Some Machines. In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 10 (this volume).

    Google Scholar 

  • McKevitt, Paul (1986). Building Embedded Representation of Queries About UNIX. Memorandum in Computer and Cognitive Science, MCCS-86-72, Computing Research Laboratory, Dept. 3CRL, Box 30001, New Mexico State University, Las Cruces, NM 88003–0001, U.S.A.

    Google Scholar 

  • Mc Kevitt, Paul (1991a). Principles and Practice in an Operating System Consultant. In Derek Partridge (ed.) Artificial Intelligence and Software Engineering, Vol. I, Chapter on ‘AI Mechanisms and Techniques in Practical Software’, Ablex Publishing Corporation: New York.

    Google Scholar 

  • Mc Kevitt, Paul (1991b). Analysing Coherence of Intention in Natural-Language Dialogue. Ph.D. Thesis, Department of Computer Science, University of Exeter, Exeter, England, U.K., EU, September.

    Google Scholar 

  • McKevitt, P. (ed.) (1992). Natural Language Processing. Artificial Intelligence Review 6(4): 327–332.

    Google Scholar 

  • Mc Kevitt, P. (ed.) (1994a). Proceedings of the Workshop on Integration of Natural Language and Vision Processing. In Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Mc Kevitt, P. (ed.) (1994b). Proceedings of the Workshop on Integration of Natural Language and speech Processing. In Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Mc Kevitt, Paul & Wilks, Yorick (1987). Transfer Semantics in an Operating System Consultant: the formalization of actions involving object transfer. In Proceedings of The Tenth International Joint Conference on Artificial Intelligence (IJCAI-87), 1: 569–575, Milan, Italy, E.U., August.

    Google Scholar 

  • Mc Kevitt, Paul, Partridge, Derek & Wilks, Yorick (1992a). Approaches to Natural Language Discourse Processing. Artificial Intelligence Review 6(4): 333–364.

    Google Scholar 

  • McKevitt, Paul, Partridge, Derek & Wilks, Yorick (1992b). Analysing Coherence of Intention in Natural Language Dialogue. Technical Report 227, Department of Computer Science, University of Exeter, GB-EX4 4PT, Exeter, England, EU.

    Google Scholar 

  • McKevitt, Paul, Partridge, Derek & Wilks, Yorick (1992c). Why Machines should analyse Intention in Natural Language Dialogue. Technical Report 233, Department of Computer Science, University of Exeter, GB-EX4 4PT, Exeter, England, EU.

    Google Scholar 

  • McKevitt, Paul, Partridge, Derek & Wilks, Yorick (1992d). Experimenting with Intention in Natural Language Dialogue. Technical Report 234, Department of Computer Science, University of Exeter, GB-EX4 4PT, Exeter, England, EU.

    Google Scholar 

  • Meini, Christina & Paternoster, Alfredo (1995). Understanding Language Through Vision. In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 10 (this volume).

    Google Scholar 

  • Nakatani, Hiromasa & Itoh, Yukihiro (1994). An Image Retrieval System that Accepts Natural Language. In Mc Kevitt, Paul (ed.) Proceedings of Workshop on Integration of Natural Language and Vision Processing, 7–13. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Narayanan, A., Ford, L., Manuel, D., Tallis, D. & Yazdani, M. (1994). Language Animation. In McKevitt, Paul (ed.) Proceedings of Workshop on Integration of Natural Language and Vision Processing, 58–65. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Oliver, Patrick & Tsujii, Jun-ichi (1994). Prepositional Semantics in the WIP system. In Mc Kevitt, Paul (ed.) Proceedings of The Workshop on Integration of Natural Language and Vision Processing, 139–144. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Partridge, Derek (1995). Language and Vision: A Single Perceptual Mechanism? In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 9: 4–5.

    Google Scholar 

  • Pentland, Alex (ed.) (1993). Looking at People: Recognition and Interpretation of Human Action. IJCAI-93 Workshop (W28) at The 13th International Conference on Artificial Intelligence (IJCAI-93), Chambéry, France, EU, August.

    Google Scholar 

  • Procter, P. (1978) Longman Dictionary of Contemporary English. Longman: London.

    Google Scholar 

  • Rajagopalan, Raman (1994). Integrating text and graphical input to a knowledge base. In Mc Kevitt, Paul (ed). Proceedings of The Workshop on Integration of Natural Language and Vision Processing, 14–21. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Reyero-Sans, Irina & Tsujii, Jun-ichi (1994). A Cognitive Approach to Interlingua Representation of Spatial Descriptions. In Mc Kevitt, Paul (ed.) Proceedings of Workshop on Integration of Natural Language and Vision Processing, 122–130. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Rowe, Jon & McKevitt, Paul (1991). An Emergent Computation Approach to Natural Language Processing. In Proceedings of The Fourth Irish Conference on Artificial Intelligence and Cognitive Science, University College Cork, IRL-Cork, Ireland, European Union (EU), September.

    Google Scholar 

  • Schank, Roger C. (1972). Conceptual Dependency: A Theory of Natural Language Understanding. Cognitive Psychology 3(4): 552–631.

    Article  Google Scholar 

  • Schank, Roger C. (1973). Identification and Conceptualizations Underlying Natural Language. In Schank, R. & Kolby, K. (eds.) Computer Models of Thought and Language. Wh Freeman and co: San Francisco, CA.

    Google Scholar 

  • Schank, Roger C. (1975). Conceptual Information Processing. Fundamental Studies in Computer Science, 3. North-Holland: Amsterdam.

    Google Scholar 

  • Schank, Roger & Fano, Andrew (1995). Memory and Expectations in Learning, Language and Visual Understanding. In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 9: 4–5.

    Google Scholar 

  • Searle, J. R. (1980). Minds, Brains and Programs. Behaviour and Brain Sciences 3: 417–424.

    Article  Google Scholar 

  • Searle, J. R. (1984). Minds, Brains and Science. Penguin Books: London.

    Google Scholar 

  • Searle, J. R. (1990). Is the Brain’s Mind a Computer Program? Scientific American 262: 26–31.

    Article  Google Scholar 

  • Sinclair, John (ed.) (1987). Looking Up: An Account of the COBUILD Project in Lexical Computing. Collins: London.

    Google Scholar 

  • Srihari, Rohini (1994). Photo Understanding Using Visual Constraints Generated from Accompanying Text. In Mc Kevitt, Paul (ed.) Proceedings of The Workshop on Integration of Natural Language and Vision Processing, 22–29. Twelfth American National Conference on Artificial Intelligence (AAAI-94), Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Srihari, Rohini & Burchans, Debra (1994). Visual Semantics: Extracting Visual Information from Text Accompanying Pictures. In Proceedings of The Twelfth American National Conference on Artificial Intelligence (AAAI-94). Seattle, Washington, U.S.A., August.

    Google Scholar 

  • Wilks, Yorick (1973). An Artificial Intelligence Approach to Machine Translation. In Schank, R. & Kolby, K. (eds.) Computer Models of Thought and Language. Wh Freeman and Co.: San Francisco, CA.

    Google Scholar 

  • Wilks, Yorick (1975a). Preference Semantics. In Keenan, Edward (ed.) Formal Semantics of Natural Language. Cambridge University Press, Cambridge, United Kingdom. Also as Memo AIM-206, Artificial Intelligence Laboratory, Stanford University, Stanford, California, U.S.A., July 1973.

    Google Scholar 

  • Wilks, Yorick (1975b). An Intelligent Analyzer and Understander of English. In Communications of the ACM 18(5): 264–274, May. Also in Grosz, Barbara, Jones, Karen Sparck & Webber, Bonnie (eds.) (1986) Readings in Natural Language Processing, 193–204. Morgan Kaufmann: Los Altos, California.

    Google Scholar 

  • Wilks, Yorick (1975c). A Preferential, Pattern-Seeking Semantics for Natural Language Inference. Artificial Intelligence 6: 53–74.

    Article  MATH  Google Scholar 

  • Wilks, Yorick (1977). Good and Bad Arguments About Semantic Primitives. Communication and cognition 10(3/4): 181–221.

    Google Scholar 

  • Wilks, Yorick (1978). Semantic Primitives in Language and vision. In Proceedings of The Second Conference on Theoretical Issues in Natural Language Processing. Champaign-Urbana, IL.

    Google Scholar 

  • Wilks, Yorick (1995). Language, Vision and Metaphor. In McKevitt, Paul (ed.) Artificial Intelligence Review, Special Volume on Integration of Natural Language and Vision Processing 9: 4–5.

    Google Scholar 

  • Wilks, Yorick & Ballim, Afzal (1987). Multiple Agents and the Heuristic Ascription of Belief. In Proceedings of The 10th International Joint Conference on Artificial Intelligence (IJCAI-87) 119–124, Milan, Italy, August.

    Google Scholar 

  • Wilks, Yorick & Oakada, N. (eds.) (in press). Computer Language & Vision Across the Pacific. Ablex: Norwood, NJ.

    Google Scholar 

  • Wittgenstein, Ludwig (1963). Philosophical Investigations (translated by G.E. Anscombe). Oxford: Blackwell.

    Google Scholar 

  • Wright, Ron E. & Young, D. A. (1990). The Cognitive Modalities (CM) System of Knowledge Representation and its Potential Application in C31 Systems. In Proceedings of The 1990 Symposium on Command and Control Research, SAIC Report SAIC-9011508, 373–381.

    Google Scholar 

  • Young, D. A. (1983). Interactive Modal Meaning as the Mental Basis of Syntax. Medical Hypothesis 10: 5.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Sheffield, Regent Court, 211 Portobello Street, GB-SI 4DP, Sheffield, England, E.U.

    Paul Mc Kevitt

  2. Computer Science Department, Tsinghua University, Beijing, 100084, China

    Chengming Guo

Authors
  1. Paul Mc Kevitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengming Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Sheffield, England, EU

    Paul Mc Kevitt (EPSRC Advanced Fellow in Information Technology) (EPSRC Advanced Fellow in Information Technology)

  2. Dún Na nGall (Donegal), Ireland, EU

    Paul Mc Kevitt (EPSRC Advanced Fellow in Information Technology) (EPSRC Advanced Fellow in Information Technology)

Rights and permissions

Reprints and permissions

Copyright information

© 1996 Kluwer Academic Publishers

About this chapter

Cite this chapter

Mc Kevitt, P., Guo, C. (1996). From Chinese Rooms to Irish Rooms: New Words on Visions for Language. In: Mc Kevitt, P. (eds) Integration of Natural Language and Vision Processing. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-1639-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-94-009-1639-5_12

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-0-7923-3944-1

  • Online ISBN: 978-94-009-1639-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation