Abstract
The Human-Machine Communication Department of LIMSI (LIMSI Report 1994), a laboratory of the French National Scientific Research Agency (CNRS), conducts research in the field of Pattern Recognition, Artificial Intelligence and Pattern Generation. Several approaches are currently investigated there for studying interactions between modalities: studies about the cognitive processes involved in human intermodal tasks (Denis and Cocude 1992; Gryl 1994; Daniel et al. 1994), image and language interactions for learning (Bordeaux 1993), human factors (Castaing and True 1993), symbolic approaches for designing multimodal human-computer interfaces (Bellik and Teil 1993; Barès et al. 1992), spatial reasoning (Ligozat 1992; Briffault 1992).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
AAAI-NLP-VP (1994). Working notes of the workshop on the Integration of Natural Language and Vision Processing. Chair P. Mc Kevitt. Twelfth National Conf. on Artificial Intelligence, august, 2nd/3rd, Seattle.
Arnold, M. (1993). Linguistic and Spatial Representation of a Scene: An Experiment in Image Synthesis. Sémiotiques 4: 7–29. In French.
Barès, M., Nèel, F., Teil, D. & Martin, J.-C. (1992). Conceptual Approach to Man-Machine Interaction: Application to Pilote’s Assistance. In Proceedings of The Interface Between Real and Virtual Worlds Conference, 571–590. Montpellier, France. In French.
Bellik, Y. & Teil, D. (1993). A Multimodal Dialogue Controller for Multimodal User Interface Management System Application: A Multimodal Window Manager. Adjunct proceedings of INTERCHI’93, 93–94. ACM Press: Amsterdam.
Béroule D. (1990). Guided Propagation: Current State of Theory and Application. In Soulié, F. Fogelman & Hérault, J. (eds.) Neurocomputing, Vol. F 68, 241–260. NATO ASI Series; Berlin Heidelberg: Springer-Verlag.
Béroule, D. (1988). The Never-Ending Learning. In Eckmiller, R. & Malsburg, C.v.d. (eds.) Neural Computers, 219–230. NATO ASI Series vil F41, Berlin: Springer-Verlag.
Béroule, D. (1989). The Adaptive, Dynamic and Associative Memory Model: A Possible Future Tool for Vocal Human-Computer Communication. In Taylor, M., Néel, F. & Bouwhuis, D. G. (eds.) The Structure of Multimodal Dialogue, 189–202. Elsevier Science Publisher B.V.: North Holland
Béroule, D., Von Hoe, R. & Ruellan, H. (1994). A Guided Propagation Model of Reading, Instituut voor Perceptie Onderzoek (IPO), Annual Progress Report NB0228, Eindhoven, The Netherlands.
Bordeaux, F. (1993). Building Categories from Visual Experiences. In Proceedings of The IEEE International Conference on Systems, Man and Cybernetics. October, Le Touquet, France.
Briffault, X. (1992). Computer Modelisation of the Expression of Localisation in Natural Language. Ph.D. diss. Paris VI University. 255 pp. In French.
Castaing, M. F. & True-Martini, D. (1993). Constraints in the Recording of Speech Data Bases. In Proceedings of The Internat. Symp. on Spoken Dialogue, 29–31. Tokyo, Japon.
Daniel, M. -P., Carité, L. & Denis, M. (1994). Modes of Linearization in the Description of Spatial Configurations. In Portugali, J. (ed.) The Construction of Cognitive Maps. Kluwer: Dordrecht, The Netherlands.
Denis, M. & Cocude, M. (1992). Structural Properties of Visual Images Constructed form Poorly or Well-structured Verbal Descriptions. Memory and Cognition 20: 497–506.
Escande, P., Béroule, D. & Blanchet, P. (1991). Speech Recognition Experiments with Guided Propagation. Proc. of IJCNN’91, Singapour.
Faure, C. & Arnold, M. (1993). Man-Machine Interaction and Minimum Principle. In Proceedings of The IHM’93, 3–8 Lyon. In French.
GDR-PRC-CHM (1993). Activity Report About Multimodal Human-Computer Interfaces, CNRS-MESR, Coutaz, J. & Caelen, J. In French. In 1994: C. Faure is at the head of the CHM Pôle, F. Poirier at the head of the multimodal working group.
Grumbach, A. (1994). Artificial Cognition. Addison-Wesley France. In French.
Gryl, A. (1994). Cognitive Approach of Route Descriptions. In Pre-proceedings of The Fith European Workshop on Imagery and Cognition (EWIC), 6–8 Saarbrücken, Germany, Univ. of the Saarland.
Huls, C, Bos, E. & Dijkstra, A. (1994). Talking Pictures. An Empirical Study into the Usefulness of Natural Language Output in a Graphical Interface. Working notes of the AAAI-94 workshop on the Integration of Natural Language and Vision Processing, 83–90. Chair P. McKevitt. Twelfth National Conf. on Artificial Intelligence, Seattle, Washington.
Hummel, J. E. & Biederman, I. (1990). Dynamic Binding: A Basis for the Representation of Shape by Neural Networks. 12th Annual Meeting of the Cognitive Science Society. Cambridge, MA.
Images and Languages. (1993). Proceedings of The Interdisciplinary Workshop “Image and Language: Multimodality and Cognitive Modelisation”. M. Denis and M. Carfantan (eds.) CNRS, Paris, 1993. In French.
Jackendoff, R. (1987). On Beyond Zebra: The Relation of Linguistic and Visual Information. Cognition 26(2): 89–114.
Lainé, A. & Béroule, D. (1994). Towards Active Perception for Automatic Speech Recognition. In Proceedings of The NSI’94 Chamonix, 23–26. In French.
Ligozat, G. (1992). Strategies for Route Description: An Interdisciplinary Approach. ECAI’92 workshop W19, Spatial Concepts: Connecting Cognitive Theories with Formal Representations. Vienna, Austria.
LIMSI, Scientific Report, Human-Machine Communication Department, Orsay, 1994.
Martin, J. C. (1994). A Study Based on Several Types of Coorperation between Modalities and Binding through Synchrony. Working notes of the AAAI-94 workshop on the Integration of Natural Language and Vision Processing 181–184. Chair P. Mc Kevitt. Twelfth National Conf. on Artificial Intelligence, Seattle, Washington.
Martin, J. C. & Béroule, D. (1993). Types and Goals of Co-Operation Between Modalities. Proceedings of The IHM’93, 17–22. Lyon. In French.
Nenov, V. I. & Dyer, M. G., (1988). DETE: Connectionist/Symbolic Model of Visual and Verbal Association. In Proceedings of The Connexionnist Models Summer School 1988. CMU.
Nigay, L. & Coutaz, J. (1993). A Design Space For Multimodal Systems: Concurrent Processing and Data Fusion, conf. Proceedings of The INTERCHI’93, 172–178. ACM Press: Amsterdam.
Nioche, C, Tassin, J-P. & Béroule, D. (1994). Towards a Functional Modelisation of Neuro-modulation. In Proceedings of The NSI’94, 231–234. Chamonix, In French.
Novak, G. S. & Bulko, W. C. (1990). Understanding Natural Language with Diagrams. In Proceedings of The 8th National Conf. on Artificial Intelligence Vol. 1, 465–470, AAAI Press/MIT Press.
ÓNualláin, S. & Smith, A. G. (1994). Artificial Intelligence Review 8(2–3): 113–122 (this volume).
Robert-Ribes, J., Schwartz, J. L. & Escudier, P. (1994). Artificial Intelligence Review 9 (this volume).
Roques, M. (1994). Dynamic Grammatical Representations in Guided Propagation Networks. In Lecture Notes in Artificial Intelligence 862, Carrasco, R. C., Oncina, J. (eds.) Grammatical Inference and Applications, 189–202. Second International Colloqium, ICGI-94, Alicante, Spain.
Shastri, L. & Ajjanagadde, V. (1993). From Simple Associations to Systematic Reasoning: A Connectionist Representation of Rules, Variables and Dynamic Bindings Using Temporal Synchrony. Behavioral and Brain Sciences 16: 417–494.
von der Malsburg, C. & Schneider, W. (1986). A Neural Cocktail-Party Processor. Biol. Cybern. 54: 29–40.
Westerlund, P., Béroule, D. & Roques, M. (1994). Experiments of Robust Parsing Using a Guided Propagation Network. In Proceedings of The International Conference on New Methods in Language Processing (NEMLAP), Manchester.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1995 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Martin, JC., Béroule, D. (1995). Temporal Codes within a Typology of Cooperation Between Modalities. In: Mc Kevitt, P. (eds) Integration of Natural Language and Vision Processing. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-0445-6_5
Download citation
DOI: https://doi.org/10.1007/978-94-011-0445-6_5
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-4199-7
Online ISBN: 978-94-011-0445-6
eBook Packages: Springer Book Archive