Skip to main content
SpringerLink
Log in

A Survey of Visual Language Specification and Recognition

  • Chapter
Visual Language Theory

Abstract

One of the most fundamental questions in visual language research is how to specify a visual language. By a visual language we mean a set of diagrams which are valid “sentences” in that language where a diagram is a collection of “symbols” in a two or three dimensional space. Which sentences are valid and what their meaning is depends on spatial relationships between the symbols. Thus, for example, mathematical expressions, plans, and musical notation are commonly used visual languages. In this chapter we survey the many formalisms which have been suggested over the last three decades for visual language specification, and also indicate their possible applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. H. Abramson and V. Dahl.Logic Grammars.Springer-Verlag, New York, 1989.

    Book  MATH  Google Scholar 

  2. A. V. Aho and S. C. Johnson. Programming utilities and libraries - LR parsing.Computing SurveysJune 1974.

    Google Scholar 

  3. A.V. Aho, R. Sethi, and J.D. Ullman.Compilers Principles Techniques and Tools.Addison-Wesley Series in Computer Science. Addison-Wesley, Reading, 1987.

    Google Scholar 

  4. J.F. Allen. Maintaining knowledge about temporal intervals.Communications of the ACM26(1):832–843, 1983.

    Article  MATH  Google Scholar 

  5. M. Anderson. Diagrammatic reasoning and cases. InThirteenth National Conference on Artificial Intelligencepages 1004–1009, Portland, 1996.

    Google Scholar 

  6. R.H. Anderson.Syntax-Directed Recognition of Hand-Printed Two-Dimensional Mathematics.PhD thesis, Dept. of Engineering and Applied Physics, Harvard University, 1968.

    Google Scholar 

  7. R.H. Anderson. Two-dimensional mathematical notation. In K.-S. Fu, editorSyntactic Pattern Recognition Applicationspages 147177. Springer-Verlag, New York, 1977.

    Google Scholar 

  8. M. Andries and G. Engels. A hybrid query language for an extended entity-relationship model.Journal of Visual Languages and Computing7(3):321–352, 1996.

    Article  Google Scholar 

  9. AVI. Workshop on Advanced Visual Interfaces.

    Google Scholar 

  10. B.O. Backlund, O. Hagsand, and B. Pehrson. Generation of visual language-oriented design environments.Journal of Visual Languages and Computing1:333–354, 1990.

    Article  Google Scholar 

  11. H. Baird. Industrial applications. In H. Bunke and A. Sanfeliu, editorsSyntactic and Structural Pattern Recognition: Theory and Applications2, pages 369–380. World Scientific, 1990.

    Google Scholar 

  12. K.G. Balke and G.L. Carter. The COLASL automatic coding language. InSymbolic Languages in Data Processing3, pages 501–537. Gordon and Breach, New York, 1962.

    Google Scholar 

  13. C. Batini, T. Catarci, M.F. Costabile, and S. Levialdi. Visual query systems for databases. Research report, Universita di Roma “La Sapienza”, 1993.

    Google Scholar 

  14. B. Bell. Chemtrains: A rule-based visual language for building graphical simulations. Technical Report CU-CS-529–92, US West Advanced Technologies, February 1992.

    Google Scholar 

  15. B. Bell and C. Lewis. Chemtrains: A language for creating behaving pictures. InIEEE Workshop on Visual Languagespages 188–195. IEEE Computer Society Press, 1993.

    Google Scholar 

  16. M.I. Bernstein and T.G. Willimans. A two-dimensional programming system. InProceedings of the IFIP Congresspages 586–592. North Holland, 1968.

    Google Scholar 

  17. M. Berthod and J.P. Maroy. Learning in syntactic recognition of symbols drawn on a graphic tablet.Computer Graphics and Image Processing9:166–182, 1979.

    Article  Google Scholar 

  18. T. Bolognesi and D. Latella. Techniques for the formal definition of the G-LOTOS syntax. InIEEE Workshop on Visual Languagespages 43–49. IEEE Computer Society Press, 1989.

    Google Scholar 

  19. J. Boyle, S. Leishman, and P.M.D. Gray. From WIMPS to 3D: The development of AMAZE.Journal of Visual Languages and Computing7(3):291–319, 1996.

    Article  Google Scholar 

  20. R.J. Brachman, D.L. McGuinness, P.F. Patel-Schneider, L.A. Resnick, and A. Borgida. Living with classic: When and how to use a KL-ONE-like language. In J.F. Sowa, editorPrinciples of Semantic Networks: Explorations in the Representation of Knowledgepages 401–456. Morgan Kaufmann, San Mateo, 1991.

    Google Scholar 

  21. R.J. Brachman and J.G. Schmolze. An overview of KL-ONE knowledge representation systems.Cognitive Sciencepages 171–216, August 1985.

    Google Scholar 

  22. F.J Brandenburg. On polynomial time graph grammars. InFifth Conference on Theoretical Aspects of Computer Science (STAGS 88)volume 294 ofLNCSpages 227–236. Springer-Verlag, New York, 1988.

    Google Scholar 

  23. J.M. Brayer. Parsing of web grammars. InProc. IEEE Workshop on Data Description and Management1977.

    Google Scholar 

  24. R. Bruk and A. Sarma, editors.SDL ‘85 with MSC in CASE: Proceedings of the Seventh SDL Forum Oslo Norway 26–29 September 1995.Elsevier, Amsterdam, New York, 1995.

    Google Scholar 

  25. H. Bunke. Attributed programmed graph grammars and their application to schematic diagram interpretation.IEEE Transactions on Pattern Analysis and Machine Intelligence4(6):574–582, 1982.

    Article  MATH  Google Scholar 

  26. H. Bunke and B. Haller. A parser for context free plex grammars. InGraph-Theoretic Concepts in Computer Sciencevolume 411 ofLNCS.Springer-Verlag, New York, 1989.

    Google Scholar 

  27. D.K. Carter.Writing a Visual Programming Language Reference Manual Using Formal Specifications.Master thesis, Univ. of Cincinnati, 1996.

    Google Scholar 

  28. T. Catarci, M. F. Costabile, S. Levialdi, and C. Batini. Visual query systems for databases: A survey.Journal of Visual Languages and Computing8(2):215–260, 1997.

    Google Scholar 

  29. S.-K. Chang. A method for the structural analysis of two-dimensional mathematical expressions.Information Sciences2:253–272, 1970.

    Article  MATH  Google Scholar 

  30. S.-K. Chang. Picture processing grammar and its applications.Information Sciences3:121–148, 1971.

    Article  MATH  Google Scholar 

  31. S.-K. ChangPrinciples of Pictorial Information Systems Design.Prentice Hall, Englewood Cliffs, 1989.

    Google Scholar 

  32. S.-K. Chang, M.J. Tauber, B. Yu, and J.-S. Yu. A visual language compiler.IEEE Transactions on Software Engineering15:506–525, 1989.

    Article  Google Scholar 

  33. S.S. Chok and K. Marriott. Automatic construction of user interfaces from constraint multiset grammars. InIEEE Symposium on Visual Languagespages 242–250. IEEE Computer Society Press, 1995.

    Google Scholar 

  34. S.S. Chok and K. Marriott. Parsing visual languages. InProc. Eighteenth Australian Computer Science Conf. - Australian Computer Science Comm.volume 17, pages 90–98, 1995.

    Google Scholar 

  35. N. Chomsky.Syntactic Structures.Mouton, The Hague, 1957.

    Google Scholar 

  36. N. Chomsky.Aspects of the Theory of Syntax.MIT Press, Cambridge, 1965.

    Google Scholar 

  37. A. Chow and R. Rubin. Topological composition systems: Specifications for lexical elements of visual languages. InIEEE Symposium on Visual Languagespages 118–124. IEEE Computer Society Press, 1991.

    Google Scholar 

  38. B.L. Clarke. A calculus of individuals based on connection.Notre Dame Journal of Formal Logic23(3):204–218, 1981.

    Article  Google Scholar 

  39. B.L. Clarke. Individuals and points.Notre Dame Journal of Formal Logic26(1):61–75, 1985.

    Article  MathSciNet  MATH  Google Scholar 

  40. W.F. Clocksin and C.S. Mellish.Programming in Prolog.Springer-Verlag, Berlin, third edition, 1987.

    Book  MATH  Google Scholar 

  41. A.G. Cohn. A more expressive formulation of many sorted logic.Journal of Automated Reasoning3:113–200, 1987.

    Article  MathSciNet  MATH  Google Scholar 

  42. Alain Colmerauer. Metamorphosis grammars. In L. Bolc, editorNatural Language Communication with Computers.Springer-Verlag, Berlin, 1978.

    Google Scholar 

  43. G. Costagliola. (Pictorial) LR parsing from an arbitrary starting point. InThird International Workshop on Parsing Technologiespages 49–59. Tilburg University, August 1993.

    Google Scholar 

  44. G. Costagliola and S-K. Chang. DR PARSERS: A generalization of LR parsers. InIEEE Symposium on Visual Languagespages 174–180. IEEE Computer Society Press, 1990.

    Google Scholar 

  45. G. Costagliola, A. De Lucia, and S. Orefice. Towards efficient parsing of diagrammatic languages. InProceedings of the 1994 International Workshop on Advanced Visual Interfacespages 162–171. ACM Press, 1994.

    Google Scholar 

  46. G. Costagliola, A. DeLucia, S. Orefice, and G. Tortora. Efficient parsing of data-flow graphs. InProceedings of the 7th International Conference on Software Engineering and Knowledge Engineeringpages 226–233, 1995.

    Google Scholar 

  47. G. Costagliola, S. Orefice, G. Polese, G. Tortora, and M. Tucci. Automatic parser generation for pictorial languages. InIEEE Symposium on Visual Languagespages 306–313. IEEE Computer Society Press, 1993.

    Google Scholar 

  48. G. Costagliola, Tomita, and S-K. Chang. A generalized parser for 2-D languages. InIEEE Symposium on Visual Languagespages 98–104. IEEE Computer Society Press, 1991.

    Google Scholar 

  49. G. Costagliola, G. Tortora, S. Orefice, and A. De Lucia. Auto-matic generation of visual programming environments.Computer28(3):56–66, March 1995.

    Article  Google Scholar 

  50. P.T. Cox, F.R. Giles, and T. Pietrzykowsky. Prograph: a step to-wards liberating programming from textual conditioning. InIEEE Workshop on Visual Languagespages 150–156. IEEE Computer So-ciety Press, 1989.

    Google Scholar 

  51. C. Crimi, A. Guercio, G. Nota, G. Pacini, G. Tortora, and M. Tucci. Relation grammars for modelling multi-dimensional structures. InIEEE Symposium on Visual Languagespages 168–173. IEEE Computer Society Press, 1990.

    Google Scholar 

  52. I. F. Cruz. Doodle: A visual language for object-oriented databases. InACM SIGMOD Conferencepages 71–80. ACM Press, 1992.

    Google Scholar 

  53. I. F. Cruz. Expressing constraints for data display specification: A visual approach. In V. Saraswat and P. Van Hentenryck, editorsPrinciples and Practice of Constraint Programmingpages 443–468. MIT Press, 1995.

    Google Scholar 

  54. M. Duerst. Prolog for structured character description and font design.Journal of Logic Programming26(2):133–146, 1996.

    Article  MATH  Google Scholar 

  55. J. Earley. An efficient context-free parsing algorithm.Communications of the ACM13(2):94–102, 1970.

    Article  MATH  Google Scholar 

  56. S. El-Kassas. Visual languages: their definition and applications in system development. In A. Nunez, editorEUROMICRO-91pages 383–391, Wien, 1991.

    Google Scholar 

  57. T. Ellis, J. Heafner, and W. Sibley. The GRAIL language and operations. Technical Report No. RM-6001-ARPA, The RAND Corp., 1969.

    Google Scholar 

  58. M. Erwig and B. Meyer. Heterogeneous visual languages - integrating textual and visual programming. InIEEE Symposium on Visual Languagespages 318–325. IEEE Computer Society Press, 1995.

    Google Scholar 

  59. J. Feder. Plex languages.Information Sciencepages 225–241, 1971.

    Google Scholar 

  60. F. Ferrucci, G. Pacini, G. Tortora, M. Tucci, and G. Vitiello. Efficient parsing of multidimensional structures. InIEEE Symposium on Visual Languagespages 105–110. IEEE Computer Society Press, 1991.

    Google Scholar 

  61. F. Ferrucci, G. Pacini, G. Tortora, M. Tucci, and G. Vitiello. A predictive parser for visual languages specified byrelation grammars InIEEE Symposium on Visual Languagespages 245–252. IEEE Computer Society Press, 1994.

    Google Scholar 

  62. M. Flasinski. Parsing of edNLC-graph grammars for scene analysis.Pattern Recognition21(6):623–629, 1988.

    Article  MathSciNet  MATH  Google Scholar 

  63. M. Flasinski. Characteristics of edNLC-graph grammar for syntactic pattern recognition.Computer Vision Graphics and Image Processing47:1–21, 1989.

    Article  Google Scholar 

  64. R. Franck. A class of linearly parseable graph grammars.Acta Informatica10:175–201, 1978.

    Article  MathSciNet  MATH  Google Scholar 

  65. K.-S. Fu.Syntactic Methods in Pattern Recognition.Academic Press, 1974.

    Google Scholar 

  66. K.-S. Fu.Syntactic Pattern Recognition and Applications.Prentice-Hall, 1982.

    Google Scholar 

  67. K.-S. Fu and B.L. Bhargava. Tree systems for syntactic pattern recognition.IEEE Transactions on Computing22:1087–1099, 1973.

    Article  MathSciNet  MATH  Google Scholar 

  68. G.W. Fumas. Formal models for imaginal deduction. InTwelfth Annual Conference of the Cognitive Science Societypages 662–669. Lawrence Erlbaum, 1990.

    Google Scholar 

  69. G.W. Fumas. New graphical reasoning models for understanding graphical interfaces. InProc. ACM Conference on Human Factors in Computing Systems (CHI)pages 71–78. ACM Press, 1991.

    Google Scholar 

  70. G.W. Fumas. Reasoning with diagrams only. InAAAI Spring Symposium on Reasoning with Diagrammatic RepresentationsStanford, 1992.

    Google Scholar 

  71. W. Gillett and T. Kimura. Parsing two-dimensional languages. InIEEE COMPSAC 86pages 472–477. IEEE Computer Society Press, 1986.

    Google Scholar 

  72. J. Gips.Shape Grammar and Their Uses.Birkhauser Verlag, 1975.

    Google Scholar 

  73. E. Golin, S. Danz, S. Larison, and D. Miller-Karlow. Palette: An extensible visual editor. InProceedings of the 1992 ACM/SIGAPP Symposium on Applied Computingpages 1208–1216, March 1992.

    Google Scholar 

  74. E.J. Golin.A Method for the Specification and Parsing of Visual Languages.PhD thesis, Dept. of Computer Science, Brown University, 1991.

    Google Scholar 

  75. E.J. Golin and T. Magliery. A compiler generator for visual languages. InProceedings of the 1993 IEEE Workshop on Visual Languagespages 314–321. IEEE Comp. Soc. Press, 1993.

    Google Scholar 

  76. E.J. Golin and S.P. Reiss. The specification of visual language syntax. InIEEE Symposium on Visual Languagespages 105–110. IEEE Computer Society Press, 1989.

    Google Scholar 

  77. J.M. Gooday and A.G. Cohn. Visual language syntax and semantics: A spatial logic approach. In K. Marriott and B. Meyer, editorsInternational Workshop on Theory of Visual LanguagesGubbio, Italy, 1996. Monash University.

    Google Scholar 

  78. H. Göttler. Graph grammars, a new paradigm for implementing visual languages. InEUROGRAPHICS 89pages 505–516. Elsevier-Science publishers (North-Holland), 1986.

    Google Scholar 

  79. H. Göttler. Graph grammars and diagram editing. InGraph-Grammars and Their Application to Computer Science (Third International Workshop)volume 291 ofLNCSpages 211–231. Springer-Verlag, New York, 1986.

    Google Scholar 

  80. M.D. Gross. Recognizing and interpreting diagrams in design. In T. Catarci, M. Costabile, S. Levialdi, and G. Santucci, editorsAdvanced Visual Interfaces (AVI)pages 88–94. ACM Press, 1994.

    Google Scholar 

  81. V. Haarslev. A fully formalized theory for describing visual notations. Inthis volume.

    Google Scholar 

  82. E. Hammer. Representing relations diagrammatically. In G. Allwein and J. Barwise, editorsWorking Papers on Diagrams and Logic.Indiana University, Bloomington, 1993.

    Google Scholar 

  83. D. Harel. On visual formalisms.Communications of the ACM31(5):514–530, 1988.

    Article  MathSciNet  Google Scholar 

  84. D. Hays. Automatic language-data processing. In H. Borko, editorComputer Applications in the Behavioral Sciences.Prentice Hall, Englewood Cliffs, 1962.

    Google Scholar 

  85. R. Helm and K. Marriott. Declarative graphics. InProc. of the 3rd International Conference on Logic Programmingvolume 225 ofLNCSpages 513–527. Springer-Verlag, New York, 1986.

    Chapter  Google Scholar 

  86. R. Helm and K. Marriott. Declarative specification of visual languages. InIEEE Symposium on Visual Languagespages 98–103. IEEE Computer Society Press, 1990.

    Google Scholar 

  87. R. Helm and K. Marriott. A declarative specification and semantics for visual languages.Journal of Visual Languages and Computing2:311–331, 1991.

    Article  Google Scholar 

  88. R. Helm, K. Marriott, and M. Odersky. Building visual language parsers. InProc. ACM Conf. Human Factors in Computing (CHI)pages 118–125. ACM Press, 1991.

    Google Scholar 

  89. J.E. Hoperoft and J.D. Ullman.Introduction to Automata Theory Languages and Computation.Addison-Wesley, Reading, 1979.

    Google Scholar 

  90. Communications Intelligence Corporation http://www.cic.com

  91. IJGIS. International Journal of Geographic Information Systems.

    Google Scholar 

  92. Adobe Systems Inc.PostScript Language Reference Manual.Addison Wesley, Reading, 1986.

    Google Scholar 

  93. J. Jaffar and J.-L. Lassez. Constraint logic programming. Technical report, Monash University, June 1986.

    Google Scholar 

  94. J. Jaffar and J.-L. Lassez. Constraint logic programming. InACM Conference on Principles of Programming Languagespages 111–119. ACM Press, 1987.

    Google Scholar 

  95. J. Jaffar and M.J. Maher. Constraint logic programming: A survey.The Journal of Logic Programming19&20:503–582, May 1994.

    Article  MathSciNet  Google Scholar 

  96. J. Jagadeesh and Y. Wang. Labview.ComputerFebruary 1993.

    Google Scholar 

  97. E. Jungert. An algebra for symbolic image manipulation and transformation. In T.L. Kunii, editorVisual Database Systemspages 301–317. Elsevier Science publishers B.V., 1989.

    Google Scholar 

  98. E. Jungert. Symbolic expressions within a spatial algebra: Unification and impact upon spatial reasoning. InIEEE Workshop on Visual Languagespages 157–162. IEEE Computer Society Press, 1989.

    Google Scholar 

  99. JVLC. Journal of Visual Languages and Computing.

    Google Scholar 

  100. K. Kahn. Toontalk - an animated programming environment for children.Journal of Visual Languages and Computing7(2):197–217, 1996.

    Article  Google Scholar 

  101. [101] K.M. Kahn, V.A. Saraswat, and V. Haarslev. Pictorial janus. InGI-Jahrestagung 19911991.

    Google Scholar 

  102. K.M. Kahn and V.A. Sraswat. Complete visualizations of concurrent programs and their execution. InIEEE Workshop on Visual Languagespages 7–15. IEEE Computer Society Press, 1990.

    Google Scholar 

  103. M. Karima, K.S. Sadhal, and T.O. McNeil. From paper drawings to computer aided design.IEEE Computer Graphics and Applicationspages 24–39, February 1995.

    Google Scholar 

  104. T. Kasami and K. Torii. A syntax-analysis procedure for unambiguous context-free grammars.Journal of the ACM16(3), 1969.

    Google Scholar 

  105. T. Kato. Cognitive view mechanism for multimedia information systems. In Richard Cooper, editorInterfaces to Database Systemspages 244–262. Springer-Verlag, London, 1992.

    Google Scholar 

  106. M. Kaul. Parsing of graphs in linear time. InGraph-Grammars and Their Application to Computer Science (Second International Workshop)volume 153 ofLNCSpages 206–218. Springer-Verlag, New York, 1983.

    Google Scholar 

  107. M. Kaul. Practical applications of precedence graph grammars. InGraph-Grammars and Their Application to Computer Science (Third International Workshop)volume 291 ofLNCSpages 326–342. Springer-Verlag, New York, 1986.

    Google Scholar 

  108. T.D. Kimura, S. Sengupta, and A. Apte. A graphic diagram editor for pen computers.Software - Concepts and Tools15(2):82–95, 1994.

    Google Scholar 

  109. J. Kirsch and R. Kirsch. The structure of paintings: Formal grammar and design.Environment and Planning B: Planning and Design13:163–176, 1986.

    Article  Google Scholar 

  110. R. Kirsch. Computer interpretation of English text and picture patterns.IEEE Transactions on Electronic Computers13:363–376, 1964.

    Article  Google Scholar 

  111. D.E. Knuth. The genesis of attribute grammars. In P. Deransart and M. Jourdan, editorsAttribute Grammars and their Applications (WAGA)volume 461 ofLNCSpages 1–12. Springer-Verlag, New York-Heidelberg-Berlin, 1990.

    Chapter  Google Scholar 

  112. K. Kojima and B.A. Myers. Parsing graphic function sequences. InIEEE Workshop on Visual Languagespages 111–117. IEEE Computer Society Press, 1991.

    Google Scholar 

  113. A. Kramer. Translucent patches - dissolving windows. InProceedings of the ACM Symposium on User Interface Software and Technology (UIST ‘84)pages 121–130, Marina del Rey, 1994. ACM Press.

    Google Scholar 

  114. F. Lakin. Spatial parsing for visual languages. In Shi-Kuo Chang, Tadao Ichikawa, and Panos A. Ligomenides, editorsVisual Languagespages 35–85. Plenum Press, New York, 1986.

    Chapter  Google Scholar 

  115. F. Lakin. Visual grammars for visual languages. InAAAI-87 7th National Conference on AIpages 683–688, 1987.

    Google Scholar 

  116. J. Landay and B. Myers. Interactive sketching for the early stages of user interface design. InProc. ACM Conf. Human Factors in Computing (CHI)pages 43–50. ACM Press, 1995.

    Google Scholar 

  117. K.H. Lee, K.-B. Eom, and R.L. Kashyap. Character recognition based on attribute-dependent programmed grammar.IEEE Transactions on Pattern Analysis and Machine Intelligence14:1122–1128, 1992.

    Article  Google Scholar 

  118. J. van Leeuwen, editor.Handbook of Theoretical Computer Sciencevolume 2. Elsevier, Amsterdam, New York, 1990.

    Google Scholar 

  119. W.C. Lin and K.-S. Fu. A syntactic approach to 3-D object recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence6:362–378, 1984.

    Article  Google Scholar 

  120. J.W. Lloyd.Foundations of Logic Programming.Symbolic Computation - Artificial Intelligence Series. Springer-Verlag, Berlin, second edition, 1987.

    Google Scholar 

  121. R.M. MacGregor. The evolving technology of classification-based knowledge representation systems. In J.F. Sowa, editorPrinciples of Semantic Networks: Explorations in the Representation of Knowledgepages 385–400. Morgan Kaufmann, San Mateo, 1991.

    Google Scholar 

  122. J. Mackinlay. Automating the design of graphical presentations of relational information.ACM Transactions on Graphics5(2):110–141, 1986.

    Article  Google Scholar 

  123. M.W. Maimone, J.D. Tygar, and J.M. Wing. Formal semantics for visual specification of security. In S.-K. Chang, editorVisual Languages and Visual Programmingpages 97–116. Plenum Press, New York, 1990.

    Google Scholar 

  124. J. Marks. A formal specification for network diagrams that facilitates automated design.Journal of Visual Languages and Computing2:395–414, 1991.

    Article  Google Scholar 

  125. K. Marriott. Constraint multiset grammars. InIEEE Symposium on Visual Languagespages 118–125. IEEE Computer Society Press, 1994.

    Google Scholar 

  126. K. Marriott and P. Stuckey.Programming with Constraints: An Introduction.MIT Press, Cambridge, 1998.

    MATH  Google Scholar 

  127. G. Martin, J. Pittman, K. Wittenburg, R. Cohen, and T. Parish. Sign here, please: State of the art, computing without keyboards.BYTEJuly 1990.

    Google Scholar 

  128. G. Masini and R. Mohr. MIRABELLE: A system for structural analysis of drawings.Pattern Recognition16(4):363–372, 1983.

    Article  Google Scholar 

  129. D. W. McIntyre and E. P. Glinert. Visual tools for generating iconic programming environments. InIEEE Workshop on Visual Languagespages 162–168. IEEE Computer Society Press, 1992.

    Google Scholar 

  130. B. Meyer. Pictures depicting pictures - on the specification of visual languages by visual grammars. InIEEE Workshop on Visual Languagespages 41–47. IEEE Computer Society Press, 1992.

    Google Scholar 

  131. B. Meyer. Logic and the structure of space: Towards a visual logic for spatial reasoning. In Dale Miller, editorInternational Symposium on Logic Programmingpage 674. MIT Press, 1993.

    Google Scholar 

  132. B. Meyer. Pictorial deduction in spatial information systems. InIEEE Symposium on Visual Languagespages 23–30. IEEE Computer Society Press, 1994.

    Google Scholar 

  133. B. Meyer.Visual Logic Languages for Spatial Information Handling (in German).Doctoral thesis, FernUni Hagen, 1994.

    Google Scholar 

  134. B. Meyer. Formalization of visual mathematical notations. InAAAI Fall Symposium on Reasoning with Diagrammatic Representationspages 58–68, Boston, 1997. AAAI Press.

    Google Scholar 

  135. D.L. Milgram and A. Rosenfeld. A note on grammars with coordinates. In F. Nake and A. Rosenfeld, editorsGraphic Languagespages 187–191. North-Holland, 1972.

    Google Scholar 

  136. M. Minas and G. Viehstaedt. Diagen: A generator for diagram editors providing direct manipulation and execution of diagrams. InIEEE Workshop on Visual Languagespages 203–210. IEEE Computer Society Press, 1995.

    Google Scholar 

  137. R. Mohr. Precompilation of syntactical descriptions and knowledge directed analysis of patterns.Pattern Recognition19(4):255–266, 1986.

    Article  Google Scholar 

  138. M. Najork and S. Kaplan. Specifying visual languages with conditional set rewrite systems. InIEEE Symposium on Visual Languagespages 12–17. IEEE Computer Society Press, 1993.

    Google Scholar 

  139. M.A. Najork. Programming in three dimensions.Journal of Visual Languages and Computing7(2):219–242, 1996.

    Article  Google Scholar 

  140. E.G. Noik. A space of presentation emphasis techniques for visualizing graphs. InGraphics Interface (GI ‘84)pages 225–234, Banff, 1994.

    Google Scholar 

  141. W. Older and A. Vellino. Extending Prolog with constraint arithmetic on real intervals. InCanadian Conference on Electrical and Computer EngineeringOttawa, 1990.

    Google Scholar 

  142. S. Orefice, G. Polese, G. Costagliola M. Tucci, G. Tortora, and S.-K. Chang. A 2D interactive parser for iconic languages. InIEEE Symposium on Visual Languagespages 207–213. IEEE Computer Society Press, 1992.

    Google Scholar 

  143. B. Pasternak, G. Gabrielides, and R. Sprengel. Wiz - a prototype for knowledge-based drawing interpretation. InInternational Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systemspages 379–384, Paderborn, 1992.

    Google Scholar 

  144. B. Pasternak and B. Neumann. Adik - an adaptable drawing interpretation kernel. InScientific Conference on AI XPS NLpages 531–540, Avignon, 1993.

    Google Scholar 

  145. C.S. Peirce.Semiotische Schriften.Suhrkamp Verlag, Frankfurt, 1995.

    Google Scholar 

  146. K.J. Peng, T. Yamamoto, and Y. Aoki. A new parsing scheme for plex grammars.Pattern Recognition23:393–402, 1990.

    Article  Google Scholar 

  147. J.L. Pfaltz. Web grammars and picture descriptions.Computer Graphics and Image Processingpages 193–220, 1972.

    Google Scholar 

  148. J.L. Pfaltz and A. Rosenfeld. Web grammars. InProc. First Int. Joint Conference on Artificial Intelligencepages 609–619, 1969.

    Google Scholar 

  149. J.J. Pfeiffer. Parsing graphs representing two dimensional figures. InIEEE Symposium on Visual LanguagesIEEE Computer Society Press, 4, pages 200–206 1992.

    Google Scholar 

  150. A. Pritsker and B. Alan.Introduction to Simulation and SLAM II.John Wiley, New York, 1994.

    Google Scholar 

  151. R. Reiter and A.K. Mackworth. A logical framework for depiction and image interpretation.Artificial Intelligence41:125–155, 1989.

    Article  MathSciNet  MATH  Google Scholar 

  152. J. Rekers and A. Schurr. A graph grammar approach to graphical parsing. InIEEE Symposium on Visual Languagespages 195–202. IEEE Press, 1995.

    Google Scholar 

  153. A. Rosenfeld.Picture Languages.Academic Press, 1979.

    Google Scholar 

  154. S. P. Roth, J. Kolojejchick, J. Mattis, and J. Goldstein. Interactive graphic design using automatic presentation knowledge. InProc. ACM Conf. Human Factors in Computing (CHI)pages 112–117. ACM Press, 1994.

    Google Scholar 

  155. D. Rubine. Specifying gestures by example.Computer Graphics25(4):329–337, 1991.

    Article  Google Scholar 

  156. A. Sanfeliu and K.-S. Fu. Tree graph grammars for pattern recognition. InGraph-Grammars and Their Application to Computer Science (Second International Workshop)volume 153 ofLNCSpages 349–368. Springer-Verlag, New York, 1983.

    Google Scholar 

  157. A.C. Shaw. A formal picture description scheme as a basis for picture processing systems.Information and Control14:9–52, 1969.

    Article  MATH  Google Scholar 

  158. A.C. Shaw. Parsing of graph representable pictures.JACM17:453481, 1970.

    Google Scholar 

  159. Q.Y. Shi and K.-S. Fu. Parsing and translation of (attributed) expansive graph languages for scene analysis.IEEE Transactions on Pattern Analysis and Machine Intelligence5:472–485, 1983.

    Article  MATH  Google Scholar 

  160. S.-J. Shin.The Logical Status of Diagrams.Cambridge University Press, Cambridge, 1995.

    Book  Google Scholar 

  161. S.-J. Shin. Situation theoretic account of valid reasoning with Venn diagrams. In G. Allwein and J. Barwise, editorsLogical Reasoning with Diagramspages 81–108. Oxford University Press, New York, 1996.

    Google Scholar 

  162. N.M. Soiffer.The Design of a User Interface for Computer Algebra Systems.PhD thesis, University of California, 1991.

    Google Scholar 

  163. J. Song. A Glotoslayout tool. Technical Report YCS-92–179, University of York, 1991.

    Google Scholar 

  164. L. Spratt and A. Ambler. Using 3D tubes to solve the intersecting line representation problem. InIEEE Symposium of Visual Languagespages 254–261. IEEE Computer Society Press, 1994.

    Google Scholar 

  165. L. Sterling and E. Shapiro.The Art of Prolog.MIT Press, Cambridge, 1986.

    MATH  Google Scholar 

  166. G. Stiny.Pictorial and Formal Aspects of Shape and Grammars.Birkhauser Verlag, 1975.

    Google Scholar 

  167. G. Stiny and W.J. Mitchell. The palladian grammar.Environment and Planning B: Planning and Design5:5–18, 1978.

    Article  Google Scholar 

  168. L. Stringa. A new set of constraint-free character recognition grammars.IEEE Transactions on Pattern Analysis and Machine Intelligence12:1210–1217, 1990.

    Article  Google Scholar 

  169. L. Spratt and A. Ambler. Using 3D tubes to solve the intersecting line representation problem. InIEEE Symposium of Visual Languagespages 254–261. IEEE Computer Society Press, 1994.

    Google Scholar 

  170. T. Tanaka. Definite clause set grammars: A formalism for problem solving.Journal of Logic Programming10:1–17, 1991.

    Article  MathSciNet  Google Scholar 

  171. G.Y. Tang and T.S. Huang. A syntactic-semantic approach to image understanding and creation.IEEE Transactions on Pattern Analysis and Machine Intelligence1:135–144, 1979.

    Article  Google Scholar 

  172. C.C. Tappert, C.Y. Suen, and T. Wakahara. On-line handwriting recognition - a survey. InNinth International Conference on Pattern Recognitionpages 1123–1132, 1989.

    Google Scholar 

  173. M. Tomita.Efficient Parsing for Natural Language.Kluwer Academic, Boston, 1985.

    Google Scholar 

  174. M. Tomita. Parsing 2-dimensional languages. In M. Tomita, editorCurrent Trends in Parsing Technologypages 277–289. Kluwer Academic, 1991.

    Google Scholar 

  175. W.-H. Tsai and K.-S. Fu. Attributed grammar - a tool for combining syntactic and statistical approaches to pattern recognition.IEEE Transactions on Systems Man and Cybernetics10:873–885, 1980.

    Article  MATH  Google Scholar 

  176. M. Tucci, G. Vitiello, and G. Costagliola. Parsing nonlinear languages.IEEE Transactions on Software Engineering20:720–739, 1994.

    Article  Google Scholar 

  177. M. Tucci, G. Vitiello, G. Pacini, and G. Tortora. Graphs and visual languages for visual interface. InAvanced Visual Interfaces (AVI 92)Series in Computer Science, pages 304–318. World Scientific, 1992.

    Google Scholar 

  178. S. Üsküdarli. Generating visual editors for formally specified languages. InIEEE Symposium on Visual Languagespages 278–287. IEEE Computer Society Press, 1994.

    Google Scholar 

  179. S. Usküdarli and T.B. Dinesh. Input and output for specified visual languages. Inthis volume.

    Google Scholar 

  180. S. Üsküdarli and T.B. Dinesh. Towards a visual programming environment generator for algebraic specifications. InIEEE Symposium on Visual Languagespages 234–241. IEEE Computer Society Press, 1995.

    Google Scholar 

  181. S. Üsküdarli and T.B. Dinesh. The VAS formalism in VASE. InIEEE Symposium on Visual Languagespages 140–147. IEEE Computer Society Press, 1996.

    Google Scholar 

  182. P. Della Vigna and C. Ghezzi. Context-free graph grammarsInformation and Control37:207–233, 1978.

    Article  MathSciNet  MATH  Google Scholar 

  183. VL. IEEE Symposium on Visual Languages.

    Google Scholar 

  184. D. Wang.Studies on the Formal Semantics of Pictures.Doctoral thesis, University of Amsterdam, 1995.

    Google Scholar 

  185. D. Wang and J.R. Lee. Pictorial concepts and a concept supporting graphical system.Journal of Visual Languages and Computing4:177–199, 1993.

    Article  Google Scholar 

  186. D. Wang and J.R. Lee. Visual reasoning• its formal semantics and applications.Journal of Visual Languages and Computing4:327356, 1993.

    Google Scholar 

  187. D. Wang andH.Zeevat. A syntax directed approach to picture semantics. Inthis volume.

    Google Scholar 

  188. L. Weitzman and K. Wittenburg. Relational grammars for interactive design. In ACM Multimedia Conference

    Google Scholar 

  189. L. Weitzman and K. Wittenburg. Automatic presentation of multimedia documents using relational grammars. InACM Multimedia Conferencepages 443–451, 1994.

    Google Scholar 

  190. L. Weitzman and K. Wittenburg. Grammar-based articulation for multimedia document design.Multimedia Systems4:99–111, 1996.

    Article  Google Scholar 

  191. M.B. Wells. MADCAP: A scientific compiler for a displayed formula textbook language.Communications of the ACM4(1):31–36, 1961.

    Article  MATH  Google Scholar 

  192. K. Wittenburg. Earley-style parsing for relational grammars. InIEEE Workshop on Visual Languagespages 192–199. IEEE Computer Society Press, 1992.

    Google Scholar 

  193. K. Wittenburg. F-PATR: Functional constraints for unification grammars. In31st Meeting of the Association for Computational Linguisticspages 216–223, 1993.

    Google Scholar 

  194. K. Wittenburg. Predictive parsing for unordered relational languages. InH.Bunt and M. Tomita, editorsRecent Advances in Parsing Technologypages 385–407. Kluwer, 1996.

    Google Scholar 

  195. K. Wittenburg and L. Weitzman. Visual grammars and incremental parsing for interface languages. InIEEE Symposium on Visual Languagespages 111–118. IEEE Computer Society Press, 1990.

    Google Scholar 

  196. K. Wittenburg, L. Weitzman, and J. Talley. Unification-based grammars and tabular parsing for graphical languages.Journal of Visual Languages and Computing2:347–370, 1991.

    Article  Google Scholar 

  197. S. Wolfram.The Mathematica Book.Cambridge University Press, 1996.

    Google Scholar 

  198. R. Yeung. MPL: A graphical programming environment for matrix processing based on logic and constraints. InIEEE Workshop on Visual Languagespages 137–143. IEEE Computer Society Press, 1988.

    Google Scholar 

  199. K.C. You and K.-S. Fu. A syntactic approach to shape recognition using attributed grammars.IEEE Transactions on Systems Man and Cybernetics9(6):334–345, 1979.

    Article  Google Scholar 

  200. K.C. You and K.-S. Fu. Distorted shape recognition using attributed grammars and error correction techniques.Computer Graphics and Image Processing13:1–16, 1980.

    Article  Google Scholar 

  201. D.H. Younger. Recognition and parsing of context free languages in time n3 . Information and Control10:189–208, 1967.

    Article  MATH  Google Scholar 

Download references

Authors
  1. Kim Marriott
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernd Meyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kent B. Wittenburg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, Monash University, Clayton, Victoria, 3168, Australia

    Kim Marriott

  2. Lehr und Forschungseinrichtung für Programmierung und Softwaretechnik Institut für Informatik, Ludwig Maximillians Universität, Oettingenstrasse 67, Munich, 80538, Germany

    Bernd Meyer

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer Science+Business Media New York

About this chapter

Cite this chapter

Marriott, K., Meyer, B., Wittenburg, K.B. (1998). A Survey of Visual Language Specification and Recognition. In: Marriott, K., Meyer, B. (eds) Visual Language Theory. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-1676-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4612-1676-6_2

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4612-7240-3

  • Online ISBN: 978-1-4612-1676-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation