SketchSPORE: A Sketch Based Domain Separation and Recognition System for Interactive Interfaces

  • Danilo Avola
  • Luigi Cinque
  • Giuseppe Placidi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8157)


Multimodal interfaces are used to interact with devices and automata using different channels of communication. In this context, the sketch modality plays a key role since it allows users to convey concepts and/or commands using freehand drawing (graphical domain) and/or handwriting (textual domain). The acquisition of the sketch modality can be performed using touch (e.g., touchscreen) or touchless (e.g., RGB-D camera) tools supporting the development of versatile and powerful interactive interfaces. Domain separation and sketch recognition are two fundamental issues of these interfaces. This paper presents SketchSPORE a novel framework designed both to automatically distinguish graphical from textual elements within the same sketch and to recognize freehand drawing as well as handwriting. The recognition processes support both on-line and off-line modes, moreover their processing can be suitably stored within an XML file to provide a means to maintain the compatibility between the framework and service and/or application targets. Extensive experiments showing the effectiveness of the proposed method are reported and discussed.


multimodal interfaces sketch recognition freehand drawing handwriting graphical domain textual domain SketchML 


  1. 1.
    Alvarado, C., Davis, R.: Sketchread: a multi-domain sketch recognition engine. In: ACM Symposium on User Interface Software and Technology, pp. 23–32 (2004)Google Scholar
  2. 2.
    Alvarado, C., Davis, R.: Dynamically constructed bayes nets for multi-domain sketch understanding. In: ACM SIGGRAPH 2007 Courses, SIGGRAPH 2007, pp. 1–6. ACM, New York (2007)Google Scholar
  3. 3.
    Avola, D., Bottoni, P., Dafinei, A., Labella, A.: Fcbd: An agent-based architecture to support sketch recognition interfaces. In: Proceedings of the 17th International Conference on Distributed Multimedia Systems, DMS 2011, pp. 295–300. Knowledge Systems Institute, Convitto della Calza (2011)Google Scholar
  4. 4.
    Avola, D., Del Buono, A., Gianforme, G., Paolozzi, S.: A novel recognition approach for sketch-based interfaces. In: Foggia, P., Sansone, C., Vento, M. (eds.) ICIAP 2009. LNCS, vol. 5716, pp. 1015–1024. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Avola, D., Del Buono, A., Del Nostro, P., Wang, R.: A novel online textual/graphical domain separation approach for sketch-based interfaces. In: Damiani, E., Jeong, J., Howlett, R.J., Jain, L.C. (eds.) New Directions in Intelligent Interactive Multimedia Systems and Services - 2. SCI, vol. 226, pp. 167–176. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Avola, D., Buono, A.D., Gianforme, G., Paolozzi, S., Wang, R.: Sketchml a representation language for novel sketch recognition approach. In: Proceedings of the 2nd International Conference on Pervasive Technologies Related to Assistive Environments, PETRA 2009, pp. 1–8. ACM, New York (2009)Google Scholar
  7. 7.
    Avola, D., Cinque, L., Placidi, G.: A novel multimodal framework to support advanced human-computer interaction. In: Proceedings of the 6th Group of Italian Researchers in Pattern Recognition, GIRPR 2012, pp. 1–12. GIRPR Press, Certosa di Pontignano (2012)Google Scholar
  8. 8.
    Bhat, A., Hammond, T.: Using entropy to distinguish shape versus text in hand-drawn diagrams. In: Proceedings of the 21st International Jont Conference on Artifical Intelligence, IJCAI 2009, pp. 1395–1400. Morgan K. Publishers Inc., USA (2009)Google Scholar
  9. 9.
    Bishop, C.M., Svensn, M.: Distinguishing text from graphics in on-line handwritten ink. In: Proceedings of the 9th International Workshop on Frontiers in Handwriting Recognition, IWFHR 2004, pp. 142–147. IEEE Computer Society, USA (2004)Google Scholar
  10. 10.
    Blagojevic, R., Plimmer, B., Grundy, J., Wang, Y.: Using data mining for digital ink recognition: Dividing text and shapes in sketched diagrams. Computers & Graphics 35(5), 976–991 (2011)CrossRefGoogle Scholar
  11. 11.
  12. 12.
    Fonseca, M., Pimentel, C., Jorge, J.: Cali: An online scribble recognizer for calligraphic interfaces. In: AAAI 2002 Spring Symposium, AAAI 2002, pp. 51–58 (2002)Google Scholar
  13. 13.
    Fonseca, M.J., Jorge, J.A.: Experimental evaluation of an on-line scribble recognizer. Pattern Recognition Letters 22(12), 1311–1319 (2001)CrossRefzbMATHGoogle Scholar
  14. 14.
    Hammond, T., Davis, R.: Ladder: A language to describe drawing, display, and editing in sketch recognition. In: Gottlob, G., Walsh, T. (eds.) Proceedings of the 8th Int. Joint Conf. on Art. Intel., IJCAI 2003, pp. 461–467. M. Kaufmann (2003)Google Scholar
  15. 15.
    Hammond, T., Davis, R.: Ladder, a sketching language for user interface developers. Comput. Graph. 29(4), 518–532 (2005)CrossRefGoogle Scholar
  16. 16.
  17. 17.
    Plamondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: A comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(1), 63–84 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Danilo Avola
    • 1
  • Luigi Cinque
    • 2
  • Giuseppe Placidi
    • 1
  1. 1.Department of Life, Health and Environmental SciencesUniversity of L’AquilaL’AquilaItaly
  2. 2.Department of Computer ScienceSapienza UniversityRomeItaly

Personalised recommendations