DCSR: A Digital Circuit Sketch Recognition System for Education

  • Shuo MaEmail author
  • Yongbin Sun
  • Pengchen Lyu
  • Seth Polsley
  • Tracy Hammond
Part of the Human–Computer Interaction Series book series (HCIS)


Digital logic is an important part of any engineering curriculum in today’s digital era, and it is often taught visually through circuit diagrams. However, for students just learning logic, this process can be non-interactive, with students typically drawing and solving diagrams that will only be evaluated by a human grader later. This paper presents DCSR (Digital Circuit Sketch Recognition), a system that recognizes hand-drawn digital logic circuits through a web interface and calculates the truth value of its output based on students’ input. It allows users to draw freely and gives immediate feedback; DCSR aims to provide an interactive, sketch-based approach for educators to assist students in learning digital logic. It was evaluated by 15 electrical engineering students.



We would like to thank all the volunteers who tested our system and provided feedback. We would also like to thank the Computer Science and Engineering department at Texas A&M University and the members of the Sketch Recognition Lab.


  1. 1.
    Alvarado, C.: Sketch recognition for digital circuit design in the classroom. In: 2007 Invited Workshop on Pen-Centric Computing Research. Citeseer (2007)Google Scholar
  2. 2.
    Alvarado, C., Davis, R.: Dynamically constructed bayes nets for multi-domain sketch understanding. In: ACM SIGGRAPH 2006 Courses, p. 32. ACM (2006)Google Scholar
  3. 3.
    Alvarado, C., Kearney, A., Keizur, A., Loncaric, C., Parker, M., Peck, J., Sobel, K., Tay, F.: Logisketch: a free-sketch digital circuit design and simulation system. In: Proceedings of the Workshop on the Impact of Pen and Touch Technologies in Education, WIPTTE’13 (2013)Google Scholar
  4. 4.
    Alvarado, C., Lazzareschi, M.: Properties of real-world digital logic diagrams. In: First International Workshop on Pen-Based Learning Technologies (PLT 2007), pp. 1–6. IEEE (2007)Google Scholar
  5. 5.
    Anthony, L., Wobbrock, J.O.: A lightweight multistroke recognizer for user interface prototypes. In: Proceedings of Graphics Interface 2010, pp. 245–252. Canadian Information Processing Society (2010)Google Scholar
  6. 6.
    Anthony, L., Wobbrock, J.O.: $ n-protractor: a fast and accurate multistroke recognizer. In: Proceedings of Graphics Interface 2012, pp. 117–120. Canadian Information Processing Society (2012)Google Scholar
  7. 7.
    Atilola, O., McTigue, E.M., Hammond, T., Linsey, J.: Mechanix: evaluating the effectiveness of a sketch recognition truss tutorin program against other truss programs. In: 120th American Society for Engineering Education Annual Conference & Exposition (ASEE), ASEE, Atlanta, 23–26 June 2013, 15 pp.Google Scholar
  8. 8.
    Atilola, O., Valentine, S., Kim, H.H., Turner, D., McTigue, E., Hammond, T., Linsey, J.: Mechanix: a natural sketch interface tool for teaching truss analysis and free-body diagrams. Artif. Intell. Eng. Des. Anal. Manuf. 28(2), 169–192 (2014)CrossRefGoogle Scholar
  9. 9.
    Calhoun, C., Stahovich, T.F., Kurtoglu, T., Kara, L.B.: Recognizing multi-stroke symbols. In: AAAI Spring Symposium on Sketch Understanding, pp. 15–23 (2002)Google Scholar
  10. 10.
    Dixon, D., Prasad, M., Hammond, T.: icandraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 897–906. ACM (2010)Google Scholar
  11. 11.
    Edwards, B., Chandran, V.: Machine recognition of hand-drawn circuit diagrams. In: Proceedings 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2000), vol. 6, pp. 3618–3621. IEEE (2000)Google Scholar
  12. 12.
    Feng, G., Sun, Z., Viard-Gaudin, C.: Hand-drawn electric circuit diagram understanding using 2D dynamic programming. In: IAPR, pp. 493–498 (2008)Google Scholar
  13. 13.
    Feng, G., Viard-Gaudin, C., Sun, Z.: On-line hand-drawn electric circuit diagram recognition using 2D dynamic programming. Pattern Recogn. 42(12), 3215–3223 (2009)CrossRefzbMATHGoogle Scholar
  14. 14.
    Field, M., Valentine, S., Linsey, J., Hammond, T.: Sketch recognition algorithms for comparing complex and unpredictable shapes. In: IJCAI, pp. 2436–2441 (2011)Google Scholar
  15. 15.
    Fitzgerald, J., Geiselbrechtinger, F., Kechadi, T., et al.: Mathpad: a fuzzy logic-based recognition system for handwritten mathematics. In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), vol. 2, pp. 694–698. IEEE (2007)Google Scholar
  16. 16.
    Hammond, T.: Ladder: a perceptually-based language to simplify sketch recognition user interface development. Ph.D. Doctoral Dissertation, Massachusetts Institute of Technology (MIT), Cambridge (2007). Advisor: Randall Davis, 495 pp.Google Scholar
  17. 17.
    Hammond, T., Davis, R.: Tahuti: a geometrical sketch recognition system for UML class diagrams. In: Technical Report SS-02-08: Papers from the 2002 Association for the Advancement of Artificial Intelligence (AAAI) Spring Symposium on Sketch Understanding. AAAI, Menlo Park, 8 p., (2002)Google Scholar
  18. 18.
    Hammond, T., Davis, R.: Ladder, a sketching language for user interface developers. Comput. Graph. 29(4), 518–532 (2005)CrossRefGoogle Scholar
  19. 19.
    Hammond, T., Paulson, B.: Recognizing sketched multistroke primitives. ACM Trans. Interact. Intell. Syst. 1(1), 4:1–4:34 (2011). doi:10.1145/2030365.2030369.
  20. 20.
    Hong, J.I., Landay, J.A.: SATIN: A Toolkit for Informal Ink-Based Applications. ACM, New York (2007)CrossRefGoogle Scholar
  21. 21.
    Kang, B., LaViola, J.: Logicpad: a pen-based application for visualization and verification of boolean algebra. In: Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces, pp. 265–268. ACM (2012)Google Scholar
  22. 22.
    Kara, L.B., Gennari, L., Stahovich, T.F.: A sketch-based interface for the design and analysis of simple vibratory mechanical systems. In: ASME 2004 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pp. 1047–1057. American Society of Mechanical Engineers (2004)Google Scholar
  23. 23.
    Kara, L.B., Stahovich, T.F.: Hierarchical parsing and recognition of hand-sketched diagrams. In: ACM SIGGRAPH 2007 Courses, p. 17. ACM (2007)Google Scholar
  24. 24.
    Lin, J., Newman, M.W., Hong, J.I., Landay, J.A.: Denim: finding a tighter fit between tools and practice for web site design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 510–517. ACM (2000)Google Scholar
  25. 25.
    Liwicki, M., Knipping, L.: Recognizing and simulating sketched logic circuits. In: International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, pp. 588–594. Springer (2005)Google Scholar
  26. 26.
    Nelligan, T., Polsley, S., Ray, J., Helms, M., Linsey, J., Hammond, T.: Mechanix: a sketch-based educational interface. In: Proceedings of the 2015 ACM International Conference on Intelligent User Interface (2015)Google Scholar
  27. 27.
    Oltmans, M.: Envisioning sketch recognition: a local feature based approach to recognizing informal sketches. Ph.D. thesis, Massachusetts Institute of Technology (2007)Google Scholar
  28. 28.
    Ouyang, T.Y., Davis, R.: Recognition of hand drawn chemical diagrams. In: Proceedings of the National Conference on Artificial Intelligence, vol. 22, p. 846. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999 (2007)Google Scholar
  29. 29.
    Polsley, S., Ray, J., Nelligan, T., Helms, M., Linsey, J., Hammond, T.: Leveraging trends in student interaction to enhance the effectiveness of sketch-based educational software. In: Hammond, A.A.T., Valentine, S., Payton, M. (eds.) The Impact of Pen and Touch Technology on Education. Part II, Chapter 6. Springer, New York (2015)Google Scholar
  30. 30.
    Polsley, S., Ray, J., Nelligan, T., Helms, M., Linsey, J., Hammond, T.: Leveraging trends in student interaction to enhance the effectiveness of sketch-based educational software. In: Revolutionizing Education with Digital Ink, pp. 103–114. Springer (2016)Google Scholar
  31. 31.
    Rubine, D.: Specifying Gestures by Example, vol. 25. ACM (1991)Google Scholar
  32. 32.
    Sezgin, T.M., Davis, R.: Sketch recognition in interspersed drawings using time-based graphical models. Comput. Graph. 32(5), 500–510 (2008)CrossRefGoogle Scholar
  33. 33.
    Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H.h., Li, W., Linsey, J., Hammond, T.: Mechanix: a sketch-based tutoring and grading system for free-body diagrams. AI Mag. 34(1), 55 (2012)Google Scholar
  34. 34.
    Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H.h., Li, W., Linsey, J., Hammond, T.: Mechanix: a sketch-based tutoring system for statics courses. In: Proceedings of the Twenty-Fourth Innovative Applications of Artificial Intelligence Conference (IAAI), pp. 2253–2260. AAAI, Toronto (2012)Google Scholar
  35. 35.
    Vatavu, R.D., Anthony, L., Wobbrock, J.O.: Gestures as point clouds: a $ p recognizer for user interface prototypes. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, pp. 273–280. ACM (2012)Google Scholar
  36. 36.
    Wais, P., Wolin, A., Alvarado, C.: Designing a sketch recognition front-end: user perception of interface elements. In: Proceedings of the 4th Eurographics Workshop on Sketch-Based Interfaces and Modeling, pp. 99–106. ACM (2007)Google Scholar
  37. 37.
    Wobbrock, J.O., Wilson, A.D., Li, Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, pp. 159–168. ACM (2007)Google Scholar
  38. 38.
    Wolin, A., Eoff, B., Hammond, T.: Shortstraw: a simple and effective corner finder for polylines. In: SBM, pp. 33–40 (2008)Google Scholar
  39. 39.
    Zesheng, S., Jing, Y., Chunhong, J., Yonggui, W.: Symbol recognition in electronic diagrams using decision tree. In: Proceedings of the IEEE International Conference on Industrial Technology, 1994, pp. 719–723. IEEE (1994)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Shuo Ma
    • 1
    Email author
  • Yongbin Sun
    • 1
  • Pengchen Lyu
    • 1
  • Seth Polsley
    • 1
  • Tracy Hammond
    • 1
  1. 1.Sketch Recognition Lab, Department of Computer Science & EngineeringTexas A&M UniversityCollege StationUSA

Personalised recommendations