A Multilingual Sketch-Based Sudoku Game with Real-Time Recognition

  • Caio D. D. MonteiroEmail author
  • Meenakshi Narayanan
  • Seth Polsley
  • Tracy Hammond
Part of the Human–Computer Interaction Series book series (HCIS)


Sudoku is one of the most popular puzzles of all time: easy to understand but still very challenging, Sudoku continues to captivate players all over the world through newspapers, puzzle books, and digital devices. The application introduced in this work is a multilingual, sketch-based version of the Sudoku game. Sketch input renders more flexibility to users and increases usability. Multilingual support paired with sketching allows the game to serve as an educational tool for those learning a new language, with the current implementation supporting Chinese and Hindi. The recognition algorithm proposed in this work, based on the Hausdorff metric, easily enables extending the application to support other languages. Preliminary results indicate an overall accuracy of over 93% when recognizing Chinese and Hindi numbers at the same time.



We thank the Texas A&M University Sketch Recognition Lab, especially Paul Taele, for his guidance, and all the volunteers that helped us train and test the supported languages. Author Caio D. D. Monteiro also gratefully acknowledges the scholarship from CAPES #11886/13-7.


  1. 1.
    Al-Omari, F.: Handwritten Indian numeral recognition system using template matching approaches. In: ACS/IEEE International Conference on Computer Systems and Applications, pp. 83–88. IEEE (2001)Google Scholar
  2. 2.
    Bengio, Y., Grandvalet, Y.: No unbiased estimator of the variance of k-fold cross-validation. J. Mach. Learn. Res. 5, 1089–1105 (2004)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Field, M., Valentine, S., Linsey, J., Hammond, T.: Sketch recognition algorithms for comparing complex and unpredictable shapes. In: IJCAI, pp. 2436–2441 (2011)Google Scholar
  4. 4.
    Hammond, T.: Ladder: a perceptually-based language to simplify sketch recognition user interface development. Ph.D. Doctoral Dissertation, Massachusetts Institute of Technology (MIT), Cambridge (2007). Advisor: Randall Davis. 495 pp.Google Scholar
  5. 5.
    Hammond, T., Davis, R.: Ladder, a sketching language for user interface developers. Comput. Graph. 29(4), 518–532 (2005)CrossRefGoogle Scholar
  6. 6.
    Hammond, T., Paulson, B.: Recognizing sketched multistroke primitives. ACM Trans. Interact. Intell. Syst. 1(1), 4:1–4:34 (2011). doi:10.1145/2030365.2030369.
  7. 7.
    Harlow, H.F., Harlow, M.K., Meyer, D.R.: Learning motivated by a manipulation drive. J. Exp. Psychol. 40(2), 228 (1950)CrossRefGoogle Scholar
  8. 8.
    Huttenlocher, D.P., Klanderman, G., Rucklidge, W.J., et al.: Comparing images using the Hausdorff distance. IEEE Trans. Pattern Anal. Mach. Intell. 15(9), 850–863 (1993)CrossRefGoogle Scholar
  9. 9.
    Iba, G.A.: Learning by discovering macros in puzzle solving. In: IJCAI, pp. 640–642 (1985)Google Scholar
  10. 10.
    Kara, L.B., Stahovich, T.F.: An image-based, trainable symbol recognizer for hand-drawn sketches. Comput. Graph. 29(4), 501–517 (2005)CrossRefGoogle Scholar
  11. 11.
    Mitchell, A., Savill-Smith, C.: The use of computer and video games for learning: a review of the literature (2004)Google Scholar
  12. 12.
    Pandey, A., Kumar, A., Kumar, R., Tiwari, A.: Handwritten devanagri number recognition using majority voting scheme. Int. J. Comput. Sci. Inform. Technol. Secur. (IJCSITS)(2), 2249–9555. ISSN (2012)Google Scholar
  13. 13.
    Paulson, B., Eoff, B., Wolin, A., Johnston, J., Hammond, T.: Sketch-based educational games: drawing kids away from traditional interfaces. In: Proceedings of the 7th International Conference on Interaction Design and Children, pp. 133–136. ACM (2008)Google Scholar
  14. 14.
    Paulson, B., Hammond, T.: Paleosketch: accurate primitive sketch recognition and beautification. In: Proceedings of the 13th International Conference on Intelligent User Interfaces, IUI’08, pp. 1–10. ACM, New York (2008). doi:10.1145/1378773.1378775.
  15. 15.
    Rubine, D.: Specifying Gestures by Example, vol. 25. ACM, New York (1991)Google Scholar
  16. 16.
    Sezgin, T.M., Stahovich, T., Davis, R.: Sketch based interfaces: early processing for sketch understanding. In: ACM SIGGRAPH 2006 Courses, p. 22. ACM (2006)Google Scholar
  17. 17.
    Shahzad, N., Paulson, B., Hammond, T.: Urdu Qaeda: recognition system for isolated urdu characters. In: Proceedings of the Workshop on Sketch Recognition at the 14th International Conference of Intelligent User Interfaces (IUI). ACM, Sanibel (2009)Google Scholar
  18. 18.
    Shapiro, M.D., Blaschko, M.B.: On Hausdorff Distance Measures, vol. 1003. Computer Vision Laboratory University of Massachusetts Amherst, Amherst (2004)Google Scholar
  19. 19.
    Sweller, J.: Cognitive load during problem solving: effects on learning. Cogn. Sci. 12(2), 257–285 (1988)CrossRefGoogle Scholar
  20. 20.
    Sweller, J.: Cognitive technology: some procedures for facilitating learning and problem solving in mathematics and science. J. Educ. Psychol. 81(4), 457 (1989)CrossRefGoogle Scholar
  21. 21.
    Taele, P., Hammond, T.: Chinese characters as sketch diagrams using a geometric-based approach. In: Proceedings of the 2008 IEEE Symposium on Visual Languages and Human-Centric Computing Workshop on Sketch Tools for Diagramming, pp. 74–82 (2008)Google Scholar
  22. 22.
    Taele, P., Hammond, T.: Using a geometric-based sketch recognition approach to sketch Chinese radicals. In: AAAI, pp. 1832–1833 (2008)Google Scholar
  23. 23.
    Taele, P., Hammond, T.: Hashigo: a next-generation sketch interactive system for Japanese kanji. In: Proceedings of the Twenty-First Innovative Applications of Artificial Intelligence Conference (IAAI), pp. 153–158. AAAI, Pasadena (2009)Google Scholar
  24. 24.
    Taele, P., Hammond, T.: Enhancing instruction of written east asian languages with sketch recognition-based intelligent language workbook interfaces. In: Hammond, T., Valentine, S., Adler, A., Payton M. (eds.) The Impact of Pen and Touch Technology on Education, 1st edn., pp. 119–126. Springer, Cham (2015)CrossRefGoogle Scholar
  25. 25.
    Taele, P., Hammond, T.: An intelligent sketch-based educational interface for learning complex written east asian phonetic symbols. In: Hammond, T., Valentine, S., Adler, A. (eds.) Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education, pp. 129–140. Springer International Publishing, Cham (2016). doi:10.1007/978-3-319-31193-7_9.
  26. 26.
    Taele, P., Hammond, T.A.: A geometric-based sketch recognition approach for handwritten mandarin phonetic symbols I. In: 2008 International Workshop on Visual Languages and Computing (VLC) at the 14th International Conference on Distributed Multimedia Systems (DMS). Knowledge Systems Institute, Boston (2008). 6 pp.Google Scholar
  27. 27.
    Torrente, J., Del Blanco, Á., Marchiori, E.J., Moreno-Ger, P., Fernández-Manjón, B.: ¡ e-adventure¿: introducing educational games in the learning process. In: Education Engineering (EDUCON), 2010 IEEE, pp. 1121–1126. IEEE (2010)Google Scholar
  28. 28.
    Virvou, M., Katsionis, G., Manos, K.: Combining software games with education: evaluation of its educational effectiveness. J. Educ. Technol. Soc. 8(2), 54–65 (2005)Google Scholar
  29. 29.
    Wolin, A., Eoff, B., Hammond, T.: Shortstraw: a simple and effective corner finder for polylines. In: Proceedings of the Fifth Eurographics Conference on Sketch-Based Interfaces and Modeling, SBM’08, pp. 33–40. Eurographics Association, Aire-la-Ville (2008). doi:10.2312/SBM/SBM08/033-040.

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Caio D. D. Monteiro
    • 1
    Email author
  • Meenakshi Narayanan
    • 1
  • Seth Polsley
    • 1
  • Tracy Hammond
    • 1
  1. 1.Sketch Recognition Lab, Department of Computer Science & EngineeringTexas A&M UniversityCollege StationUSA

Personalised recommendations