Abstract
Literacy in written phonetic symbols for East Asian languages is important but challenging for novice language students with only English language fluency, but pedagogical approaches such as rote writing practice and written technique have successfully assisted students towards written mastery. Researchers and developers are also adapting these approaches into intelligent educational apps for students to exploit interactive computing technologies for learning written East Asian language phonetic symbols, whether they are full-time students in K12 or higher education enrolled in conventional classrooms or non-traditional students studying the subject as a passionate interest. However, related pen and touch educational computing apps for sketching practice of East Asian language phonetic symbols provide limited assessment and flexibility of students’ input and sketching style, while related recognition systems either focus more on expert users’ writing styles or cannot provide assessment for more complex phonetic symbols. In this article, we describe our preliminary work on an intelligent sketch-based educational interface developed specifically for assessing students’ sketched input of complex East Asian language phonetic symbols. The interface system relies on template matching from expert users’ sketched training data and various heuristics for assessing the visual structure and technical correctness of students’ more complex written phonetic symbols. From our evaluations of separate sketching data from both novice and expert writers, we were able to achieve reasonably robust performance for both visual structure and technical correctness of our workbook interface for complex written East Asian language phonetic symbols.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anthony L, Wobbrock J (2012) \({\$}\)N-protractor: a fast and accurate multistroke recognizer. In: Proceedings of graphics interface 2012, GI ’12, Canadian Information Processing Society, Mississauga, pp 117–120
Cornerstone Studio: Hiragana - read and write [application] (2015). Accessed 30 June 2015
Dixon D, Prasad M, Hammond T (2010) iCanDraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’10. ACM, New York, pp 897–906
Hammond T, Davis R (2003) LADDER: a language to describe drawing, display, and editing in sketch recognition. In: Proceedings of the 18th international joint conference on artificial intelligence, IJCAI ’03. AAAI, Palo Alto, pp 461–467
LaViola J Jr (2007) Advances in mathematical sketching: moving toward the paradigm’s full potential. IEEE Comput Graph Appl 27(1):38–48
Legendarya/Imaginactiva: Hiragana - learn Japanese [application] (2015). Accessed 14 Jan 2016
Paulson B, Hammond T (2008) Paleosketch: accurate primitive sketch recognition and beautification. In: Proceedings of the 13th international conference on intelligent user interfaces, IUI ’08. ACM, New York, pp 1–10
Shahzad N, Paulson B, Hammond T (2009) Urdu qaeda: recognition system for isolated urdu characters. In: Proceedings of the workshop on sketch recognition at the 14th international conference of intelligent user interfaces, IUI-SR ’09
Sigma Sky, LLC: Write bopomofo [application] (2015). Accessed 14 Jan 2016
Taele P, Hammond T (2009) Hashigo: a next-generation sketch interactive system for Japanese kanji. In: Proceedings of the twenty-first innovative applications artificial intelligence conference, IAAI ’09. AAAI, Palo Alto, pp 153–158
Taele P, Hammond T (2010) Lamps: a sketch recognition-based teaching tool for mandarin phonetic symbols i. J Vis Lang Comput 21(2):109–120
Taele P, Hammond T (2015) Enhancing instruction of written east asian languages with sketch recognition-based “intelligent language workbook” interfaces. In: Hammond T, Valentine S, Adler A, Payton M (eds) The impact of pen and touch technology on education. Springer, Berlin, pp 119–126
Taele P, Barreto L, Hammond T (2015) Maestoso: an intelligent educational sketching tool for learning music theory. In: Proceedings of the twenty-seventh innovative applications of artificial intelligence conference, IAAI ’15. AAAI, Palo Alto, pp 3999–4005
Valentine S, Field M, Hammond T, Smith A (2011) A shape comparison technique for use in sketch-based tutoring systems. In: Proceedings of the 2011 intelligent user interfaces workshop on sketch recognition, IUI SR ’11
Valentine S, Vides F, Lucchese G, Turner D, Kim Hh, Li W, Linsey J, Hammond T (2012) Mechanix: A sketch-based tutoring system for statics courses. In: Proceedings of the twenty-fourth innovative applications of artificial intelligence conference, IAAI ’12. AAAI, Palo Alto, pp 2253–2260
Valentine S, Lara-Garduno R, Linsey J, Hammond T (2015) Mechanix: a sketch based tutoring system that automatically corrects hand-sketched statics homework, The impact of pen and touch technology on education
Vatavu RD, Anthony L, Wobbrock JO (2012) Gestures as point clouds: a \({\$}\)p recognizer for user interface prototypes. In: Proceedings of the 14th ACM international conference on multimodal interaction. ACM, New York, pp 273–280
Wobbrock J, Wilson A, Li Y (2007) Gestures without libraries, toolkits or training: a \({\$}\)1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on user interface software and technology, UIST ’07. ACM, New York, pp 159–168
Wolin A, Eoff B, Hammond T (2008) Shortstraw: a simple and effective corner finder for polylines. In: Proceedings of the fifth eurographics conference on sketch-based interfaces and modeling, SBIM ’08. Eurographics Association, Geneva, pp 33–40
Xiong Y, LaViola J Jr (2010) Technical section: a shortstraw-based algorithm for corner finding in sketch-based interfaces. Comput Graph 34(5):513–527
Acknowledgments
I would first like to give my great appreciation to the members of the Sketch Recognition Lab for sharing their many combined years of experience in shaping the direction of this research project. There have also been many influential East Asian language instructors in my life that allowed me to incorporate my language knowledge into intelligent sketch-based educational applications, especially to my Japanese language instructors Mr. George Adams and Ms. Kazue Kurokawa from Texas A&M University’s Department of International Studies, and my Chinese language instructor Dr. Wen-Hua Teng from the University of Texas’ Department of Asian Studies. Lastly, this project is funded in part by Microsoft (Microsoft Surface Hub Grant: Sketch and Gesture Recognition for Collaborative and Design Interfaces on the Surface Hub).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Taele, P., Hammond, T. (2016). An Intelligent Sketch-Based Educational Interface for Learning Complex Written East Asian Phonetic Symbols. In: Hammond, T., Valentine, S., Adler, A. (eds) Revolutionizing Education with Digital Ink. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-31193-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-31193-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-31191-3
Online ISBN: 978-3-319-31193-7
eBook Packages: Computer ScienceComputer Science (R0)