Abstract
The advancements in Artificial Intelligence (AI) technologies in the last decade have led many developers, universities, and researchers to invest in the research and development of digital teaching platforms. This paper compares existing features implemented in the first prototype and provides visually impaired high-school students with a system to learn and practice mathematical problems. This project describes possibilities provided by open-source APIs such as Natural Language Toolkit (NLTK) and shows how to implement them to make education universal and accessible. We aim to achieve this by introducing an open-source platform, which will provide a set of modules for personalized speech recognition and accessible online tutoring. The system is divided into two modules: First, an intelligent conversational agent to help students learn new mathematical concepts, which opens up possibilities of providing students with a personalized response. Second, a mathematical expression parser and evaluator to help the student practice mathematical questions. The implementation and development of the system shall open doors towards self-directed learning for visually impaired students.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
The effect of lockdown and social distancing on blind and partially sighted people. https://www.rnib.org.uk/campaigning/priority-campaigns/inclusive-journeys/effect-lockdown-and-social-distancing-blind-and-partially-sighted-people. Accessed 05 Feb 2021
An extensive math library for JavaScript and Node.js. https://mathjs.org/. Accessed 05 Feb 2021
Get unstuck: learn better. https://socratic.org/. Accessed 05 Feb 2021
The pandemic is undermining visually impaired people’s independence. https://theconversation.com/the-pandemic-is-undermining-visually-impaired-peoples-independence-heres-how-to-fix-this-142209. Accessed 05 Feb 2021
Stepping into math: open-sourcing our step-by-step solver. https://blog.socratic.org/stepping-into-math-open-sourcing-our-step-by-step-solver-9b5da066ae36. Accessed 05 Feb 2021
Abdul-Kader, S.A., Woods, J.: Question answer system for online feedable new born Chatbot. In: 2017 Intelligent Systems Conference (IntelliSys), pp. 863–869 (2017). https://doi.org/10.1109/IntelliSys.2017.8324231
Alajarmeh, N., Pontelli, E., Son, T.: From “reading” math to “doing” math: a new direction in non-visual math accessibility. In: Stephanidis, C. (ed.) UAHCI 2011. LNCS, vol. 6768, pp. 501–510. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21657-2_54
Arlinwibowo, J., Retnawati, H.: Developing audio tactile for visually impaired students. Int. J. New Trends Educ. Implic. 6(4), 78–90 (2015)
Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly Media, Sebastopol (2009)
Bouck, E.C., Weng, P.L.: Reading math: a comparison of reading and listening to algebraic problems. J. Spec. Educ. Technol. 29(4), 1–13 (2014). https://doi.org/10.1177/016264341402900401
Cervone, D., Sorge, V.: Adaptable accessibility features for mathematics on the web. In: Proceedings of the 16th Web for All 2019 Personalization - Personalizing the Web, W4A 2019. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3315002.3317567
DePountis, V.M., Pogrund, R.L., Griffin-Shirley, N., Lan, W.Y.: Technologies used in the study of advanced mathematics by students who are visually impaired in classrooms: teachers’ perspectives. J. Vis. Impair. Blind. 109(4), 265–278 (2015). https://doi.org/10.1177/0145482X1510900403
Gulley, A.P., Smith, L.A.C., Price, J.A., Prickett, L.C., Ragland, M.F.: Process-driven math: an auditory method of mathematics instruction and assessment for students who are blind or have low vision (2017)
Hewett, T.T., et al.: ACM SIGCHI curricula for human-computer interaction. Technical report, New York, NY, USA (1992)
Blackorby, J., Chorost, M., Garza, N., Guzman, A.: The academic performance of secondary school students with disabilities (2003)
Jariwala, A., Marghitu, D., Chapman, R.: MyA+ Math: teaching math to students with vision impairment. In: Antona, M., Stephanidis, C. (eds.) HCII 2020. LNCS, vol. 12189, pp. 200–211. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49108-6_15
Lawrence, D.O., Ashleigh, M.: Impact of Human-Computer Interaction (HCI) on users in higher educational system: Southampton University as a case study. Int. J. Manag. Technol. 6(3), 1–12 (2019)
mxparser - math expressions parser for java android c#.net/mono/xamarin - mathematical formula parser/evaluator library. http://mathparser.org/. Accessed 05 Feb 2021
Create your chatbot using Python NLTK. https://medium.com/@ritidass29/create-your-chatbot-using-python-nltk-88809fa621d1. Accessed 05 Feb 2021
Regec, V.: Mathematics in inclusive education of blind students in secondary schools in the Czech Republic. Procedia Soc. Behav. Sci. 174, 3933–3939 (2015). https://doi.org/10.1016/j.sbspro.2015.01.1136. https://www.sciencedirect.com/science/article/pii/S1877042815011957. International Conference on New Horizons in Education, INTE 2014, Paris, France, 25–27 June 2014
Sheikh, W., Schleppenbach, D., Leas, D.: MathSpeak: a non-ambiguous language for audio rendering of MathML. Int. J. Learn. Technol. 13(1), 3–25 (2018). https://doi.org/10.1504/IJLT.2018.091609
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jariwala, A., Marghitu, D., Chapman, R. (2021). A Multimodal Platform to Teach Mathematics to Students with Vision-Impairment. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Access to Media, Learning and Assistive Environments. HCII 2021. Lecture Notes in Computer Science(), vol 12769. Springer, Cham. https://doi.org/10.1007/978-3-030-78095-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-78095-1_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78094-4
Online ISBN: 978-3-030-78095-1
eBook Packages: Computer ScienceComputer Science (R0)