Abstract
In this chapter we describe “MobileEye”, a software suite which converts a camera enabled mobile device into a multi-function vision tool that can assist the visually impaired in their daily activities. MobileEye consists of four subsystems, each customized for a specific type of visual disabilities: A color channel mapper which can tell the visually impaired different colors; a software based magnifier which provides image magnification as well as enhancement; a pattern recognizer which can read currencies; and a document retriever which allows access to printed materials. We developed cutting edge computer vision and image processing technologies, and tackled the challenges of implementing them on mobile devices with limited computational resources and low image quality. The system minimizes keyboard operation for the usability of users with visual impairments. Currently the software suite runs on Symbian and Windows Mobile handsets. In this chapter we provides a high level overview of the system, and then discuss the pattern recognizer in detail. The challenge is how to build a real-time recognition system on mobile devices and we present our detailed solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Massof, R.W., Hsu, C.T., Barnett, G.D., Baker, F.H.: Visual Disability Variables. I,II: The Importance and Difficulty of Activity Goals for a Sample of Low-Vision Patients. Archives of Physical Medicine and Rehabilitation 86(5), 946–953 (2005)
Milanesi, C., Zimmermann, A., Shen, S.: Forecast: Camera phones, worldwide, 2004-2010. Gartner Inc. Report No. G00144253 (2006)
Kindberg, T., Spasojevic, M., Fleck, R., Sellen, A.: The ubiquitous camera: An in-depth study of camera phone use. IEEE Pervasive Computing 4(2), 42–50 (2005)
Karlson, A., Bederson, B., Contreras-Vidal, J.: Understanding One-Handed Use of Mobile Devices. In: Handbook of Research on User Interface Design and Evaluation for Mobile Technology (2008)
Hull, J., Erol, B., Graham, J., Ke, Q., Kishi, H., Moraleda, J., Van Olst, D.: Paper-Based Augmented Reality. In: 17th International Conference on Artificial Reality and Telexistence, pp. 205–209 (2007)
Liu, X., Doermann, D.: Mobile Retriever: access to digital documents from their physical source. International Journal on Document Analysis and Recognition 11(1), 19–27 (2008)
Freund, Y., Schapire, R.: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)
http://www.acb.org/press-releases/final-edit-paper-currency-ruling-080520.html
Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2 (2003)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)
Specht, D.: Probabilistic neural networks. Neural Networks 3(1), 109–118 (1990)
Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T., Schmalstieg, D.: Pose tracking from natural features on mobile phones. In: 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, ISMAR 2008, pp. 125–134 (2008)
Özuysal, M., Fua, P., Lepetit, V.: Fast keypoint recognition in ten lines of code. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8 (2007)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)
Viola, P., Jones, M.: Robust real-time face detection. International Journal on Computer Vision 57(2), 137–154 (2004)
Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Liu, X., Doermann, D., Li, H. (2010). Mobile Visual Aid Tools for Users with Visual Impairments. In: Jiang, X., Ma, M.Y., Chen, C.W. (eds) Mobile Multimedia Processing. WMMP 2008. Lecture Notes in Computer Science, vol 5960. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12349-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-12349-8_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12348-1
Online ISBN: 978-3-642-12349-8
eBook Packages: Computer ScienceComputer Science (R0)