Skip to main content
SpringerLink
Log in

Support System Using Microsoft Kinect and Mobile Phone for Daily Activity of Visually Impaired

  • Chapter
  • First Online:
Transactions on Engineering Technologies

Abstract

The aim of this paper is to outline a system based on Microsoft Kinect and mobile devices that will provide assistant to visually impaired people. Our primary goal is to provide navigation aid that will help visually impaired to navigate. This includes detection and identification of face, texts and chairs. This is implemented using Microsoft Kinect and machine learning methods are used for this process as it requires rough identification of object. For data acquisition and processing, OpenCV, OpenKinect, Tesseract and Espeak are used. Features that have been incorporated for building this aiding tool are object detection and recognition, face detection and recognition, object location determination, optical character recognition and audio feedback. The face recognition system showed an accuracy of 90 %, the text recognition yielded an accuracy of 65 % and the chairs are recognized with more than 74 % accuracy. To identify denominations of bank notes, more accurate recognition is required. Mobile phone is used to identify bank note denomination. The proposed system can recognize Bangladeshi paper currency notes with 89.4 % accuracy on plain paper background and with 78.4 % accuracy tested on a complex background.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. M. Ridwan, E. Choudhury, B. Poon, M.A. Amin, H. Yan, A navigational aid system for visually impaired using microsoft kinect, in Proceedings of the International MultiConference of Engineers and Computer Scientists, IMECS 2014, 12–14 Mar 2014, Hong Kong. Lecture Notes in Engineering and Computer Science, pp. 417–422

    Google Scholar 

  2. A. Opelt, M. Fussenegger, A. Pinz, P. Auer, Weak hypotheses and boosting for generic object detection and recognition, in Proceedings of the 8th European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 3022, pp. 71–84 (2004)

    Google Scholar 

  3. D.G. Lowe, Object recognition from local scale-invariant features. IEEE Trans. Pattern Anal. Mach. Intell. 2, 1150–1157 (1999)

    Google Scholar 

  4. R. Fergus, P. Perona, A. Zisserman, Object class recognition by unsupervised scale-invariant learning. Proc. Comput. Vis. Pattern Recogn. 2, 264–271 (2003)

    Google Scholar 

  5. S. Mahamud, M. Hebert, J. Shi, Object recognition using boosted discriminants. Proc. Comput. Vis. Pattern Recogn. 1, 551–558 (2001)

    Google Scholar 

  6. P. Viola, M. Jones, Rapid object detection using boosted cascade of simple features, in Proceedings of the Conference on Computer Vision and Pattern Recognition (2009)

    Google Scholar 

  7. S. Sclaroff, A. Pentland, Modal matching for correspondence and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 17(6), 545–561 (1995)

    Article  Google Scholar 

  8. L.S. Shapiro, J.M. Brady, Feature-based correspondence: an eigenvector approach. Image Vis. Comput. 10(5), 283–288 (1992)

    Article  Google Scholar 

  9. S. Umeyama, An eigen decomposition approach to weighted graph matching problems. IEEE Trans. Pattern Anal. Mach. Intell. 10(1), 71–96 (1991)

    Google Scholar 

  10. M.A. Turk, A.P. Pentland, Eigenface for recognition. J. Cogn. Neurosci. 3, 71–86 (1991)

    Google Scholar 

  11. Tesseract Open Source OCR Engine: Available http://code.google.com/p/tesseract-ocr/

  12. D.G. Lowe, Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004). http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf

  13. H. Bay, A. Ess, T. Tuytelaars, L.V. Gool, Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110, 346–359 (2008). ftp://vision.ee.ethz.ch/publications/articles/eth_biwi_00517.pdf

  14. E. Rublee, V. Rabaud, K. Konolige, G. Bradski, ORB: An efficient alternative to SIFT or SURF, in Proceedings of the International Conference on Computer Vision, pp. 2564–2571 (2011)

    Google Scholar 

  15. Open Source Computer Vision: Available http://opencv.org/

  16. Open source Open Kinect project: Available http://openkinect.org/

  17. M.A. Amin, H. Yan, An empirical study on the characteristics of gabor representations for face recognition. Int. J. Pattern Recognit Artif Intell. 23(3), 401–431 (2009)

    Article  Google Scholar 

  18. B. Poon, M.A. Amin, H. Yan, Performance evaluation and comparison of PCA based human face recognition methods for distorted images. Int. J. Mach. Learn. Cybernet. 2(4), 245–259 (2011)

    Article  Google Scholar 

  19. M.Z. Hossain, M.A. Amin, H. Yan, Rapid feature extraction for Bangla handwritten digit recognition, in Proceedings of the International Conference of Machine Learning and Cybernetics, pp. 1832–1837 (2011)

    Google Scholar 

  20. ESpeak—A Voice Synthesizer: Available http://espeak.sourceforge.net/docindex.html

  21. Central Bank of Bangledesh: Available http://www.bangladesh-bank.org/currency/note.php

  22. Samples of Sign Images: Available http://image.made-in-china.com/4f0j00NBJamowFULbd/Toilet-Sign.jpg

  23. Open CV: Feature detection and description, Available http://docs.opencv.org/modules/features2d/doc/feature_detection_and_description.html#orb-orb

  24. Issues with imgldx in Descriptor Matcher mexopencv: Stack overflow. Available http://stackoverflow.com/questions/20717025/issues-with-imgidx-in-descriptormatcher-mexopencv

  25. The CMU Multi-PIE Face Database: Available http://www.multipie.org/

  26. The Shefield (previously UMIST) Face Database: Available http://www.shef.ac.uk/eee/research/iel/research/face

  27. MIT Center for Biological and Computational Learning Face Database: Available http://cbcl.mit.edu/software-datasets/FaceData2.html

Download references

Acknowledgment

This work is jointly supported by Independent University, Bangladesh and University Grants Commission of Bangladesh under Higher Education Quality Enhancement Project (HEQEP) Number: CP-3359.

Author information

Authors and Affiliations

  1. Computer Vision and Cybernetics Group, Computer Science and Engineering, Independent University, Bangladesh, Dhaka, 1229, Bangladesh

    Mohammad M. Rahman & Md. Ashraful Amin

  2. School of Electrical and Information Engineering, University of Sydney, Sydney, NSW, 2006, Australia

    Bruce Poon

  3. Department of Electronic Engineering, City University of Hong Kong, Hong Kong, China

    Hong Yan

Authors
  1. Mohammad M. Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruce Poon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Md. Ashraful Amin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruce Poon .

Editor information

Editors and Affiliations

  1. Multimedia Engineering College of Engioneering, Mokpo National University, Chonnam, Korea, Republic of (South Korea)

    Gi-Chul Yang

  2. Unit 1, 1/F, Hung To Road, IAENG Secretariat, International Association of Engine, Hong Kong, Hong Kong SAR

    Sio-Iong Ao

  3. Faculty of Information Sciences and Engineering, University of Canberra, Canberra, Aust Capital Terr, Australia

    Xu Huang

  4. Calzada Tecnologico s/n, Instituto Tecnologico de Tijuana, Tijuana, Baja California, Mexico

    Oscar Castillo

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Rahman, M.M., Poon, B., Amin, M.A., Yan, H. (2015). Support System Using Microsoft Kinect and Mobile Phone for Daily Activity of Visually Impaired. In: Yang, GC., Ao, SI., Huang, X., Castillo, O. (eds) Transactions on Engineering Technologies. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-9588-3_32

Download citation

  • DOI: https://doi.org/10.1007/978-94-017-9588-3_32

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-017-9587-6

  • Online ISBN: 978-94-017-9588-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation