Abstract
The paper presents the preliminary results for the parametrization of 3D scene for sonification purposes in an electronic travel aid (ETA) system being built within the European Union’s H2020 Sound of Vision project. The ETA is based on the concept of sensory substitution, in which visual information is transformed into either acoustic or haptic stimuli. In this communication we concentrate on vision-to-audio conversion i.e. employing stereovision for reconstruction of 3D scenes and building a spatial model of the environment for sonification. Two prerequisite approaches for the sonification are proposed. One involves the direct sonification of the so-called “U-disparity” representation of the depth map of the environment, while the other relies on the processing of the depth map to extract obstacles present in the environment and presenting them to the user as auditory icons reflecting specific size and location of the sonified object.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hersh, M., Johnson, M.: Assistive Technology for Visually Impaired and Blind People. Springer, London (2008)
Ivanchenko, V., Coughlan J., Huiying, S.: Detecting and locating crosswalks using a camera phone. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop, CVPRW 2008, pp. 1–8, 23–28 June 2008
Huiying, S., Coughlan, J.: Reading LCD/LED displays with a camera cell phone. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2006, p. 119, 17–22 June 2006
Bach-y-Rita, P.: Brain Mechanisms in Sensory Substitution. Academic Press, New York (1972)
Strumiłło, P.: Electronic personal navigation systems for the blind and visually impaired (in Polish), Faculty of Electrical, Electronic, Computer and Control Engineering, Lodz University of Technology, Lodz (2012)
Sound of Vision: Natural sense of vision through acoustics and haptics. http://www.soundofvision.net/ . Accessed 25 Mar 2016
Flores, G., Kurniawan, S., Manduchi, R., Martinson, E., Morales, L.M., Sisbot, E.A.: Vibrotactile guidance for wayfinding of blind walkers. IEEE Trans. Haptics 8(3), 306–317 (2015)
Dakopoulos, D., Bourbakis, N.G.: Wearable obstacle avoidance electronic travel aids for blind: a Survey. IEEE Trans. Syst. Man Cybern. – Part C: Appl. Rev. 40(1), 25–35 (2010)
Garaj, V., Jirawimut, R., Ptasinski, P., Cecelja, F., Balachandran, W.: A system for remote sighted guidance of visually impaired pedestrians. Br. J. Vis. Impairment 21, 55–63 (2003)
Barański, P., Strumiłło, P.: Emphatic trials of a teleassistance system for the visually impaired. J. Med. Imaging Health Inf. 5(8), 1640–1651 (2015)
vOICe: http://www.seeingwithsound.com . Accessed 25 Mar 2016
Balakrishnan, G., Sainarayanan, G., Nagarajan, R., Sazali, Y.: A stereo image processing system for visually impaired. Int. J. Signal Process. 2(3), 136–145 (2006)
González-Mora, J., Rodríguez-Hernández, A., Rodríguez-Ramos, L., Díaz-Saco, L., Sosa, N.: Development of a new space perception system for blind people, based on the creation of a virtual acoustic space. In: Mira, J. (ed.) Engineering Applications of Bio-Inspired Artificial Neural Networks. LNCS, vol. 1607, pp. 321–330. Springer, Heidelberg (1999)
Heyes, D.: The Sonic pathfinder: a new electronic travel aid. J. Vis. Impairment Blindness 77, 200–202 (1984)
Bujacz, M., Skulimowski, P., Strumillo, P.: Naviton – a prototype mobility aid for auditory presentation of 3D scenes. J. Audio Eng. Soc. 60(9), 696–708 (2012)
Chunlei, Y., Cherfaoui, V., Bonnifait, P.: Evidential occupancy grid mapping with stereo-vision. In: IEEE Intelligent Vehicles Symposium (IV), pp. 712–717, 28 June 2015 – 1 July 2015. doi:10.1109/IVS.2015.7225768
Zhencheng, H., Uchimura, K.: U-V-disparity: an efficient algorithm for stereovision based scene analysis. In: IEEE Intelligent Vehicles Symposium, pp. 48–54, 6–8 June 2005. doi:10.1109/IVS.2005.1505076
Labayrade, R., Aubert, D., Tarel, J.-P.: Real time obstacle detection in stereovision on non flat road geometry through “V-disparity” representation. In: IEEE Intelligent Vehicle Symposium, vol. 2, pp. 646–651, 17–21 June 2002. doi:10.1109/IVS.2002.1188024
Zhencheng, H., Lamosa, F., Uchimura, K.: A complete U-V-disparity study for stereovision based 3D driving environment analysis. In: Fifth International Conference on 3-D Digital Imaging and Modeling, pp. 204–211, 13–16 June 2005. doi:10.1109/3DIM.2005.6
Suzuki, S., Abe, K.: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985)
Bujacz, M., et al.: Sound of vision - spatial audio output and sonification approaches. In: Miesenberger, K., Bühler, C., Penaz, P. (eds.) ICCHP 2016, Part II. LNCS, vol. 9759, pp. 200–207. Springer, Heidelberg (2016)
Acknowledgment
This work received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 643636 “Sound of Vision”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Owczarek, M., Skulimowski, P., Strumillo, P. (2016). Sound of Vision – 3D Scene Reconstruction from Stereo Vision in an Electronic Travel Aid for the Visually Impaired. In: Miesenberger, K., Bühler, C., Penaz, P. (eds) Computers Helping People with Special Needs. ICCHP 2016. Lecture Notes in Computer Science(), vol 9759. Springer, Cham. https://doi.org/10.1007/978-3-319-41267-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-41267-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-41266-5
Online ISBN: 978-3-319-41267-2
eBook Packages: Computer ScienceComputer Science (R0)