Keywords

1 Introduction

The ability to explore the neighborhood independently and to travel to a desired destination is required for satisfactory level of quality of life and of self-confidence. According to Golledge [11], visual impairment primarily restricts a person’s mobility. Golledge et al. [13] show that restrictions on the mobility of visually impaired people significantly reduce their travel-related activities. Although visually impaired people undergo special training to learn specific navigation and orientation techniques and strategies, it has been observed that 30 % of them never leave their homes alone [7, 34]. Moreover, only a fraction of blind people travel independently to unknown places [12]. Interestingly, the percentage of visually impaired people who never travel alone has remained constant over decades, despite the fact that more and more assistive aids have become available. This leaves a space for research in the area of blind user navigation.

The level of mobility is influenced by the efficiency of the wayfinding process, which consists of two parts: immediate environment sensing (avoiding obstacles and hazards), and navigation to remote destinations [17]. Both parts of the wayfinding process can be supported by navigation aids that will assist the visually impaired. The basic criteria for evaluating navigation aids were defined by Armstrong [1] as safety, efficiency, and stress level.

One already existing solution is a navigation aid based on a tele-assistance center with professional navigators (see Subsect. 2.2). The main problem of this solution is its scalability, as the gathering of a suitable set of landmarks for particular area often requires the physical presence of the professional navigator on the spot. According to a study by Balata et al. [3], visually impaired people memorize relatively long routes at a very high level of detail. It was also shown that 67 % of visually impaired people have experience with sharing their route with friends/family via email, phone, or messaging  [2], and that they prefer navigation provided by a blind person to that of the sighted public (also supported by [4, 16]). This opens the possibility to base a tele-assistance navigation service on visually impaired volunteers, and to build up an efficient large-scale system where one visually impaired person navigates another. In this situation, the blind navigator (navigator) forms a natural source of suitable landmarks with their descriptions and with routing strategies optimized for blind travelers (traveler).

According to the functional model of a general navigation system for the blind [18], the navigator in such tele-assistance navigation service fully covers components providing a description of the environment (typically some kind of geographic information system), route planning, auditory display and speech input. The only component that cannot be covered independently by the navigator is the component responsible for determining the traveler’s position and orientation. Here, collaboration with the traveler is needed. The traveler serves as a sensor gathering necessary data for the navigator, and/or can determine the position and orientation on her/his own.

A key feature of such tele-assistance navigation service is its non-stop availability. Here an automated dialogue-based navigation can be employed. There are several approaches to dialogue management: finite state machine, information state, grammar-based, plan-based and data-driven approach. Our case is highly complex and thus a data-driven approach like POMDP based dialogue managers is a suitable solution  [30].

Based on [17, 18], we identified the following five activities (three for navigator and two for traveler) that we wanted to observe in our experiment: The navigator describes the environment, plans the route (gives navigation instructions), and determines the blind traveler’s position [18]. The traveler travels to a remote destination (executes navigation instructions), and senses the environment (identifies landmarks) [17].

Our main goal is to investigate the process of tele-assistance-based navigation by blind people, with special reference to navigation problems that occur during these activities. Based on an analysis of the navigation problems we will develop recommendations for improving the training procedures in order to increase the efficiency of wayfinding in situations where tele-assistance takes place. Further, we will map the problem of tele-assistance-based navigation to POMDP based dialogue system in order to replace navigator with the computer system in the future.

2 Related Work

2.1 Pedestrian Navigation

For successful navigation and orientation in a space, we need to build up spatial knowledge about the given environment. According to Siegel and White [29], there are three levels of spatial knowledge: landmark knowledge, route knowledge, and overview knowledge. Route knowledge can be further subdivided into two levels [12]:

  • a procedural level, based on fixed reaction patterns that follows after exposure to a part of the route. These reactions can be automated, and do not require conscious effort. This leads to lower requirements on attention and on working memory.

  • a declarative level, based on knowledge of particular landmarks on the route and abstract rules on how to navigate between these landmarks. This level of knowledge requires greater attention and more working memory.

Overview knowledge concerns relations between objects. These relations are represented for example by angles or distances between two objects, which are not necessarily related to the route itself.

It has been shown that landmarks (representing landmark knowledge) are by far the most frequently-used category of navigation cues for pedestrians [19] (unlike junctions, distance, road type and street names or numbers). A study conducted by Ross et al. [27] states that the inclusion of landmarks within the pedestrian navigation instructions increased user confidence, and reduced or eliminated navigation errors. Rehrl et al. [26] showed that voice-only guidance in an unfamiliar environment is feasible, and that participants clearly preferred landmark-enhanced instructions.

The fact that humans rely primarily on landmarks to navigate from point A to B is reflected in many experimental designs of navigation systems, e.g. the system of Millonig and Schechtner [22]. The system designed by Hile et al. [14] presents a set of heuristics for selecting appropriate landmarks along the navigation path.

In our experiment, where the navigator instructs the traveler remotely without being physically present on the route and without any visual feedback, a declarative level of route knowledge is needed. The navigators were therefore thoroughly trained in compliance with official training methodology in the region where the tests were conducted [36]. The navigators were also introduced to objects that were not located on the test route. Finally, they checked a tactile map of the route and its environment to gain overview knowledge. In the training procedures for our experiment, we paid special attention to introducing all important landmarks and describing them to the navigators in order to support the creation of landmark knowledge (see Apparatus, Sect. 3).

2.2 Orientation and Navigation of the Blind

In large spaces where body movement is necessary, visually impaired pedestrians use different cognitive strategies from those used by sighted pedestrians for navigation and orientation, based on egocentric frames [20, 21]. Typically, they have to memorize a large amount of information [32] in the form of sequential representation [20] based on routes. Route knowledge has to be acquired on a declarative level [12]. Fortunately, it seems that visually impaired people acquire superior serial memory skills. A study by Raz et al. [25] discovered that congenitally blind people are better than sighted people in both item memory and serial memory, and that their serial memory skills are outstanding, especially for long sequences. In a study by Bradley and Dunlop [4], it was revealed that in a situation of pre-recorded verbal navigation, the blind navigator navigated the blind traveler significantly faster than a sighted navigator.

There are numerous navigation aids for visually impaired pedestrians. Some use special sensors to identify objects on the route, e.g. cameras [6], or an RFID based electronic cane [10]. Others are based on a concept described in [23], and rely on some kind of positioning system (e.g. GPS) in combination with the GIS system to identify objects and navigate the pedestrian, e.g. Ariadne GPS, BlindSquare. There have also been attempts to develop special interaction techniques for presenting navigation instructions, e.g. an auditory display [17] or a tactile compass [24].

The navigation aids based on major GIS systems (Google Maps, Apple Maps, OSM Maps, Nokia HERE Maps) suffer from an inappropriate description of the environment for visually impaired pedestrians. The available description may be imprecise (e.g. missing sidewalks or missing handrails), or may be ambiguous (e.g. an inadequate description of pedestrian crossing, meaning that it cannot be localized and identified without visual feedback) or it may ignore specific navigation cues (e.g. the surface structure of the sidewalk, acoustic landmarks such as the specific sound of a passage, the traffic noise of a busy street, or other sensory landmarks, such as the smell of a bakery). In addition, routing algorithms can encounter problems with non-trivial adjustments to the preferences and abilities of visually impaired people, e.g. their inability to cross open spaces (e.g. large squares).

Both inappropriate descriptions and unsuitable routing algorithms can be avoided by introducing navigation systems based on tele-assistance with a trained human agent. Various approaches have been proposed on the basis of various ways to identify the position and the environment of the pedestrian, like transmission of chest mounted camera view to the navigator [6], a verbal description from the pedestrian optionally combined with GPS location and GIS [8, 31], or purely based on a verbal description and knowledge of the environment [33]. Namely Navigational Centre for the Blind [8], operating since 2007, proved to be helpful tele-assistance navigation service widely used (6650 cases in years 2008–2013 [9]) by community of visually impaired people.

In our experiment navigation was performed in a way similar to that used in [4, 33].

3 Experiment

In our experiment we observed the process of navigation by a navigator navigating traveler by means of tele-assistance. The goal is to identify navigation problems in the following activities:

  1. 1.

    Navigator describing the environment,

  2. 2.

    Navigator giving navigation instructions,

  3. 3.

    Navigator determining traveler’s position,

  4. 4.

    Traveler executing navigation instructions,

  5. 5.

    Traveler identifying landmarks.

The experiment consisted of 19 sessions. There were two participants per session, one in the traveler role and the other in the navigator role. Each session lasted 100 min.

Participants. 25 visually impaired participants (12 females, 13 males) were recruited via three methods: an e-mail leaflet sent to a group of Czech Blind United [8] clients, direct recruiting of our long term collaborators, and snowball technique. The participants in the experiment were aged from 25 years to 66 years (mean = 43.44, SD = 13.27). Fourteen participants had Category 4 vision impairment (light perception); 11 participants had Category 5 vision impairment (no light perception) [35]; 12 participants were congenitally blind, 13 participants were late blind. All of the participants were native Czech speakers. None of the participants in the traveler role knew the route before the experiment, though the character of environment was familiar to them. During recruitment, the participants were asked whether they are willing to participate in both roles, as the traveler in the first session, and then as the navigator in the following session. Table 1 contains details about the participants. Table 2 contains details about the sessions and about the roles that the participants took (the session IDs do not necessarily correspond to their real order). We tried to balance onset of impairment, category of impairment and gender of the participants in the sessions as much as possible. All of the participants (except P23) were active and regularly traveled alone. Several researchers have noted that it is quite difficult to acquire blind pedestrians as a target user group for a usability study [4, 6]. However, we had established a relationship with blind communities during our previous studies, and this made it comparatively easy to recruit a considerable number of blind participants for our experiment.

Table 1. List of participants, including onset of the impairment (congenital – C, late – L), category of visual impairment [35], gender (male – M, female – F), and age.
Full size table
Table 2. List of sessions, including participants’ role in the experiment, duration of navigation (minutes), success of session, and the number of navigation problems.
Full size table

Apparatus. Training methodology. The goal of the training was to learn the navigators the route for regular independent walking, i.e. to form a declarative level of the route knowledge. We arranged several meetings with the chief methodologist from the Czech Blind United [8]. One of the chief methodologist’s fields of expertise is in the training visually impaired people in spatial orientation and in preparing itineraries for their regular routes (i.e. routes to work, to a shop, to a public transport station/stop, etc.) in accordance with their navigation strategies. In order to conform with the official training methodology [36] used by the chief methodologist, we proceeded as follows: (1) We selected the route, identified important landmarks, and consulted possible dangers on the route together with the chief methodologist. (2) Together, we prepared a tactile map of the route and printed it on a paper using foil fuser technology. (3) The experimenter observed the chief methodologist training the navigator in the first pilot session. (4) The experimenter trained the navigator according to the observed methodology under the supervision of the chief methodologist in the second pilot session. (5) The trained experimenter trained the navigators in all subsequent sessions of the experiment.

Description of the route. For our experiment, we selected a city center outdoor environment. Environments for this type of experiment are usually real environments [4, 26] rather than artificial (lab) environments, though exceptions are possible [28]. The location of the route was in a quiet area in the city center of Prague, Czech Republic (see Figs. 1 and 2). It was 256 m in total length (from S via D1–D5 to B11) and navigation via phone took place on the 105 m long final part of this route (from D1 to B11). In the initial part (from S to D1) of the route the traveler walked alone. This was done to allow the traveler to get oriented and to get familiar with the surrounding environment of D1. Along the route there were 5 decision points (D1–D5) to which the navigator tries to navigate the traveler, number of surface changes (SFx), acoustic landmarks (Ax), vertical traffic signs and columns (Cx), and doors (Bx) (see Fig. 2).

Fig. 1.
figure 1

Panorama from the beginning of the route containing decision points D1 and D2 (top), and from the end of the route containing decision points D3, D4, and D5 (bottom).

Full size image
Fig. 2.
figure 2

Schematic illustration of the route and its adjacent area. The rightmost red dot depicts the starting point of the routeS, red dots depict decision pointsDx, blue lines depict segmentsSGx of the route (bold solid – route segments traveled with the navigator, bold dashed – adjacent segmentASG3 walked without the navigator from the starting point, dashed – adjacent segmentsASGx), orange boxes depict buildingsBx, dark grey boxes and ovals depict surface landmarksSFx such as a rubber matSF1,SF3 or differences in material, e.g. cobblestoneSF2,SF4,SF6 and a broken sidewalkSF5, speaker icons depict acoustic landmarksAx, such as busy streetsA1,A2 or echoesA3, crossed circles depict vertical traffic signs and columnsCx, crossed squares depict waste containers of various sizesGx. The green area contains landmarks taught to the navigator in the training phase of the experiment. Bold black lines depict stone walls. Gray areas depict sidewalks (Color figure online).

Full size image

Equipment. Our equipment and the data we collected is based on field laboratory design presented in Hoegh et al. [15]. The traveler was equipped with a Nokia 6120 mobile phone with a lanyard which hung from his/her neck. In this way, the phone was protected from being dropped unintentionally, and the traveler was able to release it and have an empty hand when needed, and s/he could also find it again quickly. The mobile phone was set to Czech language, and it was equipped with the MobileSpeak text-to-speech (TTS) screen reader application by CodeFactory. The navigator was located in the usability lab dedicated to executing user tests equipped with a laptop and the Skype application with Skype Out capabilities for connection with the mobile phone network. The laptop speakers and the internal microphone were used as input and output devices. Communication between the traveler and the navigator was recorded using MP3 Skype Recorder v3.1 (left/right channel separated for traveler/navigator communication).

In each session, we also recorded two video streams of the traveler’s activities. The first camera (GoPro Hero 3) recorded 1st person view and was installed on a shoulder strap of the backpack that was carried by the traveler during the session, while the second camera (Panasonic SDR-S150) recorded a 3rd person view by the experimenter shadowing the traveler.

Procedure. Before the session started, both participants were briefed, and the purpose of the experiment was explained to them separately.

The experiment session consisted of two phases. In the first phase, the navigator was taught the route by the experimenter. In the second phase, the navigator navigated the traveler along the route. Both of the participants were asked to proceed as quickly and accurately as possible. The traveler was asked either to hold the phone in a hand or to leave it on the lanyard, according to his/her own preference.

The first phase of the experiment involved training the navigator. The training consisted of three walkthroughs. The first two walkthroughs of the route were done with the experimenter, and the third was done alone, with the experimenter in the vicinity. During the first two walkthroughs, the experimenter described the landmarks (see Fig. 2) along the route and offered as many details as possible. During the third walkthrough, the navigator walked alone and asked the experimenter about the landmarks in cases when s/he was uncertain, so that s/he could remember better, but mostly to verify that s/he had learned the route sufficiently. After the walkthroughs, the navigator was accompanied into our usability lab and was presented with a tactile map of the overview of the route and the destination details. From this point, the navigator waited in the usability lab with the experimenter for a call from the traveler.

The second phase of the experiment consisted of a walkthrough of a part of the route by the traveler, and of navigation of the traveler by the navigator. This phase consisted of three parts. In the first part, the traveler was accompanied to the starting point of the routeS and was given the task. The task was given as follows: “You have a meeting at Hostel Emma B11 (see Fig. 2) on Na Zderaze Street. Now you are on the corner S of Dittrichova Street and Resslova Street. Continue approximately 80 meters slightly downhill along Dittrichova Street to the first crossroad. The building will be on your right hand side, and on the left there will be cars parked on the sidewalk. Then turn right and continue approximately 80 meters uphill along Zahoranskeho Street to the crossroad D1 with Na Zderaze Street. To reach the destination, you will have to call the navigator who knows the location very well. At the crossroad, you will be assisted with dialing the phone. Proceed as if you were alone, but we will be watching for your safety from a distance.” Then the traveler started out. The second part consisted of assisting the traveler with making the phone call from the cornerD1 where the navigation with navigator starts. The phone call was initiated by the experimenter in the lab, who relayed the call to the navigator. Then the traveler accepted the phone call and started a dialog with the navigator. The third part consisted of navigating the traveler by the navigator via a phone call. The navigator described the environment, gave navigation instructions, and determined the traveler’s position. The traveler executed the navigation instructions, and identified the landmarks. The experimenter observed the whole session from nearby to ensure the safety of the traveler. If the traveler got lost beyond the possibility of finding the destination, and/or was in distress, the experimenter terminated the session. Otherwise, the traveler was not interrupted by the experimenter. After reaching the destination, the traveler was accompanied into the usability lab, where both participants were debriefed and received their payment.

Measures. During the sessions we measured the time to reach the destination in successful sessions and the number of navigation problems in all sessions. For the activities Navigator describing the environment and Navigator giving navigation instructions we define the navigation problem as deviation from the training navigators went through (see paragraph Apparatus – Training methodology). A navigation problem in the activity Navigator determining traveler’s position is defined as concurrent occurrence of two events: the traveler’s physical position differs from the navigator’s imagination of the traveler’s physical position, and the navigator is not determining the traveler’s physical position. Problems in the traveler’s activities (i.e. Traveler executing navigation instructions, Traveler identifying landmarks) are defined as fail to execute the navigation instruction and fail to identify the landmark.

Collected Data. Nineteen Skype call audio files were recorded. Nineteen video files were recorded from a 3rd person view observing the traveler, and eighteen video files were recorded using a GoPro camera on the traveler’s shoulder from the 1st person view (one file was not recorded, due to a hardware malfunction). These files were then merged and aligned by time into a single multimedia file for each session.

In order to analyze the data, we developed an application that allows time-stamped annotation of the traveler’s physical position and the navigator’s imagination of the traveler’s physical position in the map. After annotation, the multimedia file from the session and both annotated positions in the map could be browsed side-by-side.

4 Results and Discussion

Eleven sessions finished with successful arrival at the destination after between 4 min 15 s and 11 min 23 s (mean = 6 min. 48 s., SD = 2 min. 28 s.). One session was inconclusive due to intervention of the experimenter. Seven sessions failed. In this section, we describe the navigation problems observed during navigation of the traveler by the navigator. Initially, we focus on general results, and then we describe selected navigation problems in various types of situations.

We analyzed the navigation problems in all the sessions, and classified them into the corresponding activities performed by navigators and travelers, and into different types of situations. Out of the total of 96 problems, 71 were problems identified on the route, and 25 were problems identified off the route. Sixty problems were identified in the failed sessions, and 36 problems were identified in the successful sessions (see Table 2). Most of the navigation problems on the route (44 of 71) occurred in two activities: Navigator describing the environment, and Navigator giving navigation instructions (see Table 3). It was shown that the navigator’s problems on the route were greater than the traveler’s (53 vs. 18), but off the route there the difference was smaller (15 vs. 10).

Table 3. Occurrence of navigation problems in the activities performed by navigators and travelers in different situations on and off the route.
Full size table

A majority of the navigation problems that occurred in the activity Navigator describing the environment on the route were related to columnC8 (8 out of 23) and decision point [D3] (7 of 23). Navigators did not mention columnC8, doorB8, and cobblestonesSF4 while traveler was approaching decision point [D3].

Along with the navigation problems in various types of situations (i.e. reorientation at a corner, crossing from corner to corner, traveling along a building, reorientation at a building, crossing from building to building, finding a landmark), we observed four other phenomena that affected successful navigation. They were: similarities in the environment; temporary changes of environment; landmark confusions; recovery from going astray.

4.1 Reorientation at a Corner

There were 14 navigation problems in reorientation at a corner. For example, the navigator did not instruct the traveler to turn left [D1, S6,18]. The navigator did not relate the position of the traveler to the building [D1, S16]. The navigator was unable to give the traveler unambiguous instructions on how to stand at the corner: “Turn so you have the corner at your back.” [D1, S13]. The navigator could not determine the traveler’s position on the corner, i.e. which side s/he was on [D2, S12]. The navigator wrongly instructed the traveler and confused “turn left” with “have the building on your left” [D2, S11]. It seems that this situation was one of the most difficult for the participants.

However, several successful navigation strategies were used to reorient at the corner. For example, the navigator instructed the traveler to turn his/her back towards the building before s/he turned the corner [D1, S15]. The navigator instructed the traveler to check if s/he could hear a busy street from the right [D1, S4]. The navigator described the surrounding streets and gave their names at [D1, S8]. The navigator checked on which side the traveler had a building and what the slope of the sidewalk was [D2, S6]. The navigator asked the traveler on which side the downhill sidewalk was [D2, S12].

4.2 Crossing from Corner to Corner

We found 3 navigation problems when the street was crossed from corner to corner [D1, D2]. For example, the traveler did not execute the instruction to come back slightly from the cornerD1 to the street, so s/he arrived at the opposite cornerD2, while the navigator expected him/her on the left from the opposite corner [S11]. The traveler did not walk straight while crossing the street and missed the opposite corner [S17].

However, several successful navigation strategies were used to cross the street from corner to corner. For example, the navigator instructed the traveler to return back to the street and cross, in order not to miss the corner on the other side [S10,17]. The navigator instructed the traveler to walk around the cars from the left side in order not to miss the corner [S12]. The navigator instructed the traveler to cross the street to the opposite sidewalk. In this way, the navigator used traveler’s previously traveled route and the fact, that the street had sidewalk on both sides, for giving the instruction [S8].

4.3 Traveling Along a Building

In the situation when traveling along a building, we observed 14 navigation problems. For example, the traveler did not describe the slope of the sidewalk precisely when the navigator was trying to determine his/her position [SG2, S16]. The traveler did not execute the instruction to walk along the building [SG2, S7]. The navigator did not know about two restaurantsB6,B7 that the traveler asked about [SG2, S7]. The navigator did not instruct the traveler to walk along the building in order to find the rubber matSF3 [SG4, S3,4,5,15,19].

However, several successful navigation strategies were also used for traveling along building. For example, the navigator described the sidewalk made of small paving blocks [SG2, S3,14]. The navigator checked that the building was on the left-hand side of the traveler [SG2, S7,8]. The navigator checked the sound from the busy streetA1 in front of the traveler [SG2, S14]. The navigator described the restaurantsB6,B7 on the left-hand side [SG2, S3,8].

4.4 Reorientation at a Building

We found 4 navigation problems during reorientation at a building. For example, the navigator did not determine the position of the traveler when s/he reached the doorsB8 and instructed him/her to turn right instead of instructing him/her to turn about face when s/he was facing the doorB8 [D3, S3]. The navigator did not determine the traveler’s orientation when s/he reached the other side of the street [D4, S3].

However, several successful navigation strategies were also used for reorientation at the building. For example the navigator checked that the building was on the left-hand side of the traveler after s/he had crossed the street [D4, S3,8,9]. The navigator instructed the traveler to have the doorsB8 behind his/her back [D3, S4].

4.5 Crossing from Building to Building

Two navigation problems were observed during crossing the street from one building to another building [D3, D4]. For example, the navigator instructed the traveler to turn right if s/he found cars parked along the sidewalk, instead of bypassing them [S3]. The traveler did not execute the instruction to cross the street to the building, and stopped at the edge of the sidewalk [S15].

However, several successful navigation strategies were also used for crossing from building to building. For example the navigator instructed the traveler to walk around the parked cars from the left [S1].

4.6 Finding a Landmark

In the situation of finding a landmark, we observed 34 navigation problems. For example, the navigator did not describe the columnC8 [S2,4,12,13,14]. Similarly, the traveler did not identify the same columnC8 even if s/he struck it [S14]. The navigator did not describe the wooden doorsB8 with metal fittings and a handle at head level [S10,11,14]. The traveler failed to check the material of the doorsB8 and the handle [S15,18]. The traveler did not execute the instruction to stop at the cobblestonesSF4 although s/he did find them [S19]. Alternatively, the traveler did not identify the cobblestonesSF4 at all [S10,12].

However, several successful navigation strategies were also used for finding a landmark. For example, the navigator described the cobblestonesSF4 [S5,13]. The navigator described the wooden doorB8 with metal fittings and a handle at head level [S1,4,12]. The navigator described exact position of the columnC8 – 15 cm from the building on the left side [S5,7,8,9,18]. The navigator described the distance to the columnC8 from the corner [D2, S8,9,19]. The navigator described the acousticsA3 at the corner [D5, S12,13,18,19]. The navigator described a rubber matSF3 on the sidewalk [SG4, S12,14,18].

4.7 Similarities in the Environment

If the traveler was inattentive to the details of landmarks, two parts of the route can seem to be very similar. The similar parts can be characterized by the same sequences of similar landmarks (e.g., route part R consists of landmarks A, B, C and route part R′ consists of landmarks A′, B′, C′, where A is similar to A′, B to B′, and C to C′).

There was similarity between one sidewalkASG8 from restaurantB12 to place with broken sidewalkSF5 and another sidewalkSG2 from shopB5 to the cobblestonesSF4 [S5] (see Fig. 3(a)). The navigator thought that the traveler had crossed the street and returned back (from SG4 back to the other side of the road to SG2 and farther away to D2), as they could not find the destinationB11. This was because of incorrect instructions from the navigator – s/he did not stress that the traveler should go along the building to find the rubber matSF3 at the destinationB11 [SG4]. The navigator checked the acoustic landmarkA2 and the traveler acknowledged that there was indeed a busy roadA1 behind his/her back; however, it was the other oneA2. They did not check the material of sidewalks: on oneSG2 there are small paving blocks, whereas on the other oneASG8 there is asphalt.

Fig. 3.
figure 3

Similarities in sessions S5 (a), S11 (b), and S18 (c). The green figure represents the traveler’s physical position and direction (TP). The red figure represents the navigator’s imagination of the traveler’s physical position and direction (NI) (Color figure online).

Full size image

There was similarity between one sidewalkASG5 from cornerD2 to cobblestonesSF2 and another sidewalkSG2 from cornerD2 to cobblestonesSF4 [S11] (see Fig. 3(b)). The navigator confused the navigation instruction (left vs. right), and the traveler continued to the leftASG5 instead of to the rightSG2 [D2]. The navigator did not check whether the traveler had buildings on his/her left side, and did not check which side the landmarks reported by the traveler were on. Both sidewalksASG5,SG2 are downhill, but the firstASG5 is much steeper. The navigator checked for the slope and the traveler acknowledged that it was downhill but not how steep it was. Navigation continued until the traveler reached cobblestonesSF2. The material of the doorsB3 did not match the right oneB8, however the navigator instructed the traveler to continue further downhillASG5.

There was similarity between one sequence of sidewalksASG4,ASG2,ASG1 from columnC4 near stone wallASG4 to doorsB1 with rubber matSF1 and another sequence of sidewalksSG2,SG3,SG4 from cornerD2 to doorsB11 with rubber matSF3 [S18] (see Fig. 3(c)). The navigator forgot to turn the traveler to the left to cross the streetSG1 and the traveler ended up by stone wallASG4. The traveler continued along the wallASG4 and reported cars along the buildings. The navigator acknowledged, but did not check how far from the building the cars were. The traveler reported wooden doorsB2 but the navigator did not check for cobblestonesSF4, which are missing thereB2. After crossing the street from the doorsB2, the traveler did not find the corner on the right side and decided to walk to the left in the opposite directionASG1, but the navigator did not make any comment. The traveler reported that s/he was at the destination at doorsB1 with rubber matSF1.

4.8 Landmark Confusion

Travelers often had to make a further examination of a landmark that they had discovered, in order not to confuse it with another object. The traveler confused a railing with a temporary traffic sign placed next to a columnC3 [S1]. The traveler confused a garbage containerG1 with a trash canG2 [S5]. The traveler confused cars with garbage containersG1 [S18]. The traveler confused a passage with a van parked along the sidewalk [D3, S19]. Travelers confused a building with a stone wallASG4 [S6,16,18].

4.9 Temporary Changes of Environment

An urban environment has a rhythm of its own. Streets that are busy during the day are silent at night, and the shops are closed. Some shops open at 9:00 am, while some restaurants and coffee bars do not open until 11 o’clock. There is also a weekly rhythm of dustmen and periodic street cleaning. All of these changes have an impact on the environment and affect some of the landmarks. In our case, there was increased traffic in an otherwise quiet street near [D1, S14], a temporary traffic sign next to columnC3 [S1], a dustbin put outside a doorB8 for the garbage collectors [S7], a van parked on the sidewalk [D3, S19], or a missing advertising stand, which led to session failure [D2, S16].

4.10 Recovery from Going Astray

During the sessions, we observed traveler going astray. In a moment when the navigator or the traveler realized that the traveler is out of the route they started to recover from this situation and get back on the right route. The recovery process can be divided into three subsequent steps: (1) realize that the traveler went astray, (2) determine the traveler’s real position, (3) take the traveler back on the route.

The traveler walked relatively long time without mentioning that s/he went astray. This especially happened when the navigator did not determine the traveler’s physical position on regular basis and when the traveler failed to identify the landmarks. For example, at second cobblestonesSF6, the traveler realized that s/he probably did not identify the first onesSF4 as the doorB10 material did not match the navigator’s description [S4]. However, neither the traveler nor the navigator realized that the traveler went astray, but they were convinced that they reached the destinationB11, even though they were at different doorsB1 [ASG1, S18].

In order to take the traveler back on the route the navigator had to determine the traveler’s position and direction. For example, when the navigator realized the traveler went astray, s/he asked direction of busy streetA2. When the traveler confirmed the navigator determined his/her position at stone wallASG4 near 3rd columnC4 [S6]. However, neither the navigator did not determine the traveler’s position until the experimenter terminated the session [S4,5].

The last step is the attempt to take the traveler back on the route. This was typically done by backtracking to last known point on the route. For example, the navigator successfully instructed the traveler to cross the street from stone wallASG4 near 3rd columnC4 [S6]. The navigator successfully instructed the traveler to return back from end of the streetASG9 back to first cobblestonesSF4 [S12]. However, the traveler was not able to cross the street from stone wallASG4 and the session was terminated on his/her request [S17].

After successful recovery they tried to identify the error either in giving the navigation instruction by the navigator or in executing the navigation instruction by the traveler. The most common error was not identifying a cobblestonesSF4 by the traveler. The common solution was returning back and trying to identify the landmark [S4,12,14].

4.11 Guidelines

We extracted the following five guidelines based on an analysis of 96 navigation problems and successful navigation strategies collected during the experiment (see Table 4).

Table 4. Guidelines extracted from observed navigation problems and successful navigation strategies, their application, and some of the examples from the experiment.
Full size table

4.12 POMDP Based Dialogue System

The findings obtained in the study can be used for POMDP based dialogue system definition [5, 37]. In our case, the system is represented by the navigator and the environment is represented by the traveler.

A POMDP is defined by sextuplet <S, A, Z, T, O, R>, where S is a set of states (Traveler’s states), A is a set of the system’s actions (Navigator’s actions), Z is a set of observations the system can experience (a set of Navigator’s observations), T is a transition model, O is an observation model, and R is a reward model.

The state set S = <I t  × P t  × D t > is composed of three features: traveler’s action (I t ), which corresponds to Traveler executing navigation instruction activity, traveler’s 2D coordinate position (P t ), and traveler’s direction (D t ) as an absolute angle. In the future the state set can be extended with features such as traveler’s type of disability or his/her experience. The action set A = <I × L> is composed of two features: action (I), and landmarks (L). The actions are passed to traveler during the Navigator giving navigation instruction activity. The observation set Z = <OI t  × OL t  × OD t > is composed of three features: traveler’s observed action (OI t ), traveler’s observed landmarks (OL t ), which corresponds to the Traveler identifying landmarks activity, and traveler’s observed direction (OD t ) as a relative angle. The observation set is acquired during the Navigator determining Traveler’s position activity. To parameterize the transition model T(s′, s, a) = p(s′|s, a) we can use the Traveler executing navigation instruction activity. In the future the transition model can be used for personalization based on types of disability or experience (from the state set) such as adjustment of segment length, or usage of specific landmarks. The observation model O(s′, a, z) = p(z|s′, a) is represented by GIS-like data structure with probable traveler’s states. Findings from Sects. 4.7–4.9 can be used for parameterization. In the future the observation model can be used for probability distribution visualization of traveler’s position. In the future the reward model R(s, a): S × A → R can be parameterized by stress function i.e. whether to re-plan or back-track when the traveler went astray and his/her stress level became high (see Sect. 4.10).

5 Conclusion

We have gathered a set of problems that occur during the process of blind-to-blind navigation by means of tele-assistance. These problems have been classified into activities performed by the navigator and by the traveler and have been assigned to categories of situations where these problems occurred. We have also described in detail behavior of navigators and travelers in special situations (i.e. similar parts of the route, temporary changes in the environment, landmark confusion, and recovery from going astray). It seems that substantial number of problems are related to activity Navigator giving navigation instructions. These findings can serve as a basis to improve the training for visually impaired people to make the wayfinding process more efficient in situations when tele-assistance takes place. Furthermore, our results are suitable for parameterization of POMDP based dialogue systems, which can form a step towards replacement of human navigator by a computer system.

Future research should focus on experiments in different environments (e.g. city park, indoors) and on development of efficient training methods for blind-to-blind pedestrian tele-assistance-based navigation.