1 Introduction

Balance dysfunction is often associated with aging, stroke, mild traumatic brain injury (mTBI), neurological disorders and disease [1]. For patients who have balance deficit, a physical therapy (PT) program can be used to “habituate” sensory and motor systems through exercise, and “compensate” the systems through sensor reweighting and learnt skill sets [2]. Clinical studies have shown that vestibular rehabilitation training (VRT) programs resulted in significant improvement in patients with unilateral peripheral vestibular dysfunction [3]. However, these rehabilitation programs require specialist therapists and multiple intensive customized clinical sessions that are presented over the course of several weeks [4].

Real time postural feedback from external sensors can also be used to augment or replace normal internal (vestibular, visual and proprioceptive) sensory information [5]. For example, body sway information can be measured using an inertial sensor and displayed using visual, auditory or tactile means, and previous research efforts have reported improvements in balance control during standing and gait with sensory feedback [6–10]. While an ambulatory sensory feedback prosthesis is a meaningful goal for the treatment of balance disorders, that approach currently presents significant technical challenges as natural movement can be complex and stability limits are based on intent. An alternate paradigm is to consider postural sensory feedback as an instructional tool during rehabilitation physical therapy. This paper describes our development of an activity based, multimodal balance rehabilitation training device.

2 Design of a Sensory Feedback System for Rehabilitation

The primary goals for postural stance and control are equilibrium and orientation [11]. We often take the postural control system for granted because it usually operates primarily at a non-cognitive level. However it actually depends on a complex and active interaction among the sensory, muscular, and nervous systems [12]. This central-nervous-system (CNS) process is known as sensory integration.

It is well known that the human sensory system is capable of adaptation. For example, if a fixed reference (such as a fingertip on a surface) is provided to a patient who has been blindfolded, body sway has been shown to reduce [13]. Thus, the sensory system has compensated for the loss of the visual reference system and adapted to use the force feedback from the fingertip to provide the body with a spatial reference. Moreover, the brain has a built-in mechanism that allows change according to experience [14].

2.1 Multisensory Feedback Displays

Providing postural sensory feedback or context data to a patient during rehabilitation physical therapy presents a human-machine interface design problem for the rehabilitation training system. There are typically two users of the rehabilitation training system: the therapist and the patient. The rehabilitation training system user interface must track with the work flow and expectations of the physical therapist and any postural sensory feedback presented to the user (or patient) must be immediate and effective. Specifically, the feedback information to the patient must be natural and intuitive, and ideally should not increase cognitive workload.

Postural control using visual or audio displays typically requires significant cognition (i.e. concentration) as the feedback information must be recognized before deciding on an appropriate motor response reaction. Tactile events, in contrast, may be processed preattentively [15, 16]. Furthermore, proprioceptive and tactile sensory information is naturally used in postural control. Therefore, the tactile modality can potentially offer significant advantages as a feedback display during balance therapy.

Thesense of touch is arranged somatotopically so location based tactile cues are a highly effective method for providing orientation. An array of vibrotactile actuators (or tactors) that are spaced over an area of the body that is implicitly aligned with the patient’s local coordinate axis eliminates cognitive translation. Thus, the torso is preferred as it is usually aligned with the direction of intended motion.

We have previously reported on our work using an array of eight lightweight vibrotactile actuators (tactors) mounted in a lightweight, stretchable belt – this approach represents a useful compromise between localization accuracy and resolution [17]. Our current tactor belt design incorporates 8 EMR tactors as shown in Fig. 1.

Fig. 1.
figure 1

Tactor belt array containing 8 EMR tactors, a wearable controller (including Bluetooth wireless interface) and a rechargeable internal battery. The EMR tactor is a motor based design that operates at about 100 Hz.

Full size image

Visual displays are highly effective user interfaces that are potentially capable of communicating large amounts of data to a user. However, the role of vision in postural control feedback is complex, and mostly limited to low frequency postural control [18]. Typically, vision provides information regarding reference to verticality, motion of the head, self motion and navigation. The role of vision in balance rehabilitation can change depending on the postural task, balance deficits, stage of recovery, and the environment. For example, vestibular-ocular (VO) deficits often accompany blast related balance dysfunction and in these cases the vision system requires compensation and habituation during rehabilitation [19, 20].

The visual display can be used for rehabilitation task orientation and also provide visual flow (disruptive stimuli) or tasks(for example cognitive or immersive) to assist VO rehabilitation. The visual display must be mounted at eye-level and preferably have a touch screen interface for the therapist to input patient specific data and parameters.

Auditory displays can potentially convey both direction and informational components and have been used as a feedback display for postural control [21]. However, wearable audio displays use binaural headphones and synthesize apparent sound source direction. Audio localization is also subject to well known front and back localization ambiguity [22], which usually requires head or body movement to resolve. Therefore headphones were not considered and our design approach has been to limit audio to pitch and rate, and use it only as a qualitative indicator of a feedback parameter or limit in a therapeutic task.

2.2 Postural Sensor Design and Selection

A real-time measurement of the patient’s postural state is needed for the rehabilitation training system. Various sensors and biomechanical models of movement can be used depending on the chosen rehabilitation activity. Rehabilitation training tasks can be generally classified into stable (sitting and upright stance) and dynamic (gait, turns and sit-to-stand) movements.

Upright stance can be simply modeled as a single segment or as a more complex multi-segment inverted pendulum. Feedback stability during single segment upright stance is defined by the angle of lean and controlled by ankle torque [23]. In multi-link upright stance, the ankle torque and hip torque control the predominant movements. Feedback stability during multi-segment lean is then more accurately defined by the location of the center of mass with respect to the balance of support (feet).

Dynamic movements can be modeled using more complex multiple segment models, where the contact forces and movement kinematics are estimated or known. Capturing generalized dynamic movement usually requires sensors that are capable of tracking the kinematics of the multi-segment model. Rehabilitation systems require real-time measurement of multiple segment data which can usually only be implemented using a simplified model.

Feedback during dynamic activities is also more complex as the postural stability depends on previous and anticipated movements. Movements may also be relatively fast, leaving very little time to use any additional system feedback to compensate for any erroneous trajectories. In contrast, gait is cyclical and patients are usually relatively stable in the anterior/posterior axis. Therefore, gait feedback can be simplified to only the mediolateral axis, and the systemstability can be approximated using an inverted pendulum model.

Our approach has been to consider multiple sensor components as modular and scalable to meet intended activity and therapeutic needs. A combination of sensors were selected for our system including: a light weight custom force plate, a wearable inertial sensor (Microstrain® 3DM-GX3®-25), and a depth sensing camera (Microsoft Kinect®). The available sensors are fused together to provide an estimate of a user’s biomechanical state. A lightweight force plate is low cost, robust and measures location of the center of pressure (COP) of the patient standing on the plate. Our force plate design is easily scalable; multiple force plates can be attached together in order to construct a wider sensing area. Similarly, the depth sensing camera can measure the position of the patient and approximate the COG, or an inertial sensor mounted on the small of the back can provide a measure of the body tilt [24].

During static activities, any of the sensors (force plate, inertial and depth camera sensor) can be used individually, or in combination, to provide an adequate estimation of patient stability. During dynamic activities, the force plate data must be supplemented with measurements of body segment center of gravity (COG). Methods for measuring body segment COG include using one or more inertial sensors, and /or the depth sensing camera. The inertial sensor is lightweight and can be mounted on the patient’s head to track head orientation – this is useful during functional gait tasks [25] and during vestibular ocular habituation exercises [26].

The complete multimodal balance rehabilitation training device comprises a stand, computer, touch screen display, speaker, force plate, depth sensing camera, wearable inertial sensor and tactile belt as shown in Fig. 2.

Fig. 2.
figure 2

Multimodal balance rehabilitation training device showing the force plate, Microsoft Kinect camera and wearable inertial sensors, together with the touch screen and tactile belt.

Full size image

2.3 Design of Therapeutic Activities

The typical therapist workflow associated with a balance rehabilitation training protocol for the treatment and rehabilitation of subjects with balance dysfunction comprises the following components [1, 2]:

  • Assessment: Clinical examination, history, classification, diagnosis and treatment planning

  • PT Interventions:

  • Adaptation /substitution exercises

  • Postural and oculormotor control

  • Steady stand concentrating on: sensory integration, core /strength exercises, vestibular ocular reflex (VOR) visual acuity training

  • Dynamic tasks : turns /gait

  • Movement sensitivity training

  • Functional exercises

  • Immersion /dual task exercising

  • Home exercises

A range of activities is required for balance rehabilitation training, and the selection and configuration of each of the activities must be under the control of the therapists. The type and amount of feedback, as well as the task difficulty must also be selectable and individualized for a particular patient and the stage in their rehabilitation program.

Our design approach to a balance training rehabilitation system is shown in Fig. 3. The system recognizes the two users in the system; the therapist and patient. The patient forms part of a human-in-the-loop feedback system. Multiple sensors measure the patient’s postural state and a computer controller combines the therapist selected activity configurations to provide multisensory feedback to the patient.

Fig. 3.
figure 3

Block diagram relationship between the human-in-the-loop and the multi-modal sensory feedback system. The predefined functional activity and feedback settings are determined by the therapist.

Full size image

The addition of multi-modal feedback to the patient has several potential advantages: 1) feedback can be provided continuously and simultaneously along multiple channels which should increase system performance (and also bypass any possible bottle necks due to injury), 2) it should be easier to understand the feedback (cross verification of the feedback “message” leads to less confusion), and 3) the human (“plant”) is itself a non-linear adaptive (neuro) controller that can internally optimize the loop. Each of the activities was designed to make optimum use of the multi-modal feedback and each feedback mode was selected by the therapist.

The functional activities designed for the system include: steady stand, sit-to-stand, limited gait, adaptive immersive activities including movement to goals, movements with visual flow distraction and various assessment tools. Some of the activities address specific deficits within patient population groups. For example, vestibular ocular disorders are especially important for the rehabilitation of blast related balance dysfunction. Our device implemented prototype activities where the patient had to follow a visual or auditory tracking task in combination with postural control. Other patient groups such as geriatric and stroke required other functional tasks; for example, the sit-to-stand movement sequence can be trained using tactile cues.

2.4 Design of Multisensory Feedback for Activities

Various combinations of visual, auditory and tactile feedback can be configured for each therapeutic activity. Generally, visual feedback is needed at the start of rehabilitation training to provide task orientation. After orientation, tactile feedback is then primarily used to augment balance control. Tactile feedback is versatile; it can be configured as a cue that certain movement targets or limits have been reached or as an immediate indicator of the variance in postural sway.

3 Clinical Experience

Two separate groups of patients have been studied; a small group of geriatric patients (n = 12) and a small group of mTBI and TBI (n = 30) reporting balance dysfunction. Both studies were designed to be a controlled, randomized, repeated measures study. In each study, patients were randomly assigned to either a device intervention or standard care group. The studies followed the typical 8 week vestibular rehabilitation therapy (VRT) treatment pathway. A preliminary assessment was performed prior to enrollment to demonstrate inclusion criteria for the study and to diagnose specific conditions (peripheral vs. central vestibular, unilateral vs. bilateral, BPPV, post-concussive, etc.). Additional assessments were performed during and at the end of the study.

All participants showed improvements in test scores for both highly functioning (Berg Balance score of > 50) and low functioning (Berg Balance score of < 35) individuals. For the geriatric study, the device group showed statistically significant differences to the control group by the second week of treatment. The results from the mTBI /TBI study group were more variable and although the device intervention group showed good results, there were not statistically significant differences between the two groups. Further detailed analysis of this data suggests that a subgroup with known vestibular issues potentially benefited more from the device intervention, but the study numbers are too small for statistical significance.

4 Conclusions

The device was very well received by patients and therapists, and clinical measures show that the device may be more effective, especially during the early stages of rehabilitation, than standard care. Results indicate that the system, in at least one study group, initially facilitates a more rapid rehabilitation and training than the conventional approach and then, with time, the conventional approach “catches up” and the overall outcome is similar.

Another observation from clinical testing was that the system appeared to offer an increase in the efficiency of the therapist. Subjective comments suggest that by using the system, a therapist could treat multiple patients without decreasing outcomes - this result has potentially significant economic value both for the military and civilian populations and needs further investigation.