1 Introduction

1.1 Background

Soldier Visual and Cognitive Workload. Dismounted soldiers are constantly exposed to heavy cognitive and visual workloads, especially during navigation and patrol, and under conditions of high stress and time pressure. (Mitchell et al., 2004; Mitchell, 2009; Mitchell and Brennan, 2009a, 2009b; Pomranky and Wojciechowski, 2007). Additionally, a review of emerging technologies for infantry Soldier combat teams during the Army Expeditionary Warrior Experiment included aerial and ground vehicles with sensor arrays, small stationary sensors, more robust communication capabilities, and improved visual capabilities encompassing weapon sights, binoculars, night vision, and targeting aids (Scalsky et al., 2009; U.S. Army Evaluation Center, 2013). This provides evidence that cognitive task demands on dismounted Soldiers are increasing.

There are also increased physical and temporal demands placed on the Soldiers as the complexity of team communications increases. As Soldiers gain and control additional assets (command and control, robotic, etc.), the issue of weight and bulk presents itself as the additional assets must be carried. Current control interfaces for unmanned vehicles will often significantly increase the weight and quantity of equipment carried by the dismounted Soldier. Additionally, the controls and displays must be easy to understand and use. The following is an exploration of incorporating advanced concepts into smaller lightweight wearable displays and controls that reduce cognitive, physical, and temporal demands as they relate to dismounted Soldier performance.

Hands-Free Covert Communications. In order for Soldier teams to work effectively, Soldier communication is essential. Communications should be rapid, concise, and immediately understood within and across teams. However, using handheld communication devices presents some challenges. They often distract a leader’s visual attention away from the tactical battlefield environment and can sometimes hinder their ability to use their weapons or increase their response time to target engagement when transitioning from the device to the weapon. If the communication is speech-based, it is no longer functional in noisy environments. These constraints show that there are many scenarios in which hand-held devices for communication are less than optimal, however Soldier communication will always be necessary.

The use of hand and arm signals has served as a fundamental form of communication among Soldiers. Dismounted Soldiers in the field will utilize an established set of hand and arm signals to communicate with others in order to maintain noise discipline (e.g., when approaching an objective) or when noise levels are elevated to a point where radio or voice communication is not possible. These hand signals are standard practice and most military personnel are familiar with them. They can be found in documented sources such as the U.S. Army Field Manual No. 21-60 and U.S. Marine Corps Rifle Squad Manual (FMFM 6-5). Additionally, mission scenarios might dictate further requirements, such as a need for covert operations (e.g., low noise and/or electronic transmissions) or combat operations may be characterized by high stress, high time pressure, high noise, low visibility, and/or night operations. Such commands will typically be relayed from one team member to the next, not via direct communication from the Soldier initiating the command to the Soldier who is the intended recipient. This is typically due to the communication recipient being positioned outside of the line of sight of the initiating Soldier. Relaying the commands from one team member to another requires aural and visual attention to receive the commands as well as excess time. This point highlights a need for a technology-based alternative to traditional hand and arm signals.

Glove-Based Gestures. Wearable instrumented gloves present an effective solution for a technology-based solution for effective hand and arm signal communication. This approach is the most commonly used approach for wearable instrumented systems for robotic control. The glove concept is most congruent for many work situations where operators may already have to wear gloves.

Tactile Display Technology. The reception of signals by individuals beyond the LOS has been proven effective via a torso-mounted vest containing an array of vibrating tactors. This tactile modality has been proven to be a reliable and covert means of conveying critical information during infantry tactical operations (Van Erp 2005). When directly compared with visual displays, tactile displays have been identified to increase performance in select circumstances (Elliot et al., 2009). Tactile display is also effective under conditions of high cognitive workload. For navigation in field evaluations, torso-mounted tactile displays have been proven effective (Pettitt et al., 2006). When integrated with GPS, these displays enable Soldiers to navigate in low visibility conditions, hands-free (allowing the Soldier to hold his/her weapon) and eyes-free (i.e., allowing focused attention to surroundings rather than a visual display) (Elliot et al., 2011; Elliot and Redden, 2013).

1.2 Research Objectives

This evaluation serves to assess concepts and capabilities related to the use of an integrated gestural glove and a tactile vest for Soldier communications. The system, Communication-based Operational Multi-Modal Automated Navigation Device (COMMAND), integrates an instrumented glove for automated gesture-based communication and control, a tactile display vest, and a GPS-enabled ruggedized handheld computer. The study objectives are to (a) identify issues pertaining to Solder use (e.g., operational relevance) and (b) evaluate usability (human factors assessment) of system components and the system as a whole. This evaluation was situated around task demands characteristics of a rural reconnaissance mission scenario requiring inter-Soldier communications and robot control.

2 COMMAND Communication System

The COMMAND system components include a ruggedized instrumented glove, GPS-enabled ruggedized handheld computer, and tactile display vest.

2.1 Instrumented Gloves for Hand and Arm Gestures

The gesture recognition gloves used for signal communication among Soldiers consisted of a standard tactical glove with accelerometers embedded within each fingers, as well as an accelerometer, gyroscope, and digital compass embedded in the back of the hand.

2.2 Handheld Computer

Data processing and signal communication were performed by two TDS Nomad GPS-enabled ruggedized handheld computers (one carried by the individual generating hand signals and the other carried by the individual receiving communications via the haptic vest). The handheld computers included touchscreen and visual displays, and Android operating system, custom gesture recognition software and tactor controller software, as well as embedded GPS and wireless communication capabilities.

2.3 Tactile Vest

The tactile vest worn by the individual receiving communications consisted of a custom-made, ruggedized, adjustable harness with six embedded vibrotactile actuators (tactors) on the front interface, six tactors on the back, and eight tactors spaced evenly around the waist.

The vest design was previously developed under an Office of Naval Research (ONR)-funded effort, and consisted of a custom tactor solution, using original manufacturer tactor motors and a custom electronics board. It was found that the Precision MicroDrive 310-101 provided optimal performance within a small form factor and provided a cost-effective price point. The board configuration allowed for the tactor system to operate for approximately 3−5 h.

2.4 Gestural and Tactile Cues for Covert Communication

Four hand and arm signals were used to test the COMMAND system. One of the four signals (“freeze”) followed standard Army guidelines while the other three (“rally”, “double time”, “danger area”) were created to demonstrate the generative capability of the signals used in the COMMAND system.

3 Robot Control System

3.1 Robot Control Glove and Gestures

The robot control system used an instrumented glove for gestural control. The glove selected was the AcceleGlove, a commercial off-the-shelf (COTS) instrumented glove developed by AnthroTronix (ATinc). The glove was used within the robotic control tasks and was compared with a traditional gamepad controller.

The AcceleGlove consists of a nylon glove with the finger tips exposed, with accelerometers embedded within each finger and on the back of the hand. The robot control tasks consisted of maneuvering the robot through a series of paths and obstacles, driving both forward and backward, as well as approaching an object, coming as close to it as possible without coming in contact. Therefore, the necessary controls included forward, reverse, left, right, and stop. Forward control was achieved by angling the hand downward; reverse control was achieved by angling the hand upward; left turning was achieved by flexing the index finger; and right turning was achieved by flexing the middle finger. Since the robot was within view of the participants at all times during task completion, visual feedback was provided by observing the robot directly, rather than via a visual or haptic feedback display.

3.2 Baseline Robot Controller

The traditional controller used for comparison with the AcceleGlove during the robot control tasks was a COTS gamepad with thumb joysticks and binary buttons.

3.3 Robot

The robot used for the experiment was a dual tracked, skid steer mobile robot developed as a precursor to the Packbot® series of robots manufactured by iRobot®. The robot contains motorized “flipper style” arms that can be used to navigate the robot over uneven terrain that other robots might find difficult. Movement is provided by three 90-W direct current motors, which independently power the robot’s tracks, allowing for zero-point turning. For this experiment, the robot was fitted with a class 1 Bluetooth® module to allow for wireless communication with the operator control unit (OCU).

4 Experiment Method

4.1 Participants

Participants were recruited from the 11 Bravo (Infantry) or similar military occupational specialty (MOS). Initially, 36 Soldiers voluntarily and fully consented to participation as required by Code of Federal Regulations (CFR) 219 (1991) and Army Regulation (AR) 70-25 (1990). The investigators adhered to the policies for the protection of human subjects as prescribed in AR 70-25. All participants signed a Volunteer Agreement Informed Consent Form. Participants did not receive any compensation for participating in this investigation.

Data was collected on 31 of the originally consenting 36 Soldiers due to attrition from external factors (i.e. weather, equipment). Twelve Soldiers were from an Explosive Ordnance Disposal (EOD) MOS and had extensive experience with robot control. Six Soldiers were from an active Infantry unit (3rd Infantry Division). The remaining Soldiers were form the Officer Candidate School; some had previous military experience while some did not. Participants included 25 males and 6 females. Twenty-eight were right-handed or ambidextrous.

4.2 Experiment Scenario Tasks

For this experiment-based evaluation, the task demands of a typical rural reconnaissance patrol were deconstructed into specific tasks in order to better structure the data collection and performance measurement process. Task demands were separated into two data collection stations, one that included a platoon leader (PL) role and a squad leader (SL) role, and another that focused on a robot controller (RC) role.

The PL role was relatively passive but enabled the participant to use the COMMAND communications unit. The PL walked behind the SL and used the instrumented glove to send communications to the SL, who in turn received the signals via the tactile vest array. In the baseline manipulation, the PL performed traditional hand and arm signals, and the SL would perceive and recognize them visually (e.g., by turning and looking).

The RC role had the participant wear the instrumented gesture recognition glove to control robot movement and performance through four robot control courses to assess different robot control maneuvers.

Prior to each station event, participant trained with the equipment. Each Soldier was introduced to the equipment and demonstrated understanding and use of the equipment prior to the experiment session. Training evaluation information was collected during data feedback sessions.

Station 1: Covert Communication During IMT and Tactical Movement. Station 1 compared the glove/tactile system to traditional hand and arm signals during tasks associated with Soldier movement. Data collected at this station included (a) whether the Soldier perceived a signal, (b) the time taken to notice a signal, and (c) accuracy of signal interpretation. Additionally, during the tactical movement phase, the number of flags noticed by the Soldiers was also collected.

Individual Movement Technique (IMT) Phase. Soldiers used the two systems to communicate while performing standard IMT maneuvers, such as walking, climbing, and crawling. The Soldier acting in the SL role would perform the movements as the PL Soldier would follow behind the SL and provide either glove-based tactile or traditional hand and arm signals. When the traditional hand and arm signals were administered, the SL Soldiers were required to visually scan for the hand and arm signals given by the PL. It is obvious that many hand signals generated by the assigned leader can easily be missed by the designated point man. However, for the glove-based condition for IMT maneuvers, signals given by the assigned leader were able to be perceived by the designated point man immediately and without turning around.

Tactical Movement. During the tactical movement phase, one experienced data collector guided the SL Soldier through 400 m of wooded terrain. The data collector provided the glove-based signals while another recorded performance data. Each SL traversed 200 m using the glove system, and 200 m with traditional hand and arm signals. The SL walked ahead of the person generating the signals. As with the IMT Phase, using the instrumented glove and vest enabled the hand signals generated by the PL to be perceived by the SL immediately and without turning around whilst navigating in the wooded terrain and looking for hidden flag markers. Without the glove and vest, the SL was required to turn around while navigating and searching for flags in order to detect the hand signals.

Station 2: Robot Control. Station 2 was used to evaluate robot control performance using the instrumented glove against a more traditional handheld baseline controller. Collected data included (a) time and (b) driving errors. Participants were asked to navigate through each of the following robot control tasks:

  1. 1.

    Zigzag course. The robot was maneuvered between two engineering tapes outlining a zigzag pattern. The operator was required to keep the robot within the tape while accomplishing the course.

  2. 2.

    Narrow gap course. The robot was maneuvered through a gap created by engineering tape, while avoiding small flags that were situated on and around the direct route. The robot was maneuvered from start to finish, going forward, then from the finish point back to the beginning, by going backwards.

  3. 3.

    Figure-8 course. The robot was maneuvered around logs situated in a figure-8 pattern. The robot had to move around the left of one log, then the right of the next log, and so on, turning around the last one, and continuing the pattern back, going backwards.

  4. 4.

    Movement to contact. The robot was maneuvered forward towards a pole and would stop as close as possible. It was then directed backwards, to another pole, also to stop as close as possible.

4.3 Experiment Design

Orientation. Each Soldier participant was briefed on the purpose, procedures, and any risks involved in their participation. They were provided with copies of the Informed Consent Form to ensure the voluntary nature of their participation. Participants were given an opportunity to review the experiment objectives, have any of their questions answered by the investigators, and were then asked to sign he consent form indicating their informed voluntary consent to participate. All Soldiers agreed to participate. A demographic questionnaire was then administered to obtain pertinent information on his/her background.

Each of the Soldiers was assigned a unique roster number based on groups of 6 participants per day. It was ensured that each Solider would participate at each station, for each role, in a counterbalanced order.

Post-Session Evaluations. In order to obtain Soldier feedback, participants filled out feedback questionnaires after performing each station role. The questions asked the ease of use, perception, and interpretation of the COMMAND system components. Soldiers were also asked to provide ratings of workload using the National Aeronautics and Space Administration task load index (NASA TLX), ratings of operational utility for various combat missions, and additional open-ended comments, suggestions, and issues relevant to performance of the system in the field.

5 Results

5.1 Covert Communications

Detection of the four communication signals provided by the glove to a tactile vest was compared to visual recognition of the signals administered via traditional hand and arm signals. Soldiers used both system types during two performance scenarios, IMT maneuvers and tactical movement. These scenarios are described in more detail in the Experiment Method section.

Detection Rate, Accuracy, and Time (N = 31). Table 1 shows mean signal detection rate, time (seconds), and accuracy rate for the glove/tactile vest and the hand and arm signal conditions, by course type (IMT versus tactical movement). The detection rate represents the percentage of commands recognized by the Soldier, time represents the time to detect the command, and the accuracy rate represents the percentage of detected signals that were correctly identified. In some runs, only two of four signals were working (problems were associated with the ‘danger’ and ‘double-time’ signals for five of the first ten Soldiers).

Table 1 Mean values for detection rate, time to detect, and accuracy for the glove/tactile vest and the hand-arm condition, during IMT maneuvers and tactical movement.
Full size table

Using the repeated-measures analysis of variance (ANOVA) program by Statistical Package for the Social Sciences, specific comparisons within the IMT and tactical movement task demands were analyzed. The F statistic associated with degrees of freedom (df), the p-value, and the partial eta square measure of effect size (ηρ2) are reported.

A significant difference was found between glove and hand-arm means for both the IMT condition (F 1, 30 = 20.13, p = 0.00, ηρ2 = 0.40) as well as the tactical movement (F 1, 30 = 36.25, p = 0.00, ηρ2 = 0.547), where detection rates were higher for the glove/tactile system. There was also a significant difference between the systems in time for detection for IMT maneuvers (F 1, 30 = 214.84, p = 0.00, ηρ2 = 0.877) and tactical movement (F 1, 30 = 455.479, p = 0.00, ηρ2 = 0.938). However, the differences in accuracy rate were not significant for IMT tasks (F 1, 30 = 3.95, p = 0.056, ηρ2 = 0.116) or for tactical movement (F 1, 30 = 0.616, p = 0.439, ηρ2 = 0.02).

Breakdowns by Type of IMT Maneuvers. Signals were presented to the Soldiers both during obstacle events (e.g., climbing, crawling, combat roll, running) and between obstacles. Table 2 shows the descriptive statistics for the glove/tactile vest system versus hand-arm signals, for the IMT task and tactical movement demands. There was no difference between the mean detection rate (both 100 %) or mean time for the glove/tactile vest system due to type of task event (i.e., walking versus obstacle events). The percentage of correct identifications was somewhat lower with obstacle events, while with the hand and arm condition, the effect was that of lower detection rates associated with obstacle events.

Table 2 Mean performance measures by system and type of movement
Full size table

Differences between the glove/tactile vest system and the hand-arm signals are similar to overall results, in that the glove/tactile vest system was associated with higher detection rates and faster times. This breakdown examines the effect of task demand on glove/tactile system.

Effects of Task Demand on Performance Measures.

Time to detect signal. There were no significant differences for the glove system throughout task demand (F 2, 60 = 2.379, p = 0.101, ηρ2 = 0.07). The differences between the glove/tactile vest system and the hand-arm signals reflect the same trends found between the IMT and tactical movement tasks.

Effect of task demand on detection rate when using the glove/tactile vest system. There was no difference in detection rate for the glove/tactile vest system. All conditions were associated with 100 % detection.

Accuracy. The differences in time due to task demands were not significant (F 2, 60 = 1.74, p = 0.185, ηρ2 = 0.06). The same trends found between the IMT and tactical movement tasks reflect the differences between the glove/tactile vest system and the hand-arm signals (F 3, 66 = 1.18, p = 0.32, ηρ2 = 0.05).

Performance by Tactile Signal.

Differences in glove/tactile signal detection due to signal. Signal detection using the glove/tactile vest system was 100 %, regardless of signal, across all task demands.

Differences in glove/tactile signal time to detect, due to signal. Repeated measures ANOVA showed overall significant differences in time to detect due to signal (F 3, 66 = 14.55, p = 0.00, ηρ2 = 0.40). Using the Holm’s Bonferronni correction for multiple comparisons, all paired comparisons were significantly different, except for the difference between “freeze” and “double-time.”

Differences in glove/tactile signal accuracy rate, due to signal. Repeated ANOVA measures showed no overall significant differences in time to detect due to signal.

Impact of task demand on correct identifications of glove/tactile system signals. The detection rates and accuracy rate of each signal were associated by task demand. Accuracy rates remained high regardless of task demand, though rates were somewhat lower when negotiating IMT obstacles.

Number of flags detected. Soldiers detected an average of 10.61 flags with the glove/tactile system (standard deviation = 2.70) and 9.71 flags with the hand-arm signals (standard deviation = 2.77). While the mean number was higher with the glove system, the difference did not meet significance criteria, though it did come close (F 1, 30 = 3.64, p = 0.07, ηρ2 = 0.11). There was considerable variance among the Soldiers with regard to this performance.

Perceptions of Workload and Self-Efficacy. Soldiers provided self-reported ratings of workload and self-efficacy for using the glove to send signals and the tactile vest to receive signals. These were provided using the NASA-TLX rating scales. Each scale ranged from 1 (extremely low) to 10 (extremely high). Workload ratings were relatively low for both components, while performance ratings were relatively high. Direct ratings were used as they have been demonstrated as valid when compared to weighted ratings (Hart and Staveland, 1988; Hart, 2006). Soldiers tended to report confusion with the weighted process, which also confounded measures of workload with a measure of self efficacy (i.e., performance scale). For this reason, we kept these constructs separate.

5.2 Robot Control: OCU and Glove-Based Control

Performance of the robot control tasks using the OCU was compared to the Glove-Based Control. Additionally, participants were asked to provide workload ratings using the NASA TLX and their spatial ability scores were compared with their robot controller performance.

Performance Measures. Performance of the tasks was recorded through the mean and standard deviations for time to complete task (in seconds), number of minor errors, and number of major errors. There was no significant difference between the two control methods for time taken to complete the task (F 1, 27 = 1.39, p = 0.25, ηρ2 = 0.05). This could be due to the high variance around OCU time (SD = 19.30 for OCU versus 7.82 for glove control). The difference in minor errors was also not significant (F 1, 27 = 2.49, p = 0.13, ηρ2 = 0.08). However, the number of major errors associated with the glove system was significantly higher (F 1, 27 = 6.31, p = 0.02, ηρ2 = 0.19).

Perceptions of Workload and Performance. Overall, the mean values and standard deviations for NASA-TLX constructs were higher for the glove controller with regard to workload. The differences were significant for mental workload (F 1, 27 = 16.98, p = 0.00, ηρ2 = 0.39), physical workload (F 1, 27 = 7.90, p = 0.01, ηρ2 = 0.23), time pressure (F 1, 27 = 6.82, p = 0.01, ηρ2 = 0.20), effort (F 1, 27 = 15.43, p = 0.00, ηρ2 = 0.36), and frustration (F 1, 27 = 16.43, p = 0.00, ηρ2 = 0.38). Self-ratings of how well each Soldier thought they performed were also significantly different (F 1, 27 = 8.79, p = 0.01, ηρ2 = 0.25), with higher performance ratings associated with the OCU.

Spatial Ability: Robot Control. Participant spatial ability scores did not correlate significantly with any robot controller performance measure. These scores were also analyzed as a covariate in analyses regarding robot controller performance. This factor was not significant for most of the criterion performance values; however, it approached significance for forward movement distance.

6 Discussion

6.1 Glove/Tactile Vest System for Covert Communications

In comparison to traditional hand and arm signals (84 %–88 %), average percentage of signal detections of Soldiers using the instrumented glove with a tactile display was significantly higher. The glove-based signals were also detected significantly faster. This finding was not unexpected, as traditional hand and arm signals presented from behind depend upon the Soldier’s ability to periodically look at other team members while also maneuvering through the woods and visually monitoring his surroundings. This situation is common when platoon leaders or point persons are placed between two squads in formation. The differences between the glove-based systems and the hand-arm signals are more pronounced during tactical movement and IMT objectives, in comparison to signals presented during walking. Soldiers were also able to interpret the received tactile signals with similar accuracy (87 %–95 %, across task conditions) to that of hand and arm signals. The data also suggested that Soldiers were able to pay more attention to their surrounds while using the glove and tactile vest system. Cognitive workload ratings provided by the soldiers were relatively low for the glove/tactile vest system, ranging from 1.79 to 3.40 on a 10-pt scale.

We are able to assume that the strength of the presented signals was sufficient for percentage, given the high percentage of detected glove-based signals. Interpretation is also dependent on other characteristics of the tactile patterns regarding tactile salience (Hancock et al., in press; Mortimer et al., 2011). It is worth noting that only two to four signals were used; however operator accuracy of tactile signal interpretation was high. This evaluation is a preliminary effort to inform the development of the system during the course of the funded project. Suggested research would include an examination of the characteristics of tactile patterns that would make the signals more easily and correctly interpreted. Additionally, research regarding the number of tactile signals that can be easily training and discriminated is recommended for the effort.

The glove-based system demonstrated performance-based results, however it also demonstrated some limitations. Some of the glove-based cues experienced functionality failures, resulting in Soldiers only experiencing two or three of the four signals. Technical refinement of the system is necessary in order to extend capabilities to provide a greater range of signals with more reliable signaling. Additionally, some soldiers experienced difficulty performing particular gesture(s). This highlights a need for modification of the gestures for ease of execution.

6.2 Gesture-Based Robot Control

Using the glove-based robot control system and a traditional handheld robot controller, Soldiers performed several robot maneuver tasks. The task demands were designed to be difficult and challenging. Although the handheld controller was associated with lower average number of driving errors, the overall glove-based robot control concept was demonstrated as effective. While the handheld controller and glove-based controller were associated with similar times to perform the robot maneuver tasks, the handheld controller displayed greater variability in times across Soldiers; some Soldiers performed much faster while others performed much slower. In contrast, the glove-based controller was associated with less variance in timed performance, even after a short training session. These findings may reflect differences in Soldier experience with handheld controllers. They also suggest that novice operators may more easily learn the glove-based approach, as it is associated with a shorter training time. Further investigation would be necessary in regards to training content, training time, and individual differences.

Soldier perceptions of workload using the NASA-TLX were higher for the glove than the handheld system. These differences were significant for mental, physical, time pressure, effort, and frustration. Self-ratings of performance were also significantly different, with higher ratings for the handheld OCU.

Soldiers with higher spatial ability were associated with somewhat faster times for the handheld controller. In contrast, there was no association between spatial ability and difference in performance times for the glove condition. This suggests that the glove-based approach may be less difficult overall, with regard to spatial skill demand, particularly when one controls for experience with robot controllers. These results also suggest a need for further investigation regarding training content, training time, and individual differences.