Comparing Psychometric and Behavioral Predictors of Compliance During Human-AI Interactions

Abstract

Optimization of human-AI teams hinges on the AI’s ability to tailor its interaction to individual human teammates. A common hypothesis in adaptive AI research is that minor differences in people’s predisposition to trust can significantly impact their likelihood of complying with recommendations from the AI. Predisposition to trust is often measured with self-report inventories that are administered before interactions. We benchmark a popular measure of this kind against behavioral predictors of compliance. We find that the inventory is a less effective predictor of compliance than the behavioral measures in datasets taken from three previous research projects. This suggests a general property that individual differences in initial behavior are more predictive than differences in self-reported trust attitudes. This result also shows a potential for easily accessible behavioral measures to provide an AI with more accurate models without the use of (often costly) survey instruments.

Part of the effort behind this work was sponsored by the Defense Advanced Research Projects Agency (DARPA) under contract number W911NF2010011. The content of the information does not necessarily reflect the position or the policy of the U.S. Government or the Defense Advanced Research Projects Agency, and no official endorsements should be inferred.

Appendices

Appendix A Figures

Fig. 1.

Disposition to Trust Inventory Items

Appendix B Data and Models

1.1 B.1 Models

We used linear regression to model and test the predictive value of the various behavioral measures. The basic approach is to fit a reference model (Model 1) in which the outcome measure, future behavior, is predicted by the treatment conditions alone, such as:

$$\begin{aligned} Y_{iFB} = \beta _0 + \beta _\textrm{Treat} X_\textrm{iTreat} + \epsilon _i \end{aligned}$$

(1)

We use this general form for the reference (null) model across the experiments. Note that there are different FB measures; which one is used in a given set of models depends on the independent variable in question. A given reference and test model, however, always use the same dependent variable.

Fig. 2.

The robot only communicated differently in study 3—it always made the same mistakes, in the same order. These features mean that the data lend themselves to concise, straightforward visual representation, which we present in the top panel. Note that the colors in this panel indicate the robot’s recommendation type. Specifically, the robot made two deadly and three innocuous mistakes which are highlighted above. Overall, there is a trend of increasing compliance. The figures for studies 1 and 2, which are respectively the bottom left and right panels, only indicate compliance percentage, not the type of recommendations made by the robot.

Model 2: The first category of test models incorporate participants’ DTI score as an independent variable:

$$\begin{aligned} Y_{iFB} = \beta _0 + \beta _\textrm{Treat} X_\textrm{iTreat} + \beta _\textrm{DTI} X_\textrm{iDTI} + \epsilon _i \end{aligned}$$

(2)

Note that the reference model is nested within this model. In other words, Eq. (2) represents the alternative hypothesis that adding the predictor variable to the model will result in accounting for more variance, or a significant improvement in model performance. Comparing the two models is as simple as applying an F-test, which in this case tells us whether the more complex model results in a statistically different residual sum of squares value. If it does, we can reject the null hypothesis, i.e. that the reference model is sufficient, in favor of the alternative hypothesis, or that adding the additional predictor variable(s) was warranted. The F-test, in this instance, can be thought of as allowing us to investigate the utility of adding DTI or other measures to the model.² When this test returns a p-value less than 0.05, we can conclude that the residual sum of squares values for the two models are significantly different at the level of $\alpha = 0.05$, or in other words, the new explanatory variable is warranted as it significantly increases the amount of variance explained in the model (i.e. RSS is lower).

Model 3: The next category of test models incorporate participants’ past behavior as independent variables, for example, the model:

$$\begin{aligned} Y_{iFB} = \beta _0 + \beta _\textrm{Treat} X_\textrm{iTreat} + \beta _\textrm{FC} X_\textrm{iFC} + \epsilon _i \end{aligned}$$

(3)

This uses participants’ first choice to predict their compliance for all of the remaining choices that they faced. Again, this has the reference model embedded in it, and a simple F-test will reveal the utility of the FC predictor. We construct similar models for M1C, AFM, and AC-AFM (but using appropriate FB measures).

Model 4: Since it is likely that DTI and the past behavior measures are accounting for different variance in the models, directly comparing equations (2) and (3) via $R^2$ is not entirely informative. Thus, we introduce a third category of test models in which both DTI and a past behavior measure are included. In the case of FC, the model is:

$$\begin{aligned} Y_{iFB} = \beta _0 + \beta _\textrm{Treat} X_\textrm{iTreat} + \beta _\textrm{DTI} X_\textrm{iDTI} + \beta _\textrm{FC} X_\textrm{iFC} + \epsilon _i \end{aligned}$$

(4)

These models facilitate assessing whether the added complexity of including both DTI and the past behavior measure is warranted, again using F-tests. Finally, for readability, we refer to relevant statistics within the text of the manuscript but place regression and other tables for each set of models and their associated tests in the appendix. These tables include, in the same order as above, models that facilitate comparing DTI and the behavioral measures. Each regression table is followed by a table presenting the results of the relevant F-tests.

1.2 B.2 Data

We used data from experiments conducted as part of a long-term research project on explainability and AI. Participants of these experiments team with a simulated robot during reconnaissance missions. The missions involve entering buildings to determine whether threats are present. The robot goes first and is equipped with a camera, microphone, and sensors for nuclear, biological, and chemical threats. These sensors are not perfectly reliable. Based on the data that it collects using its sensors, the robot makes a recommendation to the participant about putting on protective gear. The participant then makes a choice about wearing the gear, i.e., whether or not to comply with the robot’s recommendation. When participants wear the gear, it always neutralizes any threat. If they do not wear it and encounter a threat, they die in the virtual world, but in reality, incur a prohibitive time penalty. Finally, participants incur a slight time delay (much smaller than that for death in the virtual world) when equipping the gear.

In all three studies, the robot based its recommendations on the noisy sensor readings as input to a policy computed through either Partially Observable Markov Decision Processes (POMDPs) [19] or model-free reinforcement learning (RL) [20, 40] using the reward signal of the time cost and deaths incurred. The robot performed significantly better than chance across the studies. This means that compliance from the participants was highly correlated with making the normative choice, i.e., wearing protective equipment at the right time.

Participants of all three studies completed the 12-item DTI before starting their assigned mission(s). Gross experimental details and the results of these conditions are reported in the original publications. As such, we do not replicate those findings here for brevity’s sake. Do note, however, that the n’s we report may vary from the original papers because of incomplete observations (some participants chose to not complete the DTI).

Study 1 participants ($n=198$, Amazon Mechanical Turk) completed three missions, each with eight buildings [44]. They were randomly paired with one of two POMDP-based robot types: a high-ability robot that was never wrong or a low-ability robot that made mistakes 20% of the time (or was 80% reliable). Both types of robots were crossed with four different recommendation explanation conditions: none, confidence level, sensor readings version 1, and sensor readings version 2. This experiment was fully between subjects, meaning that each participant interacted with only one robot type and received only one type of explanation throughout the missions. The coefficient $\beta _\textrm{Treat}$ in the models for Study 1 captures which information condition participants experienced. First compliance choice (FC) takes 1 if participants heeded the robot’s recommendation for the first building and 0 if not. Mission 1 compliance (M1C), on the other hand, is the fraction of times that a participant complied with the robot’s recommendations during the first mission. The compliance future behavior (FB) measure associated with FC for study one is thus the fraction of times a participant complied for the remaining 23 buildings. Similarly, for M1C it is the fraction of times that a participant complied with the robot’s advice during missions two and three. It should be noted that participants were not told whether they were interacting with the same robot across missions; instead, the robot started each mission as if it had not previously interacted with a participant.

Study 2 participants ($n=53$, cadets at West Point Academy) completed eight missions, each with a different POMDP-based robot [46]. In each mission, the human-robot team carried out a reconnaissance task of 15 buildings. The mission order was fixed (i.e., always searched the buildings in the same order and across missions), but the robot order was randomized. The $2\times 2\times 2$ design crossed robot acknowledgment of mistakes (none/acknowledge), recommendation explanation (none/confidence), and embodiment (robot-like/doglike). Unlike Study 1, participants interacted with different robots during each mission. Nevertheless, to demonstrate the robustness of the simple behavioral measures, we rely on the same first compliance choice (FC) as Study 1 and a similar mission 1 compliance (M1C). The compliance measures, obviously, cover a longer horizon: 119 missions and 105 missions, respectively. The $\beta _\textrm{Treat}$ of the models for Study 2 data captures the robot type of the first mission. It is possible that ordering of robot advisors mattered; however, the data are insufficient to specify a hierarchical model that would uncover such a feature.

Study 3 participants ($n=148$, Amazon Mechanical Turk) completed one mission that covered 45 buildings with an RL-based (RL: reinforcement learning) robot in a fully between design [14, 15, 31]. The treatment conditions held the robot’s ability constant but varied how it explained its recommendations: no explanation, explanation of decision, explanation of decision and learning. Again, the first compliance choice (FC) is the same as the previous two studies and the FC outcome measure is the compliance fraction for the remaining 44 buildings. Mission 1 compliance (M1C) is not applicable given that the entire experiment consisted of a single mission. Given that building order and robot performance were fixed across treatment conditions, however, the two additional compliance measures, choice after the first mistake (AFM) and average compliance through the first mistake (AC-AFM), become meaningful. The first mistake occurred during building six, thus participants’ decision for building seven is the AFM measure and the fraction of times they complied during the first seven buildings is AC-AFM. Relatedly, the dependent variable is the fraction of times that a given participant complied during the remaining 38 buildings.

Appendix C Tables

Table 1. Study 1 FC Models

Table 2. Study 1 FC Model Comparisons

Table 3. Study 1 M1C Models

Table 4. Study 1 M1C Model Comparisons

Table 5. Study 2 FC Models

Table 6. Study 2 FC Model Comparison

Table 7. Study 2 M1C Models

Table 8. Study 2 M1C Model Comparison

Table 9. Study 3 FC Models

Table 10. Study 3 FC Model Comparison

Table 11. Study 3 AFM Models

Table 12. Study 3 AFM Model Comparison

Table 13. Study 3 AC-AFM Models

Table 14. Study 3 AC-AFM Model Comparison

Table 15. DTI as a Predictor of First Compliance Choice