Keywords

1 Introduction

In many health care situations, patients and their providers must choose a course of treatment from among many viable options. When there is not a clearly superior course of treatment, there are often major discrepancies between patient preferences and care received and these discrepancies can challenge patient autonomy, quality of care and can result in costly and unnecessary treatments. Shared decision-making (SDM) can help to better align patient preferences, values and health care goals with the care they receive [1].

Shared Decision-Making (SDM) is a collaborative process of interaction and communication between patients and their providers allowing them to make health care decisions together. SDM takes into account the clinician’s knowledge and experience, the best scientific evidence available as well as the patient’s goals, preferences and values. Research demonstrates that SDM can increase patients’ knowledge, reduce uncertainty, improve quality of care and reduce costs, often by limiting overuse of treatments that patients do not value [1].

Implementation of SDM has been limited by the available time for patients to explore treatment options with their physicians. This process is further complicated by the presence of low health literacy. Patient decision aids can help foster SDM, however, decision aids often do not address low health literacy users [2]. The usage of ECAs has been identified as a possible solution to facilitate shared and knowledge transfer to patients with low health literacy [3].

The primary goal of this study was to identify the characteristics of an ECA to help patients understand the benefits and drawbacks of different treatment choices in response to prostate cancer using a user-centered design process. Through this process, an African-American character was created in a stylized, two-dimensional animation style and with facial, hand and body gestures intended to convey empathetic emotions such as optimism and concern. Facial expressions and body poses and gestures were authored to display in conjunction with a physician-authored patient dialogue between the ECA and patient.

This paper outlines the design process and challenges of creating this character for our implementation of an ECA (eCoach), specifically the diverse views of focus groups consisting of different socio-economic groups and levels of health literacy. Initial focus groups consisting of members representing both higher and lower socio-economic status and levels of health literacy found the stylized ECA to be unappealing. Later input after completion of the educational intervention accepted the portrayal of the ECA, reporting it to be acceptable.

We also present the qualitative and quantitative findings of a series of user focus groups conducted during the development of the eCoach ECA and the design implications and lessons learned for future work using ECAs within a shared decision-making context, particularly when designing to accommodate differing health literacy and socio-economic backgrounds.

2 Health Literacy and Shared Decision-Making

Numerous decision aids have been developed to foster shared decision-making, but most fail to address the needs of patients with low health literacy, which is particularly prevalent among racial and ethnic minorities. Health literacy is an individual’s ability to read and comprehend a range of health-related materials required to successfully function in the healthcare environment [4]. Health literacy includes the ability to perform both basic reading and numerical tasks and requires a complex combination of analytical and decision-making skills to be applied to health situations.

Low health literacy disproportionately affects minorities – more than half of African American adults and two-thirds of Hispanic adults have low health literacy compared to less than one-third of white adults [5]. Decision aids utilizing embodied conversational agents have shown promise at addressing health disparities due to low health literacy [3, 6].

3 Embodied Conversational Agents for Health Decision Aids

There is evidence to consider face-to-face consultation with a health provider, coupled with well designed, written instructional materials, a health care best practice, especially when communicating health information to patients with low health literacy [7]. The many affordances of face-to-face consultation include the use of verbal and non-verbal cues and behaviors, such as empathy and immediacy, which can foster patient trust and satisfaction and enable better health communication and understanding. Because they simulate face-to-face communication, embodied conversational agents have shown promise for delivering health information in decision aids [3, 6].

An embodied conversational agent is a user interface which simulates face-to-face conversation, typically by presenting the user with an animated, human character who talks to the user and often also uses other naturalistic modes of communication such as facial expressions and hand, head and body gestures [8]. The use of an ECA in a health decision aid offers many potential advantages:

  1. 1.

    Patients can learn essential health information without requiring time from their provider.

  2. 2.

    The interactive, conversational modes of communication used by ECAs can overcome passivity limitations of traditional health decision aids and can promote active learning and decision-making.

  3. 3.

    ECAs can allow patients to take adequate time to understand important information, repeating content or explaining content in simpler terms as necessary, all critical features for patients with low health literacy.

4 Decision Aids for Prostate Cancer

Prostate cancer is an ideal candidate to test interventions intended to increase shared decision-making and decrease decisional regret. Prostate cancer is a leading cause of morbidity and mortality in men. Its direct treatment costs alone are estimated to be $11.9 billion annually [9]. There are several different treatment options for prostate cancer patients that do not differ greatly in efficacy. However, the potential side effects and possibilities for adverse events vary significantly among the various treatment options. Patients can have difficulty understanding the large range of treatment options and each option’s adverse event and possible side effects profile, often leading to distress and decisional regret. SDM can address these issues by seeking to better match patients’ preferences to the treatment option ultimately chosen.

Although decision aids and educational materials can reduce some of the time and cost burdens to physicians for SDM, research shows that those for localized prostate cancer are inadequate [10]. We have chosen to explore the use of an ECA-based decision aid for localized prostate cancer as a possible better alternative.

5 Effects of Appearance in Embodied Conversational Agents

The physical appearance of an ECA and the use of embodied, non-verbal cues has been shown to impact patients’ perceived trust and understanding [11].

5.1 Agent Realism

McDonnell, et al. investigated how different rendering styles, ranging from abstract to realistic, affect users’ perception of a virtual human character, finding that more abstractly depicted, cartoon characters were often considered highly appealing and more pleasant than realistically rendered characters and that they were rated as more friendly and trustworthy and therefore may be more appropriate for certain virtual interactions (e.g. health care decision aids, motivational agents, etc.) [12]. These findings also confirm the Uncanny Valley hypothesis [13] that as realism is increased in rendering and animating a virtual human character, at some point, the character begins to trigger increasingly negative reactions: Study participants experienced relatively negative reactions to several versions of a moderately realistic character compared to highly realistic characters or cartoon characters.

5.2 Agent Gender, Race and Age

Other studies have shown that gender, race and age of pedagogical agents can have significant effects on a learner’s motivation, self-efficacy, engagement and satisfaction [14, 15]. Social psychology research suggests that people are more persuaded by members of their in-group and research with ECAs generally confirms this finding, though with some context and task-dependent caveats [14–16].

For example, Baylor et al. found that Caucasian students who were assigned an African-American virtual agent “expert” in an education task had greater self-efficacy (confidence) and interest towards the topic than those who were assigned a Caucasian agent, perhaps because their expectations of what a domain expert should look like were challenged. In contrast, they found that African-American students have strong affiliations with same-race agents, performed better and were more satisfied with an African-American agent [16].

5.3 Interactions Between Agent Appearance and Task Domain

A series of experiments by Ring, et al. demonstrate an interaction between virtual agent appearance and task domain [17]. A cartoon-rendered character was rated as more likeable and caring for a social dialogue task and more friendly for a health counseling task, but a realistic rendering of the character was rated higher for appropriateness, trustworthiness and familiarity for the health counseling task. The effects of character proportions (i.e. realistic cartoon rendering vs. exaggerated, stylized cartoon rendering) were also explored, showing similar results: a highly exaggerated cartoon character was rated as being more friendly regardless of task, but a more realistic cartoon rendering was rated as being more appropriate for a health counseling task.

Studies by Gulz and Haake demonstrate that when learners are given a choice between a more realistic versus stylized visual appearance of virtual pedagogical agents and also a choice between engaging with an agent via a strictly task oriented communication style versus a more socially oriented communication style, there was a significant correlation between preferences for agents with a social communication style and a more stylized visual appearance [18, 19].

6 eCoach: An ECA-Based Prostate Cancer Decision Aid

In order to investigate the feasibility and acceptability of an ECA as a health decision aid, we developed a prototype system consisting of a brief conversation with a virtual agent health advisor (eCoach) to inform and advise prostate cancer patients of the range of treatment options available to them as well as the risk factors and possible side effects associated with each option. We developed an animated, 2D character using the Unity game engine (see Fig. 1). The agent interacts with the user via a turn-based conversation driven by a branching dialogue tree and state-machine-based dialogue engine. Recorded voice-over audio clips were used for the agent’s speech rather than speech synthesis, since we hypothesized that more realistic speech reproduction would enhance users’ affinity with the agent. The agent’s mouth motions were synchronized to the speech recordings by sampling the audio amplitude (not viseme/phoneme synching).

Fig. 1.
figure 1figure 1

Screenshots from the eCoach prostate cancer decision aid

Full size image

A range of non-verbal, embodied cues were also incorporated, such as mouth movements and eyebrow raises synchronized to speech, head nods, facial displays of emotion (concern, empathy, hope, etc.), posture changes, deictic gestures (attention-directing), and idle behavior (blinking, etc.). Various supplemental illustrations and animations were included, such as mortality and side effect risk probabilities, animated visualization of procedures, etc. User participation in the conversation is achieved by selecting from multiple-choice responses and questions. The dialogue, both the agent’s speech and the users’ responses, was authored to closely model the conversational style of a face-to-face, patient-provider encounter, with the intention of establishing rapport, trust and affinity with the agent.

Synthesizing the findings of previous research on agent appearance, and considering that our ECA would be designed to both explain health information (task oriented communication style) as well as engage in social dialogue and present an empathetic demeanor (social oriented communication style), we hypothesized that a moderately realistic, stylized agent with African-American appearance would appeal to our target demographic. eCoach was designed as a decision aid to facilitate shared decision-making for prostate cancer patients with low health literacy, which disproportionately affects minorities.

6.1 User-Centered Design Process for Creating an ECA

We utilized a user-centered design process during the development of the prototype prostate cancer decision aid (eCoach). This process included:

  1. 1.

    Requirements gathering and functional specifications: utilized input from stakeholders, including patients, providers and domain experts (medical experts and ECA experts).

  2. 2.

    Design and development: iterative design of ECA, dialogue script, medical and risk visuals with input and feedback from a series of user focus groups and key informant interviews.

During the design and development phase of our study, we created several versions of the eCoach agent in response to feedback and ratings by our user focus groups. We tested a number of different agent designs, including highly realistic (photographic) to highly stylized renderings, younger and older looking agents, different racial appearances, etc. Figures 2 and 3 illustrate some of the variants of the eCoach agent visual design that we evaluated.

Fig. 2.
figure 2figure 2

Early design sketches and ECA design alternatives showing a range of facial, hair and clothing features, some more stylized than others, and a range of skin tones indicating ethnicity.

Full size image
Fig. 3.
figure 3figure 3

Four rendering styles presented in our focus groups, ranging from photorealistic to abstract. Based on prior research findings and interviews and focus group feedback (with greater weight given to focus groups meeting our target demographic composition), the cartoon option was chosen for the prototype decision aid implementation.

Full size image

Early focus groups were conducted at the Emory Clinic and were largely composed of Caucasian men diagnosed with prostate cancer, with increasing participation by African American men in later groups. A few women, wives of patients, also participated in these focus groups. Participants in the Emory groups also tended to have higher health literacy, education level and socio-economic status than our target demographic. Contrary to our expectations, these participants uniformly expressed a negative reaction to the stylized versions of the eCoach agent. Some participants even expressed anger about the “cartoon” versions of the agent. Anecdotes collected from these groups include:

  • “Prostate cancer is not like a cartoon.”

  • “This is a serious matter and having a cartoon seems to minimize it.”

  • “The cartoon character is very brown.”

Focus groups were also conducted at Grady Memorial Hospital and in these groups participants were more racially diverse and tended to better match our target demographic (lower health literacy and socio-economic status). The final focus group at Grady, in fact, was the only all-male, all African-American group convened. While the Grady groups also largely expressed dislike for the stylized, “cartoon” agent, the final Grady focus group bucked this trend and collectively expressed approval for this version of the agent. Many participants said the agent was “good” and comments on the visual appearance of the stylized agent included:

  • “The cartoon character is good.”

  • “[He] looks professional and has an agreeable look.”

  • “[He] looks like a comic strip, but he is alright.”

This group was also asked for their reactions to more realistic renderings of the agent, as seen in Fig. 2. The group agreed that the vectorized photo version looked “fake”, a reaction shared by all of the Grady focus groups. The photographic agent was considered “good” and “looked cool”, however, when asked to compare the photographic agent with the most stylized, cartoon agent, the group preferred the cartoon version of the agent with participants saying that this version “drew you in” and that “he was softer; he was not serious so, in a sense, you’re more willing to listen [to him].”

6.2 Prototype ECA Evaluation

A prototype of the eCoach decision aid was developed based on prior ECA research and feedback from our focus groups. Though only a minority of our focus group participants had a positive assessment of the stylized, cartoon agent rendering, with some Emory participants even expressing dislike for any ECA at all, we chose to continue development of a prototype decision aid using this version of the ECA in order to test our original hypothesis that it would be appealing to our target demographic. To evaluate the eCoach decision aid prototype, focus groups were held at both Grady Memorial Hospital and Emory Clinic, composed of men diagnosed with prostate cancer who had previously served in the earlier focus groups to provide input on the design and content of the decision aid. The eCoach prototype was developed based on initial clinical and patient feedback in order to test the algorithms, animation, content and usability by patients as well as their satisfaction and affinity for the ECA. The prototype consisted of an animated ECA with recorded voice-over and closed-captioned dialogue presented in a conversational style intended to simulate a typical face-to-face consultation with a doctor.

Focus group participants were shown, as a group, an example walk-through of the decision aid dialogue. A computer running the eCoach decision aid was connected to a video projector and speakers and a focus group facilitator demonstrated a typical user session with the tool. Satisfaction with the eCoach decision aid was measured by single items on seven-point scales and feasibility and acceptability of the tool was measured on 7.5-point scales. In addition, semi-structured interview questions were asked of the participants and their responses were manually transcribed.

6.3 Results

Table 1 summarizes quantitative results of questions related to their level of satisfaction with the ECA decision aid as well as their assessment of its feasibility and acceptability as a prostate cancer decision aid.

Table 1. Quantitative results for prototype focus groups
Full size table

The quantitative results reveal that the Grady participants, who better matched our target demographic, had a higher level of satisfaction with eCoach. In general, statements regarding the feasibility and acceptability of eCoach as a prostate cancer decision aid were rated somewhat lower, however, when asked how effective eCoach would be as a patient decision aid, both Grady and Emory participants gave a significantly lower rating (2.5 for Grady, 3.2 for Emory).

Focus group participants were also asked to comment on their impressions of eCoach, including its usability, how a tool like eCoach might augment usual care for patients with newly diagnosed prostate cancer and how eCoach might be changed to be more usable or acceptable. Grady participants found the stylized ECA character to be “OK” in contrast to the Emory participants who disliked the ECA’s cartoon appearance and stated a preference for a “real person”, a difference of opinion that was not surprising considering feedback from prior focus groups.

When asked to comment on their impressions of the eCoach tool, the Grady group offered the following:

  • The program helped focus information for them.

  • They would rather use eCoach than explore information on their own.

  • eCoach took fear out of decision making.

7 Conclusion

Designing an embodied conversational agent as a health decision aid for patients with low health literacy requires careful consideration of visual design parameters. Our findings suggest that, when designing an ECA for a health decision aid, there is no optimal set of appearance parameters that will be appealing and acceptable to every user. Rather, we find that the ECA’s demographic appearance (e.g. gender, race, age) should align with the target user population’s demographics and further, that additional demographic factors, such as socioeconomic status and level of health literacy should often be considered. The most appropriate rendering style of an ECA is also challenging to determine, especially in the context of a health decision aid. Though prior research on ECAs suggested that we utilize a stylized ECA, given our intention of presenting a health counseling task but using a social dialogue style of communication, we encountered considerable resistance to stylized versions of the ECA among the majority of design-phase and prototype focus groups. Many of the Emory focus group participants, in fact, did not like having an ECA at all, whether realistic or stylized.

Our findings provide evidence that, in certain contexts, such as advising newly diagnosed cancer patients on treatment options and associated risks, presenting a cartoon or stylized avatar in an attempt to appear friendly, empathetic, trustworthy, etc. may actually backfire due to users’ sense of the extreme seriousness of the subject. It may be the case that a stylized, cartoon ECA would be better accepted in a less serious health context.

8 Future Work

Our study has a number of limitations, including exploring a small subset of the design space for ECAs used for health decision aids. Our study was limited to a series of focus groups as part of a user-centered design process and we did not test the completed eCoach decision aid as an intervention to promote shared decision-making. Future work should evaluate the eCoach decision aid against currently available decision aids for prostate cancer with outcome measures to include validated measures of decisional conflict.