BACKGROUND

eConsult programs are becoming more widespread as health care systems seek innovative ways of lowering costs and improving specialty access. An eConsult is an asynchronous exchange through which a PCP places a consult electronically, which is then reviewed and replied to by the consultant. The service can enhance the appropriate use of specialist time, allowing more access for face-to-face visits for patients who need to be evaluated in person.

There is a growing body of literature to support the rationale for implementing eConsults with regard to cost, access, patient satisfaction, and provider satisfaction. eConsult has been a facilitator of improved PCP and specialist communication.1 Though there are limited studies to date, there is potential cost savings for healthcare systems.2, 3 Benefits of such programs for patients include improved access to care, greater convenience, and fewer visits and out of pocket costs.4,5,6,7 Patients are generally satisfied with eConsult programs1, 3, 8,9,10 and appreciate the enhanced PCP role in their care.11

Studies of clinician experiences with eConsults show that they perceive improved quality of care, timeliness of care, costs savings, and comfort in managing clinical problems1, 12, 13 and that eConsults provide educational opportunities.3, 14, 15 In addition, eConsults enable PCPs to keep the patient in their medical home and provide more comprehensive care. On the other hand, PCPs have complained that eConsult programs generate more work.16 PCPs report mixed perceptions of the increased work; while some say the benefits outweigh the negatives, others feel the burden is too great.15 To date, the majority of studies of clinician experiences with eConsults have been conducted in national health systems like the Veterans Health Administration (VA), and Canada and Finland’s national health care systems, integrated delivery systems, and public “safety net” institutions.1, 13,14,15,16 By contrast, the US academic medical centers (AMCs) in our study operate as fee-for-service institutions in an environment of increased attention to value-based care and use of innovative care delivery. AMCs also place high priority on education and teaching, and many specialists and PCPs are part-time clinicians. There has been no comprehensive evaluation of an opt-in eConsult service at multiple US AMCs, and much remains to be learned about how best to implement them.

In 2014, the Association of American Medical Colleges (AAMC) formed a collaborative, funded by a Center for Medicare and Medicaid Innovation (CMMI) grant, to extend an eConsult model developed at the University of California San Francisco (UCSF). The program is called Coordinating Optimal Referral Experiences (CORE) and includes an opt-in electronic medical record (EMR)-based eConsult service with both PCP and specialist remuneration that enables PCP-to-specialist asynchronous consultation. To date, 30 AMCs have joined the CORE collaborative and adopted the eConsult model.

OBJECTIVE

Our aim was to understand PCP experiences with this opt-in eConsult model at CORE site AMCs. In particular, we sought to understand (a) the impact of eConsult integration into routine clinical practice, (b) barriers and facilitators to the successful adoption of eConsults, and (c) implementation strategies that were most effective at building local support for the model.

DESIGN

eConsult Model

A PCP-led team at UCSF developed the CORE eConsult service in 2012. The EMR-based platform enabled PCPs to submit a clinical question to a specialist for a patient whose condition the PCP felt might not warrant an in-person visit with the specialist. The expected response time from the specialist was three business days. Specialists were able to convert the eConsult to an in-person visit if deemed appropriate. At UCSF, both the specialist and PCP received a modest relative value unit-based credit for the eConsult, recognizing the work of both clinicians. EMR-based referral templates were also introduced alongside the eConsult service. Although not the focus of this study, the templates were designed to improve the traditional referral process by providing pre-referral guidance and asking for detailed clinical information from the PCP at the time of placing a referral order for an in-person visit with a specialist.

The AAMC convened a group of AMCs to adopt the UCSF eConsult model, called the CORE collaborative. A designated PCP at each CORE site led implementation efforts and was given flexibility to tailor the model to local needs and conditions. For example, participating AMCs developed their own approach to compensating PCPs and specialists for eConsult-related work. Key aspects of the model are summarized in Text box 1. The first cohort of five AMCs (CORE-1) formed in 2014, and a second cohort of seven AMCs (CORE-2) formed in 2016. Additional AMCs joined after this, and to date, there are 30 AMCs participating in the CORE collaborative, in addition to UCSF, and three additional University of California AMCs adopted the model independently of the collaborative.

Text box 1 Key Features of the CORE eConsult Model

1) Opt-in: The decision to place an eConsult is left with the PCP, not mandated by the specialty as a gateway for referral.

2) Primary Care Lead: A PCP leads efforts to tailor and implement the eConsult model and templates.

3) PCP and Specialist Compensation: Participating AMCs are encouraged to provide compensation or RVU (related value unit) credit to participating PCPs and specialists for recognition of work.

4) Quality Assurance: review of PCP questions and specialist response are used to tailor education and improve the quality of eConsults.

PARTICIPANTS

Participants and Setting

We conducted semi-structured interviews and focus groups with PCPs at six CORE sites and UCSF (Table 1). Interviews were conducted at UCSF in 2014, and at the University of Wisconsin, University of Iowa, University of California San Diego, University of Virginia, and Dartmouth-Hitchcock (CORE-1) in 2015. Focus groups were conducted in 2017 at the University of Washington (CORE-2). The different methods used at UCSF and the CORE-1 and CORE-2 site reflect available resources and feasibility at the time of data collection.

Table 1 Interview Participants at AAMC CORE Academic Medical Centers
Full size table

APPROACH

Recruitment

A purposeful sampling strategy was used for one-on-one interviews, with the goal of including approximately 4–5 interviews per site covering a range of primary care provider training backgrounds, level of experience and stage of career, location, and experience using eConsults. A larger number of interviews were planned at UCSF as part of a broader effort to evaluate the original eConsult program. Medical directors were contacted at clinical sites to assist with recruitment. PCPs were invited to be interviewed for this project through email and by phone. Focus group interviews at the University of Washington were held in place of prescheduled conferences and participation was voluntary.

Data Collection

Researchers used guides for semi-structured interviews (Appendix 1) and focus groups (Appendix 2). The guides included questions that aimed to assess knowledge of and experiences with eConsults and feedback on the eConsult and structured referral program. Interviews lasted 20–30 min and were conducted by a medical anthropologist (SA) and medical sociologist. Interviews at UCSF took place in person, were conducted by a medical anthropologist (SA) and medical sociologist, and were recorded and transcribed. Interviews with providers at CORE-1 sites took place by telephone and were conducted by SA, who also took detailed notes including verbatim quotes. Data saturation was pre-determined as “informational redundancy,” or the point at which no new themes or concepts were apparent in the data.17 Focus groups at the University of Washington (CORE-2) were facilitated by the eConsult program’s primary care leaders and took place in person. A project manager or co-primary care lead took detailed notes including verbatim quotes whenever possible.

Analysis

The authors used the constant comparative method, a grounded theory-based approach to data analysis in which an iterative process of coding and interpretation is used to derive themes.18 After repeatedly reading interview transcripts and notes, all authors met to develop a coding framework and codebook. All transcripts and notes were coded independently by 3 authors (SD, SA, KD), and differences were resolved in group discussions. Further discussions identified key themes related to commonalities and differences across sites, and across interviews and focus groups, with emphasis on factors that impacted PCPs’ response to and use of eConsults.

This study was IRB approved or exempted from review at all participating sites.

KEY RESULTS

One-on-one interviews were conducted at six CORE sites, including UCSF, with 35 total PCPs: 27 attending physicians, 4 advanced nurse practitioners (ARNPs), 2 residents, 2 physician assistants (Table 1). Six semi-structured focus groups were conducted at the University of Washington with 24 total PCPs: 10 attending physicians, 2 ARNPs, and 12 residents. All clinicians were either internal medicine or family medicine trained.

Data saturation was reached before all interviews were completed. We identified three major themes related to implementation, adoption, and impact: (1) eConsult use results in enhanced comprehensiveness of primary care and provides opportunities for case-based education; (2) factors that influence PCPs’ decision about whether to order an eConsult include prior reliance on “curbside” consultation, patient preference, case complexity, and perceived need for specialist guidance; and (3) implementation and incorporation of eConsult into routine practice requires addressing barriers and building relationships between PCPs and specialists (Table 2).

Table 2 Summary of Three Major Themes from Provider Interviews: Experiences with and Unintended Consequences of eConsult Use
Full size table

1) Enhanced Comprehensiveness and Educational Impact

Comprehensiveness

Among PCPs who reported using the service, eConsult increased responsibility for patient management on the part of the PCP. Along with more clinical care being handled by the PCP, there was an increase in need for communication with the patient about the specialist recommendations. One PCP said, “…definitely more responsibility stays on me, for sure, because it is my responsibility to take that information, to interpret that information in the context of everything else and then to communicate that to the patient” (Interview #22). Providers commented on the positive impact that the increased responsibility had on patient relationships. One participant said, “I think it keeps patients in the office and I think when you are managing most of their problems it builds confidence in your patient-physician relationship and with your own practice” (Interview #24).

An unintended consequence of greater responsibility is an intensified workload for PCPs, “It is more time on my part because I’ve got to order all the stuff that they are recommending, get back to the patient, evaluate that. There is definitely more time spent...” (Interview #21). Most PCPs found the shift in management to be an acceptable part of their role: “…as a primary care provider I feel like that is my responsibility anyway” (Interview #22). Some PCPs told us that eConsult actually helped to reduce workload because of a decrease in the care coordination burden: “I think the workload has decreased, strangely enough... Because I feel like I can take care of the issue myself…” (Interview #20).

Educational Impact

The educational impact of eConsult was one of the most important motivations for PCPs to continue using the system. Not only did participants describe feeling reassured by specialists’ responses to their eConsult questions, many were able to extend the knowledge gained through eConsult to their management of other patients’ conditions: “…I can extrapolate some of that knowledge to taking care of other patients” (Interview #30). Many PCPs told us that they saved specialists’ eConsult responses and referred to them later to inform both clinical care and teaching: “I always copy a lot of their helpful notes… so that I can always pull it up later on and then I will teach the residents” (Interview #20). We learned from our participants that eConsult provided a particularly effective venue for learning because of its discrete, case-based approach and immediate relevance. One PCP said, “I like the just in time learning. It’s just relevant to me right now…retention is better. It’s better than a talk [lecture]” (Interview #40).

Factors influencing decision to use eConsult

For most of the PCPs, eConsult presented an entirely new way of communicating with specialists within their institution. We found that PCPs faced two key decisions: whether to submit an eConsult question or use a more informal channel to ask a specialist colleague for advice and whether to use eConsult or place a referral order for an in-person appointment with a specialist.

Many PCPs told us that they welcomed eConsults as an alternative to informal curbside consultations. A curbside consult is an unofficial solicitation of medical advice from a colleague regarding a patients’ care, without the consultant seeing the patient or being directly involved in their care. Curbsides are often conducted over email, by phone, or in hallway conversations. Although some PCPs told us that they continued to use curbside consultations after the eConsult service was made available, many others found eConsult a welcome formalization of the informal system previously used. eConsult not only enabled specialist access to patient records in the EMR, it included formal documentation of the eConsult exchange: “…the big downside of sending an e-mail is that they don’t have direct access to some of the information…” (Interview #22). It also enabled PCPs without an extensive professional network at the institution to benefit from specialists’ expertise, in particular new and junior clinicians. As one PCP explained, “I don’t have a go-to person to just phone and so… I will eConsult them, let them get back to me at their leisure” (Interview #26).

When deciding whether to submit an eConsult in place of a traditional face-to-face referral, we found that PCPs took several key factors into consideration (Table 3).

Table 3 Primary Care Provider Expressions of Provider Decision-Making
Full size table

First, patient preference and clinical complexity played a large role in decisions to use eConsult. If patients voiced a strong preference for a traditional referral, PCPs often conceded, even if eConsult appeared to be a reasonable option: “There are definitely some [patients] who are very adamant about being seen by somebody else…” (Interview #22). Similarly, if a patient’s condition was thought to be very complex, PCPs were more likely to refer the patient for an in-person consult: “If you have a more complex decision or need complex counseling, then a standard referral is helpful” (Interview #40).

We also found that PCPs were drawn to the eConsult service when seeking to resolve uncertainties related to diagnosis and treatment. Examples include requesting advice about a borderline laboratory finding or choosing between different treatment options. One provider said. “I used it for next level processes that [are] not guideline based… more the art side of medicine.” (Interview #14) We also learned that many PCPs used eConsult as a means of deferring the decision about whether a traditional consult is needed to the specialist: “…if they disagree then they can say, ‘No, I really think I should see the patient’” (Interview #19).

Interestingly, we discovered that some PCPs believed that submitting an eConsult would expedite access to a specialty visit for the patient, via the “convert to an in-person visit” option provided to the specialist. Although the conversion option was not designed to give an eConsult patient a special advantage in obtaining a specialty appointment, this did appear to be an unintended consequence of eConsult at some locations. As one PCP explained the phenomenon: “If the specialty attending is worried about it, they can get the patients in” (Interview #14).

3) Implementation and Incorporation into Routine Practice

Challenges incorporating eConsults into routine use included ensuring that PCPs were aware of and able to use the eConsult service, ensuring that specialists who are enthusiastic and enjoy teaching were recruited to serve as “eConsultants,” and recognizing and mitigating unintended consequences of eConsult use.

Education, Training, and Program Goals

Across the CORE sites, there was no standard introduction to the program or training for PCPs. While this allowed for flexibility at each site, many of the providers who were interviewed did not appreciate the goals of the program, which was a large barrier to use. Additionally, many learned about the program through word of mouth or self-discovery in the EMR. A PCP reported that, “There was a period of time where I actually, unfortunately, did not even know it existed… and then I realized I was getting e-mails about it and it made sense” (Interview #24). A PCP involved in the implementation at their site told us, “The biggest challenge is educating all the providers about why we’re doing it and what the ultimate goal is” (Interview #6). Informing PCPs who supervise trainees may be particularly impactful, given that residents often described their initial use of eConsult as prompted by the advice of a preceptor.

Engaged Specialists

Engaged specialists are a key component to program success. PCPs felt that eConsults allowed them to form new relationships with specialists where previously there may have been none and to become comfortable working with that specialist. A provider said, “the key for me was establishing the relationship” (Interview #26). Overwhelmingly, PCPs expressed the desire for timely, non-judgmental, and educational responses. Some PCPs were hesitant to use eConsult due to concerns about being judged by the specialist, one provider said, “I’m sometimes embarrassed to ask a question, but sometimes I just swallow my pride and say, “I’m going to ask anyway”” (Interview #40). PCPs reported that some consultant responses are more helpful than others, and appreciate consults that include a clear recommendation, such as a “…stepwise process for what they would do… if this, then this” (Interview #8) and that are educational; “It is important for the e-consultant to be a very good educator” (Interview #30).

Strategies for Implementation Success

Several successful strategies were identified and used to facilitate use of eConsult, improve the quality of eConsult exchanges, and strengthen the working relationship between PCPs and specialists. Two particularly popular approaches included co-management conferences and eConsult newsletters. Co-management conferences are informal meetings in which a specialist serving as an eConsultant reviews and discusses common clinical cases with PCPs. PCPs reported that co-management conferences helped them get to know their specialist colleagues and overcome their hesitation about using eConsult: “After sitting in on one of those [conferences], I felt more comfortable sending [an eConsult]” (Interview #5).

Newsletters provided an opportunity to communicate updates about the program, including the addition of new specialties and exemplar eConsult questions and responses. A provider stated that newsletters helped to “give people some ideas of what appropriate eConsult questions are” and provide “education for both residents and primary care faculty” (Interview #34). Together, co-management conferences and newsletters helped to create norms guiding appropriate eConsult use and promoted collegiality among PCPs and specialist.

DISCUSSION

Key lessons learned from our interviews with PCPs were the following: (1) eConsults enhanced PCPs’ ownership of patient care and provide cased-based educational opportunities, but this requires engaged specialists; (2) the importance of the PCP driving the decision on when to use eConsult; and (3) successful program implementation necessitates ongoing outreach and education efforts, and recruiting specialists who enjoy the educational aspect of eConsult work.

Comprehensiveness is considered a pillar of high-performing primary care delivery,19 and refers to the extent to which the primary care team meets most patient’s needs. With a changing landscape in healthcare, primary care providers now provide mostly office based care and less comprehensive care of the past. eConsult programs allow for more care to be delivered within the primary care setting. Increased primary care comprehensiveness means improved continuity and less care fragmentation, and has been correlated with lower costs and hospitalizations.20

Introducing an eConsult program shifts more patient management responsibility to PCPs,15, 16 adding to between-visit work and potentially contributing to provider burn out. Lee et al. reported on provider perspectives of an eConsult program at a safety net system and found mixed reviews of the enhanced comprehensiveness, with some PCPs perceiving the burden as “worth it” and others frustrated by increased workload.15 It is important to note that this and other eConsult programs7, 21 require all consults and referrals to first pass through an eConsult triage system. By contrast, the CORE model provides PCPs with a choice between eConsult or referral for a traditional face-to-face visit with the specialist.

Our findings demonstrate that most providers perceived the shift in management to be acceptable; it gave them an opportunity to work more closely and comprehensively with patients and enhanced their clinical knowledge. The VA and Mayo clinic are also academic affiliated centers with opt-in eConsult programs that report high rates of PCP satisfaction;22, 23 however, there have not been in depth reports on PCP perspectives of workload and comprehensiveness at these sites. We believe that PCP control over whether to use eConsult or a traditional referral is a key factor in PCPs’ satisfaction with the program. It is also possible that the remuneration to the PCP, which differs from other academic models, might mitigate the perceived burden by PCPs participating in CORE.

Similar to the reports of others, our interviews revealed the importance of the educational impact of eConsult programs.3, 14, 15 This is a core component for a successful program, and requires engagement of specialists in this educational role. PCPs told us that they wanted consultant responses that had some explanation of the clinical reasoning behind the recommendation, included clear, step-by-step plans, and were non-judgmental in tone. These preferences are now incorporated into AAMC materials used for clinician training at CORE sites. Additionally, each site in our study incorporated a quality assurance process to review eConsult questions and consultant responses to identify training needs and ensure that the eConsult program was meeting providers’ needs and expectations.

Our findings also provide insights into the eConsult decision-making process, which have not previously been reported. Many PCPs used eConsults to replace face-to-face referrals as well as informal curbside consults. Although the use of curbside consultation persists at CORE sites, eConsult was preferred over curbsides among PCPs, especially those without an extensive social network at an institution. eConsult is also favored because, like the VA and Mayo clinic models, the eConsult is embedded in the EHR and the consultant has direct access to the chart for review and documentation. While eConsult has been reported to reduce face-to-face visits by up to 25–50%,4, 5, 13 there may be a compensatory increase in electronic consultation for simple clinical questions that previously may have been answered through a more informal method, such as an email or phone call with a colleague. In deciding to send an eConsult, PCPs considered patient factors, clinical complexity, and sought specialist input (Table 3).

Implementation required sites to assess for unintended consequences and issues with the program, and to perform quality assurance. Many PCPs were hesitant to use the program, due to misunderstandings of how it worked or lack of knowledge about the program. Some PCPs also believed—contrary to the purpose of the program—that eConsult could be used to expedite scheduling of a face-to-face visit with the specialist. To reduce this and other unintended uses of eConsult, we recommend aligning program goals (i.e., improved access and care coordination) with local priorities and circumstances, and providing tailored education and training via conferences, emails, and continual outreach. At many CORE sites, informal co-management conferences provide an opportunity for PCPs and specialist to meet face-to-face in order to discuss the management of commonly seen conditions. Although not the explicit focus of these meetings, appropriate uses of the eConsult service are often discussed. In addition, periodic newsletters with exemplary cases and updates on the program have proven a popular method to promote eConsult uptake, enhance PCP learning, and foster PCP-specialist collaboration at clinic sites.

This study has several limitations. Our interviews were conducted with PCPs at AMCs and may not be representative of PCPs at other clinical sites, since many PCPs at AMCs are part-time clinicians who also engage in education and/or research. We tried to account for this by interviewing PCPs at AMC-affiliated clinics that differed with regard to provider mix, patient population, and clinical experience. There may be something about the academic environment that may have contributed to program success. It is possible that having primary care leads serve as focus group facilitators may have influenced participants’ responses. However, focus group and interview findings were concordant, suggesting minimal if any influence on responses. Funding for recording and transcription of interviews and focus groups was unavailable during CORE-1 and CORE-2. However, detailed notetaking resulted in rich data including verbatim quotes. Despite these limitations, this study shows important lessons for successful eConsult implementation.

CONCLUSIONS

Our findings demonstrate that for successful implementation of an eConsult program, key program features should include PCP control and choice in consultation practices, engaged and education-oriented specialists, alignment of program goals with those of providers, and mitigation of unintended consequences through sustained quality assurance efforts.