Impact of Project ECHO Models of Medical Tele-Education: a Systematic Review
Extension for Community Healthcare Outcomes (ECHO) and related models of medical tele-education are rapidly expanding; however, their effectiveness remains unclear. This systematic review examines the effectiveness of ECHO and ECHO-like medical tele-education models of healthcare delivery in terms of improved provider- and patient-related outcomes.
We searched English-language studies in PubMed, Embase, and PsycINFO databases from 1 January 2007 to 1 December 2018 as well as bibliography review. Two reviewers independently screened citations for peer-reviewed publications reporting provider- and/or patient-related outcomes of technology-enabled collaborative learning models that satisfied six criteria of the ECHO framework. Reviewers then independently abstracted data, assessed study quality, and rated strength of evidence (SOE) based on Cochrane GRADE criteria.
Data from 52 peer-reviewed articles were included. Forty-three reported provider-related outcomes; 15 reported patient-related outcomes. Studies on provider-related outcomes suggested favorable results across three domains: satisfaction, increased knowledge, and increased clinical confidence. However, SOE was low, relying primarily on self-reports and surveys with low response rates. One randomized trial has been conducted. For patient-related outcomes, 11 of 15 studies incorporated a comparison group; none involved randomization. Four studies reported care outcomes, while 11 reported changes in care processes. Evidence suggested effectiveness at improving outcomes for patients with hepatitis C, chronic pain, dementia, and type 2 diabetes. Evidence is generally low-quality, retrospective, non-experimental, and subject to social desirability bias and low survey response rates.
The number of studies examining ECHO and ECHO-like models of medical tele-education has been modest compared with the scope and scale of implementation throughout the USA and internationally. Given the potential of ECHO to broaden access to healthcare in rural, remote, and underserved communities, more studies are needed to evaluate effectiveness. This need for evidence follows similar patterns to other service delivery models in the literature.
The authors thank the funders of this project at ASPE. We thank Caryn Marks, Nancy De Lew, and Rose Chu for their support. Among our colleagues at the RAND Corporation, we are grateful to Christine Eibner, Jody Larkin, Justin Timbie, Lisa Turner, Lori Uscher-Pines, Monique Martineau, Paul Koegel, and Tricia Soto.
This work was supported by the Office of the Assistant Secretary for Planning and Evaluation, US Department of Health and Human Services, under master contract, Building Analytic Capacity for Monitoring and Evaluating the Implementation of the ACA, HHSP23320095649WC.
Compliance with Ethical Standards
Conflict of Interest
Drs. McBain, Rose, Faherty, and Fischer, as well as Ms. Sousa and Ms. Baxi, report support from ASPE through a contract with the RAND Corporation during the conduct of the study. All remaining authors declare that they do not have a conflict of interest.
The authors of this manuscript are responsible for its content. Statements in the manuscript should not be construed as endorsements of ASPE or HHS.
- 8.Kinfu Y, Dal Poz MR, Mercer H, Evans DB. The health worker shortage in Africa: Are enough physicians and nurses being trained? : SciELO Public Health; 2009.Google Scholar
- 9.ECHO Act, Pub L No. 114–270, 130 Stat 1395 (2016).Google Scholar
- 10.University of New Mexico. Project ECHO. Available at: https://echo.unm.edu/. Accessed June 16, 2019.
- 11.Fischer SH, Rose AJ, McBain RK, Faherty LJ, Sousa J, Martineau M. Evaluation of technology-enabled collaborative learning and capacity building models: Materials for a report to Congress. Santa Monica, CA: RAND Corporation, 2019. Available at: https://www.rand.org/pubs/research_reports/RR2934.html.
- 12.Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The impact of Project ECHO on participant and patient outcomes: A systematic review. Academic medicine : journal of the Association of American Medical Colleges 2016;91(10):1439–61. doi: https://doi.org/10.1097/acm.0000000000001328 CrossRefGoogle Scholar
- 13.Ryan R, Hill S. How to GRADE the quality of the evidence. Cochrane Consumers and Communication Group. 2016. Available at: http://cccrg.cochrane.org/author-resources. Version 3.0. Accessed June 16, 2019.
- 18.Beste LA, Glorioso TJ, Ho PM, Au DH, Kirsh SR, Todd-Stenberg J, et al. Telemedicine specialty support promotes hepatitis C treatment by primary care providers in the Department of Veterans Affairs. Am J Med. 2017;130(4):432–8.e3. doi: https://doi.org/10.1016/j.amjmed.2016.11.019 CrossRefGoogle Scholar
- 19.Beste LA, Mattox EA, Pichler R, Young BA, Au DH, Kirsh SF, et al. Primary care team members report greater individual benefits from long- versus short-term specialty telemedicine mentorship. Telemedicine journal and e-health : the official journal of the American Telemedicine Association 2016;22(8):699–706. doi: https://doi.org/10.1089/tmj.2015.0185 CrossRefGoogle Scholar
- 22.Chaple MJ, Freese TE, Rutkowski BA, Krom L, Kurtz AS, Peck JA, Warren P, Garrett S. Using ECHO Clinics to Promote Capacity Building in Clinical Supervision. American Journal of Preventive Medicine, 54:S275-S280. 2018.Google Scholar
- 23.Cofta-Woerpel L, Lam C, Reitzel LR, Wilson W, Karam-Hage M, Beneventi D, et al. A tele-mentoring tobacco cessation case consultation and education model for healthcare providers in community mental health centers. Cogent Med. 2018;5(1). doi: https://doi.org/10.1080/2331205X.2018.1430652
- 24.Cordasco KM, Zuchowski JL, Hamilton AB, Kirsh S, Veet L, Saavedra JO, et al. Early lessons learned in implementing a women’s health educational and virtual consultation program in VA. Med Care 2015;53(4 Suppl 1):S88–92. doi: https://doi.org/10.1097/mlr.0000000000000313 CrossRefPubMedPubMedCentralGoogle Scholar
- 26.Eaton LH, Godfrey DS, Langford DJ, Rue T, Tauben DJ, Doorenbos AZ. Telementoring for improving primary care provider knowledge and competence in managing chronic pain: A randomised controlled trial. J Telemed Telecare. 2018:1357633x18802978. doi: https://doi.org/10.1177/1357633x18802978
- 27.Farris G, Sircar M, Bortinger J, Moore A, Krupp JE, Marshall J, et al. Extension for Community Healthcare Outcomes-Care Transitions: Enhancing geriatric care transitions through a multidisciplinary videoconference. J Am Geriatr Soc 2017;65(3):598–602. doi: https://doi.org/10.1111/jgs.14690 CrossRefPubMedGoogle Scholar
- 31.Jansen BD, Brazil K, Passmore P, Buchanan H, Maxwell D, McIlfatrick SJ, Morgan SM, Watson M, Parsons C. Evaluation of the impact of telementoring using ECHO© technology on healthcare professionals’ knowledge and self-efficacy in assessing and managing pain for people with advanced dementia nearing the end of life. BMC Helath Services Research, 18:228. 2018.Google Scholar
- 32.Johnson KL, Hertz D, Stobbe G, Alschuler K, Kalb R, Alexander KS, et al. Project Extension for Community Healthcare Outcomes (ECHO) in multiple sclerosis: Increasing clinician capacity. Int J MS Care 2017;19(6):283–9. doi: https://doi.org/10.7224/1537-2073.2016-099 CrossRefPubMedPubMedCentralGoogle Scholar
- 34.Kauth MR, Shipherd JC, Lindsay JA, Kirsh S, Knapp H, Matza L. Teleconsultation and training of VHA providers on transgender care: Implementation of a multisite hub system. Telemedicine journal and e-health : the official journal of the American Telemedicine Association. 2015;21(12):1012–8. doi: https://doi.org/10.1089/tmj.2015.0010 CrossRefGoogle Scholar
- 36.Komaromy M, Ceballos V, Zurawski A, Bodenheimer T, Thom DH, Arora S. Extension for Community Healthcare Outcomes (ECHO): A new model for community health worker training and support. J Public Health Policy 2018;39(2):203–16. doi: https://doi.org/10.1057/s41271-017-0114-8 CrossRefPubMedGoogle Scholar
- 40.Mazurek MO, Brown R, Curran A, Sohl K. ECHO Autism. Clinical Pediatrics, 56: 247-256. 2017.Google Scholar
- 41.Mehrotra K, Chand P, Bandawar M, Rao Sagi M, Kaur S, G A, et al. Effectiveness of NIMHANS ECHO blended tele-mentoring model on Integrated Mental Health and Addiction for counsellors in rural and underserved districts of Chhattisgarh, India. Asian J Psychiatr. 2018;36:123–7. doi: https://doi.org/10.1016/j.ajp.2018.07.010 CrossRefPubMedGoogle Scholar
- 42.Meins AR, Doorenbos AZ, Eaton L, Gordon D, Theodore B, Tauben D. TelePain: A community of practice for pain management. J Pain Relief. 2015;4(2). doi: https://doi.org/10.4172/2167-0846.1000177
- 45.Parsons C, Mattox EA, Beste LA, Au DH, Young BA, Chang MF, Palen BN. Development of a Sleep Telementorship Program for Rural Department of Veterans Affairs Primary Care Providers: Sleep Veterans Affairs Extension for Community Healthcare Outcomes. Annals of American Thoracic Society, 14: 267-274.Google Scholar
- 49.Salgia RK, Mullan PB, McCurdy H, Sales A, Moseley RH, Su GL. The educational impact of the Specialty Care Access Network-Extension of Community Healthcare Outcomes program. Telemedicine Journal and E-Health, 20: 1004-8.Google Scholar
- 51.Sockalingam S, Arena A, Serhal E, Mohri L, Alloo J, Crawford A. Building Provincial Mental Health Capacity in Primary Care: An Evaluation of a Project ECHO Mental Health Program. Academic psychiatry : the journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry 2018;42(4):451–7. doi: https://doi.org/10.1007/s40596-017-0735-z CrossRefGoogle Scholar
- 55.White C, McIlfatrick S, Dunwoody L, Watson M. Supporting and improving community health services--a prospective evaluation of ECHO technology in community palliative care nursing teams. BMJ Supportive & Palliative Care. 9: 202-208.Google Scholar
- 57.Wood BR, Mann MS, Martinez-Paz N, Unruh KT, Annese M, Spach DH, Scott JD, Stekler JD. Project ECHO: telementoring to educate and support prescribing of HIV pre-exposure prophylaxis by community medical providers. Sexual Health, 15: 601-605.Google Scholar
- 60.Gordon SE, Dufour AB, Monti SM, Mattison ML, Catic AG, Thomas CP, et al. Impact of a videoconference educational intervention on physical restraint and antipsychotic use in nursing homes: Results from the ECHO-AGE pilot study. J Am Med Dir Assoc 2016;17(6):553–6. doi: https://doi.org/10.1016/j.jamda.2016.03.002 CrossRefPubMedPubMedCentralGoogle Scholar
- 61.Katzman JG, Qualls CR, Satterfield WA, Kistin M, Hofmann K, Greenberg N, et al. Army and Navy ECHO Pain telementoring improves clinician opioid prescribing for military patients: An observational cohort study. J Gen Intern Med 2018. doi: https://doi.org/10.1007/s11606-018-4710-5 CrossRefGoogle Scholar
- 63.Moore AB, Krupp JE, Dufour AB, Sircar M, Travison TG, Abrams A, et al. Improving transitions to postacute care for elderly patients using a novel video-conferencing program: ECHO-Care Transitions. Am J Med 2017;130(10):1199–204. doi: https://doi.org/10.1016/j.amjmed.2017.04.041 CrossRefPubMedGoogle Scholar
- 65.Watts SA, Roush L, Julius M, Sood A. Improved glycemic control in veterans with poorly controlled diabetes mellitus using a Specialty Care Access Network-Extension for Community Healthcare Outcomes model at primary care clinics. J Telemed Telecare 2016;22(4):221–4. doi: https://doi.org/10.1177/1357633x15598052 CrossRefPubMedGoogle Scholar
- 67.Metz A, Louison L, Ward C, Burke K. Global Implementation Specialist Practice Profile: Skills and competencies for implementation practitioners. Chapel Hill, N.C.: National Implementation Research Network and Centre for Effective Services, 2017. Available at: https://fpg.unc.edu/node/9178.
- 68.Agency for Healthcare Research and Quality. Types of health care quality measures. 2011. Available at: http://www.ahrq.gov/talkingquality/measures/types.html. Accessed June 16, 2019.
- 69.U.S. Department of Health & Human Services. Office of the Assistant Secretary for Planning and Evaluation. Available at: https://aspe.hhs.gov/. Accessed June 16, 2019.
- 71.Ricketts EJ, Goetz AR, Capriotti MR, Bauer CC, Brei NG, Himle MB, et al. A randomized waitlist-controlled pilot trial of voice over Internet protocol-delivered behavior therapy for youth with chronic tic disorders. J Telemed Telecare 2016;22(3):153–62. doi: https://doi.org/10.1177/1357633x15593192 CrossRefPubMedGoogle Scholar
- 72.Baxter AJ, Tweed EJ, Katikireddi SV, Thomson H. Effects of Housing First approaches on health and well-being of adults who are homeless or at risk of homelessness: Systematic review and meta-analysis of randomised controlled trials. J Epidemiol Community Health. 2019:jech-2018-210981.Google Scholar