Strategies for Building Delivery Science in an Integrated Health Care System

  • Tracy A. LieuEmail author
  • Philip R. Madvig


Health systems today have increasing opportunities and imperatives to conduct delivery science, which is applied research that evaluates clinical or organizational practices that systems can implement or encourage. Examples include research on eliminating racial/ethnic disparities in hypertension management and on identifying the types of patients who can successfully use video visits. Clinical leaders and researchers often face barriers to delivery science, including limited funding, insufficient leadership support, lack of engagement between operational and research leaders, limited pools of research expertise, and lack of pathways to identify and develop ideas. We describe five key strategies we employed to address these barriers and develop a portfolio of delivery science programs in Kaiser Permanente Northern California. This portfolio now includes small and medium-sized grant programs, training programs for postdoctoral research fellows and experienced physician researchers, and a dedicated team that partners with clinicians to develop high-priority ideas and conduct small projects. Most of our approaches are consistent with frameworks used to develop delivery science by other health systems; some are innovative. Most of these strategies are adaptable by other health systems prepared to make long-range organizational commitments to mechanisms that foster partnerships between clinical leaders and researchers.


implementation research health care delivery program evaluation medical informatics physician decision support 



We are grateful to the executive leaders and physicians of The Permanente Medical Group (TPMG), including Robert Pearl, MD (Executive Director and CEO, 1999-2017), Richard Isaacs, MD (current Executive Director and CEO), and Yi-Fen Irene Chen, MD (current Associate Executive Director for quality and research) for their strong support and guidance of our delivery science portfolio. We acknowledge Richard Grant, MD, MPH, Julie Schmittdiel, PhD, Douglas Corley, MD, PhD (current Director of Delivery Science and Applied Research), Andy Avins, MD, MPH, and Lisa Herrinton, PhD, for their capable leadership of the programs described here. We thank Steven Schroeder, MD, Grace Lee, MD, MPH, Hal Sox, MD, and the anonymous reviewers for insightful suggestions on shaping this paper. This portfolio’s development has relied on the skillful contributions of our research scientists, project managers, and programmer/analysts, especially Beth Dameron, MS, Suzanne Furuya, MPH, MBA, and Stephanie Prausnitz, MPH.

Funding Information

This work was funded by TPMG, Kaiser Permanente Northern California.

Compliance with Ethical Standards

Conflict of Interest

The authors are employees of TPMG and have no other relationships or conflicts of interest to declare.


  1. 1.
    Whicher D. Accelerating Progress Toward Continuous Learning: A National Academy of Medicine Meeting Summary. Washington, DC: National Academy of Medicine; 2017. Accessed 5 Jan 2019.
  2. 2.
    Jena AB, Stevens W, McWilliams JM. Turning evidence into practice under payment reform: the new frontier of translational science. J Gen Intern Med. 2014;29(11):1542–1545.CrossRefGoogle Scholar
  3. 3.
    Lieu TA, Platt R. Applied Research and Development in Health Care - Time for a Frameshift. N Engl J Med. 2017;376(8):710–713.CrossRefGoogle Scholar
  4. 4.
    Rubenstein LV, Pugh J. Strategies for promoting organizational and practice change by advancing implementation research. J Gen Intern Med. 2006;21 Suppl 2:S58–64.CrossRefGoogle Scholar
  5. 5.
    Atkins D, Kilbourne AM, Shulkin D. Moving From Discovery to System-Wide Change: The Role of Research in a Learning Health Care System: Experience from Three Decades of Health Systems Research in the Veterans Health Administration. Annu Rev Public Health. 2017;38:467–487.CrossRefGoogle Scholar
  6. 6.
    Fisher ES, Shortell SM, Savitz LA. Implementation Science: A Potential Catalyst for Delivery System Reform. JAMA. 2016;315(4):339–340.CrossRefGoogle Scholar
  7. 7.
    Bindman AB, Pronovost PJ, Asch DA. Funding Innovation in a Learning Health Care System. JAMA. 2018;319(2):119–120.CrossRefGoogle Scholar
  8. 8.
    Kilbourne AM, Atkins D. Partner or perish: VA health services and the emerging bi-directional paradigm. J Gen Intern Med. 2014;29 Suppl 4:817–819.CrossRefGoogle Scholar
  9. 9.
    King G, Currie M, Smith L, Servais M, McDougall J. A framework of operating models for interdisciplinary research programs in clinical service organizations. Eval Program Plann. 2008;31(2):160–173.CrossRefGoogle Scholar
  10. 10.
    Greene SM, Reid RJ, Larson EB. Implementing the learning health system: from concept to action. Ann Intern Med. 2012;157(3):207–210.CrossRefGoogle Scholar
  11. 11.
    Kilbourne AM, Elwy AR, Sales AE, Atkins D. Accelerating Research Impact in a Learning Health Care System: VA’s Quality Enhancement Research Initiative in the Choice Act Era. Med Care. 2017;55(7 Suppl 1):S4–S12.Google Scholar
  12. 12.
    Selby JV, Slutsky JR. Practicing partnered research. J Gen Intern Med. 2014;29 Suppl 4:814–816.CrossRefGoogle Scholar
  13. 13.
    Division of Research, Kaiser Permanente Northern California. Delivery Science Portfolio. Accessed 8 Sept 2018.
  14. 14.
    Wu RR, Kinsinger LS, Provenzale D, et al. Implementation of new clinical programs in the VHA healthcare system: the importance of early collaboration between clinical leadership and research. J Gen Intern Med. 2014;29 Suppl 4:825–830.CrossRefGoogle Scholar
  15. 15.
    Schmittdiel JA, Dlott R, Young JD, Rothmann MB, Dyer W, Adams AS. The Delivery Science Rapid Analysis Program: A Research and Operational Partnership at Kaiser Permanente Northern California. Learn Health Syst. 2017;1(4).
  16. 16.
    Schmittdiel JA, Grant RW. Crossing the Research to Quality Chasm: A Checklist for Researchers and Clinical Leadership Partners. J Gen Intern Med. 2018;33(1):9–10.CrossRefGoogle Scholar
  17. 17.
    Suh-Burgmann EJ, Flanagan T, Osinski T, Alavi M, Herrinton L. Prospective Validation of a Standardized Ultrasonography-Based Ovarian Cancer Risk Assessment System. Obstet Gynecol. 2018;
  18. 18.
    Marcus JL, Hurley LB, Chamberland S, et al. No Difference in Effectiveness of 8 vs 12 Weeks of Ledipasvir and Sofosbuvir for Treatment of Hepatitis C in Black Patients. Clin Gastroenterol Hepatol. 2018;16(6):927–935.CrossRefGoogle Scholar
  19. 19.
    Hochman M, Asch SM. Disruptive Models in Primary Care: Caring for High-Needs, High-Cost Populations. J Gen Intern Med. 2017;32(4):392–397.CrossRefGoogle Scholar
  20. 20.
    James BC, Poulsen GP. The Case for Capitation. Harv Bus Rev. 2016;94(7-8):102–111, 134.Google Scholar
  21. 21.
    Gould MK. Annual Report 2017 of the Care Improvement Research Team. Pasadena, CA: Kaiser Permanente Southern California Department of Research and Evaluation; 2018.Google Scholar
  22. 22.
    National Institute for Health Research. NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRCs). Accessed 8 Sept 2018.
  23. 23.
    Penn Medicine. Center for Health Care Innovation. Accessed 8 Sept 2018.
  24. 24.
    Gonzales R, Moriates C, Lau C, et al. Caring Wisely: A Program to Support Frontline Clinicians and Staff in Improving Healthcare Delivery and Reducing Costs. J Hosp Med. 2017;12(8):662–667.CrossRefGoogle Scholar
  25. 25.
    Reed ME, Parikh R, Huang J, Ballard DW, Barr I, Wargon C. Real-Time Patient-Provider Video Telemedicine Integrated with Clinical Care. N Engl J Med. 2018;379(15):1478–1479.Google Scholar
  26. 26.
    Vinson DR, Mark DG, Chettipally UK, et al. Increasing Safe Outpatient Management of Emergency Department Patients With Pulmonary Embolism. Ann Intern Med 2018;169(12):855–865.

Copyright information

© Society of General Internal Medicine 2019

Authors and Affiliations

  1. 1.Division of ResearchKaiser Permanente Northern CaliforniaOaklandUSA
  2. 2.The Permanente Medical GroupOaklandUSA

Personalised recommendations