Review of Statistical Issues in Pragmatic Clinical Trials in Current Drug Development Environment

  • Dingfeng JiangEmail author
  • Kun Chen
  • Saurabh Mukhopadhyay
  • Nareen Katta
  • Lanju Zhang
Part of the ICSA Book Series in Statistics book series (ICSABSS)


Facing increasing resource pressure, stakeholders in healthcare are seeking real-world evidence (RWE) of effectiveness and safety for informed decision-making. Increasing attention is being drawn to pragmatic clinical trials (PCTs) for their emerging roles in generating high-quality RWE. Principles of PCTs challenge some well-established guidelines in randomized clinical trials (RCTs), as open-label and treatment switch in intention-to-treat (ITT) population being the most pronounced ones. This review discusses key methodological and statistical implications of PCTs in the context of drug development and reimbursement, with emphasis on study design and analyses to maximize external validity. Many of those aspects are very challenging and are expected to crystallize over time after more research is done.


Pragmatic clinical trial Real-world evidence Study design and analysis Open-label Treatment switch 


  1. Bakerly, N., Woodcock, A., New, J., Gibson, M., Wu, W., Leather, D., Vestbo, J.: The Salford lung study protocol, a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease. Respir. Res. 16, 101 (2015)CrossRefGoogle Scholar
  2. Berger, M., Daniel, G., Frank, K., Hernandex, A., McClellan, M., Okun, S., Overhage, M., Platt, R., Romine, M., Tunis, S., Wilson, M.: A framework for regulatory use of real world evidence. Duke-Margolis Center for Health Policy. (2017). Accessed 15 Aug 2018
  3. Beyer-Westendorf, J., Buller, H.: External and internal validity of open label or double-blind trials in oral anticoagulation, better, worse or just different? J. Thromb. Haemost. 9, 2153–2158 (2011)CrossRefGoogle Scholar
  4. Black, N.: Why we need observational studies to evaluate the effectiveness of health care. BMJ. 312, 1215–1218 (1996)CrossRefGoogle Scholar
  5. Branson, M., Whitehead, J.: Estimating a treatment effect in survival studies in which patients switch treatment. Stat. Med. 21, 2449–2463 (2002)CrossRefGoogle Scholar
  6. Bulpitt, C.: Randomized Controlled Clinical Trials, 2nd edn. Springer, New York (1996)CrossRefGoogle Scholar
  7. Califf, R.: Pragmatic clinical trials, emerging challenges and new roles for statisticians. Clini. Trials. 13(5), 471–477 (2016)CrossRefGoogle Scholar
  8. Califf, R., Sugarman, J.: Exploring the ethical and regulatory issues in pragmatic clinical trials. Clin. Trials. 12(5), 436–441 (2015)CrossRefGoogle Scholar
  9. Chalkidou, K., Tunis, D., Whicher, D., Fowler, R., Zwarenstein, M.: The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research. Clin. Trials. 9, 436–446 (2012)CrossRefGoogle Scholar
  10. Correspondence: Effectiveness of fluticasone furoate–vilanterol in COPD. N. Engl. J. Med. 375(26), 2605–2607 (2016)CrossRefGoogle Scholar
  11. Desgagné, A., Castilloux, A., Angers, J., LeLorier, J.: The use of the bootstrap statistical method for the pharmacoeconomic cost analysis of skewed data. PharmacoEconomics. 13, 487–497 (1998)CrossRefGoogle Scholar
  12. EMA.: Guidance for companies considering the adaptive pathways approach (2016)Google Scholar
  13. Fayers, P., Hand, D.: Generalisation from phase III clinical trials, survival, quality of life, and health economics. Lancet. 350, 1025–1027 (1997)CrossRefGoogle Scholar
  14. FDA.: Guidance for industry, an E9 statistical principle for clinical trials (1998)Google Scholar
  15. FDA.: PDUFA (Prescription Drug User Fee Act 2016) VI on enhancing use of real world evidence for use in regulatory decision making. (2016). Accessed 15 Aug 2018
  16. FDA.: Use of real-world evidence to support regulatory decision-making for medical devices guidance for industry and Food and Drug Administration Staff (2017)Google Scholar
  17. Grill, J., Karlawish, J.: Addressing the challenges to successful recruitment and retention in Alzheimer’s disease clinical trials. Alzheimers Res. Ther. 2, 34 (2010)CrossRefGoogle Scholar
  18. Guisasola, F.: Glycaemic control among patients with type 2 diabetes mellitus in seven European countries, findings from the Real-Life Effectiveness and Care Patterns of Diabetes Management (RECAP-DM) study. Diabetes. Obes. Metab. 10, 8–15 (2008)CrossRefGoogle Scholar
  19. Gupta, S.: Intention-to-treat concept: a review. Perspect. Clin. Res. 2(3), 109–112 (2011)CrossRefGoogle Scholar
  20. Hannan, E.: Randomized clinical trials and observational studies: Guidelines for assessing respective strengths and limitations. J. Am. Coll. Cardiol. Intv. 1, 211–217 (2008)CrossRefGoogle Scholar
  21. Hemming, K., Haines, Y., Chilton, P., Girling, A., Lilford, R.: The stepped wedge cluster randomised trial, rationale, design, analysis, and reporting. BMJ. 350, h391 (2015)CrossRefGoogle Scholar
  22. Hernan, M., Brumback, B., Robins, J.: Marginal structural models to estimate the joint causal effect of nonrandomized treatments. J. Am. Stat. Assoc. 96, 440–448 (2001)MathSciNetCrossRefGoogle Scholar
  23. Hoel, A.: Under-representation of women and ethnic minorities in vascular surgery randomized controlled trials. J. Vasc. Surg. 50, 349–354 (2009)CrossRefGoogle Scholar
  24. Hotopf, M.: The pragmatic randomised controlled trial. Adv. Psychiatr. Treat. 8, 326–333 (2002)CrossRefGoogle Scholar
  25. Hussey, M., Hughes, J.: Design and analysis of stepped wedge cluster randomized trials. Contemp. Clin. Trials. 28, 182–191 (2007)CrossRefGoogle Scholar
  26. Konrat, C.: Underrepresentation of elderly people in randomised controlled trials the example of trials of 4 widely prescribed drugs. PLoS One. 7(3), 33559 (2012)CrossRefGoogle Scholar
  27. Latimer, N., Abrams, K., Lambert, P.: Adjusting for treatment switching in randomised controlled trials - a simulation study and a simplified two-stage method. Stat. Methods Med. Res. 26, 724–751 (2014)MathSciNetCrossRefGoogle Scholar
  28. Latimer, N., Abrams, K., Lambert, P., Morden, J., Crowther, M.: Assessing methods for dealing with treatment switching in clinical trials: a follow-up simulation study. Stat. Methods Med. Res. 27(3), 765–784 (2018)MathSciNetCrossRefGoogle Scholar
  29. Loudon, K., Treweek, S., Sullivan, F., Donnan, P., Thorpe, K., Zwarenstein, M.: The PRECIS-2 tool, designing trials that are fit for purpose. BMJ. 350, h2147 (2015)CrossRefGoogle Scholar
  30. Mark, S., Robins, J.: A method for the analysis of randomized trials with compliance information - an application to the multiple risk factor intervention trial. Control. Clin. Trials. 14, 79–97 (1993)CrossRefGoogle Scholar
  31. Mosca, L., Barrett-Connor, E., Wenger, N.: Sex/gender differences in cardiovascular disease prevention what a difference a decade makes. Circulation. 124, 2145–2154 (2011)CrossRefGoogle Scholar
  32. Mullins, D., Whicher, D., Reese, E., Tunis, S.: Generating evidence for comparative effectiveness research using more pragmatic randomized controlled trials. PharmacoEconomics. 28(10), 969–976 (2010)CrossRefGoogle Scholar
  33. New, J., Bakerly, N., Leather, D., Woodcock, A.: Obtaining real-world evidence, the Salford lung study. Thorax. 69, 1152–1154 (2014)CrossRefGoogle Scholar
  34. Patsopoulos, N.: A pragmatic view on pragmatic trials. Dialogues Clin. Neurosci. 13, 2 (2011)Google Scholar
  35. Reynolds, R., Lem, J., Gatto, N., Eng, S.: Is the large simple trial design used for comparative, post-approval safety research? Drug Saf. 34(10), 799–820 (2011)CrossRefGoogle Scholar
  36. Robins, J., Tsiatis, A.: Correcting for noncompliance in randomized trials using rank preserving structural failure time models. Commun. Stat. - Theory Methods. 20, 2609–2631 (1991)MathSciNetCrossRefGoogle Scholar
  37. Saad, E., Katz, A., Hoff, P., Buyse, M.: Progression-free survival as surrogate and as true end point, insights from the breast and colorectal cancer literature. Ann. Oncol. 21, 7–12 (2010)CrossRefGoogle Scholar
  38. Schulz, K., Altman, D., Moher, D., CONSORT Group: CONSORT 2010 statement, updated guidelines for reporting parallel group randomised trials. BMC Med. 8, 18 (2010)CrossRefGoogle Scholar
  39. Schulz, K., Grimes, D.: Blinding in randomised trials, hiding who got what. Lancet. 359, 696–700 (2002)CrossRefGoogle Scholar
  40. Schwartz, D., Lellouch, J.: Explanatory and pragmatic attitudes in therapeutical trials. J. Chronic Dis. 20, 637–648 (1967)CrossRefGoogle Scholar
  41. The Digitalis Investigation Group: Rationale, design, implementation and baseline characteristics of patients in the dig trial, a large simple long-term trial to evaluate the effect of digitalis on mortality in heart failure. Control. Clin. Trials. 17, 77–97 (1996)CrossRefGoogle Scholar
  42. The DREAM Trial Investigators: Rationale, design and recruitment characteristics of a large, simple international trial of diabetes prevention, the DREAM trial. Diabetologia. 47, 1519–1527 (2004)CrossRefGoogle Scholar
  43. Thompson, S., Barber, J.: How should cost data in pragmatic randomised trials be analysed? BMJ. 320, 1197–1200 (2000)CrossRefGoogle Scholar
  44. Thorpe, K., Zwarenstein, M., Oxman, A., Treweek, S., Furberg, C., Altman, D., Tunis, S., Bergel, E., Harvey, I., Magid, D., Chalkidou, K.: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J. Clin. Epidemiol. 180, E47–E57 (2009)Google Scholar
  45. Treweek, S., Zwarenstein, M.: Making trials matter, pragmatic and explanatory trials and the problem of applicability. Trials. 10, 37 (2009)CrossRefGoogle Scholar
  46. Tunis, S., Stryer, D., Clancy, C.: Practical clinical trials, increasing the value of clinical research for decision making in clinical and health policy. JAMA. 290, 1624–1632 (2003)CrossRefGoogle Scholar
  47. Vestbo, J., Leather, D., Bakerly, N., New, J., Gibson, M., McCorkindale, S., Collier, S., Crawford, J., Frith, L., Harvey, C., Svedsater, H., Woodcock, A., Salford Lung Study Investigators: Effectiveness of fluticasone furoate-vilanterol for COPD in clinical practice. N. Engl. J. Med. 375, 1253–1260 (2016)CrossRefGoogle Scholar
  48. Ware, J., Hamel, M.: Pragmatic trials - Guides to better patient care? N. Engl. J. Med. 364(18), 1685 (2011)CrossRefGoogle Scholar
  49. Williams, H., Burden-The, E., Nunn, A.: What is a pragmatic clinical trial? J. Investig. Dermatol. 135, e33 (2015)Google Scholar
  50. Wong, S., North, S., Sweeney, C., Stockler, M., Sridhar, S.: Screen failure rates in contemporary phase II/III therapeutic trials in genitourinary malignancies. J. Clin. Oncol. 34(Suppl 2S), 176 (2016)CrossRefGoogle Scholar
  51. Zwarenstein, M., Treweek, S., Gagnier, J., Altman, D., Tunis, S., Haynes, B., Oxman, A., Moher, D., CONSORT and Pragmatic Trials in Healthcare (Practihc) Groups: Improving the reporting of pragmatic trials, an extension of the CONSORT statement. BMJ. 337, a2390 (2008)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Dingfeng Jiang
    • 1
    Email author
  • Kun Chen
    • 1
  • Saurabh Mukhopadhyay
    • 1
  • Nareen Katta
    • 1
  • Lanju Zhang
    • 2
  1. 1.AbbVie Inc.North ChicagoUSA
  2. 2.Data and Statistical Sciences, AbbVie Inc.North ChicagoUSA

Personalised recommendations