Evaluating Potential Subpopulations Using Stochastic SIDEScreen in a Cross-Over Trial

  • Ilya Lipkovich
  • Bohdana Ratitch
  • Bridget Martell
  • Herman Weiss
  • Alex DmitrienkoEmail author
Part of the ICSA Book Series in Statistics book series (ICSABSS)


In this paper we enhance existing SIDES and SIDEScreen methods for biomarker discovery (Lipkovich et al., Stat. Med. 30:2601–2621, 2011; Lipkovich and Dmitrienko, J. Biopharm. Statist. 24:130–153, 2014; Lipkovich et al. Stat. Biopharm. Res. 9:368–378, 2017b) and apply it to a small Phase 2 clinical trial in patients with recurrent dysmenorrhea. We argue that incorporating stochastic elements in computing the variable importance, expected treatment effect and replicability index is particularly useful when dealing with relatively small data sets, so as to properly account for the uncertainty of the subgroup selection process. To demonstrate improved operating characteristics of the Stochastic SIDEScreen compared with the corresponding deterministic procedure, we conducted a small simulation study that mimics data from our Phase 2 trial. As analytical formulas for power calculations are not available for machine learning methods of biomarker/subgroup discovery, simulations utilizing existing early phase data should be conducted routinely for obtaining realistic estimates of power.


Subgroup identification Exploratory analysis Biomarkers Recurrent dysmenorrhea Cross-over trials 



This work is dedicated to the memory of James (Chip) Hackett.


  1. Ashley, E.A.: The precision medicine initiative: a new national effort. JAMA. 313, 2119–2120 (2015)CrossRefGoogle Scholar
  2. Basile, J.: Blood pressure responder rates versus goal rates: which metric matters? Ther. Adv. Cardiovasc. Dis. 3, 157–174 (2009)CrossRefGoogle Scholar
  3. Breiman, L.: Bagging predictors. Mach. Learn. 24, 123–140 (1996)zbMATHGoogle Scholar
  4. Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)CrossRefGoogle Scholar
  5. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth, Belmont (1984)zbMATHGoogle Scholar
  6. Chen, G., Zhong, H., Belousov, A., Viswanath, D.: PRIM approach to predictive-signature development for patient stratification. Stat. Med. 34, 317–342 (2015)MathSciNetCrossRefGoogle Scholar
  7. Coco, A.S.: Primary dysmenorrhea. Am. Fam. Physician. 60, 489–496 (1999)Google Scholar
  8. Dawood, M.Y.: Primary dysmenorrhea: advances in pathogenesis and management. Obstet. Gynecol. 108, 428–441 (2006)CrossRefGoogle Scholar
  9. Efron, B.: Estimation and accuracy after model selection. J. of Am. Stat. Assoc. 109, 991–1007 (2014)MathSciNetCrossRefGoogle Scholar
  10. Foster, J.C., Taylor, J.M.C., Ruberg, S.J.: Subgroup identification from randomized clinical trial data. Stat Med. 30, 2867–2880 (2011)MathSciNetCrossRefGoogle Scholar
  11. Friedman, J.H., Fisher, N.I.: Bump hunting in high-dimensional data. Stat. Comput. 9, 123–143 (1999)CrossRefGoogle Scholar
  12. Huang, X., Sun, Y., Trow, P., Chatterjee, S., Chakravatty, A., Tian, L., Devanarayan, V.: Patient subgroup identification for clinical drug development. Stat. Med. 36, 1414–1428 (2017)MathSciNetCrossRefGoogle Scholar
  13. Kehl, V., Ulm, K.: Responder identification in clinical trials with censored data. Comput. Statist. Data Anal. 50, 1338–1355 (2006)MathSciNetCrossRefGoogle Scholar
  14. Lamont, A., Lyons, M.D., Jaki, T., Stuart, E., Feaster, D.J., Tharmaratnam, K., Oberski, D., Ishwaran, H., Wilson, D.K., Horn, M.L.W.: Identification of predicted individual treatment effects in randomized clinical trials. Stat. Methods Med. Res. 27, 142–157 (2016)MathSciNetCrossRefGoogle Scholar
  15. Lipkovich, I., Dmitrienko, A., Denne, J., Enas, G.: Subgroup identification based on differential effect search (SIDES): a recursive partitioning method for establishing response to treatment in patient subpopulations. Stat. Med. 30, 2601–2621 (2011)MathSciNetGoogle Scholar
  16. Lipkovich, I., Dmitrienko, A.: Strategies for identifying predictive biomarkers and subgroups with enhanced treatment effect in clinical trials using SIDES. J. Biopharm. Statist. 24, 130–153 (2014)MathSciNetCrossRefGoogle Scholar
  17. Lipkovich, I., Dmitrienko, A., D’Agostino, R.B.: Tutorial in biostatistics: data-driven subgroup identification and analysis in clinical trials. Stat. Med. 36, 136–196 (2017a)MathSciNetCrossRefGoogle Scholar
  18. Lipkovich, I., Dmitrienko, A., Patra, K., Ratitch, B., Pulkstenis, E.: Subgroup identification in clinical trials by stochastic SIDEScreen methods. Stat. Biopharm. Res. 9, 368–378 (2017b)CrossRefGoogle Scholar
  19. Loh, W.Y., He, X., Man, M.: A regression tree approach to identifying subgroups with differential treatment effects. Stat. Med. 34, 1818–1833 (2015)MathSciNetCrossRefGoogle Scholar
  20. Simon, R.M., Subramanian, J., Li, M.C., Menezes, S.: Using cross validation to evaluate the predictive accuracy of survival risk classifiers based on high dimensional data. Brief. Bioinform. 12, 2013–2214 (2011)CrossRefGoogle Scholar
  21. Shen, L., Ding, Y., Battioui, C.: A framework for statistical methods for identification of subgroups with differential treatment effect in randomized trials. In: Chen, Z., Liu, A., Qu, Y., Tang, L., Ting, N., Tsong, Y. (eds.) Applied Statistics in Biomedicine and Clinical Trials Design. Springer, New York (2015)Google Scholar
  22. Su, X., Tsai, C.L., Wang, H., Nickerson, D.M., Li, B.: Subgroup analysis via recursive partitioning. J. Mach. Learn. Res. 10, 141–158 (2009)Google Scholar
  23. Wager, S., Hastie, T., Efron, B.: Intervals for random forests: the jackknife and the infinitesimal jackknife. J. Mach. Learn. Res. 15, 1625–1651 (2014)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Ilya Lipkovich
    • 1
  • Bohdana Ratitch
    • 2
  • Bridget Martell
    • 3
  • Herman Weiss
    • 4
  • Alex Dmitrienko
    • 5
    Email author
  1. 1.Eli Lilly and CompanyIndianapolisUSA
  2. 2.Eli Lilly and CompanyMontrealCanada
  3. 3.Yale University School of MedicineNew HavenUSA
  4. 4.Juniper PharmaceuticalsBostonUSA
  5. 5.Mediana, IncOverland ParkUSA

Personalised recommendations