Lifetime Data Analysis

, Volume 15, Issue 1, pp 59–78 | Cite as

Validation of prognostic indices using the frailty model

  • C. Legrand
  • L. Duchateau
  • P. Janssen
  • V. Ducrocq
  • R. Sylvester


A major issue when proposing a new prognostic index is its generalisibility to daily clinical practice. Validation is therefore required. Most validation techniques assess whether “on average” the results obtained by the prognostic index in classifying patients in a new sample of patients are similar to the results obtained in the construction set. We introduce a new important aspect of the generalisibility of a prognostic index: the heterogeneity of the prognostic index risk group hazard ratios over different centers. If substantial variability between centers exists, the prognostic index may have no discriminatory capability in some of the centers. To model such heterogeneity, we use a frailty model including a random center effect and a random prognostic index by center interaction. Statistical inference is based on a Bayesian approach using a Laplacian approximation for the marginal posterior distribution of the variances of the random effects. We investigate different ways to summarize the information available from this marginal posterior distribution. Our approach is applied to a real bladder cancer database for which we demonstrate how to investigate and interpret heterogeneity in prognostic index effect over centers.


Prognostic index Validation Frailty model Multicenter clinical trial Bladder cancer 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Allard P, Bernard P, Fradet Y, Tetu B (1998) The early clinical course of primary Ta and T1 bladder cancer: a proposed prognostic index. Brit J Urol 81: 692–698Google Scholar
  2. Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19: 453–473CrossRefGoogle Scholar
  3. Blamey RW, Ellis IO, Pinder SE, Lee AH, Macmillan RD (2007) Survival of invasive breast cancer according to the Nottingham Prognostic Index in case diagnosed in 1990–1999. Eur J Cancer 43: 1548–1555CrossRefGoogle Scholar
  4. Blanchon F, Grivaux M, Asselain B et al (2006) 4-year mortality in patients with non-small-cell lung cancer: development and validation of a prognostic index. Lancet Oncol 7: 829–836CrossRefGoogle Scholar
  5. Bouffioux C, Denis L, Oosterlinck W, Viggiano G, Vergison B, Keuppens F, De Pauw M, Sylvester R, Cheuvart B (1992) Adjuvant chemotherapy of recurrent superficial transitional cell carcinoma: results of a European Organization for Research and Treatment of Cancer randomized trial comparing intravesical instillation of thiotepa, doxorubicin and cisplatin. J Urol 148: 297–301Google Scholar
  6. Bouffioux C, Kurth KH, Bono A, Oosterlinck W, Kruger CB, De Pauw M, Sylvester R (1995) Intravesical adjuvant chemotherapy for superficial transitional cell bladder carcinoma: results of two European Organization for Research and Treatment of Cancer randomized trials with mitomycin C and doxorubicin comparing early versus delayed instillations and short-term versus long-term treatment. European Organization for Research and Treatment of Cancer Genitourinary Group. J Urol 153(Supplement): 934–941Google Scholar
  7. Cox DR (1972) Regression models in life-tables (with discussion). J Roy Stat Soc Ser B 34: 187–220MATHGoogle Scholar
  8. Cramer H (1971) Mathematical methods of statistics. Princeton University Press, PrincetonGoogle Scholar
  9. Duchateau L, Janssen P (2005) Understanding heterogeneity in mixed, generalized mixed and frailty models. Am Stat 59: 143–146CrossRefMathSciNetGoogle Scholar
  10. Ducrocq V, Casella G (1996) A Bayesian analysis of mixed survival models. Genet Sel Evol 28: 509–529CrossRefGoogle Scholar
  11. Ducrocq V, Sölkner J (1994) The Survival Kit, a FORTRAN package for the analysis of survival data. In: 5th World congress on genetics applied to livestock production, vol 22. Department of Animal Poultry Science, University of Guelph, Guelph, Ontario, Canada, pp 51–52Google Scholar
  12. Ducrocq V, Sölkner J (1998) The Survival Kit-V3.0, a package for large analysis of survival data. In: 6th World congress on genetics applied to livestock production, vol 27. Animal Genetics and Breeding Unit, University of New England, Armidale, Australia, pp 447–448Google Scholar
  13. Glidden DV, Vittinghoff E (2004) Modelling clustered survival data from multicentre clinical trials. Stat Med 23: 369–388CrossRefGoogle Scholar
  14. Graf E, Schmoor C, Sauerbrei W, Schumacher M (1999) Assessment and comparison of prognostic classification schemes for survival data. Stat Med 18: 2529–2545CrossRefGoogle Scholar
  15. Harrell FE, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15: 361–387CrossRefGoogle Scholar
  16. Ibrahim JG, Chen MH, Sinha D (2001) Bayesian survival analysis. Springer-Verlag Inc, New-YorkMATHGoogle Scholar
  17. Justice AC, Covinsky KE, Berlin JA (1999) Assessing the generalisibility of prognostic information. Ann Internal Med 130: 515–524Google Scholar
  18. Kendall M, Stuart A (1977) The advanced theory of statistics, vol 1: Distribution theory, 4th edn. London & High Wycombe, Charles Griffin & Company LimitedGoogle Scholar
  19. Kurth KH, Schröder FH, Tunn U, Ay R, Pavone-Macaluso R, Debruyne F, De Pauw M, Dalesio O, Ten Kate F (1984) Adjuvant treatment of superficial transitional cell bladder carcinoma: preliminary results of an European Organization for Research and Treatment of Cancer randomized trial comparing doxorubicin hydrochloride, ethoglucid and transurethral resection alone. J Urol 132: 258–262Google Scholar
  20. Legrand C (2005) Assessing heterogeneity in multicenter clinical trials using the frailty model. PhD Thesis. Center for Statistics, Hasselt UniversityGoogle Scholar
  21. Legrand C, Ducrocq V, Janssen P, Sylvester R, Duchateau L (2005) A Bayesian approach to jointly estimate center and treatment by center heterogeneity in a proportional hazards model. Stat Med 24: 3789–3804CrossRefMathSciNetGoogle Scholar
  22. Matsuyama Y, Sakamoto J, Ohashi Y (1998) A Bayesian hierarchical survival model for the institutional effects in a multi-centre cancer clinical trial. Stat Med 17: 1893–1908CrossRefGoogle Scholar
  23. Nelder JA, Mead R (1965) A Simplex method for function minimization. Comput J 7: 308–313MATHGoogle Scholar
  24. Newling DW, Robinson MR, Smith PH, Byar D, Lockwood R, Stevens I, De Pauw M, Sylvester R (1995) Tryptophan metabolites, pyridoxine (vitamin B6) and their influence on the recurrence rate of superficial bladder cancer Results of a prospective randomized phase III study performed by the EORTC GU Group. EORTC Genitourinary Tract Cancer Cooperative Group. Eur Urol 27: 110–116Google Scholar
  25. Oosterlinck W, Kurth KH, Schröder FH, Butlinck J, Hammond B, Sylvester R (1993) A prospective European Organization for Research and Treatment of Cancer Genitourinary Group randomized trial comparing transurethral resection followed by a single intravesical instillation of epirubicin or water in single stage TaT1 papillary carcinoma of the bladder. J Urol 149: 749–752Google Scholar
  26. Putter H, Fiocco M, Geskus RB (2007) Tutorial in biostatistics: competing risks and multi-state models. Stat Med 26: 2389–2430CrossRefMathSciNetGoogle Scholar
  27. Ripatti S, Larsen K, Palmgren J (2002) Maximum likelihood inference for multivariate frailty models using an automated Monte Carlo EM algorithm. Lifetime Data Anal 8: 349–360MATHCrossRefMathSciNetGoogle Scholar
  28. Royston P, Parmar MKB, Sylvester R (2004) Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer. Stat Med 23: 907–926CrossRefGoogle Scholar
  29. Schemper M (2003) Predictive accuracy and explained variation. Stat Med 22: 2299–2308CrossRefGoogle Scholar
  30. Schemper M, Henderson R (2000) Predictive accuracy and explained variation in Cox regression. Biometrics 56: 249–255MATHCrossRefGoogle Scholar
  31. Schemper M, Stare J (1996) Explained variation in survival analysis. Stat Med 15: 1999–2012CrossRefGoogle Scholar
  32. Simon R, Altman DG (1994) Statistical aspects of prognostic factor studies in oncology. Brit J Cancer 69: 979–985Google Scholar
  33. Sinha D, Ibrahim JG, Chen MH (2003) A Bayesian justification of Cox’s partial likelihood. Biometrika 90: 629–641CrossRefMathSciNetGoogle Scholar
  34. Smith A, Skene AM, Shaw J, Naylor J, Dransfield M (1985) The implementation of the Bayesian paradigm. Commun Stat Theory Meth 14: 1079–1102MATHCrossRefMathSciNetGoogle Scholar
  35. Smith CT, Williamson PR, Marson AG (2005) Investigating heterogeneity in an individual patient data meta-analysis of time to event outcomes. Stat Med 24(9): 1307–1319CrossRefMathSciNetGoogle Scholar
  36. Solal-Celigny P, Roy P, Colombat P et al (2004) Follicular lymphoma international prognostic index. Blood 104: 1258–1265CrossRefGoogle Scholar
  37. Spiegelhalter DJ, Abrams K, Myles JP (2004) Bayesian approaches to clinical trials and health care evaluation. Wiley, ChichesterGoogle Scholar
  38. Sylvester RJ, Van der Meijden APM, Oosterlinck W, Witjes JA, Bouffioux C, Denis L, Newling DWW, Kurth K (2006) Predicting recurrence and progression in individual patients with stage TaT1 bladder cancer using EORTC risk tables: a combined analysis of 2596 patients from seven EORT trials. Eur Urol 49: 466–477CrossRefGoogle Scholar
  39. Tierney L, Kadane JB (1986) Accurate approximations for posterior moments and marginal densities. J Am Stat Assoc 81(391): 82–86MATHCrossRefMathSciNetGoogle Scholar
  40. Vaida F, Xu R (2000) Proportional hazards model with random effects. Stat Med 19: 3309–3324CrossRefGoogle Scholar
  41. Van Houwelingen HC (2000) Validation, calibration, revision and combination of prognostic survival models. Stat Med 19: 3401–3415CrossRefGoogle Scholar
  42. Witjes JA, Van der Meijden AP, Sylvester RJ, Debruyne FM, Van Aubel A, Witjes WP (1998) Long term follow up of an EORTC randomized prospecive trial comparing intravesical bacille Calmette-Guérin-RIVM and Mitomycin C in superficial bladder cancer. Urology 52: 403–410CrossRefGoogle Scholar
  43. Yamaguchi T, Ohashi Y (1999) Investigating centre effects in a multi-centre clinical trial of superficial bladder cancer. Stat Med 18: 1961–1971CrossRefGoogle Scholar
  44. Yamaguchi T, Ohashi Y, Matsuyama Y (2002) Proportional hazards models with random effects to examine centre effects in multicentre cancer clinical trials. Stat Methods Med Res 11: 221–236MATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • C. Legrand
    • 1
    • 2
  • L. Duchateau
    • 3
  • P. Janssen
    • 4
  • V. Ducrocq
    • 5
  • R. Sylvester
    • 1
  1. 1.European Organisation for Research and Treatment of CancerBrusselsBelgium
  2. 2.Université catholique de LouvainLouvain-la-NeuveBelgium
  3. 3.Department of Physiology and Biometrics, Faculty of Veterinary MedicineGhent UniversityMerelbekeBelgium
  4. 4.Center for StatisticsHasselt UniversityDiepenbeekBelgium
  5. 5.Institut National de la Recherche AgronomiqueJouy-en-JosasFrance

Personalised recommendations