Skip to main content

Enhanced Method for Diagnosing Pharmacometric Models: Random Sampling from Conditional Distributions



For nonlinear mixed-effects pharmacometric models, diagnostic approaches often rely on individual parameters, also called empirical Bayes estimates (EBEs), estimated through maximizing conditional distributions. When individual data are sparse, the distribution of EBEs can “shrink” towards the same population value, and as a direct consequence, resulting diagnostics can be misleading.


Instead of maximizing each individual conditional distribution of individual parameters, we propose to randomly sample them in order to obtain values better spread out over the marginal distribution of individual parameters.


We evaluated, through diagnostic plots and statistical tests, hypothesis related to the distribution of the individual parameters and show that the proposed method leads to more reliable results than using the EBEs. In particular, diagnostic plots are more meaningful, the rate of type I error is correctly controlled and its power increases when the degree of misspecification increases. An application to the warfarin pharmacokinetic data confirms the interest of the approach for practical applications.


The proposed method should be implemented to complement EBEs-based approach for increasing the performance of model diagnosis.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5






Empirical Bayes estimates


Maximum a posteriori


Markov Chain Monte Carlo






Posterior predictive checks


Visual predictive checks


  1. Bonate PL. Pharmacokinetic-pharmacodynamic modeling and simulation. Springer. 2011.

  2. Lavielle M. Mixed effects models for the population approach: models, tasks, methods and tools. Chapman and Hall/CRC. 2014.

  3. Comets E, Brendel K, Mentré F. Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: the npde add-on package for R. Comput Methods Prog Biomed. 2008;90(2):154–66.

    Article  Google Scholar 

  4. Comets E, Brendel K. Model evaluation in nonlinear mixed effect models, with applications to pharmacokinetics. Journal de la Société Française de Statistique. 2010;151(1):106–28.

    Google Scholar 

  5. Karlsson M, Savic R. Diagnosing model diagnostics. Clin Pharmacol Therapeut. 2007;82(1):17–20.

    CAS  Article  Google Scholar 

  6. Lavielle M, Bleakley K. Automatic data binning for improved visual diagnosis of pharmacometric models. J Pharmacokinet Pharmacodyn. 2011;38(6):861–71.

    Article  PubMed  Google Scholar 

  7. Yano Y, Beal SL, Sheiner LB. Evaluating pharmacokinetic/pharmacodynamic models using the posterior predictive check. J Pharmacokinet Pharmacodyn. 2001;28(2):171–92.

    CAS  Article  PubMed  Google Scholar 

  8. Combes F, Retout S, Frey N, Mentré F. Powers of the likelihood ratio test and the correlation test using empirical bayes estimates for various shrinkages in population pharmacokinetics. CPT: Pharmacom Syst Pharmacol. 2014;3(4):1–9.

    Google Scholar 

  9. Savic R, Karlsson M. Importance of shrinkage in empirical Bayes estimates for diagnostics: problems and solutions. AAPS J. 2009;11(3):558–69.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Drikvandi R, Verbeke G, Khodadadi A, Nia VP. Testing multiple variance components in linear mixed-effects models. Biostatistics. 2013;14(1):144–59.

    Article  PubMed  Google Scholar 

  11. Li Z, Zhu L. A new test for random effects in linear mixed models with longitudinal data. J Stat Plan Infer. 2013;143(1):82–95.

    Article  Google Scholar 

  12. Mun J, Lindstrom MJ. Diagnostics for repeated measurements in linear mixed effects models. Stat Med. 2013;32(8):1361–75.

    Article  PubMed  Google Scholar 

  13. Alonso A, Litière S, Molenberghs G. A family of tests to detect misspecifications in the random-effects structure of generalized linear mixed models. Comput Stat Data Anal. 2008;52(9):4474–86.

    Article  Google Scholar 

  14. Huang X. Diagnosis of random-effect model misspecification in generalized linear mixed models for binary response. Biometrics. 2009;65(2):361–8.

    Article  PubMed  Google Scholar 

  15. Vonesh EF, Chinchilli VM, Pu K. Goodness-of-fit in generalized nonlinear mixed-effects models. Biometrics. 1996;52(2):572–87.

    CAS  Article  PubMed  Google Scholar 

  16. Claeskens G, Hart J. Goodness-of-fit tests in mixed models. Test. 2009;18(2):213–39.

    Article  Google Scholar 

  17. Ritz C. Goodness-of-fit tests for mixed models. Scand J Stat. 2004;31(3):443–58.

    Article  Google Scholar 

  18. Meintanis SG, Portnoy S. Specification tests in mixed effects models. J Stat Plan Infer. 2011;141(8):2545–55.

    Article  Google Scholar 

  19. Parke J. A procedure for generating bootstrap samples for the validation of nonlinear mixed-effects population models. Comput Methods Prog Biomed. 1999;59(1):19–29.

    CAS  Article  Google Scholar 

  20. Laffont CM, Concordet D. A new exact test for the evaluation of population pharmacokinetic and/or pharmacodynamic models using random projections. Pharm Res. 2011;28(8):1948–62.

    CAS  Article  PubMed  Google Scholar 

  21. Robert CP, Casella G. Monte Carlo statistical methods. Springer texts in statistics. 2004.

  22. Holford N. Clinical pharmacokinetics and pharmacodynamics of warfarin. understanding the dose-effect relationship. Clin Pharmacokinet. 1986;11(6):483–504.

    CAS  Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Benjamin Ribba.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplemental Figure S1
figure 6

Warfarin PK data. (GIF 30 kb)

Supplemental Figure S2
figure 7

Empirical distribution of the individual parameters. The estimated pdf’s are displayed in solid line. Top: EBEs, bottom: sampled from the conditional distributions. (GIF 40 kb)

Supplemental Figure S3
figure 8

Sampled individual parameters versus weight. The correlation coefficient and the p-value of the test r = 0 are displayed for each parameters. (GIF 37 kb)

Supplemental Figure S4
figure 9

Joint distributions of the sampled random effects. The correlation coefficient and the p-value of the test r = 0 are displayed for each pair of parameters. (GIF 53 kb)

High resolution image (EPS 161 kb)

High resolution image (EPS 94.4 kb)

High resolution image (EPS 358 kb)

High resolution image (EPS 505 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lavielle, M., Ribba, B. Enhanced Method for Diagnosing Pharmacometric Models: Random Sampling from Conditional Distributions. Pharm Res 33, 2979–2988 (2016).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • model diagnostics
  • modeling and simulation
  • pharmacokinetics and pharmacodynamics