Application of Stochastic Control Theory to Optimal Design of Dosage Regimens

  • Alan Schumitzky


Designing a dosage regimen for a pharmacokinetic/pharmacodynamic system involves defining: i) a patient-dependent model, which includes structure, parameter, and measurement uncertainties; ii) the choice of controls, which can include dose amounts, dose times and /or sampling times; and iii) an appropriate performance index to evaluate achievement of a clinically chosen therapeutic goal. The control problem then is to choose the dosage regimen that optimizes the expected value of the performance index. This problem fits within the framework of stochastic control theory. Examples are given to illustrate the variety of this class of problems, including: optimal dose regimens for target level and target window cost; and optimal sampling schedules for maximal information. By varying the class of admissible controls, different strategies are generated. Control strategies to be discussed include: open loop, open loop feedback, separation principle, and iteration in policy space. Monte Carlo simulation studies of a terminal cost type problem are presented.


Open Loop Stochastic Control Admissible Control Dose Time Policy Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. H. Jazwinski. Stochastic Processes and Filtering Theory, Academic Press, New York, 1970.Google Scholar
  2. 2.
    A. Racine-Poon and A. F. M. Smith. Population models. In D. A. Berry and Marcel Dekker (eds.), Statistical Methodology in the Pharmaceutical Sciences, New York, 1990, pp. 139-162.Google Scholar
  3. 3.
    J. L. Steimer, A. Mallet, and F. Mentré. Estimating interindividual pharmacokinetic variability. In M. Rowland et al. (eds.), Variability in Drug Therapy: Description, Estimation, and Control, Raven Press, New York, 1985, pp. 65–111.Google Scholar
  4. 4.
    R. W. Jelliffe. A simulation study of factors affecting aminoglycoside therapeutic precision. In C. Cobelli and L. Mariani (eds.), Proc. First Symposium on Modeling and Control in Biomedical Systems, Venice, Italy, 1988, pp. 86-88.Google Scholar
  5. 5.
    R. W. Jelliffe, A. Schumitzky, and M. Van Guilder. A simulation study of factors affecting aminoglycoside therapeutic precision, Technical Report: 90-3, Laboratory of Applied Pharmacokinetics, USC School of Medicine, Los Angeles, 1990.Google Scholar
  6. 6.
    D. P. Bertsekas. Dynamic Programming: Deterministic and Stochastic Models, Prentice Hall, Englewood Cliffs, 1987.Google Scholar
  7. 7.
    R. Bellman. Adaptive Control Processes: A Guided Tour, Princeton University Press, Princeton, 1961.Google Scholar
  8. 8.
    D. S. Bayard. A forward method for optimal stochastic nonlinear and adaptive control. In Proc. 27th IEEE Conference on Decision and Control, Austin, 1988. To appear: IEEE T. Automat. Contr. Google Scholar
  9. 9.
    D. S. Bayard. Aspects of stochastic adaptive control synthesis. Doctoral Thesis, Electrical Engineering Department, State University of New York, Stony Brook, 1984.Google Scholar
  10. 10.
    Y. Bar-Shalom. Stochastic dynamic programming: caution and probing. IEEE T. Automat. Contr. AC-10:1184–1195 (1981).CrossRefGoogle Scholar
  11. 11.
    D. S. Bayard and M. Eslami. Implicit dual control for general stochastic systems. Opt. Cont. Appl. Meth. 6:265–279 (1985).CrossRefGoogle Scholar
  12. 12.
    W. F. Powers, P. H. Abbrecht, and D. G. Covel. Systems and microcomputer approach to anticoagulant therapy. IEEE T. Bio-med. Eng. 27:520–523 (1980).CrossRefGoogle Scholar
  13. 13.
    S. Amrani, E. Walter, Y. Lecourtier, and R. Gomeni. Robust control of uncertain pharmacokinetic models. Proc. IFAC 9th Triennial World Congress, Budapest, 3079-3083 (1984).Google Scholar
  14. 14.
    A. Schumitzky, M. Milman, P. Khademi, and R. Jelliffe. Approximate optimal closed loop control of pharmacokinetic systems. In Proc. IFAC Workshop on Decision Support for Patient Management: Measurement, Modeling, and Control, British Medical Informatics Society, London, 1989, pp. 338-347.Google Scholar
  15. 15.
    J. Gaillot, J-L. Steimer, A. Mallet, J. Thebault, and A. Beider. A prior lithium dosage regimen using population characteristics of pharmacokinetic parameters. J. Pharmacokin. Biopharm. 7:579–628 (1979).CrossRefGoogle Scholar
  16. 16.
    O. Richter and D. Reinhardt. Methods for evaluating optimal dosage regimens and their application to theophylline. Int. J. Clin. Pharm. Th. 20:564–575 (1982).Google Scholar
  17. 17.
    A. Mallet, F. Mentré, J. Giles, A. W. Kelman, A. H. Thomson, S. M. Bryson, and B. Whiting. Handling covariates in population pharmacokinetics with an application to gentamicin. Biomed. Meas. Infor. Contr. 2:138–146 (1988).Google Scholar
  18. 18.
    D. Katz and D. Z. D’Argenio. Stochastic control of pharmacokinetic systems: Open loop strategies. In C. Cobelli and L. Mariani (eds.), Proc. First Symposium on Modelling and Control in Biomedical Systems, Venice, 1988, pp. 560-566.Google Scholar
  19. 19.
    D. Z. D’Argenio and D. Katz. Implementation and evaluation of control strategies for individualizing dosage regimens with application to the aminoglycoside antibiotics. J. Pharmacokin. Biopharm. 14:523–37 (1986).CrossRefGoogle Scholar
  20. 20.
    L. B. Sheiner, B. Rosenberg, and K. L. Melmon. Modelling of individual pharmacokinetics for computer-aided drug dosage. Comput. Biomed. Res. 5:441–459 (1972).CrossRefGoogle Scholar
  21. 21.
    R. W. Jelliffe. Clinical applications of pharmacokinetics and control theory: planning, monitoring, and adjusting dosage regimens of aminoglycosides, lidocaine, digitoxin, and digoxin. In R. Maronde (ed.), Selected Topics in Clinical Pharmacology, Springer-Verlag, New York, 1986, pp. 26–82.CrossRefGoogle Scholar
  22. 22.
    R. W. Jelliffe, A. Schumitzky, and L. Hu, M. Liu. PC computer programs for Bayesian adaptive control of drug dosage regimens. Technical Report: 90-5, Laboratory of Applied Pharmacokinetics, USC School of Medicine, Los Angeles, 1990.Google Scholar
  23. 23.
    S. Vozeh and J. L. Steimer. Feedback control methods for drug dosage optimization. Clin. Pharmacokinet. 10:457–476 (1985).PubMedCrossRefGoogle Scholar
  24. 24.
    A. Schumitzky. Stochastic control of pharmacokinetic systems. In R. Maronde (ed.), Selected Topics in Clinical Pharmacology and Therapeutics, Springer-Verlag, New York, 1986, pp. 13–25.CrossRefGoogle Scholar
  25. 25.
    A. Schumitzky. Adaptive control in drug therapy. In H. Ducrot et al. (eds.) Computer Aid to Drug Therapy and to Drug Monitoring, Berne, Switzerland, March 6–10, 1978, North Holland, Amsterdam, 1978, pp. 357–360.Google Scholar
  26. 26.
    Y. Bar-Shalom and E. Tse. Concepts and methods in stochastic control. In C. Leondes (ed.), Control and Dynamic Systems, Vol. 12, Academic Press, New York, 1976, pp. 99–172.Google Scholar
  27. 27.
    D. Z. D’Argenio. Incorporating prior parameter uncertainty in the design of sampling schedules for pharmacokinetic parameter estimation experiments. Math. Biosci. 99:105–118 (1990).PubMedCrossRefGoogle Scholar
  28. 28.
    E. Walter and L. Pronzato. Robust experimental design via stochastic approximation. Math. Biosci. 75:103–120 (1985).CrossRefGoogle Scholar
  29. 29.
    D. Z. D’Argenio. Optimal sampling times for pharmacokinetic experiments. J. Pharmacokin. Biopharm. 9:739–56 (1981).CrossRefGoogle Scholar
  30. 30.
    L. Pronzato and E. Walter. Qualitative and quantitative experiment design for phenomenological models-A survey. Automatica 26:195–213 (1990).CrossRefGoogle Scholar
  31. 31.
    D. S. Bayard and A. Schumitzky. A stochastic control approach to optimal sampling schedule design. Technical Report: 90-1, Laboratory of Applied Pharmacokinetics, USC School of Medicine, Los Angeles, 1990.Google Scholar

Copyright information

© Springer Science+Business Media New York 1991

Authors and Affiliations

  • Alan Schumitzky
    • 1
  1. 1.Department of MathematicsUniversity of Southern CaliforniaUSA

Personalised recommendations