A Paradigmatic Example

Abstract

The theoretical foundations for this class of models have been summarized in Easley and Kiefer (1988). A discrete time decision problem is considered where the decisionmaker chooses an action r in each period to maximize total expected discounted reward depending on the action chosen and the outcome, a random variable. The conditional distribution f(.|r, ß) of the outcome given the action depends on an initially unknown parameter ß. The decisionmaker begins with a prior belief about the unknown parameter and at the end of each period updates it via Bayes’ rule utilizing the latest observations on the action taken and the outcome. Easley and Kiefer take the additional simplifying step of integrating out the outcome and redefining the maximand to be the total expected discounted mean reward where the mean is calculated with respect to the conditional distribution f(.|r,ß) and the belief distribution.