Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Decision-makers are increasingly willing to consider the uncertainty associated with model predictions of the economic, environmental, or social impacts associated with possible decisions. Information on uncertainty does not make decision-making easier, but to ignore it is to ignore reality. Incorporating what is known about the uncertainty of input parameters and variables used in optimization and simulation models can help in quantifying the uncertainty in the resulting model output . This chapter outlines and illustrates some approaches for doing this.

7.1 Introduction

Water resource planners and managers work in an environment of change and uncertainty. Water supplies are always uncertain, if not in the short term at least in the long term. Water demands and the multiple purposes and objectives water serve tend to change over time, and these changes cannot always be predicted. Many of the parameters of models used to predict the multiple hydrologic, economic, environmental, ecological, and social impacts are also uncertain. Indeed, models used to predict these impacts are, at least in part, based on many uncertain assumptions. This uncertainty associated with planning and managing cannot be avoided (WWAP 2012).

To the extent that probabilities can be included where appropriate in models and their inputs at least some of the uncertainty of their outputs can be identified and quantified. These models are called probabilistic or stochastic models. Most probabilistic models provide a range of possible values for each output variable along with their probabilities. Stochastic models attempt to model the random processes that occur over time, and provide alternative time series of outputs along with their probabilities. In other cases sensitivity analyses (solving models under different assumptions) can be carried out to estimate the impact of any uncertainty on the decisions being considered. In some situations uncertainty may not significantly impact the decisions that should be made. In other situations it will. Sensitivity analyses can help guide efforts needed to reduce that uncertainty. Model sensitivity and uncertainty analysis are discussed in more detail in the next chapter.

This chapter is divided into two main sections. The first section introduces a number of approaches to probabilistic optimization and simulation modeling . Probabilistic models will be developed and applied to some of the same water resources management problems used to illustrate deterministic modeling in previous chapters. These modeling methods could be, and have been, applied to numerous other water resources planning and management problems as well. The purpose here, however, is simply to illustrate some of these commonly used approaches to probabilistic modeling and show how they can be applied to water resources system design and operating problems.

7.2 Generating Values from Known Probability Distributions

As discussed in the previous chapter, variables whose values cannot be predicted with certainty are called random variables. Often inputs to hydrologic simulation models are observed or synthetically generated values of rainfall or streamflow. Other examples of such random variables could be evaporation losses, point and nonpoint source wastewater discharges , demands for water, spot prices for energy that may impact the amount of hydropower to produce, etc. Random processes are considered stationary if the statistical attributes of the process are not changing. If there is no serial correlation in the spatial or temporal sequence of observed values, then such stationary random processes can be characterized by single probability distributions. These probability distributions are often based on past observations of the values of the random variable. These past observations or measurements are used either to define the probability distribution itself or to estimate parameter values of an assumed type of distribution.

Let R be a random variable whose probability density distribution, f R (r), is as shown in Fig. 7.1. This distribution indicates the probability or likelihood of an observed value of the random variable R being between any two values of r on the horizontal axis. For example, the probability of an observed value of R being between 0 and r* is p*, the shaded area to the left of r* in Fig. 7.1. The entire area under a probability density distribution, as shown in Fig. 7.1, is 1.

Fig. 7.1
figure 1

Probability density distribution of a random variable R. The probability that r is less than or equal r* is p*

Full size image

Integrating this function over the entire range of r, converts the density function to a cumulative distribution function, F R (r*), ranging from 0 to 1, as illustrated in Fig. 7.2.

Fig. 7.2
figure 2

Cumulative distribution function of a random variable R showing the probability of any observed random value of R being less than or equal to a given value r. The probability of an observed value of R being less than or equal to r* is p*

Full size image
$$ \int\limits_{0}^{{r^{*} }} {f_{R} {(}r{)d}r} = { \Pr }(r^{*} \le R) = F_{R} \left( {r^{*} } \right) $$
(7.1)

Given any value of p* from 0 to 1, one can find its corresponding random variable value r* from the inverse of the cumulative distribution function.

$$ F_{R}^{ - 1} \left( {p^{*} } \right) = r^{*} $$
(7.2)

From the distribution shown in Fig. 7.1 it is obvious that the likelihood of different values of the random variable varies; values in the vicinity of r* are much more likely to occur than are values at the tails of the distribution. A uniform distribution is one that looks like a rectangle; any value of the random variable between its lower and upper limits is equally likely. Using Eq. 7.2, together with a series of uniformly distributed (all equally likely) values of p* over the range from 0 to 1 (i.e., along the vertical axis of Fig. 7.2), a corresponding series of random variable values, r*, associated with any distribution can be generated. These random variable values will have a cumulative distribution as shown in Fig. 7.2, and hence a density distribution as shown in Fig. 7.1, regardless of the types or shapes of those distributions. The mean and variance of the distributions will be maintained.

The mean and variance of continuous distributions are

$$ \int {rf_{R} {(}r{)d}r} = {E[}R{]} $$
(7.3)
$$ \int {\left( {r{-}{E[}R{]}} \right)^{2} f_{R} {(}r{)d}r} = {\text{Var}}\left[ R \right] $$
(7.4)

The mean, variance and serial correlations of discrete distributions having possible values denoted by r i with probabilities p i are

$$ \sum\limits_{i} {r_{i} p_{i} } = {E[}R{]} $$
(7.5)
$$ \sum\limits_{i} {\left( {r_{i} {-}{E}\left[ R \right]} \right)^{2} p_{i} } = {\text{Var[}}R{]} $$
(7.6)

If a time series of T random variable values, r t , generated from the distribution of the same stationary random variable , R, exist, then the serial or autocorrelations of r t and r t+k in this time series for any positive integer k are

$$ \rho_{R} (k) = \sum\limits_{t = 1,T - k} {\left[ {(r_{t} {-}{ E[}R{]})(r_{{{t} + k}} {-}{ E}[R])} \right]/\sum\limits_{t = 1,T} {\left( {r_{t} {-}{ E[}R{]}} \right)^{2} } } $$
(7.7)

The probability density and corresponding cumulative probability distributions can be of any shape, not just those named distributions commonly found in probability and statistics books.

The process of generating a time sequence t = 1, 2, … of inputs, r t , from the probability distribution of a random variable R where the lag 1 serial correlation , ρ R (1) = ρ, is to be preserved is a little more complex. The expected value of the random variable R t+1 depends on the observed value, r t , of the random variable R t , together with the mean of the distribution, E[R], and the correlation coefficient ρ. If there is no correlation (ρ is 0), the expected value of R t+1 is the mean of the population, E[R]. If there is perfect correlation (ρ is 1), the expected value of R t+1 is r t . In general, the expected value of R t+1 given an observed value r t of R t is

$$ {E}\left[ {R_{t + 1} |R_{t} = r_{t} } \right] = {E[}R{]} + \rho (r_{t} {-}{E[}R{]}). $$
(7.8)

The variance of the random variable R t+1 depends on the variance of the distribution, Var[R], and the lag one correlation coefficient, ρ.

$$ {\text{Var}}\left[ {R_{t + 1} |R_{t} = r_{t} } \right] = {Var}[R](1 - \rho^{2} ). $$
(7.9)

If there is perfect correlation (ρ = 1) the process is deterministic and there is no variance. The value for r t+1 is r t . If there is no correlation, i.e., serial correlation does not exist (ρ = 0), the generated value for r t+1 is its mean, E[R], plus some randomly generated deviation from a normal distribution having a mean of 0 and a standard deviation of 1, denoted as N(0, 1). In this case the value r t+1 is not dependent on r t .

When the serial correlation is more than 0 but less than 1, then both the correlation and the standard deviation (the square root of the variance) influence the value of r t+1. A sequence of random variable values from a multivariate normal distribution that preserves the mean, E[R], overall variance, Var[R], and lag one correlation ρ, can be obtained from Eq. 7.10.

$$ r_{t + 1} = E[R]{ } + \rho (r_{t} - E[R]) + N(0,1)\sigma (1 - \rho^{2} )^{1/2} . $$
(7.10)

The term N(0, 1) in Eq. 7.10 is a random number generated from a normal distribution having a mean of 0 and a variance of 1. The process involves selecting a random number from a uniform distribution ranging from 0 to 1, and using it in Eq. 7.2 for a N(0, 1) distribution to obtain a value of random number for use in Eq. 7.10. This positive or negative number is substituted for the term N(0, 1) in Eq. 7.10 to obtain a value r t+1. This is shown on the graph in Fig. 7.3.

Fig. 7.3
figure 3

Diagram showing the calculation of a sequence of values of the random variable R from a multivariate normal distribution in a way that preserves the mean, variance and correlation of the random variable

Full size image

Simulation models that have random inputs, such as a series of r t values, will generally produce random outputs. After many simulations, the probability distributions of each random output variable value can be defined. These then can be used to estimate reliabilities and other statistical characteristics of those output distributions. This process of generating multiple random inputs for multiple simulations to obtain multiple random outputs is called Monte Carlo simulation .

7.3 Monte Carlo Simulation

To illustrate Monte Carlo simulation, consider the allocation problem involving three firms, each of which receives a benefit, B i (x it ), from the amount of water, x it , allocated to it in each period t. This situation is shown in Fig. 7.4. Monte Carlo simulation can be used to find the probability distribution of the benefits to each firm associated with the firm’s allocation policy.

Fig. 7.4
figure 4

Stream flow allocations in each period t result in benefits, B i (x it ), to each firm i. The flows, Q it , at each diversion site i are the random flows Q t less the upstream withdrawals , if any

Full size image

Suppose the policy is to keep the first two units of flow in the stream, to allocate the next 3 units to Firm 3, and the next 4 units to firms 1 and 2 equally. The remaining flow is to be allocated to each of the three firms equally up to the limits desired by each firm, namely 3.0, 2.33, and 8.0, respectively. Any excess flow will remain in the stream. The plots in Fig. 7.5 illustrate this policy. Each allocation plot reflects the priorities given to the three firms and the users further downstream.

Fig. 7.5
figure 5

a Water allocation policy for Firm 1 based on the flow at its diversion site. This policy applies for each period t. b Water allocation policy for Firm 2 based on the flow at its diversion site for that firm. This policy applies for each period t. c Water allocation policy for Firm 3 based on the flow at its diversion site. This policy applies for each period t. d Streamflow downstream of site 3 given the streamflow Q 3t at site 3 before the diversion. This applies for each period t

Full size image

A simulation model can now be created. In each of a series of discrete time periods t, the flows Q t are drawn from a probability distribution, such as from Fig. 7.2 using Eq. 7.2. Once this flow is determined, each successive allocation, x it , is computed. Once an allocation is made it is subtracted from the streamflow and the next allocation is made based on that reduced streamflow, in accordance with the allocation policy defined in Fig. 7.5. After numerous time steps the probability distributions of the allocations to each of the firms can be defined.

Figure 7.6 shows a flow chart for this simulation model.

Fig. 7.6
figure 6

Monte Carlo simulation to determine probability distributions of allocations to each of three water users, as illustrated in Fig. 7.4. The dash lines represent information (data) flows

Full size image

Having defined the probability distribution of the allocations, based on the allocation policy, one can now consider each of the allocations as random variables , X 1, X 2, and X 3 for firms 1, 2 and 3, respectively.

7.4 Chance Constrained Models

For models that include random variables it may be appropriate in some situations to consider constraints that do not have to be satisfied all the time. Chance constraints specify the probability of a constraint being satisfied, or the fraction of the time a constraint has to apply. Consider, for example, the allocation problem shown in Fig. 7.4. For planning purposes, the three firms may want to set allocation targets , not expecting to have those targets met 100% of the time. To insure, for example, that an allocation target, T i , of firm i will be met at least 90% of the time, one could write the chance constraint

$$ { \Pr }\{ T_{i} \le X_{i} \} \ge 0.90\quad i = 1,2,\;{\text{and}}\;3 $$
(7.11)

In this constraint, the allocation target T i is an unknown decision variable, and X i is a random variable whose distribution has just been computed and is known.

To include chance constraints in optimization models , their deterministic equivalents must be defined. The deterministic equivalents of these three chance constraints in Eq. 7.11 are

$$ T_{i} \le x_{it}^{0.10} \quad i = 1,2,\;{\text{and}}\;3 $$
(7.12)

where \( x_{it}^{0. 10} \) is the particular value of the random variable X i that is equaled or exceeded 90% of the time. This value is shown on the probability distribution for X i in Fig. 7.7.

Fig. 7.7
figure 7

Probability density distribution of the random allocation X i to firm i. The particular allocation value \( x_{it}^{0.10} \) has a 90% chance of being equaled or exceeded, as indicated by the shaded region

Full size image

To modify the allocation problem somewhat, assume the benefit obtained by each firm is a function of its target allocation and that the same allocation target applies in each time period t. The equipment and labor used in the firm is presumably based on the target allocations . Once the target is set assume there are no benefits gained by excess allocations of water. If the benefits obtained are to be based on the target allocations, rather than the actual allocations, then the optimization problem is one of finding the values of the three targets that maximize the total benefits obtained with a reliability of, say, at least 90%.

$$ {\text{Maximize}}\left( {6T_{1} {-}T_{1}^{2} } \right) + \left( {7T_{2} - 1.5T_{2}^{2} } \right) + \left( {8T_{3} - 0.5T_{3}^{2} } \right) $$
(7.13)

Subject to:

$$ { \Pr }\left\{ {T_{1} + T_{2} + T_{3} \le \left[ {Q_{t} - { \hbox{min} }\left( {Q_{\text{t}} ,2} \right)} \right]} \right\} \ge 0.90\quad {\text{for}}\,{\text{all}}\,{\text{periods}}\,t $$
(7.14)

where Q t is the random streamflow variable upstream of all diversion sites. If the same unconditional probability distribution of Q t applies for each period t then only one Eq. 7.14 is needed.

Assuming the value of the streamflow, \( q_{t}^{0.10} \), that is equaled or exceeded 90% of the time, is greater than 2 (the amount that must remain in the stream), the deterministic equivalent of chance constraint Eq. 7.14 is

$$ T_{1} + T_{2} + T_{3} \le \left[ {q_{t}^{0.10} - { \hbox{min} }\left( {q_{t}^{0.10} ,2} \right)} \right] $$
(7.15)

The value of the flow that is equaled or exceeded 90% of the time, \( q_{t}^{0.10} \), can be obtained from the cumulative distribution of flows as illustrated in Fig. 7.8.

Fig. 7.8
figure 8

Example cumulative probability distribution showing the particular value of the random variable, \( q_{t}^{0.10} \), that is equaled or exceeded 90% of the time

Full size image

Assume this 90% reliable flow is 8. The deterministic equivalent of the chance constraint Eq. 7.14 for all periods t is simply T 1 + T 2 + T 3 ≤ 6. The optimal solution of the chance constrained target allocation model, Eqs. 7.13 and 7.15, is, as seen before, T 1 = 1, T 2 = 1, and T 3 = 4. The next step would be to simulate this problem to see what the actual reliabilities might be for various sequences of flows q t.

7.5 Markov Processes and Transition Probabilities

Time series correlations can be incorporated into models using transition probabilities. To illustrate this process, consider the observed flow sequence shown in Table 7.1.

Table 7.1 Sequence of flows for 31 time periods t
Full size table

The estimated mean , variance and correlation coefficient of the observed flows shown in Table 7.1 can be calculated using Eqs. 7.16, 7.17 and 7.18.

$$ E[Q] = \sum\limits_{t = 1}^{31} q_{t} /31 = 3.155 $$
(7.16)
$$ {\text{Var}}[Q] = \sum\limits_{t = 1}^{31} {\left( {q_{t} - 3.155} \right)^{2} /31} = 1.95 $$
(7.17)
$$ \begin{aligned} & {\text{Lag-one}}\,{\text{correlation}}\,{\text{coefficient}} = \rho \\ & \quad = \left[ {\sum\limits_{{{t} = 1}}^{30} {\left( {q_{t + 1} - 3.155} \right)\left( {q_{t} - { }3.155} \right)} } \right]/\sum\limits_{{{t} = 1}}^{31} {\left( {q_{t} - 3.155} \right)^{2} } = 0.50 \\ \end{aligned} $$
(7.18)

The probability distribution of the flows in Table 7.1 can be approximated by a histogram. Histograms can be created by subdividing the entire range of random variable values, e.g., flows, into discrete intervals. For example, let each interval be 2 units of flow. Counting the number of flows in each interval and then dividing those interval counts by the total number of counts results in the histogram shown in Fig. 7.9. In this case, just to compare this with what will be calculated later, the first flow, q 1, is ignored.

Fig. 7.9
figure 9

Histogram showing an equal 1/3 probability that the values of the random variable Q t will be in any one of the three 2-flow unit intervals

Full size image

Figure 7.9 shows a uniform unconditional probability distribution of the flow being in any of the possible discrete flow intervals. It does not show the possible dependency of the probabilities of the random variable value, q t+1, in period t + 1 on the observed random variable value, q t , in period t. It is possible that the probability of being in a flow interval j in period t + 1 depends on the actual observed flow interval i in period t.

To see if the probability of being in any given interval of flows is dependent on the past flow interval one can create a matrix. The rows of the matrix are the flow intervals i in period t. The columns are the flow intervals j in the following period t + 1. Such a matrix is shown in Table 7.2. The numbers in the matrix are based on the flows in Table 7.1 and indicate the number of times a flow in interval j followed a flow in interval i.

Table 7.2 Matrix showing the number of times a flow in interval i in period t was followed by a flow in interval j in period t + 1
Full size table

Given an observed flow in an interval i in period t, the probabilities of being in one of the possible intervals j in the next period t + 1 must sum to 1. Thus each number in each row of the matrix in Table 7.2 can be divided by the total number of flow transitions in that row (the sum of the number of flows in the row) to obtain the probabilities of being in each interval j in t + 1 given a flow in interval i in period t. In this case there are 10 flows that followed each flow interval i, hence by dividing each number in each row of the matrix by 10 defines the transition probabilities P ij .

$$ P_{ij} = { \Pr }\left\{ {Q_{{{t} + {1}}} \;{\text{in}}\;{\text{interval}}\;j|Q_{t} \;{\text{in}}\;{\text{interval}}\;i} \right\} $$
(7.19)

These conditional or transition probabilities, shown in Table 7.3, correspond to the number of transitions shown in Table 7.2.

Table 7.3 Matrix showing the probabilities P ij of having a flow in interval j in period t + 1 given an observed flow in interval i in period t
Full size table

Table 7.3 is a matrix of transition probabilities. The sum of the probabilities in each row equals 1. Matrices of transition probabilities whose rows sum to one are also called stochastic matrices or first-order Markov chains .

If each row’s probabilities were the same, this would indicate that the probability of observing any flow interval in the future is independent of the value previous flows. Each row would have the same probabilities as the unconditional distribution shown in Fig. 7.9. In this example the probabilities in each row differ, showing that low flows are more likely to follow low flows, and high flows are more likely to follow high flows. Thus the flows in Table 7.1 are positively correlated, as indeed has already determined from Eq. 7.18.

Using the information in Table 7.3, one can compute the probability of observing a flow in any interval at any period on into the future given the present flow interval. This can be done one period at a time. For example assume the flow in the current time period t = 1 is in interval i = 3. The probabilities, PQ j,2, of being in any of the three intervals in the following time period t + 1 are the probabilities shown in the third row of the matrix in Table 7.3.

The probabilities of being in an interval j in the following time period t = 3 is the sum over all intervals i of the joint probabilities of being in interval i in period t = 2 and making a transition to interval j in period t = 3.

$$ \begin{aligned} & \Pr \left\{ {Q_{3} \,{\text{in}}\,{\text{interval}}\,j} \right\} = \text{PQ}_{j,3} \\ & \quad = \sum\limits_{i} {\Pr \left\{ {Q_{2} \,{\text{in}}\,{\text{interval}}\,i} \right\}\Pr \left\{ {Q_{3} \,{\text{in}}\,{\text{interval}}\,j|Q_{2} \,{\text{in}}\,{\text{interval}}\,i} \right\}} \\ \end{aligned} $$
(7.20)

The last term in Eq. 7.20 is the transition probability , from Table 7.3, that in this example remains the same for all time periods t. These transition probabilities, Pr{Q t+1 in interval j | Q t in interval i} can be denoted as P ij .

Referring to Eqs. 7.19, 7.20 can be written in a general form as

$$ PQ_{j,t + 1} = \sum\limits_{i} {PQ_{it} P_{ij} } \quad {\text{for}}\,{\text{all}}\,{\text{intervals}}\,j\,{\text{and}}\,{\text{periods}}\,t $$
(7.21)

This operation can be continued to any future time period. Table 7.4 illustrates the results of such calculations for up to six future periods, given a present period (t = 1) flow in interval i = 3.

Table 7.4 Probabilities of observing a flow in any flow interval i in a future time period t given a current flow in interval i = 3
Full size table

Note that as the future time period t increases, the flow interval probabilities are converging to the unconditional probabilities, in this example 1/3. 1/3, 1/3, as shown in Fig. 7.9. The predicted probability of observing a future flow in any particular interval at some time in the future becomes less and less dependent on the current flow interval as the number of time periods increases between the current period and that future time period.

When these unconditional probabilities are reached, PQ it will equal PQ i,t+1 for each flow interval i. To find these unconditional probabilities directly, Eq. 7.21 can be written as

$$ PQ_{j} = \sum\limits_{i} {PQ_{i} \,P_{ij} \quad {\text{for}}\,{\text{all}}\,{\text{intervals}}\,j\,{\text{less}}\,{\text{one}}} $$
(7.22)

Equation 7.22 along with Eq. 7.23 can be used to calculate all the unconditional probabilities PQ i directly.

$$ \sum\limits_{i} {PQ_{i} } = 1 $$
(7.23)

Conditional or transition probabilities can be incorporated into stochastic optimization models of water resource systems.

7.6 Stochastic Optimization

To illustrate the development and use of stochastic optimization models consider first the allocation of water to a single user. Assume the flow in the stream where the diversion takes place is not regulated and can be described by a known probability distribution based on historical records. Clearly the user cannot divert more water than is available in the stream. A deterministic model would include the constraint that the diversion x cannot exceed the available water Q. But Q is a random variable. Some discrete value, q, of the random variable Q will have to be selected, knowing that there is some probability that in reality, or in a simulation model , the actual flow may be less than the selected value q. Hence if the constraint x ≤ q is binding (x = q), the actual allocation may be less than the value of the allocation or diversion variable x produced by the optimization model.

If the value of x affects one of the system’s performance indicators, e.g., the net benefits, B(x), to the user, a more accurate estimate of the user’s net benefits will be obtained from considering a range of possible allocations x, depending on the range of possible values of the random flow Q. One way to do this is to divide the known probability distribution of flows q into discrete ranges, i, each range having a known probability PQ i . Designate a discrete flow q i for each range. Associated with each known flow q i is an unknown allocation x i . Now the deterministic constraint x ≤ q can be replaced with the set of constraints x i  ≤ q i and the term B(x) in the original objective function can be replaced by its expected value , \( \sum\nolimits_{i} {{\text{PO}}_{i} B(x_{i} )} \).

Note, when dividing a continuous known probability distribution into discrete ranges, the discrete flows q i selected to represent each range i having a given probability PQ i , should be selected so as to maintain at least the mean and variance of that known distribution as defined by Eqs. 7.5 and 7.6.

To illustrate this consider a slightly more involved example involving the allocation of water to consumers upstream and downstream of a reservoir. Both the policies for allocating water to each user and the reservoir release policy are to be determined. This example problem is shown in Fig. 7.10.

Fig. 7.10
figure 10

Example water resource system involving water diversions from a river both upstream and downstream of a reservoir of known capacity

Full size image

If the allocation of water to each user is to be based on a common objective, such as the minimization of the total sum, over time, of squared deviations from prespecified target allocations , each allocation in each time period will depend in part on the reservoir storage volume.

Consider first a deterministic model of the above problem, assuming known flows Q t and upstream and downstream allocation targets UT t and DT t in each of T within-year periods t in a year. Assume the objective is to minimize the sum of squared deviations of actual allocations, u t and d t , from their respective target allocations, UT t and DT t in each within-year period t.

$$ {\text{Minimize}}\,\sum\limits_{t}^{T} {\left\{ {\left( {UT_{t} {-}u_{t} } \right)^{2} + \left( {DT_{t} {-}d_{t} } \right)^{2} } \right\}} $$
(7.24)

The constraints include:

  1. (a)

    Continuity of storage involving initial storage volumes S t , net inflows Q t  − u t , and releases R t . Assuming no losses

$$ S_{t} + Q_{t} - u_{t} - R_{t} = S_{t + 1} \quad {\text{for}}\,{\text{each}}\,{\text{period}}\,t,\,T + 1 = 1 $$
(7.25)
  1. (b)

    Reservoir capacity limitations. Assuming a known active storage capacity K,

$$ S_{t} \le K\quad {\text{for}}\,{\text{each}}\,{\text{period}}\,t $$
(7.26)

Allocation restrictions for each period t:

$$ u_{t} \le Q_{t} $$
(7.27)
$$ d_{t} \le R_{t} $$
(7.28)

Equations 7.25 and 7.28 could be combined to eliminate the release variable R t since in this problem knowledge of the total release in each period t is not required. In this case Eq. 7.25 would become an inequality.

The solution for this model, Eqs. 7.24–7.28, would depend on the known variables (the targets UT t and DT t , flows Q t and reservoir capacity K). It would identify the particular upstream and downstream allocations and reservoir releases in each period t. It would not provide a policy that defines what allocations and releases to make for a range of different inflows and initial storage volumes in each period t. A backward-moving dynamic programming model can provide such a policy. This policy will identify the allocations and releases to make based on various initial storage volumes, S t , as well flows, Q t , as discussed in Chap. 4.

This deterministic discrete dynamic programming allocation and reservoir operation model can be written for different discrete values of storage volumes S t from 0 ≤ S t ≤ capacity K as

$$ \begin{aligned} F_{t}^{n} \left( {S_{t} ,Q_{t} } \right) = & \hbox{min} \left\{ {\left( {{\text{UT}}_{t} - u_{t} } \right)^{2} + \left( {{\text{DT}}_{t} - d_{t} } \right)^{2} + F_{t + 1}^{n - 1} \left( {S_{t + 1} ,Q_{t + 1} } \right)} \right\} \\ & u_{t} ,R_{t} ,d_{t} \\ & u_{t} \le Q_{t} \\ & R_{t} \le S_{t} + Q_{t} - u_{t} \\ & R_{t} \ge S_{t} + Q_{t} - u_{t} - K \\ & d_{t} \le R_{t} \\ & S_{t + 1} = S_{t} + Q_{t} - u_{t} - R_{t}\\ \end{aligned} $$
(7.29)

There are three variables to be determined at each stage or time period t in the above dynamic programming model. These three variables are the allocations u t and d t and the reservoir release R t . Each decision involves three discrete decision variable values. The functions \( F_{t}^{n} \left( {S_{t} ,Q_{t} } \right) \) define the minimum sum of squared deviations given an initial storage volume S t and streamflow Q t in time period or season t with n time periods remaining until the end of reservoir operation.

One can reduce this three-decision variable model to a single variable model by realizing that for any fixed discrete initial and final storage volume states, there can be a direct tradeoff between the upstream and downstream allocations given the particular streamflow in each period t. Increasing the upstream allocation will decrease the resulting reservoir inflow and this in turn will reduce the release by the same amount. This reduces the amount of water available to allocate to the downstream use.

Hence for this example problem involving these upstream and downstream allocations, a local optimization can be performed at each time step t for each combination of storage states S t and S t+1. This optimization finds the allocation decision variables u t and d t that

$$ {\text{minimize}}\left( {{\text{UT}}_{t} - u_{t} } \right)^{2} + \left( {{\text{DT}}_{t} - d_{t} } \right)^{2} $$
(7.30)

where

$$ u_{t} \le Q_{t} $$
(7.31)
$$ d_{t} \le S_{t} + Q_{t} - u_{t} - S_{t + 1} $$
(7.32)

This local optimization can be solved to identify the u t and d t allocations for each feasible combination of S t and S t+1 in each period t.

Given these optimal allocations, the dynamic programming model can be simplified to include only one discrete decision variable, either R t or S t+1. If the decision variable S t+1 is used in each period t, the releases R t in those periods t do not need to be considered. Thus the dynamic programming model expressed by Eq. 7.29 can be written for all discrete storage volumes S t from 0 to K and for all discrete flows Q t as

$$ F_{t}^{n} \left( {S_{t}, Q_{t} } \right) = \mathop{\hbox{min}}\limits_{\begin{subarray}{l} S_{t + 1} \\ S_{t + 1} \le K \end{subarray} } \left\{ \begin{array}{ll} \left( \text{UT}_{t} - u_{t} \left( {S_{t}, S_{t + 1} } \right) \right)^{2} + \\ \left( \text{DT}_{t} - d_{t} \left({S_{t}, S_{t + 1} } \right) \right)^{2} \end{array}\right. $$
(7.33)

where the functions u t (S t , S t+1) and d t (S t , S t+1) have been determined using Eqs. 7.30–7.32.

As the total number of periods remaining, n, increases, the solution of this dynamic programming model will converge to a steady or stationary state. The best final storage volume S t+1 given an initial storage volume S t will likely differ for each within-year period or season t, but for a given season t it will be the same in successive years. In addition, for each storage volume S t , streamflow, Q t , and within-year period t the difference between \( F_{t}^{n + T} \left( {S_{t} ,Q_{t} } \right) \) and \( F_{t}^{n} \left( {S_{t} ,Q_{t} } \right) \) will be the same constant regardless of the storage volume S t , flow Q t and period t. This constant is the optimal, in this case minimum, annual value of the objective function, Eq. 7.24.

There could be additional limits imposed on storage variables and release variables, such as for flood control storage or minimum downstream flows, as might be appropriate in specific situations.

The above deterministic dynamic programming model (Eq. 7.33) can be converted to a stochastic model . Stochastic models consider multiple discrete flows as well as multiple discrete storage volumes, and their probabilities, in each period t. A common way to do this is to assume that the sequence of flows follow a first-order Markov process . Such a process involves the use of transition or conditional probabilities of flows as defined by Eq. 7.20.

To develop these stochastic optimization models it is convenient to introduce some additional indices or subscripts. Let the index k denote different initial storage volume intervals. These discrete intervals divide the continuous range of storage volume values from 0 to the active reservoir capacity K. Each S kt is a discrete storage volume that represents the range of storage volumes in interval k at the beginning of each period t.

Let the index l denote different final storage volume intervals. Each S l,t+1 is a discrete volume that represents the storage volume interval l at the end of in each period t or equivalently at the beginning of period t + 1. As previously defined, let the indices i and j denote the different flow intervals, and each discrete q it and q j,t+1 represent the flows in those flow intervals i and j in periods t and t + 1, respectively.

These subscripts and the volume or flow intervals they represent are illustrated in Fig. 7.11.

Fig. 7.11
figure 11

Discretization of streamflows and reservoir storage volumes. The area within each flow interval i below the probability density distribution curve is the unconditional probability , PQ it , associated with the discrete flow q it

Full size image

With this notation it is now possible to develop a stochastic dynamic programming model that will identify the allocations and releases that are to be made given both the initial storage volume, S kt , and the flow, q it . It follows the same structure as the deterministic models defined by Eqs. 7.30–7.32, and 7.33.

To identify the optimal allocations in each period t for each pair of feasible initial and final storage volumes S kt and S l,t+1, and inflows q it, one can solve Eqs. 7.34–7.36.

$$ {\text{minimize}}\left( {{\text{UT}}_{t} - u_{kit} } \right)^{2} + \left( {{\text{DT}}_{t} - d_{kilt} } \right)^{2} $$
(7.34)

where

$$ u_{kit} \le q_{it} \quad {\text{for}}\,{\text{all}}\,k,i,t. $$
(7.35)
$$ d_{kilt} \le S_{kt} + q_{it} - u_{kit} - S_{l,t + 1} \quad {\text{for}}\,{\text{all}}\,{\text{feasible}}\,k,i,l,t. $$
(7.36)

The solution to these equations for each feasible combination of intervals k, i, l, and period t defines the optimal allocations that can be expressed as u t (k, i) and d t (k, i, l).

The stochastic version of Model 7.33, again expressed in a form suitable for backward moving discrete dynamic programming, can be written for different discrete values of S kt from 0 to K and for all q it as

$$ \begin{aligned} F_{t}^{n} ( {S_{kt} ,q_{it}}) = & \hbox{min} \{ \left( {{\text{UT}}_{t} - u_{t} \left( {k,i} \right)} \right)^{2} + \left( {{\text{DT}}_{t} - d_{t} \left( {k,i,l} \right)} \right)^{2} +\\ & \sum\nolimits_{j} P_{ij}^{t} F_{t + 1}^{n - 1} (S_{l,t+1}, q_{j,t+1})\} \\ & S_{l,t+1} \\ & S_{l,t + 1} \le K \\ & S_{l,t + 1} \le S_{kt} + q_{it} \\ \end{aligned} $$
(7.37)

Each \( P_{ij}^{t} \) in the above recursive equation is the known conditional or transition probability of a flow q j,t+1 within interval j in period t + 1 given a flow of q it within interval i in period t.

$$ \begin{aligned} F_{t}^{n} (k,i) = & \hbox{min} \{ \left( {{\text{UT}}_{t} - u_{t} \left( {k,i} \right)} \right)^{2} + \left( {{\text{DT}}_{t} - d_{t} \left( {k,i,l} \right)} \right)^{2} +\\& \sum\nolimits_{j} P_{ij}^{t} F_{t + 1}^{n - 1} (l,j)\} \\ & l\,{\text{such}}\,{\text{that}} \\ & S_{l,t + 1} \le K \\ & S_{l,t + 1} \le S_{kt} + q_{it} \\ \end{aligned} $$

The sum over all flow intervals j of these conditional probabilities times the \( F_{t + 1}^{n - 1} \left( {S_{l,t + 1} ,q_{j,t + 1} } \right) \) values is the expected minimum sum of future squared deviations from allocation targets with n − 1 periods remaining given an initial storage volume of S kt and flow of q it and final storage volume of S l,t+1. The value \( F_{t}^{n} \left( {S_{kt} ,q_{it} } \right) \) is the expected minimum sum of squared deviations from the allocation targets with n periods remaining given an initial storage volume interval of S kt and flow interval of q it . Stochastic models such as these provide expected values of objective functions.

Another way to write the recursion equations of this model, Eq. 7.37, is by using just the indices k and l to denote the discrete storage volume variables S kt and S l,t+1 and indices i and j to denote the discrete flow variables q it and q j,t+1:

$$ \begin{aligned} F_{t}^{n} (k,i) = & \hbox{min} \{ \left( {{\text{UT}}_{t} - u_{t} \left( {k,i} \right)} \right)^{2} + \left( {{\text{DT}}_{t} - d_{t} \left( {k,i,l} \right)} \right)^{2} + \\& \sum\nolimits_{j} P_{ij}^{t} F_{t + 1}^{n - 1} (l,j)\} \\ & l\,{\text{such}}\,{\text{that}} \\ & S_{l,t + 1} \le K \\ & S_{l,t + 1} \le S_{kt} + q_{it} \\ \end{aligned} $$
(7.38)

The steady-state solution of this dynamic programming model will identify the preferred final storage volume S l,t+1 in period t given the particular discrete initial storage volume S kt and flow q it . This optimal policy can be expressed as a function ℓ that identifies the best interval l given intervals k, i and period t.

$$ l = \ell \left( {k,i,t} \right) $$
(7.39)

All values of l given k, i, and t, defined by Eq. 7.39, can be expressed in a matrix, one for each period t.

Knowing the best final storage volume interval l given an initial storage volume interval k and flow interval i, the optimal downstream allocation, d t (k, i), can, like the upstream allocation, be expressed in terms of only k and i in each period t. Thus knowing the initial storage volume S kt and flow q it is sufficient to define the optimal allocations u t (k, i) and d t (k, i), final storage volume S l,t+1, and hence the release R t (k, i).

$$ S_{kt} + q_{it} - u_{t} (k,i) - R_{t} (k,i) = S_{l,t + 1} \quad \forall k,i,t\;\;{\text{where}}\,l = \ell (k,i,t) $$
(7.40)

7.6.1 Probabilities of Decisions

Knowing the function l = ℓ(k, i, t) permits a calculation of the probabilities of the different discrete storage volumes, allocations, and flows. Let

  • PS kt  = the unknown probability of an initial storage volume S kt being within some interval k in period t.

  • PQ it  = the steady-state unconditional probability of flow q it within interval i in period t.

  • P kit  = the unknown probability of the upstream and downstream allocations u t (k, i) and d t (k, i) and reservoir release R t (k, i) in period t.

As previously defined

  • \( P_{ij}^{t} \) = the known conditional or transition probability of a flow within interval j in period t + 1 given a flow within interval i in period t.

These transition probabilities \( P_{ij}^{t} \) can be displayed in matrices, similar to Table 7.3, but as a separate matrix (Markov chain ) for each period t.

The joint probabilities of an initial storage interval k, an inflow in the interval i, P kit in each period t must satisfy two conditions. Just as the initial storage volume in period t + 1 is the same as the final storage volume in period t, the probabilities of these same respective discrete storage volumes must also be equal. Thus

$$ \sum\limits_{j} {P_{l,j,t + 1} = \sum\limits_{k} {\sum\limits_{i } {P_{kit} } } } \quad \forall l,t $$
(7.41)

where the sums in the right hand side of Eq. 7.41 are over only those combinations of k and i that result in a final volume interval l. This relationship is defined by Eq. 7.39 (l = ℓ(k, i, t)).

While Eq. 7.41 must apply, it is not sufficient. The joint probability of a final storage volume in interval l in period t and an inflow j in period t + 1 must equal the joint probability of an initial storage volume in the same interval l and an inflow in the same interval j in period t + 1. Multiplying the joint probability P kit times the conditional probability \( P_{ij}^{t} \) and then summing over all k and i that results in a final storage interval l defines the former, and the joint probability P l,j,t+1 defines the latter.

$$ P_{l,j,t + 1} = \sum\limits_{k} {\sum\limits_{i} {P_{kit} P_{ij}^{t} } } \quad \forall l,j,t\quad l = \ell (k,i,t) $$
(7.42)

Once again the sums in Eq. 7.42 are over all combinations of k and i that result in the designated storage volume interval l as defined by the policy ℓ(k, i, t).

Finally, the sum of all joint probabilities P kit in each period t must equal 1.

$$ \sum\limits_{k} {\sum\limits_{i} {P_{kit} = 1} } \quad \forall t $$
(7.43)

Note the similarity of Eqs. 7.42 and 7.43 to the Markov steady-state flow Eqs. 7.22 and 7.23. Instead of only one flow interval index considered in Eqs. 7.22 and 7.23, Eqs. 7.42 and 7.43 include two indices, one for storage volume intervals and the other for flow intervals. In both cases, one of Eqs. 7.22 and 7.42 can be omitted in each period t since it is redundant with that period’s Eqs. 7.23 and 7.43, respectively.

The unconditional probabilities PS kt and PQ it can be derived from the joint probabilities P kit .

$$ {\text{PS}}_{kt} = \sum\limits_{i} {P_{kit} } \quad \forall k,t $$
(7.44)
$$ {\text{PQ}}_{it} = \sum\limits_{k} {P_{kit} } \quad \forall i,t $$
(7.45)

Each of these unconditional joint or marginal probabilities, when summed over all their volume and flow indices, will equal 1. For example

$$ \sum\limits_{k} {{\text{PS}}_{kt} } = \sum\limits_{i} {{\text{PQ}}_{it} } = 1 $$
(7.46)

Note that these probabilities are determined based only on the relationships among flow and storage intervals as defined by Eq. 7.39, l = ℓ(k, i, t) in each period t, and the Markov chains defining the flow interval transition or conditional probabilities, \( P_{ij}^{t} \). It is not necessary to know the actual discrete storage values representing those intervals. Thus assuming any relationship among the storage volume and flow interval indices, l = ℓ(k, i, t) and a knowledge of the flow interval transition probabilities \( P_{ij}^{t} \), one can determine the joint probabilities P kit and their marginal or unconditional probabilities PS kt . One does not need to know what those storage intervals are to calculate their probabilities. (Amazing, isn’t it?)

Given the values of these joint probabilities P kit , the deterministic model defined by Eqs. 7.24–7.28 can be converted to a stochastic model to identify the best storage and allocation decision variable values associated with each storage interval k and flow interval i in each period t.

$$ {\text{Minimize}}\sum\limits_{k} {\sum\limits_{i} {\sum\limits_{t}^{T} {P_{kit} \left\{ {\left( {{\text{UT}}_{t} - u_{kit} } \right)^{2} + \left( {{\text{DT}}_{t} - d_{kit} } \right)^{2} } \right\}} } } $$
(7.47)

The constraints include

  1. (a)

    Continuity of storage involving initial storage volumes S kt , net inflows q it  − u kit , and at least partial releases d kit . Again assuming no losses:

$$ S_{kt} + q_{it} - u_{kit} - d_{kit} \ge S_{l,t + 1} \quad \forall k,i,t\quad l = \ell (k,i,t) $$
(7.48)

Reservoir capacity limitations.

$$ S_{kit} \le K\quad \forall k,i,t $$
(7.49)

Allocation restrictions.

$$ u_{kit } \le q_{it} \quad \forall k,i,t $$
(7.50)

7.6.2 A Numerical Example

A simple numerical example may help to illustrate how these stochastic models can be developed without getting buried in detail. Consider, for simplicity, two within-year periods each year. The random flows Q t in each period t are divided into two intervals. These flow intervals are represented by discrete flows of 1 and 3 volume units per second in the first period and 3 and 6 volume units per second in the second period. Their transition probabilities are shown in Table 7.5.

Table 7.5 Transition probabilities for two ranges of flows in two within-year periods
Full size table

Assuming equal within-year period durations, these three discrete flow rates are equivalent to about 16, 47 and 95 million volume units per period.

Assume the active storage volume capacity K in the reservoir equals 50 million volume units. This capacity can be divided into different intervals of storage. For this simple example assume three storage volume intervals represented by 10, 25, and 40 million volume units. Assume the allocation targets remain the same in each period at both the upstream and downstream sites. The upstream allocation target is approximately 2 volume units per second or 30 million volume units in each period. The downstream allocation target is approximately 5 volume units per second or 80 million volume units in each period.

With these data we can use Eqs. 7.34–7.36 to determine the allocations that minimize the sum of squared deviations from targets and what that sum is, for all feasible combinations of initial and final storage volumes, and flows. Table 7.6 shows the results of these optimizations. These results will be used in the dynamic programming model to determine the best final storage volumes given initial volumes and flows.

Table 7.6 Optimal allocations associated with given initial storage, S k , flow, Q i , and final storage, S l , volumes
Full size table

With the information in Tables 7.5 and 7.6, the dynamic programming model, Eq. 7.38 or as expressed in Eq. 7.51, can be solved to find the optimal final storage volumes given an initial storage volume and flow. The iterations of the recursive equation, sufficient to reach a steady state, are shown in Table 7.7.

Table 7.7 First four iterations of dynamic programming model, Eq. 7.51, moving backward in successive periods n, beginning in season t = 2 with n = 1
Full size table
$$ \begin{aligned} F_{t}^{n} (k, \, i) = & { \hbox{min} }\{ {\text{SD}}_{kil } + \sum\limits_{j } {P_{ij}^{t } F_{t + 1}^{n - 1} \left( {l,j} \right)\} } \\ & {\text{over}}\,{\text{all}}\,l\,{\text{such}}\,{\text{that}} \\ & S_{l,t + 1} \le K \\ & S_{l,t + 1} \le S_{kt} + Q_{it} \\ \end{aligned} $$
(7.51)

This process can continue until a steady-state policy is defined. Table 7.8 summarizes the next five iterations. At this stage, the annual differences in the objective values associated with a particular state and season have come close to a common constant value.

Table 7.8 Summary of objective function values \( F_{t}^{n} \left( {k, \, i} \right) \) and optimal decisions for stages n = 5–9 periods remaining
Full size table

While the differences between corresponding \( F_{t}^{n + T} \;{\text{and}}\;F_{t}^{n} \) have not yet reached a common constant value to the nearest unit deviation (they range from, 3475.5 to 3497.1 for an average of 3485.7), the policy has converged to that shown in Tables 7.8 and 7.9.

Table 7.9 Optimal reservoir policy l = ℓ(k, i, t) for the example problem
Full size table

Given this policy, the probabilities of being in any of these volume and flow intervals can be determined by solving Eqs. 7.42–7.45. Table 7.10 shows the results of these equations applied to the data in Tables 7.5 and 7.8. It is obvious that if the policy from Table 7.9 is followed, the steady-state probabilities of being in storage interval 1 in period 1 and in interval 3 in period 2 are 0.

Table 7.10 Probabilities of flow and storage volume intervals associated with policy as defined in Table 7.9 for the example problem
Full size table

Multiplying these joint probabilities by the corresponding SD kit values in the last column of Table 7.6 provides the annual expected squared deviations, associated with the selected discrete storage volumes and flows. This is done in Table 7.11 for those combinations of k, i, and l that are contained in the optimal solution as listed in Table 7.9.

Table 7.11 The optimal operating policy and the probability of each state and decision
Full size table

The sum of products of the last two columns in Table 7.11 for each period t equals the expected squared deviations in the period. For period t = 1 the expected sum of squared deviations are 1893.3 and for t = 2 they are 1591.0. The total annual expected squared deviations are 3484.3. This compares with the expected squared deviations derived from the dynamic programming model, after nine iterations, ranging from 3475.5 to 3497.1 (as calculated from data in Table 7.8).

The policy for reservoir releases is a function not only of the initial storage volumes, but also of the current inflow, i.e., the total water available in the period. Reservoir release rule curves now must become two-dimensional. However, the inflow for each period usually cannot be predicted at the beginning of each period. Thus the reservoir release policy has to be expressed in a way that it can be followed without knowledge of the current inflow. One way to do this is to compute the expected value of the release for each discrete storage volume, and show it in a release rule. This is done in Fig. 7.12. The probability of each discrete release associated with each discrete river flow is the probability of the flow itself. Thus in period 1 when the storage volume is 40, the expected release is 46(0.41) + 56(0.59) = 52. These discrete expected releases can be used to define a continuous range of releases for the continuous range of storage volumes from 0 to full capacity, 50. Figure 7.12 also shows the hedging that might take place as the reservoir storage volume decreases.

Fig. 7.12
figure 12

Reservoir release rule showing an interpolated release, increasing as storage volumes increase

Full size image

These and modifications of these policies can be simulated to determine improved release rules. Simulation modeling is the subject of the following chapter.

7.7 Summary

This chapter has introduced some approaches for including risk into optimization and simulation models. The discussion began with ways to obtain values of random variables whose probability distributions are known. These values, for example streamflows or parameter values , can be inputs to simulation models . Monte Carlo simulation involves the use of multiple simulations using these random variable values to obtain the probability distributions of outputs, including various system performance indicators.

Two methods were reviewed for introducing random variables along with their probabilities into optimization models . One involves the use of chance constraints . These are constraints that must be met, as all constraints must be, but now with a certain probability. As in any method there are limits to the use of chance constraints. These limitations were not discussed, but in cases where chance constraints are applicable, and if their deterministic equivalents can be defined, they are probably the only method of introducing risk into otherwise deterministic models that do not add to the model size.

Alternatively, the range of random variable values can be divided into discrete ranges. Each range can be represented by a specific or discrete value of the random variable. These discrete values and their probabilities can become part of an optimization model. This was demonstrated using transition probabilities incorporated into both linear and dynamic programming models.

The examples used in this chapter to illustrate the development and application of stochastic optimization and simulation models are relatively simple. These and similar probabilistic and stochastic models have been applied to numerous water resources planning and management problems. They can be a much more effective screening tool than deterministic models based on the mean or other selected values of random variables . But sometimes they are not. Clearly if the system being analyzed is very complex, or just very big in terms of the number of variables and constraints, the use of deterministic models for a preliminary screening of alternatives prior to a more precise probabilistic screening is often warranted.