Abstract
Preface. A preface to a chapter is unusual, but dynamic programming requires some motivation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In 1979 Richard Bellman received the IEEE Medal of Honor (the highest award in electrical engineering) for his work in dynamic programming. In 1985 the Bellman Prize in Mathematical Bioscience was established to honor his contributions.
- 2.
The earliest use of the term “dynamic programming” I have found is Richard Bellman, “On the Theory of Dynamic Programming,” Proceedings of the National Academy of Sciences, 38 (8), 716–719 (1952), which is available online. Here he stated, “The theory of dynamic programming is intimately related to the theory of sequential analysis (1947) due to Wald [Wald’s Statistical Decision Functions, John Wiley & Sons, 1950.]” Abraham Wald died in 1950 at age 48 in an airplane crash. In this paper, Bellman referenced several other technical papers dealing with decision processes—e.g., Arrow, K.J., Blackwell, D., and Girshick, M.A., “Bayes and Minimax Solutions of Sequential Decision Problems,” Econometrica, 17, 214–244 (1949).
In the DVD the bellman equation (Shami Media, 2013), one of Bellman’s wives said Bellman told her that dynamic programming was in the wind at the time. And if he had not discovered it (actually formalized the method, named it, and wrote a book expounding its use), then someone else would have. Harold J. Kushner, one of Bellman’s colleagues at RAND, once stated in a speech, “Bellman did not quite invent dynamic programming, and many others contributed to its early development. But no one grasped its essence, isolated its essential features, and showed its full potential in control and operations research as well as in applications to the biological and social sciences, as did Bellman.”
- 3.
Fortran replaced assembler language in many programs, thereby reducing the size of those programs by an average factor of 20. See Wikipedia, s.v. Fortran. There were few computers in 1957, partly because they were so expensive, and the ones that existed were computationally feeble. Processing speed and memory size were both extremely limited. Computer memory was being converted from mercury tubes to iron cores. The operating systems and editors were crude. The machines were programmed in assembly languages. The first commercial computer (the UVIVAC I with 5000 vacuum tubes) was not shipped until 1952, and was priced at $159,000. Eventually the price rose to $1,500,000. As a comparison, I remember my mother complaining in the late 50s that she had difficulty buying groceries for a family of four on twenty dollars a week. Bellman worked at the RAND Corporation and their computer was the JOHNNIAC, hand-built by their engineers with funds from the Air Force and first working in 1953. The mean free time between (machine) failures was 500 seconds. It is not easy to communicate how difficult it was to run a complicated program on such a machine. Search the Internet for the 20-page History of the JONNIAC by Fred Joseph Gruenberger (1968).
- 4.
In 1973 Bellman developed a brain tumor, which when removed left him severely disabled. Nevertheless, he continued to publish at a high rate until he died in 1984 at age 63.
“Hal Shaperio asked me [Bellman], do you think you will be a better mathematician than Erdős?” “Far better,” I said. Immediately four pairs of incredulous eyes fastened upon me. I explained. “Erdős has great talent, even genius, but he has no judgement. He does not match the problems he works on with his ability.” I doubt whether at the time those listening got the point. I think they understand now.—Richard Bellman, Eye of the Hurricane (World Scientific, 1984), page 109.
This statement was made around 1946. Bellman (pre-PhD) was 26, and the Hungarian Paul Erdős was 35. Erdős later became one of the world’s most prolific, respected, and admired mathematicians. His field was analytic number theory, one of the most difficult areas of mathematics. Bellman who initially specialized in the same field eventually gave it up for applied mathematics. In my opinion, the two mathematicians cannot be compared. The world needs both. Notice that Bellman’s 1946 comment echoes this chapter’s preface.
- 5.
By the same reasoning, the probability of f (7,3) should work out to be ¾, which it does.
- 6.
The word “memoization” was coined from the root word “memo” by British AI pioneer Donald Michie in 1968.
- 7.
Those who cannot remember the past are condemned to repeat it.—George Santayana, (1905) Reason in Common Sense, p. 284, volume 1 of the Life of Reason.
- 8.
I found the solution in Martin Gardner, My Best Mathematical and Logic Puzzles (Dover, 1994). The solution had appeared in an earlier book of Gardner’s (1961). I met Martin Gardner twice and found him to be an extraordinarily warm and modest person. Gardner died in 2010. To this day (2017), there are meetings each called Gathering4Gardner .
- 9.
Tech. Note. Any physics teacher insists on writing f (t) = d, with both letters containing units (fuel tanks and miles). Most math teachers tend to keep the units implicit to focus more on the mathematical structure: f (1) = d. The physics teachers are correct.
- 10.
The term “acyclic” means a loop (cycle) is impossible, and “directed” means all links are one way. Here we use the terms nodes and links (arcs) instead of vertices and edges, and we use graph instead of network. Any acyclic directed graph may have its nodes labeled so that any link (i,j) [from node i to j] will have the property that i < j. Why? If the graph is acyclic, then there must be at least one node with no incoming links. Label that node 1 and remove all outgoing links from that node. Then the remaining network must have at least one node with no incoming links (for the same reason as before). Then repeat.
- 11.
Reference: R. Bellman and S. Dreyfus, Applied Dynamic Programming (Princeton, 1962), page 229, A Routing Problem.
- 12.
Tech. Note. This particular recursive functional equation (*) is called a Bellman equation or, more accurately, the Bellman equation for this problem. See Wikipedia, s.v. Bellman equation. Some problems in dynamic programming do not require a max or a min—e.g., the Fibonacci function, the jeep problem, and Problem 3, given later.
- 13.
I had forgotten to deconstruct the dictionary (dict) before the beginning of each call. Consequently, neighbor was always in dict after the first call. A Bellman equation never had to be evaluated for the final 999,999 calls. Oops!
- 14.
An early reference to this problem is Richard Bellman, Dynamic Programming (Princeton, 1957), page 45, problem 21. Bellman referred to loading cargo on a ship, not a knapsack. On page 117 of The Art and Theory of Dynamic Programming (Academic Press, 1977), the authors (Dreyfus and Law) imply that the knapsack problem is only the 0-1 version of the cargo-loading problem. Today entire books have been written about the cargo-loading problem and its variants.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2018 Michael Stueben
About this chapter
Cite this chapter
Stueben, M. (2018). Dynamic Programming. In: Good Habits for Great Coding. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-3459-4_21
Download citation
DOI: https://doi.org/10.1007/978-1-4842-3459-4_21
Published:
Publisher Name: Apress, Berkeley, CA
Print ISBN: 978-1-4842-3458-7
Online ISBN: 978-1-4842-3459-4
eBook Packages: Professional and Applied ComputingApress Access BooksProfessional and Applied Computing (R0)