Abstract
Dynamic programming is an important algorithm design technique. It is used for solving problems whose solutions involve recursively solving subproblems that share subsubproblems. While a straightforward recursive program solves common subsubproblems repeatedly and often takes exponential time, a dynamic programming algorithm solves every subsubproblem just once, saves the result, reuses it when the subsubproblem is encountered again, and takes polynomial time. This paper describes a systematic method for transforming programs written as straightforward recursions into programs that use dynamic programming. The method extends the original program to cache all possibly computed values, incrementalizes the extended program with respect to an input increment to use and maintain all cached results, prunes out cached results that are not used in the incremental computation, and uses the resulting incremental program to form an optimized new program. Incrementalization statically exploits semantics of both control structures and data structures and maintains as invariants equalities characterizing cached results. The principle underlying incrementalization is general for achieving drastic program speedups. Compared with previous methods that perform memoization or tabulation, the method based on incrementalization is more powerful and systematic. It has been implemented and applied to numerous problems and succeeded on all of them.
This work is supported in part by NSF under Grant CCR-9711253 and ONR under Grant N0014-99-1-0132.
Chapter PDF
Keywords
- Dynamic Programming
- Dynamic Programming Algorithm
- Original Program
- Recursive Call
- Functional Programming
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
M. Abadi, B. Lampson, and J.-J. Lévy. Analysis and caching of dependencies. In Proceedings of the 1996 ACM SIGPLAN International Conference on Functional Programming. ACM, New York, May 1996.
A. V. Aho, J. E. Hopcroft, and J. D. Ullman. The Design and Analysis of Computer Algorithms. Addison-Wesley, Reading, Mass., 1974.
F. L. Bauer, B. Möller, H. Partsch, and P. Pepper. Formal program construction by transformations — Computer-aided, intuition-guided programming. IEEE Trans. Softw. Eng., 15(2):165–180, Feb. 1989.
R. E. Bellman. Dynamic Programming. Princeton University Press, Princeton, New Jersey, 1957.
R. S. Bird. Tabulation techniques for recursive programs. ACM Comput. Surv., 12(4):403–417, Dec. 1980.
R. S. Bird. The promotion and accumulation strategies in transformational programming. ACM Trans. Program. Lang. Syst., 6(4):487–504, Oct. 1984.
R. S. Bird and O. de Moor. From dynamic programming to greedy algorithms. In B. Möller, H. Partsch, and S. Schuman, editors, Formal Program Development, volume 755 of Lecture Notes in Computer Science, pages 43–61. Springer-Verlag, Berlin, 1993.
E. A. Boiten. Improving recursive functions by inverting the order of evaluation. Sci. Comput. Program., 18(2):139–179, Apr. 1992.
R. M. Burstall and J. Darlington. A transformation system for developing recursive programs. J. ACM, 24(1):44–67, Jan. 1977.
W.-N. Chin. Towards an automated tupling strategy. In Proceedings of the ACM SIGPLAN Symposium on Partial Evaluation and Semantics-Based Program Manipulation, pages 119–132. ACM, New York, June 1993.
W.-N. Chin and M. Hagiya. A bounds inference method for vector-based memorization. In ICFP 1997 [23], pages 176–187.
W.-N. Chin and S.-C. Khoo. Tupling functions with multiple recursion parameters. In P. Cousot, M. Falaschi, G. Filé, and A. Rauzy, editors, Proceedings of the 3rd International Workshop on Static Analysis, volume 724 of Lecture Notes in Computer Science, pages 124–140. Springer-Verlag, Berlin, Sept. 1993.
N. H. Cohen. Eliminating redundant recursive calls. ACM Trans. Program. Lang. Syst., 5(3):265–299, July 1983.
T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. The MIT Press/McGraw-Hill, 1990.
S. Curtis. Dynamic programming: A different perspective. In R. Bird and L. Meertens, editors, Algorithmic Languages and Calculi, pages 1–23. Chapman & Hall, London, U.K., 1997.
O. de Moor. A generic program for sequential decision processes. In M. Hermenegildo and D. S. Swierstra, editors, Programming Languages: Implementations, Logics, and Programs, volume 982 of Lecture Notes in Computer Science, pages 1–23. Springer-Verlag, Berlin, 1995.
O. de Moor and J. Gibbons. Bridging the algorithm gap: A linear-time functional program for paragraph formatting. Technical Report CMS-TR-97-03, School of Computing and Mathematical Sciences, Oxford Brookes University, July 1997.
J. Field and T. Teitelbaum. Incremental reduction in the lambda calculus. In Proceedings of the 1990 ACM Conference on LISP and Functional Programming, pages 307–322. ACM, New York, June 1990.
D. P. Friedman, D. S. Wise, and M. Wand. Recursive programming through table look-up. In Proceedings of the 1976 ACM Symposium on Symbolic and Algebraic Computation, pages 85–89. ACM, New York, 1976.
Y. Futamura and K. Nogi. Generalized partial evaluation. In B. Bjø rner, A. P. Ershov, and N. D. Jones, editors, Partial Evaluation and Mixed Computation, pages 133–151. North-Holland, Amsterdam, 1988.
Z. Hu, H. Iwasaki, M. Takeichi, and A. Takano. Tupling calculation eliminates multiple data traversals. In ICFP 1997 [23], pages 164–175.
J. Hughes. Lazy memo-functions. In Proceedings of the 2nd Conference on Functional Programming Languages and Computer Architecture, volume 201 of Lecture Notes in Computer Science, pages 129–146. Springer-Verlag, Berlin, Sept. 1985.
Proceedings of the 1997 ACM SIGPLAN International Conference on Functional Programming. ACM, New York, June 1997.
R. M. Keller and M. R. Sleep. Applicative caching. ACM Trans. Program. Lang. Syst., 8(1):88–108, Jan. 1986.
H. Khoshnevisan. Efficient memo-table management strategies. Acta Informatica, 28(1):43–81, 1990.
Y. A. Liu. CACHET: An interactive, incremental-attribution-based program transformation system for deriving incremental programs. In Proceedings of the 10th Knowledge-Based Software Engineering Conference, pages 19–26. IEEE CS Press, Los Alamitos, Calif., Nov. 1995.
Y. A. Liu. Principled strength reduction. In R. Bird and L. Meertens, editors, Algorithmic Languages and Calculi, pages 357–381. Chapman & Hall, London, U.K., 1997.
Y. A. Liu. Dependence analysis for recursive data. In Proceedings of the IEEE 1998 International Conference on Computer Languages, pages 206–215. IEEE CS Press, Los Alamitos, Calif., May 1998.
Y. A. Liu and G. Gómez. Automatic accurate time-bound analysis for high-level languages. In Proceedings of the ACM SIGPLAN 1998 Workshop on Languages, Compilers, and Tools for Embedded Systems, volume 1474 of Lecture Notes in Computer Science, pages 31–40. Springer-Verlag, June 1998.
Y. A. Liu and S. D. Stoller. Loop optimization for aggregate array computations. In Proceedings of the IEEE 1998 International Conference on Computer Languages, pages 262–271. IEEE CS Press, Los Alamitos, Calif., May 1998.
Y. A. Liu, S. D. Stoller, and T. Teitelbaum. Discovering auxiliary information for incremental computation. In Conference Record of the 23rd Annual ACM Symposium on Principles of Programming Languages, pages 157–170. ACM, New York, Jan. 1996.
Y. A. Liu, S. D. Stoller, and T. Teitelbaum. Static caching for incremental computation. ACM Trans. Program. Lang. Syst., 20(3):546–585, May 1998.
Y. A. Liu and T. Teitelbaum. Systematic derivation of incremental programs. Sci. Comput. Program., 24(1):1–39, Feb. 1995.
D. Michie. “memo” functions and machine learning. Nature, 218:19–22, Apr. 1968.
D. J. Mostow and D. Cohen. Automating program speedup by deciding what to cache. In Proceedings of the 9th International Joint Conference on Artificial Intelligence, pages 165–172. Morgan Kaufmann Publishers, San Francisco, Calif., Aug. 1985.
R. Paige. Programming with invariants. IEEE Software, pages 56–69, Jan. 1986.
R. Paige. Symbolic finite differencing — Part I. In Proceedings of the 3rd European Symposium on Programming, volume 432 of Lecture Notes in Computer Science, pages 36–56. Springer-Verlag, Berlin, May 1990.
R. Paige and S. Koenig. Finite differencing of computable expressions. ACM Trans. Program. Lang. Syst., 4(3):402–454, July 1982.
H. A. Partsch. Specification and Transformation of Programs — A Formal Approach to Software Development. Springer-Verlag, Berlin, 1990.
A. Pettorossi. A powerful strategy for deriving efficient programs by transformation. In Conference Record of the 1984 ACM Symposium on LISP and Functional Programming. ACM, New York, Aug. 1984.
A. Pettorossi and M. Proietti. Rules and strategies for transforming functional and logic programs. ACM Comput. Surv., 28(2):360–414, June 1996.
A. Pettorossi and M. Proietti. Program derivation via list introduction. In R. Bird and L. Meertens, editors, Algorithmic Languages and Calculi. Chapman & Hall, London, U.K., 1997.
W. Pugh. An improved cache replacement strategy for function caching. In Proceedings of the 1988 ACM Conference on LISP and Functional Programming, pages 269–276. ACM, New York, July 1988.
W. Pugh. The Omega Test: A fast and practical integer programming algorithm for dependence analysis. Commun. ACM, 31(8), Aug. 1992.
W. Pugh and T. Teitelbaum. Incremental computation via function caching. In Conference Record of the 16th Annual ACM Symposium on Principles of Programming Languages, pages 315–328. ACM, New York, Jan. 1989.
P. W. Purdom and C. A. Brown. The Analysis of Algorithms. Holt, Rinehart and Winston, 1985.
T. Reps and T. Teitelbaum. The Synthesizer Generator: A System for Constructing Language-Based Editors. Springer-Verlag, New York, 1988.
W. L. Scherlis. Program improvement by internal specialization. In Conference Record of the 8th Annual ACM Symposium on Principles of Programming Languages, pages 41–49. ACM, New York, Jan. 1981.
D. R. Smith. KIDS: A semiautomatic program development system. IEEE Trans. Softw. Eng., 16(9):1024–1043, Sept. 1990.
D. R. Smith. Structure and design of problem reduction generators. In B. Möller, editor, Constructing Programs from Specifications, pages 91–124. North-Holland, Amsterdam, 1991.
M. Sniedovich. Dynamic Programming. Marcel Dekker, New York, 1992.
B. Wegbreit. Goal-directed program transformation. IEEE Trans. Softw. Eng., SE-2(2):69–80, June 1976.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Y.A., Stoller, S.D. (1999). Dynamic Programming via Static Incrementalization. In: Swierstra, S.D. (eds) Programming Languages and Systems. ESOP 1999. Lecture Notes in Computer Science, vol 1576. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49099-X_19
Download citation
DOI: https://doi.org/10.1007/3-540-49099-X_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65699-9
Online ISBN: 978-3-540-49099-9
eBook Packages: Springer Book Archive