Abstract
Several nonlinear optimization code comparisons have been published, providing data for picking a code for a given application or developing new codes. The common performance measures (number of function evaluations, standardized computer time, number of problems solved) are intuitively machine independent, which encourages such use. Unfortunately, the relative performance of optimization codes does depend on the computer and compiler used for testing, and this dependence is evident regardless of the performance measure. In addition, relative performance measured on a single machine may depend significantly on the desired degree of accuracy, the choice of test problem(s), the chosen performance measure, and even the time of day (machine workload) when the tests are run. Numerical evidence of these difficulties is presented, based on tests of the same problem and algorithm decks on several different computers, with various compilers, problem sets, accuracy levels, and performance measures.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Colville, A.R., “A Comparative Study on Nonlinear Programming Codes,” IBM New York Scientific Center Report No. 320–2949, June 1968, summarized in Proceedings of the Princeton Symposium on Mathematical Programming, Kuhn, H.W., ed., Princeton, NJ, 1970, pp. 487–501.
Eason, E.D. and Fenton, R.G., “A Comparison of Numerical Optimization Methods for Engineering Design,” Journal of Engineering for Industry, Trans. ASME Series B, Vol. 96, No. 1, February 1974, pp. 191–196.
Stocker, D. C., A Comparative Study of Nonlinear. Programming Codes, MS Thesis, University of Texas at Austin, 1969.
Himmelblau, York, 1972. D.M., Applied Nonlinear Programming, McGraw-Hill, New
Sandgren, E., The Utility of Nonlinear Programming Algorithms, PhD Thesis, Purdue University, 1977.
Schittkowski, K., “A Numerical Comparison of 13 Nonlinear Program- ming Codes with Randomly Generated Test Problems,” Preprint, March 1979.
Eason, E.D., “Validity of Colville’s Time Standardization for Comparing Optimization Codes,” ASME Paper No. 77-DET-116, September 1977.
Dembo, R.S. and Mulvey, J.M., “On the Analysis and Comparison of Mathematical Programming Algorithms and Software,” Proceedings of the Bicentennial Conference on Mathematical Programming, Gaithersberg, Maryland, 1976.
Hillstrom, K.E., “A Simulation Test Approach to the Evaluation and Comparison of Unconstrained Nonlinear Optimization Algorithms,” Argonne National Laboratories Report No. ANL-76–20, 1976.
Nazareth, L. and Schlick, F., “The Evaluation of Unconstrained Optimization Routines,” Proceedings of the Bicentennial Conference on Mathematical Programming, Gaithersberg, Maryland, December 1976.
Dembo, R.S., “A Set of Geometric Programming Test Problems and Their Solutions,” Working Paper #87, Department of Management Sciences, University of Waterloo, Ontario, June 1974.
Hock, W. and Schittkowski, K., “Test Examples for the Solution of Nonlinear Programming Problems,” Parts 1 and 2, Preprints 44, 45, Institut fur Angewandte Mathematik und Statistik, Universitat Wurzburg, 1979.
Crowder, 11.P., Dembo, R.S., and Mulvey, J.M., “Reporting Computational Experiments in Mathematical Programming,” Mathematical Programming, Vol. 15, #3, November 1978, pp. 316–329.
Bell, T.E., “Computer Performance Variability,” Computer Performance Evaluation, National Bureau of Standards Special Publication 406, August 1975.
Eason, E.D. and Fenton, R.G., “Testing and Evaluation of Numerical Methods for Design Optimization,” University of Toronto Report UTME-TP 7204, September 1972.
Miele, A. and Gonzalez, S., “On the Comparative Evaluation of Algorithms for Mathematical Programming Problems,” presented at Nonlinear Programming Symposium - 3, Madison, Wisconsin, July 1977.
Curnow, H.J. and Wichmann, B.A., “A Synthetic Benchmark,” The Computer Journal, Vol. 19, No. 1, February 1976, pp. 43–49.
Schittkowski, K., Nonlinear Programming Codes - Information, Tests, Performance, Lecture Notes in Economics and Mathematical Systems, No. 183, Springer Verlag, New York, 1980.
Sandgren, E., “A Statistical Review of the Sandgren-Ragsdell Comparative Study,” Proceedings, U.S. Bureau of Standards/Mathematical Programming Sociéty Conference on Testing and Validating Algorithms and Software, Boulder, Colorado, January 1981.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1982 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Eason, E.D. (1982). Evidence of Fundamental Difficulties in Nonlinear Optimization Code Comparisons. In: Mulvey, J.M. (eds) Evaluating Mathematical Programming Techniques. Lecture Notes in Economics and Mathematical Systems, vol 199. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-95406-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-95406-1_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-11495-6
Online ISBN: 978-3-642-95406-1
eBook Packages: Springer Book Archive