Smooth Numerical Solutions of Ordinary Differential Equations

Abstract

When the function evaluation in parameter estimation by error norm minimization or in a range of related optimization problems involves the numerical solution of differential equations, the smoothness of that solution with respect to parameter changes is critical to the behavior of the minimization code (which might be trying to estimate partial derivatives numerically). If fixed stepsize, fixed order methods are used for integration, it is easy to see that the integrands have the desired smoothness, but such methods are not always efficient or even possible. Modern automatic integrators have very poor smoothness properties-some-times not even continuous. If the solution of y′ = f(y, t, p), y(0) =y₀(p) is denoted by y(t, p) where p is a parameter, and the numerical solution from a code using tolerance ε is y(t, p;ε), we hope that

$$||\;{\rm{y(t,p)}}\;{\rm{ - }}\;{\rm{y(t,p;\varepsilon )}}\;{\rm{||}}\;{\rm{ = }}\;{\rm{0(\varepsilon )}}$$

In a “smooth“ method we also want

$$||\;{{{\partial ^{\rm{S}}}{\rm{y}}\left( {{\rm{t,p}}} \right)} \over {\partial {{\rm{p}}^{\rm{S}}}}}\; - \;{{{\partial ^{\rm{S}}}{\rm{y}}\left( {{\rm{t,p;\varepsilon }}} \right)} \over {\partial {{\rm{p}}^{\rm{S}}}}}\;||\; \leqq \;0(\delta )$$

In practice, the second term is computed numerically. If the integrator is not smooth, we are forced to use ε= 0(δS(Δp)^S) which can be very expensive. In this paper we examine methods for which the above inequality can hold with = 5. Preliminary experiments with Runge-Kutta like codes show some promise.