Three Numerical Reproducibility Issues That Can Be Explained as Round-Off Error

Abstract

We explore an application from the author’s work in neuroscience. A code used to investigate neural development modeled 100 neurons with all-to-all excitatory connectivity. We used a simple ordinary differential equation system to model each neuron, and this 100-neuron model was used to produce a paper published in the Journal of Neurophysiology. Later a colleague used our code to continue this work, and found he could not reproduce our results. This lead us to thoroughly investigate this code and we discovered that it offered many different ways to thwart reproducibility that could be explained by round-off error arising from floating-point arithmetic.

Numerical reproducibility is considered a task that directly follows from the determinism in computations. However, reproducibility has become an intense concern and issue for research. We will show how this particular code provides a lack of reproducibility from the following three mechanisms: (i) the introduction of floating-point errors in an inner product; (ii) introduction of floating-point errors at each an increasing number of time steps during temporal refinement (ii); and (iii) differences in the output of library mathematical functions at the level of round-off error. This code’s sensitivity makes it a very powerful tool to explore many different manifestations of numerical reproducibility. However, this code is by no means exceptional, as in neuroscience these types of models are used extensively to gain insights on the functioning of the nervous system. In addition, these types of models are widely used in many other fields of study as they just nonlinear evolution equations.

I wish to thank Wilfredo Blanco Figuerola from the Universidade do Estado do Rio Grande do Norte, Brazil, who brought the behavior of this code to my attention.