# An evaluation of R^{2} as an inadequate measure for nonlinear models in pharmacological and biochemical research: a Monte Carlo approach

- 15k Downloads
- 120 Citations

## Abstract

### Background

It is long known within the mathematical literature that the coefficient of determination R^{2} is an inadequate measure for the goodness of fit in nonlinear models. Nevertheless, it is still frequently used within pharmacological and biochemical literature for the analysis and interpretation of nonlinear fitting to data.

### Results

The intensive simulation approach undermines previous observations and emphasizes the extremely low performance of R^{2} as a basis for model validity and performance when applied to pharmacological/biochemical nonlinear data. In fact, with the 'true' model having up to 500 times more strength of evidence based on Akaike weights, this was only reflected in the third to fifth decimal place of R^{2}. In addition, even the bias-corrected R^{2}_{adj} exhibited an extreme bias to higher parametrized models. The bias-corrected AICc and also BIC performed significantly better in this respect.

### Conclusion

Researchers and reviewers should be aware that R^{2} is inappropriate when used for demonstrating the performance or validity of a certain nonlinear model. It should ideally be removed from scientific literature dealing with nonlinear model fitting or at least be supplemented with other methods such as AIC or BIC or used in context to other models in question.

### Keywords

Nonlinear Model Bayesian Information Criterion Residual Variance Weibull Model Akaike Weight## Background

Fitting nonlinear models to data is frequently applied within all fields of pharmaceutical and biochemical assay quantification. A plethora of nonlinear models exist, and chosing the right model for the data at hand is a mixture of experience, knowledge about the underlying process and statistical interpretation of the fitting outcome. While the former are of somewhat individual nature, there is a need in quantifying the validity of a fit by some measure which discriminates a 'good' from a 'bad' fit. The most common measure is the coefficient of determination R^{2} used in linear regression when conducting calibration experiments for samples to be quantified [1]. In the linear context, this measure is very intuitive as values between 0 and 1 give a quick interpretation of how much of the variance in the data is explained by the fit. Although it is known now for some time that R^{2} is an inadequate measure for nonlinear regression, many scientists and also reviewers insist on it being supplied in papers dealing with nonlinear data analysis. Several initial and older descriptions for R^{2} being of no avail in nonlinear fitting had pointed out this issue but have probably fallen into oblivion [2, 3, 4, 5, 6, 7, 8]. This observation might be due to differences in the mathematical background of trained statisticians and biochemists/pharmacologists who often apply statistical methods but lack detailed statistical insight.

We made the observation that R^{2} is still frequently being used in the context of performance or validity of a certain model when fit to nonlinear data. R^{2} is not an optimal choice in a nonlinear regime as the the total sum-of-squares (TSS) is not equal to the regression sum-of-squares (REGSS) plus the residual sum-of-squares (RSS), as is the case in linear regression, and hence it lacks the above interpretation (see Additional File 1, paragraphs 1 & 2). To our observation, there is still a high occurrence in the present literature of all biomedical fields where the validity of nonlinear models is based solely on R^{2} values, which might be a result of authors or reviewers not being aware of this fallacy. Additionally, almost all of the commercially available statistical software packages (i.e. Prism, Origin, Matlab, SPSS, SAS) calculate R^{2} values for nonlinear fits, which is bound to unintentionally corroborate its frequent use. A further example is the TableCurve2 D software (Systat, USA) which can fit hundreds of nonlinear models to a given dataset automatically and then rank these by means of R^{2}. Noted 25 years ago by Kvalseth [8], the user is usually not able to identify which of the eight different definitions of R^{2} that are commonly being used in the literature is chosen for the analysis output in statistical software (see Additional File 1, Remark 4).

We thus aimed to point out the low performance of R^{2} and its inappropriateness for nonlinear data analysis by basing our analysis on an extensive Monte Carlo simulation approach. This approach has fundamental advantages in the analysis of nonlinear data analysis [9] and can reveal tendencies within statistical methods by supplying the models and measures in question with thousands of generated datasets.

## Methods

### Creation of the 'true' model

In a first step, we fitted a three-parameter log-logistic model (L3, see Formula 3 below) by nonlinear least-squares to sigmoidal data that was taken from quantitative real-time polymerase chain reaction (qPCR). This yielded a sigmoidal model with the parameters b = -9.90, d = 11.07 and e = 24.75. We used the fitted values of this model and x-values from 10-35 as the 'true' model with sample size n = 26 for the following Monte Carlo simulation. This essentially gave a sigmoidal curve that can be encountered in many different areas of pharmacological/biochemical analysis. Specific to qPCR data, the x-values ("Cycles") are equidistant and not on a log-scale, as often encountered in dose-response analysis. For mathematical details, see Remark 7 in Additional File 1.

### Perturbation of data (Monte Carlo Simulation)

Using the fitted values as above, all datapoints were perturbed 2000 times by adding six different magnitudes (very low to high) of homoscedastic noise from a gaussian distribution with mean = 0 and standard deviations = 0.01, 0.02, 0.05, 0.1, 0.2 or 0.4. The noise of the data was therefore between 0.1% and 4% of the data range. This way, for each of six different settings (determined by different standard deviations), we obtained 2000 new data sets of sample size n = 26 with true model L3. For each of these data sets, nine different sigmoidal models differing in model type and number of parameters (Formulas 1-9) were fit. For mathematical details, see Remark 7 in Additional File 1.

**Formula 1:**Five parameter log-logistic model (L5):

**Formula 2:**Four-parameter log-logistic model (L4):

**Formula 3:**Three-parameter log-logistic model (L3):

**Formula 4:**Five parameter logistic model (B5):

**Formula 5:**Four-parameter logistic model (B4):

**Formula 6:**Three-parameter logistic model (B3):

**Formula 7:**Four-parameter Weibull model (W4):

**Formula 8:**Three-parameter Weibull model (W3):

**Formula 9:**Five-parameter baroreflex model (baro5):

### Calculation of measures for goodness-of-fit

For each simulation, we calculated the following measures for goodness-of-fit:

R^{2}, using the most general definition [5, 8]:

**Formula 10:**

with *RSS* = residual sum-of-squares, *TSS* = total sum-of-squares, *y* = response values, Open image in new window = fitted values and Open image in new window = the mean of response values. For a more detailed description see Remarks 1-6 in Additional File 1.

We chose to use the adjusted R^{2} to compensate for possible bias due to different number of parameters:

**Formula 11:**

with *n* = sample size and *p* = number of parameters.

The Akaike Information Criterion (AIC, [10, 11, 12]), a measure that is widely accepted for measuring the validity within a cohort of nonlinear models and frequently used for model selection [13].

**Formula 12:**

with *p* = number of parameters and ln(*L)* = maximum log-likelihood of the estimated model. The latter, in the case of a nonlinear fit with normally distributed errors [13], is calculated by

**Formula 13:**

with *x*_{ 1 }, ..., *x*_{ n } = the residuals from the nonlinear least-squares fit and *N* = their number.

To provide a fair playing ground, we employed an AIC variant that corrects for small sample sizes, the bias-corrected AIC (AICc):

**Formula 14:**

with *n* = sample size and *p* = number of parameters.

In order to obtain values for the validity of a fit, we used Akaike weights which calculate the *weight of evidence* for each model within a cohort of models in question [12, 13, 14]:

**Formula 15:**

with *i*, *k* = model numbers, Δ_{i}(*AIC*) = the difference in AIC of each model in comparison to the model with the lowest AIC, subsequently normalized to their sum (denominator).

Also here, we used the bias-corrected AICc for calculating the Akaike weights.

We also chose to employ the Bayesian Information Criterion (BIC), which gives a higher penalty on the number of parameters [15]:

**Formula 16:**

with *p* = number of parameters, *n* = sample size and *L* = maximum likelihood of the estimated model.

Furthermore, the residual variance as the part of the variance that cannot be accounted for by the model:

**Formula 17:**

with *RSS* = residual sum-of-squares, *n* = sample size and *p* = number of parameters.

**Formula 18:**

where *y*_{ i } = response values, *f(x*_{ i }*)* = the fitted values and Open image in new window = the uncertainty in the individual measurements *y*_{ i }. We further define the reduced chi-square as a useful measure [16] by

**Formula 19:**

with *ν* = *n - p* (degrees of freedom). If the fitting function is a good approximation to the parent function, then the variances of both should agree well, and the reduced chi-square should be approximately unity. If the reduced chi-square is much larger than 1 (i.e. 10 or 100), it means that one is either overly optimistic about the measurement errors or that one selected an inappropriate fitting function. If reduced chi-square is too small (i.e. 0.1 or 0.01) it may mean that one has been too pessimistic about measurement errors. For this work, models were selected based on reduced chi-square by being closest to 1.

### Analysis of the simulation data

Two different approaches were pursued within the Monte Carlo simulated data. To reveal general tendencies, we averaged the values of R^{2}_{adj}, AICc, BIC, residual variance and reduced chi-square from all 2000 simulated data sets (for each noise magnitude and each of the 9 models). To permit a more detailed insight for each simulation and to compare the measures on the single model level, we selected the best model in each iteration based on the highest R^{2}_{adj}, lowest AICc, lowest BIC, lowest residual variance and smallest difference to unity for reduced chi-square. For the latter, we used as the measurement uncertainty Open image in new window the *a priori* known squared standard deviation from the homoscedastic noise that was applied to produce the random data of the Monte Carlo simulations.

Finally, we calculated the percentage of selecting the 'true' model L3 in all iterations.

### Code for the simulations

All simulations were conducted using *R*, a well reputed and open-source statistical programming language [17]. Nonlinear curve fitting was done by using functionality from the R package *qpcR* [18]. The commented code for the simulations can be obtained from Additional File 2.

## Results and Discussion

^{2}

_{adj}were compared by averaging the output of all 2000 simulations and for three different magnitudes (low, middle, high) of homoscedastic gaussian noise (Figure 3). Not surprisingly, the R

^{2}

_{adj}values decrease and the AICc values increase with higher noise (left to right). But the major problem in the use of R

^{2}is clearly evident within the simulation setup: the AIC differences (delta-AIC) between some models can be 78 (compare L3 and B3 models at s.d. = 0.02), which when transferred to Akaike weights result in a

*weight of evidence*16 (!) orders of magnitude in favor of L3. One might be inclined to say that this is major evidence for the first model being in favor of the second one, but in respect to the corresponding R

^{2}

_{adj}values, only the fourth decimal place is affected. This tendency is also found for higher noise, despite AICc values increasing and R

^{2}

_{adj}values decreasing. Even in the presence of relatively high noise (s.d. = 0.4), at least for the simulation setup in this work, R

^{2}

_{adj}would hardly drop below 0.99. Comparing the Akaike weight of the models with the R

^{2}

_{adj}values in Table 1 shows the strong discrepancy in scale changes of these two measures when comparing a cohort of different models.

Summary of the Monte-Carlo simulation study.

s.d = 0.01 | Model | R | AICc | Akaike weights | BIC | resVar | red. Chi |
---|---|---|---|---|---|---|---|

| 0.99999471 | -156.22 | 0.0818 | -156.35 | 0.00009933 | 0.9924 | |

| 0.99999471 | -158.36 | 0.2389 | -158.33 | 0.00009935 | 0.9930 | |

| 0.99999470 | -160.20 | 0.5969 | -160.33 | 0.00009959 | 0.9927 | |

| 0.99999258 | -147.31 | 0.0010 | -147.44 | 0.00013945 | 1.4739 | |

| 0.99975702 | -57.72 | 0.0000 | -57.69 | 0.00456696 | 42.5939 | |

| 0.99959911 | -47.29 | 0.0000 | -47.42 | 0.00734455 | 63.5805 | |

| 0.99853046 | -10.89 | 0.0000 | -10.85 | 0.02762048 | 282.3939 | |

| 0.99829856 | -7.91 | 0.0000 | -8.04 | 0.03336157 | 334.1353 | |

| 0.99999471 | -156.21 | 0.0814 | -156.34 | 0.00009936 | 0.9927 | |

| | 0.99997869 | -119.98 | 0.0774 | -120.11 | 0.00040059 | 0.9946 |

| 0.99997868 | -122.13 | 0.2273 | -122.10 | 0.00040068 | 0.9959 | |

| 0.99997871 | -124.06 | 0.5954 | -124.19 | 0.00040007 | 0.9957 | |

| 0.99997653 | -117.49 | 0.0224 | -117.62 | 0.00044108 | 1.1135 | |

| 0.99974038 | -56.14 | 0.0000 | -56.11 | 0.00487955 | 11.4098 | |

| 0.99958227 | -46.30 | 0.0000 | -46.43 | 0.0076527 | 16.6958 | |

| 0.99851513 | -10.64 | 0.0000 | -10.61 | 0.02790803 | 71.2971 | |

| 0.99828395 | -7.71 | 0.0000 | -7.84 | 0.03364791 | 84.1649 | |

| 0.99997869 | -119.98 | 0.0775 | -120.11 | 0.00040044 | 0.9956 | |

| | 0.99986676 | -72.33 | 0.0765 | -72.46 | 0.00250459 | 0.9897 |

| 0.99986656 | -74.44 | 0.2194 | -74.41 | 0.00250829 | 0.9904 | |

| 0.99986662 | -76.34 | 0.5674 | -76.47 | 0.00250720 | 0.9882 | |

| 0.99986439 | -71.87 | 0.0608 | -72.00 | 0.00254924 | 1.0096 | |

| 0.99962966 | -47.44 | 0.0000 | -47.40 | 0.00696139 | 2.6343 | |

| 0.99946832 | -40.44 | 0.0000 | -40.57 | 0.00973888 | 3.4859 | |

| 0.99839915 | -8.85 | 0.0000 | -8.81 | 0.03009194 | 12.3020 | |

| 0.99817327 | -6.21 | 0.0000 | -6.34 | 0.03582530 | 14.3447 | |

| 0.99986669 | -72.32 | 0.0759 | -72.45 | 0.00250592 | 0.9902 | |

| | 0.99947371 | -36.57 | 0.0742 | -36.70 | 0.00989448 | 0.9984 |

| 0.99947362 | -38.73 | 0.2180 | -38.70 | 0.00989607 | 0.9972 | |

| 0.99947374 | -40.62 | 0.5618 | -40.75 | 0.00989674 | 0.9972 | |

| 0.99947135 | -36.46 | 0.0701 | -36.59 | 0.00993888 | 1.0037 | |

| 0.99923746 | -29.03 | 0.0017 | -28.99 | 0.01433573 | 1.4052 | |

| 0.99907025 | -26.35 | 0.0004 | -26.48 | 0.01703305 | 1.6200 | |

| 0.99800791 | -3.50 | 0.0000 | -3.47 | 0.03745226 | 3.8282 | |

| 0.99779177 | -1.55 | 0.0000 | -1.68 | 0.04333951 | 4.3370 | |

| 0.99947355 | -36.56 | 0.0737 | -36.69 | 0.00989740 | 0.9982 | |

| | 0.99786138 | -0.07 | 0.0675 | -0.20 | 0.04025347 | 0.9948 |

| 0.99785879 | -2.18 | 0.1942 | -2.15 | 0.04030142 | 0.9940 | |

| 0.99785563 | -4.04 | 0.4930 | -4.17 | 0.04035658 | 0.9928 | |

| 0.99785959 | -0.05 | 0.0668 | -0.18 | 0.04028754 | 0.9960 | |

| 0.99762513 | 0.49 | 0.0512 | 0.52 | 0.04470201 | 1.1027 | |

| 0.99740761 | 0.20 | 0.0592 | 0.06 | 0.04751535 | 1.1543 | |

| 0.99640798 | 11.41 | 0.0002 | 11.44 | 0.06760434 | 1.6764 | |

| 0.99625943 | 11.71 | 0.0002 | 11.58 | 0.07350697 | 1.8059 | |

| 0.99786149 | -0.07 | 0.0676 | -0.20 | 0.04025146 | 0.9941 | |

| | 0.99160836 | 35.58 | 0.0490 | 35.45 | 0.15887711 | 0.9987 |

| 0.99157911 | 33.50 | 0.1387 | 33.53 | 0.15941969 | 0.9981 | |

| 0.99158878 | 31.58 | 0.3613 | 31.45 | 0.15928956 | 0.9980 | |

| 0.99154493 | 35.68 | 0.0466 | 35.55 | 0.15991031 | 1.0001 | |

| 0.99135309 | 34.16 | 0.0996 | 34.19 | 0.16370200 | 1.0242 | |

| 0.99098926 | 32.68 | 0.2084 | 32.55 | 0.16621846 | 1.0372 | |

| 0.99017401 | 37.58 | 0.0180 | 37.61 | 0.18602148 | 1.1663 | |

| 0.99028174 | 36.53 | 0.0305 | 36.40 | 0.19208006 | 1.1995 | |

| 0.99159379 | 35.62 | 0.0480 | 35.49 | 0.15915529 | 0.9989 |

However, differences in scale changes would be unimportant if the direction of change is always reciprocally (i.e. a lower AICc always corresponding with a higher R^{2}_{adj}). As can be deduced from Table 1, this is not always the case. Using the averaged measures from the simulations, R^{2}_{adj} and AICc did not always behave reciprocally. For instance, at s.d. = 0.05, 0.2 and 0.4 the 'true' model L3 has a lower AICc than L5 (and ~ 8 times more *weight of evidence* by Akaike weights), but R^{2}_{adj} values are also lower in the fifth to eighth decimal place. Using R^{2}_{adj} for model selection would therefore have resulted in a model which is clearly not in favor based on Akaike weights.

On the averaged values, BIC essentially shows the same characteristics as AICc. Interestingly, the *residual variance* is higher in the 'true' model L3 than L5 for most noise regimes (although also only in the third to eighth decimal place), indicating this to be a relatively unfavorable measure. Likewise, the reduced chi-square exhibited a tendency to be closest to unity for higher parameter models (L4, L5, B5) with increasing noise, here also only affected in the third decimal place.

^{2}

_{adj}, residual variance (resVar) and reduced chi-square performed not optimally, selecting the true model L3 only in 28-43% of the iterations. Both measures exhibited a severe bias in the selection of models with a higher number of parameters (L4, L5, B5, baro5). It is interesting to note that although R

^{2}

_{adj}and the residual variance 'correct' against the number of parameters, there is no positive effect on the ability to select the 'true' model. This may be due largely to the setup in this work which features a relatively high sample size (

*n*= 26) compared to the number of parameters (p = 3-5) and leaves the denominator

*n - p*in both measures relatively unaffected. In contrast, both AICc and BIC performed superior in the selection of the 'true' model L3 at these magnitudes of noise with over 80% of all iterations, but with a slight bias to models with a lower number of parameters at medium noise. At high noise (s.d. = 0.4, corresponding to 4% noise of the data range) the performance of all measures decreased markedly, most probably from the effect of the simulated data losing the structural features typical of the L3 model when high noise is added. Despite this negative trend, AICc and BIC displayed increased performance even at high noise. The two measures selected also a significant number of iterations (about 40%) of other models with the same number of parameters (B3, W3), which might be an indication for a small bias to lower parametrized models. Consequently, and based on the analysis of a sigmoidal nonlinear setup as described here, we feel compelled to give the following summary:

Model selection frequency for the different measures of goodness-of-fit.

Model | R | AICc | BIC | resVar | red. Chi | |
---|---|---|---|---|---|---|

| | 228 | 40 | 41 | 231 | 227 |

| 341 | 170 | 154 | 347 | 261 | |

| 867 | 1648 | 1660 | 857 | 567 | |

| 305 | 90 | 91 | 304 | 689 | |

| ||||||

| ||||||

| ||||||

| ||||||

| 259 | 52 | 54 | 261 | 256 | |

| 43.4 | 82.4 | 83.0 | 42.9 | 28.4 | |

| | 124 | 10 | 10 | 122 | 118 |

| 349 | 161 | 145 | 365 | 288 | |

| 958 | 1668 | 1684 | 939 | 576 | |

| 279 | 30 | 30 | 283 | 308 | |

| 35 | 53 | 51 | 34 | 361 | |

| 5 | 43 | 44 | 6 | 144 | |

| ||||||

| ||||||

| 250 | 35 | 36 | 251 | 205 | |

| 47.9 | 83.4 | 84.2 | 47.0 | 28.8 | |

| | 42 | 5 | 6 | 43 | 69 |

| 234 | 72 | 65 | 243 | 187 | |

| 698 | 982 | 989 | 667 | 374 | |

| 55 | 3 | 3 | 61 | 67 | |

| 257 | 81 | 76 | 194 | 208 | |

| 303 | 575 | 582 | 406 | 335 | |

| 51 | 46 | 43 | 89 | 305 | |

| 221 | 226 | 226 | 145 | 357 | |

| 139 | 10 | 10 | 152 | 98 | |

| 34.9 | 49.1 | 49.5 | 33.4 | 18.7 |

- 1)
The use of highly inferior nonlinear models is reflected only in the third or fourth decimal place of R

^{2}and thus the description of single models when using R^{2}is not meaningful, as this measure tends to be uniformly high when a set of models is inspected. This has also been noted by others [20]. Additionally, R^{2}and even its 'bias-corrected' counterpart R^{2}_{adj}are severely biased in favor of models with more parameters when it comes to model selection. The same accounts for the residual variance, which is also commonly used. AICc and BIC do not exhibit this bias and provide a much clearer picture and improved performance when it comes to selecting the 'true' model. The Akaike weights are especially useful in obtaining an overview of the*weight of evidence*of one model over the other, which is impossible with the*per se*high R^{2}values. This approach requires anyhow that it is mandatory to supply several models in question. - 2)
In a background of low and medium experimental noise, R

^{2}_{adj}and AICc selected different models with AICc selecting the 'true model' twice as often (82.2% versus 43%). This finding emphasizes the importance of this measure in nonlinear model selection. At a high noise level rarely encountered in the modelling of pharmacological/biochemical data (4% of data range), AICc still performed superior to R^{2}_{adj}. These results tempt us to conclude that the*degree of freedom*term*n - p*in the denominator of the residual variance and R^{2}_{adj}is not sufficient alone to compensate the effect of the number of increasing parameters. The same seems to be the case for the reduced chi-square, which is also frequently used for model selection purposes.

In this work we show that R^{2} is an inappropriate measure when used in the field of nonlinear fitting. Efforts have been made to develop R^{2}-like measures for the most common nonlinear regression models [21], but here we focused on the inadequateness of its use by using a data perturbation approach and comparing its performance in comparison to AICc, BIC, residual variance and reduced chi-square. Model selection in nonlinear statistical literature is usually divided into the frequentist methods, for example F-tests on the residual variance that are restricted to nested models [13], or measures from information theory such as AIC which are often used to compare non-nested models. Indeed, it has been shown that the latter approach can often perform better than F-tests [22]. In the field of biochemical and pharmacological literature there is a reasonably high occurrence in the use of R^{2} as the basis of arguing against or in favor of a certain model. As a result from this work, we would like to advocate that R^{2} should not be reported or demanded in phamacological/biochemical literature when discussing nonlinear data analysis. Authors as well as reviewers should be aware that demonstrating the validity of single nonlinear models by using solely R^{2} is not state-of-the-art and should be replaced or supplemented by AIC/AICc/BIC values (or their corresponding *weights of evidence*) of several possible models that are in question. The latter will give the dedicated reader the possibility to obtain information about how much a certain model is in favor over others, a feature that will not be evident by minor changes in R^{2} which tends to be uniformly high and is rarely affected more than in the third or fourth decimal place.

## Conclusions

Although frequently being used in the present pharmocological/biochemical literature for describing the validity of a nonlinear fit, R^{2} is an unfavorable measure that is rarely affected more than in the third or fourth decimal place, even in scenarios with highly inferior models. Our Monte Carlo simulations have shown that AIC, AICc or BIC perform significantly better in this respect so that authors as well as reviewers should be aware of this issue and refrain from using or asking for R^{2} values when nonlinear models are under investigation.

## Notes

### Acknowledgements

ANS is supported by grant Sp721/1-3 of from the German Research Foundation (DFG). We thank Dr. Heike Cappallo-Obermann for fruitful discussions and Dr. Caroline Feig for supplying the quantitative real-time PCR data. Many thanks also to the four reviewers whose comments helped to improve the manuscript substantially.

## Supplementary material

### References

- 1.Montgomery DC, Peck EA, Vining GG: Introduction to Linear Regression Analysis. 2006, Wiley & Sons, HobokenGoogle Scholar
- 2.Juliano SA, Williams FM: A comparison of methods for estimating the functional response parameters of the random predator equation. J Anim Ecol. 1987, 56: 641-653. 10.2307/5074.CrossRefGoogle Scholar
- 3.Miaou SP, Lu A, Lum HS: Pitfalls of using r-squared to evaluate goodness of fit of accident prediction models. Transportation Research Record #1542. 1996, 6-13. 10.3141/1542-02.Google Scholar
- 4.Willet JB, Singer JD: Another cautionary note about R-square: Its use in weighted least-squares regression analysis. Amer Stat. 1988, 42: 236-238. 10.2307/2685031.Google Scholar
- 5.Anderson-Sprecher R: Model comparisons and R-square. Amer Stat. 1994, 48: 113-117. 10.2307/2684259.Google Scholar
- 6.Nagelkerke NJD: A note on a general definition of the coefficient of determination. Biometrika. 1991, 78: 691-692. 10.1093/biomet/78.3.691.CrossRefGoogle Scholar
- 7.Magee L: R2 measures based on Wald and likelihood ratio joint singnificance tests. Amer Stat. 1990, 44: 250-253. 10.2307/2685352.Google Scholar
- 8.Kvalseth TO: Cautionary note about R2. Amer Stat. 1985, 39: 279-285. 10.2307/2683704.Google Scholar
- 9.Bates DM, Watts DG: Nonlinear regression analysis and its applications. 1988, New York, John Wiley & Sons, 1CrossRefGoogle Scholar
- 10.Akaike H: Information theory and an extension of the maximum likelihood principle. Proc 2nd Inter Symposium on Information Theory. 1973, 267-281.Google Scholar
- 11.Akaike H: A new look at the statistical model identification. IEEE Transaction on Automatic Control. 1974, 19: 716-723. 10.1109/TAC.1974.1100705.CrossRefGoogle Scholar
- 12.Akaike H: On the likelihood of a time series model. The Statistician. 1978, 27: 217-235. 10.2307/2988185.CrossRefGoogle Scholar
- 13.Burnham KP, Anderson DR: Model selection and inference: a practical information-theoretic approach. 2003, Springer Verlag, New York, USA, 2Google Scholar
- 14.Wagenmakers EJ, Farrell S: AIC model selection using Akaike weights. Psych Bull & Review. 2004, 11: 192-196.CrossRefGoogle Scholar
- 15.Schwarz G: Estimating the dimension of a model. Ann Stat. 1978, 6: 461-464. 10.1214/aos/1176344136.CrossRefGoogle Scholar
- 16.Bevington PR, Robinson DK: Data Reduction and Error Analysis for the Physical Sciences. 2003, McGraw-Hill, New York, Chapter 11: 3Google Scholar
- 17.The R project homepage. [http://www.r-project.org]
- 18.Ritz C, Spiess AN:
*qpcR*: an*R*package for sigmoidal model selection in quantitative real-time polymerase chain reaction analysis. Bioinformatics. 2008, 24: 1549-1551. 10.1093/bioinformatics/btn227.CrossRefPubMedGoogle Scholar - 19.Gottschalk PG, Dunn JR: The five-parameter logistic: A characterization and comparison with the four-parameter logistic. Anal Biochem. 2005, 343: 54-65. 10.1016/j.ab.2005.04.035.CrossRefPubMedGoogle Scholar
- 20.Zeng QC, Zhang E, Tellinghuisen J: Univariate calibration by reversed regression of heteroscedastic data: a case study. The Analyst. 2008, 133: 1649-1655. 10.1039/b808667b.CrossRefPubMedGoogle Scholar
- 21.Cameron AC, Windmeijer FAG: An R-squared measure of goodness of fit for some common nonlinear regression models. J Econometrics. 1997, 77: 329-342. 10.1016/S0304-4076(96)01818-0.CrossRefGoogle Scholar
- 22.Glatting G, Kletting P, Reske SN, Hohl K, Ring C: Choosing the optimal fit function: comparison of the Akaike Information Criterion and the f-test. Med Phys. 2007, 34: 4285-4292. 10.1118/1.2794176.CrossRefPubMedGoogle Scholar

## Copyright information

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.