A Poisson Parable: Bias in Linear Least Squares Estimation

• Wm. A. Wheaton
Conference paper

Abstract

A standard problem in high-energy astronomy data analysis is the decomposition of a set of I observed counts, ni, described by Poisson statistics, for i = 1,…, I, according to some known J-component linear model,
$$\overline {{n_i}} = E\left[ {{n_i}} \right] = \sum\limits_{j = 1}^J {{A_{ij}}{r_j}}$$
(41.1)

with underlying physical count rates r j , or fluxes which are to be estimated from the data, the A ij , being known experiment constants. This problem is often solved by Linear Least Squares (LLSQ), but limited to situations where the number of counts per bin i is not too small.

For the simplest possible case,J = 1, which is just a counting experiment with no background, it is interesting to attempt a direct application of the weighted average formula using$$\sqrt {{n_i}} \approx {\sigma _i} \cdot$$However, the resulting formula is completely wrong! Using, instead of the observed ni, the expected count,$$E\left[ {{n_i}} \right] = \sigma _i^2 = r{t_i}$$in the weighting, where t i is the observing time in bin i, it turns out that the unknown rate r cancels from the weighted average sums, and we recover the obviously correct estimate$$\hat r = {N \mathord{\left/{\vphantom {N T}} \right.\kern-\nulldelimiterspace} T} = {{\sum {{n_i}} } \mathord{\left/ {\vphantom {{\sum {{n_i}} } {\sum {{t_i}} }}} \right.\kern-\nulldelimiterspace} {\sum {{t_i}} }}$$

Keywords

Count Rate Poisson Estimation Counting Experiment Weighted Average Method True Count
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

1. [EDJ+71]
W. T. Eadie, D. Drijard. F. E. James, M. Roos. and B. Sadoulet. Statistical Methods in Experimental Physics. North-Holland, Amsterdam, 1971. See especially chapters 7 and 8.
2. [Leh59]
E. L. Lehmann. Testing Statistical Hypotheses. John Wiley and Sons, New York, 1959.
3. [WDJ+95]
Wm. A. Wheaton, Alfred D. Dunklee, Allen S. Jacobson, James C. Ling, William A. Mahoney. and Robert G. Radocinski. Multiparameter linear least squares fitting to poisson data one count at a time. The Astrophysical Journal, 438(322). jan 1995.