Advertisement

Some new statistical methods for a class of zero-truncated discrete distributions with applications

  • Guo-Liang Tian
  • Xiqian Ding
  • Yin LiuEmail author
  • Man-Lai Tang
Original paper
  • 21 Downloads

Abstract

Counting data without zero category often occurs in various fields. A class of zero-truncated discrete distributions such as the zero-truncated Poisson, zero-truncated binomial and zero-truncated negative-binomial distributions are proposed in literature to model such count data. In this paper, three main contributions have been made for better studying the zero-truncated discrete distributions: First, a novel unified expectation–maximization (EM) algorithm is developed for calculating the maximum likelihood estimates (MLEs) of parameters in general zero-truncated discrete distributions and an important feature of the proposed EM algorithm is that the latent variables and the observed variables are independent, which is unusual in general EM-type algorithms; Second, for those who do not understand the principle of latent variables, a unified minorization–maximization algorithm, as an alternative to the EM algorithm, for obtaining the MLEs of parameters in a class of zero-truncated discrete distributions is discussed; Third, a unified method is proposed to derive the distribution of the sum of i.i.d.zero-truncated discrete random variables, which has important applications in the construction of the shortest Clopper–Pearson confidence intervals of parameters of interest and in the calculation of the exact p value of a two-sided test for small sample sizes in one sample problem.

Keywords

EM algorithm MM algorithm Shortest confidence intervals Stochastic representation Zero-truncated discrete models 

Notes

Acknowledgements

Yin Liu’s research was fully supported by Grants (Nos. 11601524 and 61773401) from the National Natural Science Foundation of China and a Grant (31721811206) from the Young Teachers Innovation Project of Zhongnan University of Economics and Law. Guo-Liang Tian’s research was fully supported by a Grant from the National Natural Science Foundation of China (No. 11771199). The work of Man-Lai Tang was supported by the Research Fund of the Project (Grant No.UGC/FDS14/P06/17) and National Natural Science Foundation of China (Grant No. 11871124).

Supplementary material

180_2018_860_MOESM1_ESM.r (1 kb)
Supplementary material 1 (R 0 KB)
180_2018_860_MOESM2_ESM.r (1 kb)
Supplementary material 2 (R 0 KB)
180_2018_860_MOESM3_ESM.r (7 kb)
Supplementary material 3 (R 6 KB)
180_2018_860_MOESM4_ESM.r (7 kb)
Supplementary material 4 (R 7 KB)
180_2018_860_MOESM5_ESM.r (1 kb)
Supplementary material 5 (R 0 KB)
180_2018_860_MOESM6_ESM.r (3 kb)
Supplementary material 6 (R 3 KB)
180_2018_860_MOESM7_ESM.r (1 kb)
Supplementary material 7 (R 1 KB)
180_2018_860_MOESM8_ESM.r (2 kb)
Supplementary material 8 (R 2 KB)
180_2018_860_MOESM9_ESM.r (2 kb)
Supplementary material 9 (R 1 KB)
180_2018_860_MOESM10_ESM.r (1 kb)
Supplementary material 10 (R 1 KB)
180_2018_860_MOESM11_ESM.r (1 kb)
Supplementary material 11 (R 1 KB)
180_2018_860_MOESM12_ESM.r (10 kb)
Supplementary material 12 (R 9 KB)
180_2018_860_MOESM13_ESM.r (10 kb)
Supplementary material 13 (R 9 KB)

References

  1. Ahuja JC (1970) On the distribution of sum of independent positive binomial variables. Can Math Bull 13(1):151–152MathSciNetzbMATHGoogle Scholar
  2. Ahuja JC (1971) Distribution of the sum of independent decapitated negative binomial variables. Ann Math Stat 42(1):383–384zbMATHGoogle Scholar
  3. Ahuja JC (1972) Convolution of the truncated Poisson distribution. Am Stat 26(5):46Google Scholar
  4. Best DJ, Rayner JCW, Thas O (2007) Goodness of fit for the zero-truncated Poisson distribution. J Stat Comput Simul 77(7):585–591MathSciNetzbMATHGoogle Scholar
  5. Bliss CI, Fisher RA (1953) Fitting the negative binomial distribution to biological data. Biometrics 9(2):176–200MathSciNetGoogle Scholar
  6. Cacoullos T (1961) A combinatorial derivation of the distribution of the truncated Poisson sufficient statistic. Ann Math Stat 32(3):904–905MathSciNetzbMATHGoogle Scholar
  7. Cacoullos T (1972) On the convolution of Poisson distributions with the zero class missing. Am Stat 26(2):44Google Scholar
  8. Cohen AC Jr (1954) Estimation of the Poisson parameter from truncated samples and from censored samples. J Am Stat Assoc 49:158–168MathSciNetzbMATHGoogle Scholar
  9. Cohen AC Jr (1960) Estimating the parameter in a conditional Poisson distribution. Biometrics 16(2):203–211MathSciNetzbMATHGoogle Scholar
  10. Consul PC (1989) Generalized Poisson distributions. Marcel Dekker, New YorkzbMATHGoogle Scholar
  11. Consul PC, Famoye F (2006) Lagrangian probability distributions. Birkhäuser, BostonzbMATHGoogle Scholar
  12. Consul PC, Jain GC (1973) A generalization of the Poisson distribution. Technometrics 15(4):791–799MathSciNetzbMATHGoogle Scholar
  13. Dahiya RC, Gross AJ (1973) Estimating the zero class from a truncated Poisson sample. J Am Stat Assoc 68:731–733MathSciNetzbMATHGoogle Scholar
  14. David FN, Johnson NL (1952) The truncated Poisson. Biometrics 8(4):275–285MathSciNetGoogle Scholar
  15. Finney DJ (1949) The truncated binomial distribution. Ann Eugen 14:319–328MathSciNetzbMATHGoogle Scholar
  16. Finney DJ, Varley GC (1955) An example of the truncated Poisson distribution. Biometrics 11(3):387–394Google Scholar
  17. Grogger JT, Carson RT (1991) Models for truncated counts. J Appl Econ 6:225–238Google Scholar
  18. Gross AJ (1970) A note on the convolution of Poisson distributions with the zero class missing. Am Stat 24(5):42–43Google Scholar
  19. Gupta RC (1974) Distribution of the sum of independent decapitated generalized negative binomial variables. Sankhyā Indian J Stat Ser B 36(1):67–69MathSciNetzbMATHGoogle Scholar
  20. Gurmu S (1991) Tests of detecting overdispersion in the positive Poisson regression model. J Bus Econ Stat 9(2):215–222MathSciNetGoogle Scholar
  21. Hartley HO (1958) Maximum likelihood estimation from incomplete data. Biometrics 14(2):174–194zbMATHGoogle Scholar
  22. Irwin JO (1959) On the estimation of the mean of a Poisson distribution from a sample with the zero class missing. Biometrics 15(2):324–326MathSciNetzbMATHGoogle Scholar
  23. Jain GC, Consul PC (1971) A generalized negative binomial distribution. SIAM J Appl Math 21(4):501–513MathSciNetzbMATHGoogle Scholar
  24. Johnson NL, Kotz S (1969) Distributions in statistics: discrete distributions. Wiley, New York, pp 104–109zbMATHGoogle Scholar
  25. Karlis D (2003) An EM algorithm for multivariate Poisson distribution and related models. J Appl Stat 30(1):63–77MathSciNetzbMATHGoogle Scholar
  26. Kumar CS, Sreejakumari S (2012) On intervened negative binomial distribution and some of its properties. Statistica 72(4):395–404Google Scholar
  27. Malik HJ (1969) Distribution of the sum of truncated binomial variates. Can Math Bull 12:334–336Google Scholar
  28. Maritz JS (1950) On the validity of inferences drawn from the fitting of Poisson and negative binomial distributions to observed accident data. Psychol Bull 47(5):434–443Google Scholar
  29. Matthews JNS, Appleton DR (1993) An application of the truncated Poisson distribution to immunogold assay. Biometrics 49(2):617–621Google Scholar
  30. Medhi J (1975) On the convolutions of left-truncated generalised negative binomial and Poisson variables. Sankhyā Indian J Stat Ser B 37(3):293–299MathSciNetzbMATHGoogle Scholar
  31. Meng XL (1997) The EM algorithm and medical studies: a historical linik. Stat Methods Med Res 6:3–23Google Scholar
  32. Meng XL, Rubin DB (1993) Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80:267–278MathSciNetzbMATHGoogle Scholar
  33. Moore PG (1954) A note on truncated Poisson distributions. Biometrics 10(3):402–406MathSciNetzbMATHGoogle Scholar
  34. Patil GP (1963) Minimum variance unbiased estimation and certain problems of additive number theory. Ann MathStat 34(3):1050–1056MathSciNetzbMATHGoogle Scholar
  35. Plackett RL (1953) The truncated Poisson distribution. Biometrics 9(4):485–488MathSciNetGoogle Scholar
  36. Rider PR (1953) Truncated Poisson distributions. J Am Stat Assoc 48:826–830MathSciNetzbMATHGoogle Scholar
  37. Rider PR (1955) Truncated binomial and negative-binomial distributions. J Am Stat Assoc 50(271):877–883MathSciNetzbMATHGoogle Scholar
  38. Sampford MR (1955) The truncated negative binomial distribution. Biometrika 42:58–69MathSciNetzbMATHGoogle Scholar
  39. Scollnik DPM (2006) On the intervened generalized Poisson distribution. Commun Stat Theory Methods 35:953–963MathSciNetzbMATHGoogle Scholar
  40. Shanmugam R (1985) An intervened Poisson distribution and its medical application. Biometrics 41:1025–1029MathSciNetzbMATHGoogle Scholar
  41. Singh J (1978) A characterization of positive Poisson distribution and its statistical application. SIAM J Appl Math 34(3):545–548MathSciNetzbMATHGoogle Scholar
  42. Springael J, Van Nieuwenhuyse I (2006) On the sum of independent zero-truncated Poisson random variables. University of Antwerp, Faculty of Applied EconomicsGoogle Scholar
  43. Tate RF, Goen RL (1958) Minimum variance unbiased estimation for the truncated Poisson distribution. Ann Math Stat 29(3):755–765MathSciNetzbMATHGoogle Scholar
  44. Xie TL, Aickin M (1997) A truncated Poisson regression model with applications to occurrence of adenomatous polyps. Stat Med 16:1845–1857Google Scholar
  45. Zhou H, Lange K (2010) MM algorithms for some discrete multivariate distributions. J Comput Graph Stat 19(3):645–665MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of MathematicsSouthern University of Science and TechnologyShenzhenPeople’s Republic of China
  2. 2.Department of Statistics and Actuarial ScienceThe University of Hong KongLung Fu ShanPeople’s Republic of China
  3. 3.School of Statistics and MathematicsZhongnan University of Economics and LawWuhanPeople’s Republic of China
  4. 4.Department of Mathematics and Statistics, School of Decision SciencesThe Hang Seng University of Hong KongShatin, New TownPeople’s Republic of China

Personalised recommendations