This chapter introduces three families of LNRE models: Carroll's (1967) log-normal model, Sichel's (1975) generalized inverse Gauss-Poisson model, and Orlov and Chitashvili's (1983a,b) extended Zipf's law. By enriching the non- parametric expressions of the preceding chapter with parametric assumptions about the shape of the structural distribution, we can avoid the technical problems associated with the non-parametric methods. Section 3.2 describes the three LNRE models. Sections 3.3 and 3.6 discuss techniques for evaluating the goodness of fit of the LNRE models and for testing whether word frequency distributions are different by means of statistics such as V(N) and V(1, N).
KeywordsMean Square Error Spectrum Element Vocabulary Size Lognormal Model British National Corpus
Unable to display preview. Download preview PDF.
- 3.See Luke (1962) for details on integrate of Bessel functions.Google Scholar
- 5.For the generalized inverse Gauss-Poisson model, expressions and algorithms for parameter estimation based on the log-likelihood function are available (see Stein, Zucchini, and Juritz, 1987, Heller, 1997, and Burrell and Fenton, 1993).Google Scholar