The Non-uniqueness Property of the Intrinsic Estimator in APC Models

Abstract

This article explores an important property of the intrinsic estimator that has received no attention in literature: the age, period, and cohort estimates of the intrinsic estimator are not unique but vary with the parameterization and reference categories chosen for these variables. We give a formal proof of the non-uniqueness property for effect coding and dummy variable coding. Using data on female mortality in the United States over the years 1960–1999, we show that the variation in the results obtained for different parameterizations and reference categories is substantial and leads to contradictory conclusions. We conclude that the non-uniqueness property is a new argument for not routinely applying the intrinsic estimator.

Appendices

Appendix A: Proof of the Non-uniqueness of the IE in Case of Effect Coding

To arrive at Eq. (2), we used effect coding with the last categories of age, period, and cohort omitted from the equation. In this appendix, we will show that the IE yields different estimates when the first categories are omitted. Equation (2) then changes into

$$ \begin{array}{l}Y={\upbeta}_0^F+{\upalpha}_2^F{A}_2^F+{\upalpha}_3^F{A}_3^F+{\upbeta}_2^F{P}_2^F+{\upbeta}_3^F{P}_3^F\hfill \\ {}+{\upgamma}_2^F{C}_2^F+{\upgamma}_3^F{C}_3^F+{\upgamma}_4^F{C}_4^F+{\upgamma}_5^F{C}_5^F+e,\hfill \end{array} $$

(2a)

where superscript F denotes that the first categories of age, period, and cohort are omitted. The independent variables in (2a) differ from those in (2) because they now take value –1 for cases in the first category of age, period, or cohort. Again, the variable for the fourth cohort can be expressed in terms of other variables in (2a):

$$ {C}_4^F=-{A}_3^F+{P}_3^F+{C}_2^F-2{C}_5^F. $$

(3a)

Substituting into (2a) the expression for C ^F₄ given in (3a) yields the following:

$$ \begin{array}{l}Y={\upbeta}_0^F+{\upalpha}_2^F{A}_2^F+\left({\upalpha}_3^F-{\upgamma}_4^F\right){A}_3^F+{\upbeta}_2^F{P}_2^F+\left({\upbeta}_3^F+{\upgamma}_4^F\right){P}_3^F\hfill \\ {}+\left({\upgamma}_2^F+{\upgamma}_4^F\right){C}_2^F+{\upgamma}_3^F{C}_3^F+\left({\upgamma}_5^F-2{\upgamma}_4^F\right){C}_4^F+e.\hfill \end{array} $$

(4a)

The preceding equation can be represented more compactly:

$$ Y={\boldsymbol{\upbeta}}_0^F+{\boldsymbol{\upalpha}}_2^F{A}_2^F+{\boldsymbol{\upalpha}}_3^F{A}_3^F+{\boldsymbol{\upbeta}}_2^F{P}_2^F+{\boldsymbol{\upbeta}}_3^F{P}_3^F+{\boldsymbol{\upgamma}}_2^F{C}_2^F+{\boldsymbol{\upgamma}}_3^F{C}_3^F+{\boldsymbol{\upgamma}}_5^F{C}_5^F+e. $$

(5a)

The parameters in (5a) are identified because C ^F₄ is not part of the equation. Following the same line of reasoning as outlined in the main text for the last categories omitted, we now have to minimize the sum of squares ${\left({\widehat{\upalpha}}_3^F\right)}^2+{\left({\widehat{\boldsymbol{\upbeta}}}_3^F\right)}^2+{\left({\widehat{\upgamma}}_2^F\right)}^2+{\left({\widehat{\upgamma}}_4^F\right)}^2+{\left({\widehat{\upgamma}}_5^F\right)}^2$, which finally leads to the following IE estimator for γ ^F₄ :

$$ {\widehat{\upgamma}}_4^F=\left(-{\boldsymbol{\upalpha}}_3^F+{\boldsymbol{\upbeta}}_3^F+{\boldsymbol{\upgamma}}_2^F-2{\boldsymbol{\upgamma}}_5^F\right)/8. $$

(8a)

Recall that with the last categories omitted, we found the estimator given in Eq. (8):

$$ {\widehat{\upgamma}}_4^L=\left({\boldsymbol{\upalpha}}_1^L-{\boldsymbol{\upbeta}}_1^L+2{\boldsymbol{\upgamma}}_1^L+{\boldsymbol{\upgamma}}_2^L\right)/8. $$

Apparently, the estimate ${\widehat{\upgamma}}_4^F$ depends on the value of − α ^F₃  + β ^F₃  + γ ^F₂  − 2γ ^F₅ , whereas the estimate ${\widehat{\upgamma}}_4^L$ depends on the value of α ^L₁  − β ^L₁  + 2γ ^L₁  + γ ^L₂ . In general, with actual data, − α ^F₃  + β ^F₃  + γ ^F₂  − 2γ ^F₅ will not be equal to α ^L₁  − β ^L₁  + 2γ ^L₁  + γ ^L₂ . To see this, note that the boldfaced parameters in (8) and (8a) represent deviations from the means β ^L₀ and β ^F₀ in Eqs. (5) and (5a), respectively. Both (5) and (5a) are estimable because of the constraint that the deviation for the fourth cohort is equal to 0; that is, γ ^L₄ = 0 and γ ^F₄ = 0. As a consequence of using the same constraint in (5) and (5a), all α estimates with the same subscript are equal in both equations; the same holds for all β estimates with the same subscript and for all γ estimates with the same subscript. For example, α ^L₂ = α ^F₂ . Also, α ^L₃ (to be derived as − α ^L₁  − α ^L₂ ) is equal to α ^F₃ , as estimated with Eq. (5a). The estimates of the means β ^L₀ and β ^F₀ are equal as well. Thus, proving that − α ^F₃  + β ^F₃  + γ ^F₂  − 2γ ^F₅  ≠ α ^L₁  − β ^L₁  + 2γ ^L₁  + γ ^L₂ boils down to proving that − α ^L₃  + β ^L₃  + γ ^L₂  − 2γ ^L₅  ≠ α ^L₁  − β ^L₁  + 2γ ^L₁  + γ ^L₂ . In this last inequality, the expression to the left of the inequality sign contains different deviations from the mean β ^L₀ than the expression to the right. As a consequence, the values of ${\widehat{\upgamma}}_4^F$ and ${\widehat{\upgamma}}_4^L$ will usually differ for actual data. Also, the parameter estimates that depend on the values of ${\widehat{\upgamma}}_4^F$ and ${\widehat{\upgamma}}_4^L$ will generally be different: for example, ${\widehat{\upalpha}}_1^L={\boldsymbol{\upalpha}}_1^L-{\widehat{\upgamma}}_4^L$, whereas ${\widehat{\upalpha}}_1^F=-{\widehat{\upalpha}}_2^F-{\widehat{\upalpha}}_3^F=-{\boldsymbol{\upalpha}}_2^F-\left({\boldsymbol{\upalpha}}_3^F-{\widehat{\upgamma}}_4^F\right)={\boldsymbol{\upalpha}}_1^F+{\widehat{\upgamma}}_4^F={\boldsymbol{\upalpha}}_1^L+{\widehat{\upgamma}}_4^F$.³

Appendix B

In this appendix, we show why with dummy variable coding, the IE estimates generally differ from the “default” IE estimates Yang et al. (2004) presented. Instead of standard 0 and 1 coded dummy variables, we subtract 1 / k from these dummy variables, with k denoting the number of categories—that is, 3 for age and period, and 5 for cohort. Subtracting the constant 1 / k from the original 0 and 1 coded dummy variables does not change the interpretation of their regression coefficients (i.e., the deviation from the omitted category). If we omit the last age category, the dummy variables A ^D₁ and A ^D₂ for the first two ages have the coding scheme shown in Table 2.

Table 2 Dummy variable coding for age after subtracting one-third

Note that in Table 2, the sum over the three ages for each of the dummy variables (column sum) is 0, just as with effect coding. As a result, the intercept in Eq. (2b) is equal to the intercept in Eq. (2), both representing the unweighted mean of the three predicted values for ages 1, 2, and 3 (controlling for period and cohort). With the last categories omitted, we obtain the following equation to be estimated:

$$ \begin{array}{l}Y={\upbeta}_0^D+{\upalpha}_1^D{A}_1^D+{\upalpha}_2^D{A}_2^D+{\upbeta}_1^D{P}_1^D+{\upbeta}_2^D{P}_2^D\hfill \\ {}+{\upgamma}_1^D{C}_1^D+{\upgamma}_2^D{C}_2^D+{\upgamma}_3^D{C}_3^D+{\upgamma}_4^D{C}_4^D+e.\hfill \end{array} $$

(2b)

To keep notation parsimonious, we do not use superscript DL to explicitly indicate that the last dummy variable–coded category is the reference. Like Eqs. (2) and (2a), the new Eq. (2b) also suffers from the identification problem. For example, we can write the following for C ^D₄ :

$$ {C}_4^D=-2{A}_1^D-{A}_2^D+2{P}_1^D+{P}_2^D-4{C}_1^D-3{C}_2^D-2{C}_3^D. $$

(3b)

If we substitute in (2b) for C ^D₄ the expression given in (3b), we obtain

$$ \begin{array}{l}Y={\upbeta}_0^D+\left({\upalpha}_1^D-2{\upgamma}_4^D\right){A}_1^D+\left({\upalpha}_2^D-{\upgamma}_4^D\right){A}_2^D+\left({\upbeta}_1^D+2{\upgamma}_4^D\right){P}_1^D\hfill \\ {}+\left({\upbeta}_2^D+{\upgamma}_4^D\right){P}_2^D+\left({\upgamma}_1^D-4{\upgamma}_4^D\right){C}_1^D+\left({\upgamma}_2^D-3{\upgamma}_4^D\right){C}_2^D\hfill \\ {}+\left({\upgamma}_3^D-2{\upgamma}_4^D\right){C}_3^D+e.\hfill \end{array} $$

(4b)

Recall that for effect coding with the last category omitted, we used the constraint that γ ^L₄ = 0 in Eq. (2), leading to the estimable Eq. (5). This assumption implies that the deviation of cohort 4 from the mean is 0. Because the deviation of cohort 5 from the mean equals γ ^L₅ , the assumption γ ^L₄ = 0 implies that the deviation of cohort 4 from cohort 5 equals − γ ^L₅ (the value of − γ ^L₅ can be derived from the estimates of Eq. (5): − γ ^L₅ = γ ^L₁  + γ ^L₂  + γ ^L₃  + γ ^L₄ = γ ^L₁  + γ ^L₂  + γ ^L₃ ). In terms of the regression coefficients of Eq. (2b), the above implies: γ ^D₄ = − γ ^L₅ . If, in Eq. (2b), we plug in − γ ^L₅ for γ ^D₄ , substitute the expression for C ^D₄ given in (3b), and rearrange terms we obtain the following estimable equation:⁴

$$ \begin{array}{l}Y={\boldsymbol{\upbeta}}_0^D+\left({\boldsymbol{\upalpha}}_1^D+2{\boldsymbol{\upgamma}}_5^L\right){A}_1^D+\left({\boldsymbol{\upalpha}}_2^D+{\boldsymbol{\upgamma}}_5^L\right){A}_2^D+\left({\boldsymbol{\upbeta}}_1^D-2{\boldsymbol{\upgamma}}_5^L\right){P}_1^D\hfill \\ {}+\left({\boldsymbol{\upbeta}}_2^D-{\boldsymbol{\upgamma}}_5^L\right){P}_2^D+\left({\boldsymbol{\upgamma}}_1^D+4{\boldsymbol{\upgamma}}_5^L\right){C}_1^D+\left({\boldsymbol{\upgamma}}_2^D+3{\boldsymbol{\upgamma}}_5^L\right){C}_2^D\hfill \\ {}+\left({\boldsymbol{\upgamma}}_3^D+2{\boldsymbol{\upgamma}}_5^L\right){C}_3^D+e.\hfill \end{array} $$

(5b)

Because Eqs. (5) and (5b) are based on the same constraint with respect to cohort 4, the effect coding parameters in (5) can be translated into deviations from the reference categories resulting from estimating (5b). This is relevant when we compare the effect-coded IE with the dummy variable–coded IE later in this section. The parameters in (4b) and (5b) are related as follows:

$$ \begin{array}{l}{\upbeta}_0^D={\boldsymbol{\upbeta}}_0^D,\hfill \\ {}{\upalpha}_1^D={\boldsymbol{\upalpha}}_1^D+2{\boldsymbol{\upgamma}}_5^L+2{\upgamma}_4^D,\kern0.5em {\upalpha}_2^D={\boldsymbol{\upalpha}}_2^D+{\boldsymbol{\upgamma}}_5^L+{\upgamma}_4^D,\hfill \\ {}{\upbeta}_1^D={\boldsymbol{\upbeta}}_1^D-2{\boldsymbol{\upgamma}}_5^L-2{\upgamma}_4^D,\kern0.5em {\upbeta}_2^D={\boldsymbol{\upbeta}}_2^D-{\boldsymbol{\upgamma}}_5^L-{\upgamma}_4^D,\hfill \\ {}{\upgamma}_1^D={\boldsymbol{\upgamma}}_1^D+4{\boldsymbol{\upgamma}}_5^L+4{\upgamma}_4^D,\kern0.5em {\upgamma}_2^D={\boldsymbol{\upgamma}}_2^D+3{\boldsymbol{\upgamma}}_5^L+3{\upgamma}_4^D,\kern0.5em {\upgamma}_3^D={\boldsymbol{\upgamma}}_3^D+2{\boldsymbol{\upgamma}}_5^L+2{\upgamma}_4^D.\hfill \end{array} $$

Having an estimate for the parameter of the fourth cohort leads to estimates for the remaining parameters (except for β ^D₀ ). The IE now employs the criterion of minimizing the following sum of squares:

$$ {\left({\widehat{\upalpha}}_1^D\right)}^2+{\left({\widehat{\upalpha}}_1^D\right)}^2+{\left({\widehat{\upbeta}}_1^D\right)}^2+{\left({\widehat{\upbeta}}_2^D\right)}^2+{\left({\widehat{\upgamma}}_1^D\right)}^2+{\left({\widehat{\upgamma}}_2^D\right)}^2+{\left({\widehat{\upgamma}}_3^D\right)}^2+{\left({\widehat{\upgamma}}_4^D\right)}^2. $$

This is a completely different criterion than the one used in the effect-coded IE, proposed by Yang et al. (2004). Not only are more parameters (eight instead of five) involved in the sum of squares to be minimized, but the parameters also have a different meaning (i.e., distances to the reference category instead of distances to the mean). For the preceding sum of squares, we can write

$$ \begin{array}{l}{\left({\boldsymbol{\upalpha}}_1^D+2{\boldsymbol{\upgamma}}_5^L+2{\widehat{\upgamma}}_4^D\right)}^2+{\left({\boldsymbol{\upalpha}}_2^D+{\boldsymbol{\upgamma}}_5^L+{\widehat{\upgamma}}_4^D\right)}^2+{\left({\boldsymbol{\upbeta}}_1^D-2{\boldsymbol{\upgamma}}_5^L-2{\widehat{\upgamma}}_4^D\right)}^2+{\left({\boldsymbol{\upbeta}}_2^D-{\boldsymbol{\upgamma}}_5^L-{\widehat{\upgamma}}_4^D\right)}^2\hfill \\ {}+{\left({\boldsymbol{\upgamma}}_1^D+4{\boldsymbol{\upgamma}}_5^L+4{\widehat{\upgamma}}_4^D\right)}^2+{\left({\boldsymbol{\upgamma}}_2^D+3{\boldsymbol{\upgamma}}_5^L+3{\widehat{\upgamma}}_4^D\right)}^2+{\left({\boldsymbol{\upgamma}}_3^D+2{\boldsymbol{\upgamma}}_5^L+2{\widehat{\upgamma}}_4^D\right)}^2+{\left({\widehat{\upgamma}}_4^D\right)}^2.\hfill \end{array} $$

Taking the first-order derivative of this sum of squares with respect to ${\widehat{\upgamma}}_4^D$ and setting it to 0 finally leads to the following IE estimate for γ ^D₄ :

$$ {\widehat{\upgamma}}_4^D=\left(-4{\boldsymbol{\upalpha}}_1^D-2{\boldsymbol{\upalpha}}_2^D+4{\boldsymbol{\upbeta}}_1^D+2{\boldsymbol{\upbeta}}_2^D-8{\boldsymbol{\upgamma}}_1^D-6{\boldsymbol{\upgamma}}_2^D-4{\boldsymbol{\upgamma}}_3^D-78{\boldsymbol{\upgamma}}_5^L\right)/80. $$

(8b)

Because the boldfaced parameters in this expression for ${\widehat{\upgamma}}_4^D$ are compatible with the ones in Eq. (5), we can formulate ${\widehat{\upgamma}}_4^D$in terms of the effect coding parameters of Eq. (5). For example, coefficient α ^D₁ , which is the deviation of age 1 from reference category age 3, can be written as the difference of the two corresponding effect coding parameters: namely, α ^L₁  − α ^L₃ .⁵ Building on this relation between the coefficients of effect and dummy coding, we can write the above estimate of ${\widehat{\upgamma}}_4^D$ in terms of the effect coding parameters of Eq. (5):

$$ \begin{array}{l}{\widehat{\upgamma}}_4^D=\left(-4\left({\boldsymbol{\upalpha}}_1^L-{\boldsymbol{\upalpha}}_3^L\right)-2\left({\boldsymbol{\upalpha}}_2^L-{\boldsymbol{\upalpha}}_3^L\right)+4\left({\boldsymbol{\upbeta}}_1^L-{\boldsymbol{\upbeta}}_3^L\right)+2\left({\boldsymbol{\upbeta}}_2^L-{\boldsymbol{\upbeta}}_3^L\right)\right.\\ {}\left.-8\left({\boldsymbol{\upgamma}}_1^L-{\boldsymbol{\upgamma}}_5^L\right)-6\left({\boldsymbol{\upgamma}}_2^L-{\boldsymbol{\upgamma}}_5^L\right)-4\left({\boldsymbol{\upgamma}}_3^L-{\boldsymbol{\upgamma}}_5^L\right)-78{\boldsymbol{\upgamma}}_5^L\right)/80.\end{array} $$

Using our knowledge that α ^L₃  = − α ^L₁ − α ^L₂ , β ^L₃  = − β ^L₁ − β ^L₂ , and γ ^L₅  = − γ ^L₁ − γ ^L₂ − γ ^L₃ , we finally get the following:

$$ {\widehat{\upgamma}}_4^D=\left(-{\boldsymbol{\upalpha}}_1^L-0.8{\boldsymbol{\upalpha}}_2^L+{\boldsymbol{\upbeta}}_1^L+0.8{\boldsymbol{\upbeta}}_2^L+5.2{\boldsymbol{\upgamma}}_1^L+5.4{\boldsymbol{\upgamma}}_2^L+5.6{\boldsymbol{\upgamma}}_3^L\right)/8. $$

(8b*)

If we compare this IE estimator of ${\widehat{\upgamma}}_4^D$with ${\widehat{\upgamma}}_4^L=\left({\boldsymbol{\upalpha}}_1^L-{\boldsymbol{\upbeta}}_1^L+2{\boldsymbol{\upgamma}}_1^L+{\boldsymbol{\upgamma}}_2^L\right)/8$of Eq. (8), it is obvious that the IE estimate for the fourth cohort effect will usually differ between the effect-coded IE as proposed by Yang et al. (2004) and the dummy variable–coded IE (with the last categories as the omitted ones for both codings). The same holds for the IE estimates of the remaining categories of the three APC variables. Similar to the IE with effect coding, each triplet of reference categories of age, period, and cohort leads to different estimates when dummy variable coding is used, which we do not elaborate here.