Further Analyses Reinforce Our Conclusions About Extreme Poverty

Our study (presented earlier in this issue of Demography) shows that deep and extreme poverty have increased in the United States from 1993 to 2016, particularly for childless households. In his response, James Sullivan raises concerns that (1) the Transfer Income Model, version 3 (TRIM3) overallocates Supplemental Nutrition Assistance Program (SNAP) benefits to lower-income households, and (2) earnings among lower-income households are underreported. Sullivan largely sidesteps that as a direct response to his reviewer concerns, we had already incorporated these issues in our study. Our online appendix demonstrates that our conclusions hold even after we address these issues.

Sullivan claims that extreme poverty must be lower than we find even though his own critique undermines this claim. Although Sullivan argues that the underreporting of earnings leads us to overestimate deep/extreme poverty, the bulk of his critique argues that TRIM3 leads us to underestimate deep/extreme poverty. Although Sullivan takes this contradictory position about our estimates at one point in time, he provides no challenge to our finding of significant over-time increases. Sullivan’s critique relies heavily on a National Bureau of Economic Research working paper by Meyer et al. (2019). However, Sullivan overstates their evidence, neglects major differences between our study and theirs, and overlooks our unique contributions that surpass the Meyer et al. study.

Before proceeding, we note that Sullivan’s contributions (Meyer et al. 2015) on the crisis in household surveys and underreporting partly inspired our study. We agree on the salience of underreporting, the crucial role of SNAP, the prominence of childless adults among the deep/extreme poor, and the need to improve income measurement. Like Sullivan (and Meyer et al. 2019), we find that levels of $2/day poverty are far lower than estimates that Edin and Shaefer (2015) provided. Sullivan ignores that figure 6 in our article reports nearly identical results to Meyer et al. (2019). Specifically, we estimate that 0.29% of the United States lived in $2/day poverty in 2011 (after accounting for SNAP benefits using TRIM3). Meyer et al.’s estimates from the same year range from 0.18% to 0.29%.

While responding to Sullivan, we present new analyses that reinforce our conclusions. We reaffirm that the best approach to analyzing deep/extreme poverty is to conduct open science, report a range of estimates under a variety of transparent assumptions, follow prevailing international standards in income and poverty measurement, and use justifiable thresholds (Brady and Burton 2016; Brady et al. 2018).

The Potential Overallocation of SNAP in TRIM3

Our study explicitly addresses Stevens et al.’s (2018) valuable analysis showing that TRIM3 appears to overallocate SNAP benefits toward lower-income households in the CPS Annual Social and Economic Supplement (ASEC). In section 5 of the online appendix to our earlier article, we present evidence that even after accounting for this possibility, our conclusions hold. In his comment, Sullivan cites the same research that we cite. Here, we provide additional evidence that overallocation of SNAP benefits from TRIM3 does not affect our conclusions.

We underline that Sullivan’s critique of TRIM3 contradicts his broader claim that deep and extreme poverty must be lower than our estimates. To the extent that TRIM3 overallocates SNAP benefits to lower-income households, it should lead to a downward bias in our estimates. Indeed, figures 6–8 in our article confirm this: if we do not use TRIM3, levels of deep and extreme poverty are higher and rise more quickly over time. That we still find an increase in deep and extreme poverty over time even when applying TRIM3 demonstrates that our use of benefit adjustments is not biasing our results in the direction that Sullivan claims. Moreover, Sullivan ignores the evidence we provide (see figures 7–8 in our earlier article) that households without children—a group much less likely to receive SNAP benefits and thus less likely to be affected by TRIM3—are the primary driver behind the aggregate rise in deep/extreme poverty (see also Parolin and Brady 2019). Correcting SNAP benefits will not change levels of deep/extreme poverty among people who do not receive SNAP.

Nevertheless, in both our earlier article and here, we take seriously the claim that TRIM3 may overallocate SNAP benefits to lower-income households. In section 5 of our online appendix, we scale back TRIM3’s assignment of SNAP benefits among zero-earnings households to meet participation rates that fall halfway between the rates estimated in the unadjusted CPS ASEC and the TRIM3-adjused CPS ASEC. This follows evidence that administrative data on SNAP participation place the “real” participation rate near the midpoint of those two estimates (also demonstrated by Sullivan’s two figures). Our results hold even after scaling back TRIM3’s imputations this way. Here, we extend this procedure to higher brackets in the income distribution and demonstrate again that potential measurement error in TRIM3 does not meaningfully affect our conclusions.

Figure 1 in this reply displays the share of households in each income group receiving SNAP benefits in 2015 when applying the unadjusted CPS ASEC, the TRIM3-adjusted CPS ASEC, and our modified version of TRIM3. The figure shows that our modifications to TRIM3 successfully meet the midpoint of SNAP participation rates in TRIM3 and the unadjusted CPS ASEC for each of 21 income bins. Although we show only the 2015 pattern, we repeat this process for each year.

Fig. 1

Share of households by income bins receiving SNAP benefits in 2015. Income group 1 = zero countable income for SNAP benefit purposes. Subsequent income groups are divided into 20 income ranks. Modified TRIM3 benefits are set to meet the midpoint between TRIM3 and unadjusted SNAP participation for each income grouping.

Figure 2 in this reply presents updated trends in extreme poverty when applying the scaled-back version of TRIM3’s SNAP imputations. The top line presents the trend in extreme poverty when SNAP is excluded from household resources. As detailed in our study, excluding SNAP leads to a clear rise in extreme poverty. When adding SNAP in, but not yet using TRIM3, we still see a rise in extreme poverty. The third and fourth lines show, respectively, the trends when applying our modified version of TRIM3 adjustments and the full TRIM3 adjustments. Both show rising levels of extreme poverty over time. Obviously, the modified TRIM3 estimates show slightly higher rates of extreme poverty than the full TRIM3 adjustments.

Fig. 2

Trends in extreme poverty (<10% federal median) before and after SNAP and TRIM3 adjustments

Put simply, our conclusions hold whether we exclude TRIM3, apply an adjusted version of TRIM3, or apply the full version of TRIM3.¹ Indeed, Fig. 2 in this reply is consistent with our general approach to report a range of estimates with transparent assumptions. Although we share Sullivan’s concerns about TRIM3 potentially overimputing SNAP benefits (hence, section 5 of our online appendix to our article), his claim that our use of TRIM3 affects our conclusions is not substantiated.

The Underreporting of Earnings

Sullivan claims that “evidence from linked administrative and survey data has indicated that underreported earnings are by far the most important reason why survey-based estimates of extreme poverty are biased upward.” He cites Meyer et al. (2019) as evidence that “earnings are significantly underreported for households with very low reported income.”² If so, we may overestimate deep/extreme poverty. However, Sullivan overstates Meyer et al.’s evidence while ignoring our analyses aimed at addressing his concern.

In particular, Meyer et al.’s survey-based adjustments for earnings underreporting are relevant specifically for the Survey of Income and Program Participation (SIPP). With the CPS ASEC data that we use, Meyer et al. (2019:33) found that “households almost never report positive hours worked and extremely low earnings in the CPS . . . . In the CPS, all households that report 0 earnings also report 0 hours worked across all members.” Put simply, Meyer et al. did not find that survey-based adjustments to earnings underreporting are consequential for estimates of extreme poverty in the CPS ASEC.

Meyer et al. did find that earnings adjustments based on administrative records reduce levels of $2/day poverty in the CPS. However, even this evidence cannot be used to directly critique our analyses. First, because they analyzed only 2011, Meyer et al. provided no evidence that their adjustments affect trends in deep/extreme poverty. Second, Meyer et al. did not analyze extreme poverty at our much higher (and more justifiable) threshold of 10% of medians. Any effect of earnings underreporting is likely to be less consequential to the share of respondents below 10% of medians than the share below $2/day. Third, Meyer et al.’s particular sequence of corrections makes it impossible to say that earnings underreporting matter after the use of TRIM3.³ Because our estimate of $2/day poverty in 2011 is nearly identical to that of Meyer et al., it seems unlikely that underreported earnings affect our conclusions after using TRIM3. Nonetheless, we now present additional evidence to validate our conclusions.

Following Meyer et al., section 6 of our online appendix to the earlier article multiplies hours worked by the minimum wage for any respondent reporting zero earnings. As we mentioned, this is a rather heroic assumption that hours worked are reported accurately but earnings are not. It also assumes that all self-employed individuals receive earnings at the minimum wage level or higher. Regardless, section 6 of our online appendix shows that survey-based adjustments for earnings underreporting matter very little for estimates of deep/extreme poverty. Sullivan only has one reference to, and does not engage with, section 6 of our online appendix.

Here, we provide further evidence that the underreporting of earnings does not change our conclusions. Specifically, we adjust earnings for all individuals below the product of hours worked and the minimum wage. This bottom-codes every respondent as having a minimum of earnings at the product of hours worked and the minimum wage. Given Sullivan’s concern that households reporting zero income are underreporting either earnings or transfer income, we also present estimates of deep/extreme poverty after excluding all zero-income households.

Figure 3 in this reply shows that our results are robust to making these aggressive adjustments for earnings underreporting. There is no difference between our base estimates (i.e., those reported in the earlier article) and the estimates after adjusting for potential earnings underreporting (similar to what we show in section 6 of our online appendix). Indeed, the trend lines overlap so much that readers will probably be unable to see any difference at all between our base estimates and the estimates adjusting for potential earnings underreporting. When we remove all zero-income households—the group most likely to underreport sources of income—levels of deep/extreme poverty are lower. Again, though, we still find an increase from 1993 to 2016.

Fig. 3

Trends in deep and extreme poverty after adjusting for potential earnings underreporting and zero-income households. Adjustments for potential earnings underreporting replace reported earnings with the product of hours worked and the minimum wage for any individual reporting earnings below that amount. Base estimates and estimates with earnings adjustment overlap entirely.

Toward a Better Science of Deep/Extreme Poverty

As mentioned earlier, Sullivan’s response relies heavily on the study by Meyer et al., who found that $2/day poverty is almost non-existent in survey data. Again, Sullivan does not acknowledge that figure 6 of our article shows nearly identical results.

Sullivan also does not address the ways in which our analyses surpass those of Meyer et al. We investigate trends in deep and extreme poverty from 1993 to 2016, whereas Meyer looked at levels only in 2011.⁴ Moreover, we follow best practices in the international income and poverty literatures (Brady and Burton 2016; Brady et al. 2018). We present a range of poverty thresholds (i.e., 20% and 10% of federal, state, and anchored median incomes as well as $2/day poverty) and apply an equivalence scale. Meyer et al. did neither. While raising measurement concerns that have little effect on our conclusions, Sullivan does not address the far more consequential measurement challenge of homelessness.⁵ Although we present estimates that incorporate the homeless population, Meyer et al. omitted the homeless population. As mentioned earlier in this reply, Meyer et al.’s particular sequence of corrections makes it impossible to say exactly how much consequence each correction has for reducing estimates of $2/day poverty (see footnote 3). By contrast, we present many results with and without corrections to make the consequence of each correction transparent.

Most important, Sullivan does not make clear that Meyer et al. concentrated on $2/day (except one column in their table 6). By contrast, our article devotes several paragraphs to advocating against the $2/day threshold and for much higher thresholds (although we show some $2/day analyses). Indeed, our extreme poverty thresholds (at 10% of medians) are almost three times higher. By not making clear that we use a different, much higher, and more justifiable threshold, Sullivan’s comment conceals a crucial distinction between our analyses and those of Meyer et al. (2019). Sullivan’s comment does not mention that his claims about “extreme” poverty are based on a $2/day threshold, does not even define “extreme” poverty, and just substitutes the word “extreme” for “$2/day.” Because our thresholds are much higher, Sullivan’s critique is far less relevant. Further, Sullivan’s critique conceals how little he can say about extreme poverty with our thresholds.

This contrast in thresholds suggests that Meyer et al. and Sullivan should be far more cautious with their claims about the composition and material deprivation of the extreme poor. With the $2/day threshold, Meyer et al. had a sample of only 250 SIPP households (table A.11, column 6). This bottom 0.77% of the SIPP sample is highly unlikely to be representative of the 2.4 million people in the bottom 0.77% of the population in 2011. Indeed, Meyer and Sullivan’s (Meyer et al. 2015) own work on the crisis in household surveys suggests that it is inappropriate to draw any inferences based on the 250 households in the far left tail of the SIPP. By contrast, samples below our thresholds for extreme poverty would be much larger. Thus, an advantage of our more reasonable thresholds is that the sample will better represent the true population of extreme poor.

Finally, although Meyer et al.’s analysis of administrative data is obviously valuable, our combination of survey data and simulation also has advantages. Meyer et al. had administrative data for only 11 states and only in 2011. We are able to use nationally representative data for all income sources from 1993 to 2016. Because their administrative data are not publicly available, replication is impossible. Despite Meyer et al.’s admirably meticulous methodological appendix, replication also requires publicly available code. Even given the advantages of administrative data, our combination of public protocols (i.e., the Luxembourg Income Study and TRIM3), publicly available data, and open code is more replicable.

Conclusion

Our study finds that deep/extreme poverty have increased in the United States from 1993 to 2016, particularly among childless households. Sullivan does not directly challenge our conclusions. Instead, his response largely sidesteps that as a direct response to his reviewer concerns, we had already addressed concerns of TRIM3’s SNAP imputations and earnings underreporting among lower-income households. This reply adds new analyses that reinforce our conclusions and demonstrate that Sullivan’s concerns are misplaced.

Debates on deep/extreme poverty have become polarized in recent years. On one side, Edin and Shaefer (2015) found very high levels of $2/day poverty. On the other side, Sullivan and Meyer et al. suggested that extreme poverty is basically non-existent. In that vein, Sullivan makes the unsubstantiated claim that levels of extreme poverty must be lower than we find. By contrast, we reaffirm that the best approach to analyzing deep/extreme poverty is to conduct open science, report a range of estimates under a variety of transparent assumptions, follow prevailing international standards in income and poverty measurement, and use justifiable thresholds (Brady and Burton 2016; Brady et al. 2018). Doing so, we confirm that deep and extreme poverty in the United States remain significant and increasing social problems.