RESEARCH STUDY PROPOSAL

Presidential Address: Education and Poverty: Confronting the Evidence

Helen F. Ladd

Abstract

Current U.S. policy initiatives to improve the U.S. education system, including No Child Left Behind, test-based evaluation of teachers, and the promotion of competition are misguided because they either deny or set to the side a basic body of evidence docu- menting that students from disadvantaged households on average perform less well in school than those from more advantaged families. Because these policy initiatives do not directly address the educational challenges experienced by disadvantaged students, they have contributed little—and are not likely to contribute much in the future—to raising overall student achievement or to reducing achievement and educational attain- ment gaps between advantaged and disadvantaged students. Moreover, such policies have the potential to do serious harm. Addressing the educational challenges faced by children from disadvantaged families will require a broader and bolder approach to education policy than the recent efforts to reform schools. C© 2012 by the Association for Public Policy Analysis and Management.

INTRODUCTION

Evidence-based policymaking. That is the rallying cry for policy researchers like many of us and also for many policymakers, including the Obama administration itself. Providing a forum for researchers to present and discuss policy-relevant re- search that can provide the evidence needed for better policymaking is one of the major functions of the Association for Public Policy Analysis and Management (APPAM).

Policy-relevant evidence often comes from careful studies of specific policy in- terventions such as job training or negative income tax programs and is based on random control trials or other forms of rigorous quantitative and qualitative analysis. Many of you in the audience today have made major methodological and substantive contributions through research of this type in a range of policy areas.

I want to focus today on the policy importance of evidence of a broader type—a type that does not require any sophisticated modeling. And I will do so in the context of my main field of policy research, education policy.

Historically this country prided itself on its outstanding education system, which educated a higher proportion of its population to more advanced levels than most other countries. The Sputnik challenge from Russia in the late 1950s and the pub- lication of A Nation at Risk (1983) during the Reagan years, however, highlighted significant concerns about the quality of the U.S. education system. Concerns today are based on average test scores of U.S. students that are middling compared to Journal of Policy Analysis and Management, Vol. 31, No. 2, 203–227 (2012) C© 2012 by the Association for Public Policy Analysis and Management Published by Wiley Periodicals, Inc. View this article online at wileyonlinelibrary.com/journal/pam Supporting Information is available in the online issue at wileyonlinelibrary.com. DOI:10.1002/pam.21615

204 / Education and Poverty

those of other nations, on U.S. graduation rates that once were well above those of most other countries but now have been overtaken by rising rates in other countries, and on abysmal educational attainment and test score performance of many disad- vantaged students, especially those in urban centers. These patterns and trends, as well as recent widely publicized documentaries including for example, Waiting for Superman, have convinced many people that our education system is in crisis.1

During the decades following A Nation at Risk, U.S. education policymakers re- sponded to the perceived crisis in a variety of ways such as creating ambitious national goals and promoting standards-based reform. Of interest here are the pol- icy initiatives of the past decade, which include school accountability in the form of the federal No Child Left Behind (NCLB) Act, test-based approaches to eval- uate teachers, and promotion of expanded parental choice, charter schools, and competition.

I will argue today that these current policy initiatives are misguided because they either deny or set to the side a basic body of evidence documenting that students from disadvantaged households on average perform less well in school than those from more advantaged families. Because they do not directly address the educational challenges experienced by disadvantaged students, these policy strategies have con- tributed little—and are not likely to contribute much in the future—to raising overall student achievement or to reducing achievement and educational attainment gaps between advantaged and disadvantaged students. Moreover, such policies have the potential to do serious harm.

Addressing the educational challenges faced by children from disadvantaged fam- ilies will require a broader and bolder approach to education policy than the recent efforts to reform schools. It will also require a more ambitious research agenda, one that APPAM researchers—not just those of us who typically focus our research on education policy, but also researchers in a wide range of social policy issues—are in a good position to advance.

EVIDENCE ON THE RELATIONSHIP BETWEEN FAMILY BACKGROUND AND EDUCATIONAL OUTCOMES

Study after study has demonstrated that children from disadvantaged households perform less well in school on average than those from more advantaged households. This empirical relationship shows up in studies using observations at the levels of the individual student, the school, the district, the state, the country. The studies use different measures of family socioeconomic status (SES): income-related measures such as family income or poverty; education level of the parents, particularly of the mother; and in some contexts occupation type of the parents or employment status. Studies based on U.S. administrative data often measure SES quite crudely, using eligibility for free and reduced price lunch, for example, as a proxy for low family income, and using student race as a proxy for a variety of hard to measure charac- teristics. Studies based on longitudinal surveys often include far richer measures of family background. Regardless of the measures used and the sophistication of the methods, similar patterns emerge.

I start with differences in test scores between U.S. students whose families have high and low SES as measured by family income. The best research on income-based achievement gaps appears in a recent study by Sean Reardon for which he compiled test scores for school-aged children and family income from a large number of U.S.- based nationally representative surveys over a 55-year period. By standardizing

1 Not everyone agrees that the system is in crisis. See, for example, the critique of this view by Berliner and Biddle (1995).

Journal of Policy Analysis and Management DOI: 10.1002/pam Published on behalf of the Association for Public Policy Analysis and Management

Education and Poverty / 205

Figure 1. Trends in Income and Black-White Gaps in Reading, 1943 to 2001 Cohorts (Simplified Version of Graph in Reardon, 2011, ch. 5).

income differentials and achievement levels to make them comparable over time, he was able to estimate the trend in reading and math test scores gaps between the children in the 90th and the 10th income percentiles. As shown by the rising line in Figure 1 for reading gaps, the results are striking. The figure shows that, when first measured in the early 1940s, the gap in reading achievement between children from high- and low-income families was about 0.60 standard deviations. It subsequently more than doubled to 1.25 standard deviations by 2000.2

These income-based achievement gaps are large. To put them in perspective, consider the black-white test score gap as measured by the National Assessment of Education Progress (NAEP) for 13-year olds, depicted by the dashed line in Figure 1.3 That gap was about one standard deviation in the 1970s, then fell to about 0.50 during the 1980s where it has remained relatively constant. As a result, the achievement gap between children from high- and low-income families is now far larger than the gap between black and white children.

People can disagree about whether the relationship between family income, or broader measures of SES, on the one hand and educational outcomes on the other is correlational or causal. For example, it may be that factors correlated with low income such as poor child health or single-parent family structures account for

2 Figure 1 is a simplified version of graph 5.3 in Reardon (2011). The trend line is estimated based on the income-based achievement gaps calculated from the 12 nationally representative studies that include data on reading scores for school-age children and information on family income. The fitted regressions line is weighted by the inverse of the sampling variance of each estimate. The figure for math is similar (see Figure 5.4 in Reardon, 2011). 3 The estimated black-white gap trend line is based on all the available black-white gap information that is available in NAEP long-term trends for 13-year olds and main NAEP for eighth graders, with the latter adjusted for age differences. The line can be interpreted at the trend in the gap for 13-year olds. See footnote 6 in Reardon (2011).

Journal of Policy Analysis and Management DOI: 10.1002/pam Published on behalf of the Association for Public Policy Analysis and Management

206 / Education and Poverty

the relationship rather than income itself. Further, people may disagree about the extent to which schools and school policies contribute to the low achievement of children from low-SES households. At this point, I simply want to draw attention to the correlation. Later I will say more about the mechanisms through which low SES may translate into low academic performance.

Suffice it to say at this point that research documents a variety of symptoms of low SES that are relevant for children’s subsequent educational outcomes. These include, for example, poor health, limited access to home environments with rich language and experiences, low birth weight, limited access to high-quality preschool opportunities, less participation in many activities in the summer and after school that middle-class families take for granted, and more movement in and out of schools because of the way the housing market operates for low-income families. Differences in outcomes between high- and low-SES families may also reflect the preferences and behaviors of families and teachers. Compared to low-SES families, for example, middle- and upper-class families are better positioned to work the education system to their advantage by assuring that their children attend the best schools and get the best teachers, and they are more likely to invest in out-of-school activities that improve school outcomes such as tutoring programs, camps, and traveling.4 The preferences and behaviors of teachers are also a contributing factor in that many teachers with strong credentials tend to be reluctant to teach in schools with large concentrations of disadvantaged students than in schools with more advantaged students (Clotfelter, Ladd, & Vigdor, 2011; Jackson, 2009).

The logical implication of the low achievement of poor children relative to their better-off counterparts is that average test scores are likely to be lower in schools, districts, or states with high proportions of poor children, all else held constant, than in those with fewer poor children. Figure 2 illustrates this negative relationship be- tween child poverty and test scores across U.S. states in 2009, with eighth-grade reading scores in Figure 2a and eighth-grade math scores in Figure 2b. The achieve- ment scores in these graphs are from the NAEP and are based on random samples of students in each state while state poverty rates are from the American Community Survey.

Of course, not all else is constant. Among other things that differ across states is the quality of the states’ education systems. Test scores in Massachusetts, for exam- ple, far exceed their predicted levels given the state’s 12 percent child poverty rate, presumably in part because the state implemented an aggressive and comprehen- sive education reform strategy in 1998 that included support for young children. In contrast, test scores in California, are well below those predicted for its 20 percent poverty rate, presumably in part because of its long history of limiting spending on education. Moreover, other factors may also contribute to the patterns. Mas- sachusetts, for example, has a highly educated parental population, and California has a large immigrant population. Nonetheless, the overall negative relationship between the child poverty rate and student performance in both graphs is clear.

Consistent with the graphs, a simple bivariate regression of state test scores and state poverty rates indicates that a full 40 percent of the variation in reading scores and 46 percent of the variation in math scores is associated with variation across states in child poverty rates. The addition of one other explanatory variable related to family background, the percent of children who are members of minority groups, increases the explanatory power of the relationship to about 50 percent in reading

4 See Duncan and Murnane (2011) and the articles therein for detailed empirical analysis of many of these mechanisms.

Journal of Policy Analysis and Management DOI: 10.1002/pam Published on behalf of the Association for Public Policy Analysis and Management

Education and Poverty / 207

Figure 2. (a) State National Assessment of Education Progress (NAEP) Eighth- Grade Reading Scores and Child Poverty Rate 2009. (b) State NAEP Eighth-Grade Math Scores and Child Poverty Rate 2009.

Journal of Policy Analysis and Management DOI: 10.1002/pam Published on behalf of the Association for Public Policy Analysis and Management

208 / Education and Poverty

Table 1. Within-state changes in National Assessment of Education Progress (NAEP) test scores (standardized) as a function of within-state changes in the child poverty rate.

4th-Grade 8th-Grade

Reading Math Reading Math

Child poverty rate (%) −0.023* (0.012) −0.030*** (0.011) −0.030** (0.012) −0.030*** (0.010) Constant 0.402* (0.209) 0.514 (0.194) 0.523 (0.205) 0.518 (0.0177) State fixed effects? Yes Yes Yes Yes Observations 282 240 277 239 R2 0.908 0.932 (0.917) (0.944)

Notes: Sample is NAEP test scores (standardized across states) for years 1998, 2002, 2003, 2005, 2007, and 2009 for reading and for years 2000, 2003, 2005, 2007, and 2009 for math. Calculations are by the author. *indicator significance at the 10 percent level, **at the 5 percent level, and ***at the 1 percent level.