A recent NY Times article entitled Lauded Harlem Schools Have Their Own Problems, discusses how Harlem Children’s Zone (HCZ) charter schools’ tests scores fail to impress in light of tougher state exams. The primary concern is whether the evidence warrants the level of federal contribution promised by the current administration. Right now, no one can say for sure. However, this article raises some important points about how to measure the impact of such programs, especially when taking cost into account.
The article focuses on standardized tests (pass rates) as the main indicator of the success of the HCZ model. But test score impact should be measured on the basis of progress. While a relatively small percentage of students may be passing state exams, the question to ask is how many were passing before they enrolled in one of the HCZ schools? How many students are passing in other Harlem schools? The article alludes to these indicators, noting, for example, that while only 38% of Promise Academy’s 3rd-6th grade students passed the 2010 English test, in Harlem overall the pass rate is even lower at 29%. And change takes time. Most HCZ high school students start several grade levels behind and take three or four years to catch up. By graduation, many are performing above grade-level. Regardless of how many students are passing state tests, moving a child from three years behind to even one year behind is progress. These are the kinds of indications of impact that funders – philanthropists or the government – should look for.
Ultimately, the Harlem Children’s Zone efforts are about much more than improving test scores; they are about “cradle to college” services that improve life outcomes and opportunities for an entire community. Therefore, in many ways the more important impact measures are those directly tied to life outcomes: for example, lower incarceration rates, better health outcomes, and higher earning potential. It is these kind of impacts that can justify (or challenge) the higher than average costs of the HCZ. Despite high costs of this particular model, the potential savings to society are huge. Considering costs in isolation tells you nothing about return on investment.
As the article states, it will be many years before anyone can decisively comment on the long-term impact of the Harlem Children’s zone and its goal to break the cycle of poverty. Hence, the tug-of-war between the need to prove effectiveness and the need to take action continues. For now, what we know is that these efforts in Harlem have impacted thousands of children and that so far, these children seem to be improving according to multiple indicators (educationally and otherwise). There may still be a long way to go, but there’s no need to dismiss the Harlem Children’s Zone just yet.
Thanks to CHIP teaching quality team members Kate Barrett, Kate Hovde, Jen Landres, Kat Rosqueta, and research assistant, Masha Jones, for their contributions to this post.