Road to COVID Recovery

Across the country, school districts are implementing new strategies to support student learning loss stemming from COVID-19 disruptions. But understanding the efficacy of those interventions will be crucial to keeping kids from falling even further behind.

That’s the goal of the new Road to COVID Recovery project led by the Center for Education Policy Research (CEPR) at Harvard University. Using real-time data, the group will provide districts with evidence of which interventions are working, and which aren’t, to guide planning of catch-up efforts.

“Over the last few years, how often have we heard that science should drive decisions?” says Professor Tom Kane. “We need science to drive decisions in the K–12 world.”

Along with CALDER at the American Institutes of Research, and NWEA, an international research-based academic assessments nonprofit, Kane and his team have partnered with a dozen large, urban districts to evaluate the effectiveness of pandemic recovery efforts, including tutoring, summer enrichment, and extracurricular programs like afterschool and Saturday school. Kane says that if districts wait for test results from the end of the 2022 school year to guide their thinking, it will be too late to implement adjustments in time for the next school year, or to capitalize on federal COVID relief funds.

“This isn’t something the education research field was set up to provide, so to some extent we’re having to invent a system for doing this,” Kane says. “But if we don’t create it, districts aren’t going to have the information to help kids catch up over the next couple of years.”

This past spring, Ed. sat down with Kane, the faculty director for CEPR, to learn more about the Road to COVID Recovery project and his urgency to get districts the evidence they need to make timely decisions.

How did the Road to COVID Recovery project come together?

Last spring, we began speaking with state departments of education about their plans to measure the efficacy of different COVID academic recovery efforts and we discovered that there was really no planning going on for collecting the kind of data that would be required to learn what kinds of interventions would make a difference. After a number of those conversations, we eventually identified a partnership with NWEA as probably the best, most efficient way to generate evidence on COVID catchup and provide evidence to the field. NWEA data cover every state in the country, and we have the data now to evaluate interventions. NWEA also has fairly complete testing from the fall of 2020. A lot of states, even if they did administer tests in the spring of 2021, have incomplete data because they don’t have a baseline score to measure if kids are making progress.

In a piece in The 74, you put student learning loss in pretty stark terms, stating that if gains aren’t made now, students could lose $2 trillion in lifetime earnings. Is the situation really that dire?

I don’t think there’s a broad appreciation for the magnitude of the declines we’ve seen. There’s been a .25 standard deviation loss in math achievement, and around a .13 or .14 standard deviation loss in literacy achievement. Because not everyone is conversant in standard deviation, those numbers sort of roll off of people’s backs, but to try and make it a bit more tangible, effectively it’s as if kids missed three or four months of school last year. And if we just let those losses stand, we’ll be imposing a major debt on future generations because it’s going to have substantial impacts on kids’ careers. There’s this window in the next couple of years where districts will be making decisions about use of federal money that, if they don’t get the evidence to adjust strategies, these losses will become permanent and that loss will be, to some extent, not just a tragedy for children but a major failure by the research community to provide the evidence districts need to make these decisions. That’s the problem we’re trying to avert.

Many districts have already begun to implement interventions like tutoring. How will your research help school districts make effective choices?

There’s nearly $200 billion the federal government provided to schools for COVID measures and academic recovery, and school districts are making decisions about what interventions to try in the absence of evidence on whether they are working. There’s pre-pandemic evidence on the effects of things like tutoring or double dose math courses or afterschool programs and summer school, but districts are going to need evidence of the efficacy of the things they are implementing this year and next to guide their decisions. If they are waiting for the state test scores, they aren’t going to know until the fall just how far their current efforts have undershot. For example, research has found that high-dosage tutoring, with four or fewer students, three times a week, for an hour a day, can have an impact. But to close a .25 standard deviation gap in math, we would need to offer that intervention to two-thirds of all students. My fear from what I’ve seen so far is that the interventions aren’t nearly at the scale required to close this gap, and districts aren’t going to see their state test scores until it’s too late.

How do you balance these interventions, some of them intense as you’ve mentioned, with students already burned out and numbers showing student enrollment dropping for the second straight year?

Many of the districts we’re working with are implementing interventions to support the mental health of students and the social-emotional outcomes for students. We’ll be looking to see if some of these interventions meant to support social-emotional well-being have any effect on math and reading scores. To some extent, the debate has been framed as a false dichotomy between should we help kids with academic catch up or forget about academics for now and focus primarily on mental health? Well, of course it’s not either/or. Kids are suffering both emotionally and academically, and interventions in either sphere are likely to have spillover into the other, so if we have kids feeling more comfortable being back in school, chances are they are going to learn more in math and reading, and conversely, if kids are finding success in the classroom, that’s going to have some positive effects on mental health. I think we can and should be working on helping students recover emotionally and academically, and we shouldn’t be confused to think we can put off the academic comeback until after students are fully comfortable being back in school. We have to work on both simultaneously.

Andrew Bauld, Ed.M.’16, is a freelance writer and podcast host.