The COVID Catch-up Challenge

Many school districts are beginning to see pandemic learning losses greater than they imagined. It’s time to figure out what to do next, says Professor Tom Kane, an economist studying catch-up efforts. He wants districts to be empowered to make the best decision going forward. 

“There's no time to waste,” Kane says. “I know people are exhausted and we don't have to make up for all the loss this summer, but we do need to have a plan how over the next couple of years we're going to catch up and those plans need to be based on real numbers about the magnitude of the losses districts have experienced and our best estimate of the efficacy of the interventions they're planning.” 

In his latest research, he’s helping districts determine how to plan for these losses and shares what interventions might provide the most bang for the buck. In this episode of the Harvard EdCast, Kane shares troubling information about how these learning losses have the potential to shape the future of education and how what we do now can get learning back on track. 

TRANSCRIPT:

Jill Anderson: I'm Jill Anderson. This is the Harvard EdCast. 

As the pandemic begins to slow down. Harvard professor, Tom Kane says it's time to amp up education's efforts to address learning loss of America's schools. He's an economist studying COVID catchup efforts in schools. The pandemic learning loss has affected millions of students. And if we do nothing to target that, he says it could result in a 2 trillion earning loss in their lifetime. It's important for schools to respond. I wanted to know more about what some of these ketchup efforts look like and what is most effective. First I asked Tom about how significant this loss is across the country, 

Tom Kane: Unless we act over the next couple of years to reverse these losses. This will be the first significant widening of the racial achievement gap in 30 years. It's not just that the pandemic affected students learning. It was that it had disproportionate impacts in schools that shifted to remote or hybrid instruction. By the way, high poverty schools were more likely to shift to remote or hybrid instruction. On top of that, when they did shift to remote or hybrid instruction, the losses were much larger for students in high poverty schools. So it's sort of that double effect of high poverty schools were more likely to go remote. And when they did go remote, the consequences for student achievement were much larger. It's really an important moment. Now that we develop a plan, how we're gonna reverse these losses.

Jill Anderson: Districts right now seem to be getting data. And it's kind of confirming perhaps what they feared about learning loss among their students. How do they not get so stuck on the numbers that it keeps them from really making any kind of progress? 

Tom Kane: So, Jill, one of the huge problems in this is that the units that we use to measure achievement losses are just in a totally different set of units for thinking about interventions. So we've not given school districts good, you know, information to sort of make sure that they're right sizing their catch up activity. So if I'm a school district and I hear, oh, we've had a 10%, you know, decline in our proficiency rate or a, you know, a eight percentile point loss in our growth percentiles or something like that, those kinds of statistics, you can't convert that into action. So like the magnitudes of the impacts and the magnitudes of the effects we should expect from things like tutoring or double dose, you know, math or summer school, they're just measured in different units. And so people have had no way to see that the efforts they're currently planning are nowhere near as big as it's gonna be required. 

Now we can't be thinking, okay, we've gotta make up for all of this loss this summer or even next year. Honestly, I think most districts are not gonna make up loss even by this time next year, instead we should be thinking, okay, over the next couple of years, what are the range of things that we can be doing? One thing that's been paralyzing folks is thinking, Hey, look, we're exhausted. Our teachers are exhausted. We can't go out and hire the thousands of tutors we would need. So there's sort of frozen in this moment because the logistical challenges are so tremendous, but here is something that districts could be planning for. That feels much more manageable. If they started planning now to lengthen the school year next year, you know, they're gonna have to pay teachers more to work those say extra five weeks at the end of next school year, but that is one sort of like concrete activity where they already have the buildings they already have of buses, schedules running they already have, or will have the teachers for next school year. 

Jill Anderson: Can you tell me about maybe some of the other interventions that might be helpful? 

Tom Kane: Yes. There's a lot of promising evidence from before the pandemic on the efficacy of high dosage tutoring, where it's groups of three students or less three times a week over the course of the whole school year, actually the pre pandemic evidence suggested that the effect size for that kind of intervention is about equivalent to the magnitude of loss, that high poverty school districts who were for all of, or a majority of 2021, the losses they saw were about the effect sizes of tutoring. So that gives you another sense of just the magnet to the challenge here. So basically high poverty school districts that were remote for most of 2021 would have to provide a one-on-one tutor to every student, right? To generate the magnitude, you know, effect size to eliminate this loss. That's how big this loss is. So we see a bunch of districts, planning, tutoring exercises. 

I haven't seen a single district planning to provide tutors that all students there are other things people are planning or things like, you know, double dose math. So giving an extra period of math or an extra period of, of reading. That's not free either. I mean, it's, it may not be as expensive as a tutor, but you still have to, you can't like just double math teachers, teaching requirements, you'd have to go out and hire math teachers. That's why I was arguing for extending the school year. You're not there. You wouldn't be hiring a lot more teachers, which is gonna be challenging for a lot of districts. A third kind of thing that districts are are planning are after school interventions. Again, there is research on afterschool programs, but it's not super promising. Kids are tired at the end of the school day. And afterschool programs tend to do enrichment activities rather than, you know, math or reading instruction, which seems appropriate. 

So, so we are seeing some districts implementing after school programs that do have more academic content. And one of the points of our ongoing project will be seeing whether they're having effects on the rate at which students are catching up. Many districts are planning summer school interventions, expanding summer school, the best evidence I've seen on the effects of summer school implied that for the students that attend summer school, you could expect about like one quarter of the impact that high poverty schools saw during the pandemic. So it'll help. But even if you got a hundred percent of kids to show up at summer school, you'd only close about a quarter of the magnitude gap that we've seen. That's assuming that these programs have the same effect post pandemic, as they had in the prior research, which may not turn out to be true. 

Jill Anderson: Hmm. So there's a lot of unknowns that we're working with here. 

Tom Kane: The way to start be, to say, okay, what's the magnitude of gap that our district has suffered? What are the range of interventions and what are the effects sizes for those interventions from the pre pandemic research literature. And then what is a plan over the course of the next two that has a hope of, you know, producing the size academic catch up that we need. So what I've been trying to say, Jill, is that I haven't seen anybody put those two together yet saying, okay, we've seen losses. What's a realistic plan for eliminating the at magnitude of loss. People know their losses, but they're having trouble putting together a plan that has a realistic chance of eliminating the losses. And the losses are so large that most districts that I have seen are way underestimating the amount of, of additional academics support students are gonna need. 

Jill Anderson: You know, it's interesting how you mentioned the longer school day or the longer school year, because those have been things in education there's been advocacy for that and resistance to that for a long time. And I'm wondering if districts are maybe afraid to go there because there has been so much controversy around those interventions in particular. 

Tom Kane: Yeah. First of all, I think these wouldn't have to be permanent measures. These could be measures over the next couple of years. Mm. And number two, school districts should be prepared to pay teachers a lot more than their usual weekly rate to add on like five or six weeks at the end of the coming school year. But obviously school districts have federal dollars that they've received that are meant for COVID catch up. And actually it turns out almost by coincidence because, you know, nobody knew the magnitude of the achievement losses at the time that the American rescue plan was passed. But the amount of federal aid that high poverty school districts have received is about 40% percent of an annual year's budget. So remember I started out saying that it would cost, you know, about 40% of an annual budget to make up for these gaps and school districts have received at least high poverty school districts have received about that much on average in federal aid. 

So it's just means they'd have to spend close hundred percent of it on academic catch up. Unfortunately, the federal law only requires them to spend 20% on academic catch up. And I think that's about the range that a lot of districts are thinking, but that's not very useful guidance because that's about one fifth, the size that's gonna be needed. Uh, the danger Jill is like here is the scenario I'm very nervous about playing out. Most states are gonna be doing testing this spring. And when those test scores come back in August school districts around the country are gonna see large losses, but especially in places that were remote or hybrid last year, they'll say, oh, okay, well, we've got a set of interventions that we're planning for the fall. Cause by the time those scores come out in August, it'll be too late to have like major new plants. 

They'll go through all next year. I can tell you right now that what most districts are, is not anywhere near close enough to produce the magnitude catchup that we need. So when kids take the test next spring, you know, April, May, 2023, then they'll get results back in August 23 that, Hey, wait, like our kids are still way behind. The problem is at that point, the 23-24 school year is about to start. And that's the last year that after that year, basically the federal dollars run out, unless we get folks starting to plan more, realistically. Now the prospect that these losses become permanent, you know, is not so farfetched on the path we're on right now. It seems likely 

Jill Anderson: There's only a limited amount of time to do catch up essentially is what you're ...

Tom Kane: Saying. Yeah. Right. Part of that is coming from the federal dollars. But also part of it is, you know, marches on the more and more people are graduating from high school. Like they're just graduating from high school with a lot less academic preparation than kids who graduated before the pandemic had. Now they're going off to college. And first of all, there has been a big decline in college enrollment. But even if that rebounds, that means that a lot a of kids are gonna be having to do remedial courses in college and so forth. So there's no time to waste. I, I know people are exhausted and we don't have to make up for all the loss this summer, but we do need to have a plan how over the next couple of years we're gonna catch up in those plans need to be based on real numbers about the magnitude of the losses districts have experienced and our best estimate of the efficacy of the interventions they're planning, 

Jill Anderson: Right? If you're waiting until the fall to do this, then you've waited too long. Essentially you need to do it now. And you should just prepare for cuz what's the worst thing. It could happen. You create a plan and, and the kids exceed whatever 

Tom Kane: That's exactly right. Jill, economists call this option value. Like if I'm wrong and kids bounce back faster than anybody predicted. Okay. Then that means say we have a two year plan and we only need to do one year of it. But if you undershoot now it's much harder to adjust later. Like it's easier to scale down than to discover too late that wait a minute. Like what we planned was nowhere near enough. And actually unfortunately the federal guidance is gonna lead. If most district think all we, all we need to do is spend 20% on academic recovery. In some districts, that'll be fine, but not in districts that were remote or hybrid for much of 2021. Those districts will need to be spending considerably more than the 20% of the federal aid on academic catch up to successfully catch up. 

Jill Anderson: How do we act actually get this out there so that educators and districts can make these shifts. 

Tom Kane: Here's one thing Jill, that we're working on, we don't have it, you know, ready yet. We're gonna try to like create a worksheet where districts could enter statistics on just the magnitude of their achievement losses, you, and then we could say, okay, well here's the best evidence that we have on the effects of tutors. Here's the best evidence that we have on the effect of double dose math after school, summer school, you fill in you district fill in what proportion of your students that you're planning to serve with each of these different kinds of interventions. And here's our best guess for how much of a gap those would add up to closing and then compare that to your achievement losses. Like just to make it easier for people to do this planning. I think the research world has done a decent job of saying, okay, here are the evidence based interventions like tutoring. The thing that we've done, a terrible job at in fact, we've basically done. None of is allowing people to compare the efficacy for those kinds of interventions, to the size of the losses they've achieved and make it easier for people to recognize the inadequacy of most of their catch up plans. 

Jill Anderson: Some of these interventions might be deemed a little controversial because people do feel burnt out, getting a teen to go to school longer, might be a challenge. We have all these social emotional challenges with kids. Yeah. Yeah. So how do you balance all of these things with the fallout of the pandemic to really create something and get the community on board? 

Tom Kane: So I think it'll help if it gets pitched as, okay, this is not something we're gonna be able to do. We have to do by the end of the year or by the end of the next school year, it's gotta be at least a couple a year plan. And so if we think about it that way, it'll a, we have more time to do the negotiating for how much it will cost. Like I don't think school districts should be thinking, oh, we'll just pay the normal weekly rate to get people to work, you know, a longer school year. No, I, you know, it's time and a half say for those extra weeks, but if we recognize that it's a two year timeline, if school districts start recognizing, okay, we're not gonna be able to do this on the cheap, we're gonna have to be using a huge chunk of our federal dollars to do this. 

I do you think that, you know, most districts would be able to come up with a plan that would achieve this. I was just doing a plot the other day, like over the last 30 years, we have been actually I think in the education world, we've been under appreciating this, but over the last 30 years, we have made progress in of narrowing achievement gaps. And if we allow these losses to become permanent, this would set us back decades. So it really is sort of the most pressing equity challenge. Our country faces cuz the pandemic had very inequitable impacts on students. in high poverty schools were hit much harder and racial gaps have widened. And unless we can develop a plan over the next couple of years to reverse this, we'll be talking about the great U-turn in the us effort to provide educational opportunities for all students. 

Jill Anderson: But the good news is we have still have the time and we still have the money. 

Tom Kane: We have the time we have the money, we just gotta have a plan and we can't be spending on things that aren't helpful for reducing these gaps. 

Jill Anderson: Tom Kane is a professor at the Harvard Graduate School of Education and the faculty director of the Center for Education Policy Research. I'm Jill Anderson. This is the Harvard EdCast produced by the Harvard Graduate School of Education. Thanks for listening.