Some School Districts Saw More Learning Loss During the Pandemic. Researchers Want to Know Why

Researchers at the Harvard Graduate School of Education are still learning many lessons from the impact the pandemic had on learning in school districts around the country.

A recent study cosponsored by the National Center for Education Statistics (NCES) and conducted by Professor Thomas Kane and researchers from Stanford, Dartmouth, Johns Hopkins, and the University of Chicago explored the varying impact remote learning and other pandemic-era policies had on student test scores.

The researchers, including Stanford economist Sean Reardon, Ed.M.’92, Ed.D.’97, studied data from 7,800 different school districts in 41 states to determine which districts saw declining test scores, which student demographics were impacted the most, and what factors may have impacted those numbers.

“Although pandemic-focused recovery efforts have dwindled, the unfinished learning remains in many states and communities,” the study explains. “Identifying where students experienced the highest rates of unfinished learning compared to their peers and probing why some communities realized different learning outcomes compared to others can help states design the next wave of educational reforms.”

Declining Scores, Uneven Impacts

Pandemic learning loss happened across the United States and around the world as schools closed, pivoted to remote learning, or embraced a hybrid approach as pandemic guidance and government regulations adapted and changed to the virus’s spread.

In the process, students learned less. According to the National Assessment of Educational Progress (NAEP), mean student achievement fell by ¾ of a grade level in eighth grade math and a quarter of a grade level in eighth grade reading between 2019 and 2022. But that decline was not shared equally; in some states learning loss was statistically insignificant, while in others that level dropped a full grade or more.

Studying those differences allowed Kane and researchers to “identify the relationships of key predictors” of those trends between districts that fell sharply and others less impacted. According to the study, score declines were larger in districts where more of the 2020–21 school year was spent in remote or hybrid learning. High poverty and high minority districts that spent more time in remote and hybrid instruction were disproportionally affected by learning loss as well.

Nearly all districts studied by Kane and the researchers saw declining scores, but the impact was different across the board. That impact ranged from “near zero” to an entire grade level in reading and math levels.

The study notes that, while there are statistical differences between urban and rural districts, and that scores dropped more in lower-income and minority districts, the differences between students with different economic or demographic circumstances within those districts were actually minimal.

“The most surprising finding was that whole districts seemed to be affected — rather than subgroups of students or schools within districts,” says Kane. “If the primary source of disparity in pandemic losses were household resources — such as access to the internet or the availability of parents working from home or a quiet study space at home — we would have seen much more within-district widening of achievement gaps.”

As Kane explains, entire districts seeing lower test scores across demographics implies that district-level factors — the quality of remote instruction, social and economic disruptions like parent job loss, and even low community trust of government institutions — played a larger role in learning loss than other factors.  

From Bad to Worse

Kane, who leads the Center for Education Policy Research at Harvard (CEPR) and is co-author of the Education Recovery Scorecard, said the study was an effort to “establish the facts regarding what happened” during the COVID-19 pandemic and help encourage districts to keep a focus on recovering learning loss. The pandemic has only “widened already-wide achievement gaps,” he says, and states and districts need to help their students recover, a proposition made more difficult moving forward as resources like ESSR funding and cuts to the Department of Education impact the ability to study these gaps.

“We did this research when states and districts still had a lot of federal pandemic relief funding, and we were trying to sound the alarm that some communities have a lot more work to do than others,” says Kane, noting that research shows high-income communities have recovered from learning loss better than their lower-income counterparts in the years that followed. “Our best hope now is to keep the spotlight on the communities which remain behind to encourage stronger state and local responses.”

The uncertainty that came with the decisions made during the pandemic is gone: districts need to help students, especially their poorest and most vulnerable ones, make up for lost time no matter how well they navigated the COVID-19 pandemic.

“It just seems unfair to force poor children to pay the price for public health measures which were taken on behalf of all of us,” says Kane. “But that’s exactly what too many states — including Massachusetts — are doing.”