‘Scorecard’ of 4,000 Schools Shows Rural Districts Fared Better in Math, Worse in Reading Than Urban, Suburban Peers

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

Rural school districts suffered the smallest academic setbacks in math during COVID compared to urban and suburban systems, yet simultaneously the largest losses in reading, new data show.

The puzzling finding comes from figures released Friday by education researchers at Harvard and Stanford universities. Their new resource, the Education Recovery Scorecard, compiles test scores from roughly 4,000 schools that serve some 12 million students in third through eighth grades across 29 states and the District of Columbia. Their work presents the first school system-level analysis of student learning through the pandemic.

Students in urban and suburban schools were 65% and 54% of a school year behind in math, respectively, according to the Scorecard; yet young people at rural schools were buffered somewhat, losing half a year. Meanwhile in reading, where drops were less dramatic overall, rural students fell behind by a third of a school year while urban and suburban students were just 29% and 24% of a year back.

Focusing on math scores, where COVID losses were more stark, Harvard education professor Thomas Kane said his findings are relatively good news for rural schools.

“The average rural district fared better than the average urban district,” he said in an email.

The reasons explaining the trend still remain something of a mystery, added his research partner, Stanford education professor Sean Reardon.

“We haven’t had a chance yet to dig into possible explanations for why we see some difference between rural and urban districts,” he said. 

The Scorecard’s analysis also includes town school districts, which are typically less remote and more populated than rural districts, but further from city centers than suburban districts. Town school systems had drops in math similar to their suburban counterparts and losses in reading slightly less severe than rural districts.

Alex Baum works as director of advocacy and research at the Community Foundation of Greater Dubuque, which supports several rural counties in Iowa and runs a regional reading initiative. He has not yet seen “clear and convincing data” explaining the differences between rural and urban schools through the pandemic, he said. (Iowa is not yet included in the Scorecard.) But he believes part of the reason for relatively larger drops in reading scores among students who live in more remote locations may be their limited access to outside-of-school learning opportunities.

“With fewer literacy programs, summer learning options [and] libraries, … rural students didn’t have the additional support that urban students had, leading to a larger decline in reading proficiency scores,” Baum said. 

Nancy Mondragon, a classroom aide in Arkansas’s Waldron School District, said the children’s struggles reflected the parents’ economic woes.

“In rural communities like ours, parents were juggling a lot in trying to make a living, and children spent more time on devices,” she said. “The reading loss definitely shows.”

Mondragon is training to become a lead teacher through Reach University, a program working to combat teacher shortages in rural districts. Reach Dean of Undergraduate Studies Kimberly Eckert believes those persistent staffing issues may also be a factor behind literacy losses.

“For reading, where different learner needs demand different interventions, the teacher shortage may be more of the issue here,” she said. “Access to reading specialists in rural communities may have been impacted by COVID, even if schools remained open.”

The length of school closures has been a commonly cited factor in the conversation around missed learning, particularly in math. Schools play an especially large role in teaching students arithmetic, and sparsely populated areas were less likely to shutter their campuses during the pandemic. 

It’s “plausible” that the difference between rural and urban districts’ scores could have hinged significantly on rural systems reopening sooner, said Reardon. But vastly different virtual learning programs from one district to the next and with young people juggling traumatic life-changing events like the death of a caregiver, he stressed that other factors also certainly played in.

“There was a lot happening in communities that shaped the magnitude of learning losses in addition to when schools were open or remote,” he said.

Overall, the Harvard and Stanford research suggests that school closures were not the primary element explaining achievement losses. Though setbacks tended to be higher where schools remained closed longer, there was wide variation in the results. Districts that remained virtual for equal amounts of time often had vastly different learning outcomes, and states and school systems that underwent extended closures in many cases outperformed those that opened sooner for in-person learning.

The Scorecard relies on state test scores for its analysis, but because many states use different exams, the researchers used results from the Oct. 24 National Assessment of Educational Progress to normalize the data so they could make apples-to-apples comparisons. States that have not yet released their test scores are not included in the analysis, including several with large drop-offs in achievement and high rates of school closures during COVID, such as Maryland, New Jersey and New Mexico.

Beyond differences between rural, urban and suburban schools, their research reveals several other key findings on pandemic learning:

• Losses were most pronounced in high-poverty districts compared to more affluent ones. The wealthiest quarter of districts in the sample maintained an extra 21% of a year’s worth of learning in math and 6% of a year’s worth of learning in reading over the most economically disadvantaged quarter. 

“We’ve seen the greatest increase in inequity in our lifetimes over the last few years,” said Kane. “Unless we act over the next couple of years, these losses could become permanent and could have long-term consequences for social mobility.”

• Math setbacks exceeded a full year’s worth of learning in the school districts of over a half million students, some 5% of those represented in the analysis. On average, students were behind about a half grade level in math and about a quarter grade level in reading. A small minority attended districts that made gains in math, 2.5%, or reading, 15%, from 2019 to 2022.

• Recovering missed learning will be expensive. In nearly two-thirds of districts included in the analysis, catching students up will likely require investments exceeding the total cash delivered to schools via the American Rescue Plan. (The researchers reached that conclusion by assuming the cost of additional recovered learning will be equivalent to the typical cost of learning during the school year. In other words, catching students up on 25% of a year’s worth of learning will require 25% of a district’s annual operating budget.)

Tequilla Brownie, CEO of TNTP, said she sees the new data as a “call to action.”

“It is clear that the problems pre-date the pandemic and, therefore, we should not confine ourselves to solutions that present a path back to pre-pandemic levels of proficiency,” she said.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter