What does educational opportunity in America look like? A research team, including Andrew Ho of the Harvard Graduate School of Education and Sean Reardon of Stanford University, has pieced together testing data from almost every public school in every state to find an answer. Their collaborative efforts, known as the Educational Opportunity Project at Stanford University, have resulted in what's being called the first national database of academic performance.
Using an interactive explorer, people can navigate charts and maps that let users visualize test scores, learning rates, and trends in test scores measured against categories like socio-economic status, race, and gender. Ho compares the data, which appear on screen in a cascade of blues and greens, to a patchwork quilt. The data lets users see the kinds of disparities that are typically hidden when looking at smaller data sets — and lets them put it in a national perspective. “It gives you a sense of the scope of inequality in this country, which is just, at a glance, astounding. And within any given state or district, you have a very limited sense of that, like you have blinders on,” Ho says.
The first data set was released in 2017 without visualizations, intended primarily for researchers. While successful in that regard (more than 30 papers that utilize this data set have been published), Ho hopes that these new interactive tools will allow a wider audience of education leaders and practitioners to explore implications as well.
Are you an educator who wants to dig into this data? Here are a few ways to navigate it, with thanks to Ho for walking us through it.
Compare Average Test Scores and Socioeconomic Status
The two graphics below show average test scores and socioeconomic status. The chart allows users to observe the correlation with wealth, and the map shows how that varies with geography. Ho says most users typically start here, asking the question, "Does my school or district have high test scores?" He hopes users will notice that the correlation is not perfect and that test scores are not the only metric to consider.
Compare Learning Rates with SES
Your expectations about socioeconomic status and achievement may shift when you look at learning rates, which are illustrated in the two graphics below. The chart shows the relationship between learning rates (how much students' learning changes over time) and socioeconmic status. You'll notice that the correlation here isn't as tight as it is between test scores and socioeconomic status.
A similar point is illustrated by the map; it depicts the geographical spread of learning rates, which perhaps do not fit as strongly with common narratives about where the best schools in the country are found. Together, these graphics show an alternative way of measuring school success.
What the Data Does
- Puts test scores in the context of socioeconomic status. While the education community has long recognized that higher socioeconomic status tends to correspond with higher test scores, the correlation is made visual here.
- Breaks the boundaries of geography. Districts next door might be geographic neighbors, but there are other places in the country that are similar in performance. This means there may be space for collaboration.
- Allows assessment of your district’s performance against national peers. A district may be high performing in its state, but does achievement hold up on a national scale? The data allows leaders to think about performance nationally — something that was not possible to do so cleanly before. Take note of outlier districts, as there may be interesting reasons and areas of potential study behind them.
- Develops understanding of school “effectiveness” that considers learning rates. “Most people don’t know, or don’t ask, or don’t think to ask how kids are learning through school systems over time,” Ho says. “That’s just not part of our criteria for thinking about our education system, it’s not part of our vocabulary. We say that we care about growth, but even when we say that, we end up just looking at [test scores].”
- Factor in variation. Just because a district has an average socioeconomic status does not mean its levels of achievement and learning rates will necessarily be average as well. Correlation does not make the data predictable. Within the districts of average socioeconomic status, there are two to three grades of variation.
The Big Picture
The visualizations in the Educational Opportunity Explorer help us understand that policymakers may need to reframe the story of school success. For one thing, test scores are not necessarily indicative of school effectiveness. According to Ho, high-scoring districts and schools have a range of effectiveness, as do low-scoring districts and schools. “This creates a really dramatic question to policymakers about what it is we are measuring with average test scores — which continue to be the dominant lever or indicator for school and district accountability,” Ho says.
Be a Critical Consumer of Data
- The website is not intended to help people look at how highly ranked their child or their neighborhood school is. Rather, this site's purpose is to allow people to think more critically about school quality, along different dimensions.
- The data are meant to highlight inequality, not perpetuate it. The proliferation of sites offering school-ranking data brings a risk of self-segregation by wealthier families, Ho says. The data presented must be considered in context, and not just as an indicator of quality.