Testing, Done Right
Dan Koretz had had enough. For nearly 30 years, Koretz had accumulated mounting evidence of score inflation — increases in test scores much larger than actual improvements in learning — across school districts following decades of federal education policy, like No Child Left Behind, that reinforced test-based reforms.
But most school leaders and policymakers ignored his evidence of not just inflation but bad test preparation, massive cheating, and more negative consequences of high stakes testing on students and teachers.
For years, Dan Koretz advocated for a more sensible and equitable accountability system. One of his colleagues, Andrew Ho, is now leading the charge Koretz initiated.
After years spent shaping the field of educational testing with his research, Koretz didn’t pull his punches in his 2017 book, The Testing Charade, excoriating high stakes testing as a fraud and a system that had undermined the goals of meaningful learning.
He wrote, “I didn’t foresee that test-based accountability would fundamentally corrupt the notion of good teaching, to the point where many people can’t see the difference between test prep and good instruction.”
And yet, in the end, Koretz didn’t blame testing itself.
“Testing done right can be tremendously useful,” he says. “I argue that a replacement for the current, failed system should include sensible testing. The problem isn’t testing; it’s the misuse and sometimes abuse of testing.”
While most school systems have yet to implement changes to testing, a new generation of researchers is continuing the quest Koretz began.
“Dan has sharpened our understanding of the weaknesses and the limitations of tests as policy tools,” says Professor Andrew Ho, whose research aims to improve the design, use, and interpretation of test scores in educational policy and practice. “He’s given everyone a ‘Koretzian’ perspective. He has a lot of trust in test scores to reveal inequity, but he has this very sophisticated theory, borne out in data, that the progress anyone chooses to report is biased.”
Koretz and Ho champion an accountability system that actually measures what matters, including multiples measurements of student and teacher success, realistic targets for educators, and stakes that are commensurate with the supports given.
Through his work with the Educational Opportunity Project at Stanford University, Ho is working to gather testing data that will offer a more comprehensive story about school and district educational progress over time.
Take Chicago, for example. Compared to the performance of other large districts across the country, Chicago may seem like it is underperforming. But a closer look at trustworthy test score data indicates that the Chicago school system has shown sustained average growth over the last decade, in large part by implementing many of the methods Koretz and Ho have laid out to avoid inflation.
“Dan cares about progress, and that’s what we’re and that’s what we’re trying to measure and understand,” Ho says. A recent presentation shows Chicago’s steady rise over its peers, “Like a sunrise over the horizon,” Ho says.
Koretz retired from HGSE in 2019, but his alarm over high stakes testing is no longer being ignored and its echo is heard through the work of researchers like Ho who continues to work to improve testing systems.
“We think of test scores as numbers that tells us if you’re a good student and go to a good school,” Ho says. “That’s reductive and reveals a fixed mindset. Data shows kids and schools make massive amounts of progress, and that’s the promise of test scores — a better understanding of where we’ve made progress and where we can make it in the future.” – Andrew Bauld