A Tribute to Dan Koretz

Professor Dan Koretz has spent his career examining the impact of high-stakes testing, including by looking at score inflation and the effects of testing on educational practice, among other areas. His most recent book, The Testing Charade, notably challenged long-held beliefs in the value of test-based accountability. This week, the HGSE community will celebrate Koretz at a reception marking his retirement from the faculty. Here, his colleague, Professor Andrew Ho, reflects on Koretz's many accomplishments and contributions to the field, to HGSE, and to his own personal development as a policy-minded scholar and researcher.

Unmasking the Testing Charade

By Andrew Ho

My colleague, mentor, and friend, Dan Koretz retires next month after 18 years at Harvard and a career critiquing and improving educational testing. Dan’s ideas have so singularly shaped the field that they are known by one word: “inflation.” Whenever test scores climb, whenever gaps close, whenever stakes rise, Dan’s work demonstrates why tests can’t be trusted and what to do about it.

Generations of students, colleagues, policymakers, and reporters now look at test scores and ask themselves, “What would Dan Koretz say?” The answers lead us to rigorous auditing of our work and the work of others, to the truth behind illusory gains and false findings.

What amazes me about Dan’s contributions is how deeply they change the way we see the world. More than a simple statistic, buzzword, or finding, Dan’s theory of inflation leads us to a clearer understanding of how tests work, not just as instruments of measurement but as tools of policy.

At first, the idea of inflation can seem like simple cynicism: Don’t trust touted improvements in test scores. Dan’s early work in states like New York and Kentucky resulted in iconic graphs unmasking inflated test score gains by comparing them to gains from other tests. I am confident that Dan’s students can sketch these graphs from memory!

But these graphs are just the hook. Dan shows us how the ingredients for inflation — the way we design tests, the way we interpret test score gains, and the way we use tests for accountability — combine to lead us astray. It is a recipe for undermining test validity, in a cake we keep on baking. His 2008 classic, Measuring Up, explains these concepts with clear language and compelling real-world examples. It is a masterwork that belongs on every shelf.

Among dozens of publications and reports, my other favorites of “Koretzian” scholarship include his 2006 chapter on accountability testing with Laura Hamilton. It includes a comprehensive treatment of the theory of inflation, placed with authority in the “bible” of our field, the fourth edition of Educational Measurement. (I have the honor of authoring the follow-up to this chapter in the fifth edition in the coming years, and I hope to do its predecessor justice.)

And, of course, there is his 2017 gem, The Testing Charade, where Dan pulls no punches in his takedown of two generations of misplaced faith in test-based accountability. These two books and this classic chapter are never more than an arm’s length from where I sit at work.

Dan chaired the search that brought me to Harvard in 2010, and he has been an essential role model, mentor, and friend to me. I am grateful for the reputation for policy-relevant testing work that he established over his first nine years at Harvard, and for the last decade that he and I have had advancing that legacy together. His clear arguments and his relentlessly high standards have inspired everyone who enters his orbit.

Dan Koretz may be retiring, but Koretzian ideas and Koretzian standards will spread on, at HGSE and beyond, carried by all who have known Dan and had the privilege of learning from him.

Highlights

1979–1987  Served as analyst at the Human Resources and Community Development Division of the Congressional Budget Office, United States Congress

1987–2001 Worked as senior social scientist, resident scholar at RAND Institute on Education and Training

2001  Joined HGSE faculty as professor on education

2008–2012  Founder and chair, International Project for the Study of Educational Accountability

2008  Elected to the National Academy of Education

2009  Named the Henry Lee Shattuck Professor of Education at HGSE

2010–present   Fellow of the American Educational Research Association

2011–2015  Co-principal investigator of study, Developing More Effective Test-Based Accountability by Improving Validity Under High-Stakes Conditions, funded by Institute of Education Sciences

Books

The Testing Charade: Pretending to Make Schools Better (2017)

Measuring Up: What Educational Testing Really Tells Us (2008)

HGSE coverage

• The Testing Charade
• Testing. Testing. 1-2-3.
• When Testing Takes Over
• Harvard EdCast: The Testing Charade
• Testing, Crime, and Punishment
• The Problems and Promise of Common Core