Testing, Testing: Doctoral Marshal John Papay

After a career that straddled the worlds of education and policy, doctoral marshal John Papay, Ed.M.’05, Ed.D.’11, decided to bring the two worlds together by enrolling in the Ed School’s Education Policy and Management master’s program. “I had such a great experience that I decided to stick around for another six years,” he says of his decision to enter doctor of education program.

A former high school history teacher, Papay focused his doctoral research on studying the effects of education policy on students and teachers, working on several projects with Professor Susan Moore Johnson and the Project on the Next Generation of Teachers.

His dissertation, “The Unintended Consequences of Standardized Test Performance Labeling on Students’ Educational Investment Decisions” grew out of his work with Professors Richard Murnane and John Willett, and explored the consequences of standardized testing in Massachusetts. Papay became intrigued by the notion that labels given to Massachusetts students’ performances — advanced, proficient, needs improvement, and failing — may have an effect on the student’s future educational attainment.

He examined the state’s high-stakes exit-examination system — which features tests that students must pass in order to graduate from high school — and state tests that are used to hold schools and districts accountable but carry no official consequences for students. What the research determined is that these labels do tend to affect students and whether they go on to graduate high school and attend college.

“Scoring ‘failing,’ not surprisingly, reduces the probability that student graduates from high school,” he says.

On the other hand, when a student scores “advanced” versus “proficient,” Papay says, it can change their future in a positive way.

“For urban, low-income students, who don’t plan to attend college, scoring ‘advanced’ instead of ‘proficient’ improves their probability of going to college by 10 percentage points,” he says.

As for what this means about such standardized testing policies, Papay says it’s hard to draw conclusions. “I only looked at kids on the border,” he says. “And we don’t know what would happen in the absence of this policy. What it does show is that these labels mean a lot. The labels are causing students and parents to have strong behavioral responses. Policymakers need to think in a more nuanced way about how students, parents, and teachers use the information embedded in these performance labels.”

Now, as Papay prepares to leave HGSE behind, he says it is a bittersweet feeling.

“I had such a great experience here,” he says. “It’s exciting to think about leaving as well, though I wish I could stick around a while longer.”

His time at HGSE, he says, “has completely shaped me. I have had an opportunity to work with so many amazing professors such as Professors Susan Moore Johnson, Dick Murnane, and John Willett. It shaped my career path and where I’m going next.”

This fall Papay will begin work as a faculty member at Brown University, where he will teach courses on program evaluation and education policy.

For a story on the work of doctoral marshal Mara Tieken.

For full Commencement coverage.