Books: Methods Matter: Improving Causal Inference in Educational and Social Science Research

Educational policymakers around the globe regularly make tough decisions about how to improve their educational systems with the scarce resources available to them. Since these decisions are rarely informed by sufficient evidence, knowledge of what works in different situations has been difficult to accrue. In Methods Matter, Professors Richard Murnane and John Willett offer guidance for those who evaluate educational policies. They cover basic principles of causal inference and introduce complex concepts previously inaccessible to nonspecialists: randomization by group, natural experiments, instrumental variables, regression discontinuity, and propensity scores. With clear prose and relevant examples, Methods Matter challenges researchers and policymakers to think more critically about the evidence and assumptions in their work.

One of the book’s most persuasive features is the wide range of research examples offered in support of each argument. Specific causal inquiries include: “Does financial aid affect students’ and families’ educational decisions?” and “Does class size influence students’ achievement?” These questions are followed by descriptions of high-quality studies that led to informed scientific conclusions. The authors highlight the fact that “evidence-based” policy proposals today are often constructed upon unreliable and invalid sources. They subsequently outline the production process that leads to good evidence, explaining how the causal impact of educational and social interventions can be estimated from quantitative data.

Methods Matter is linked to real-world problems and solutions rather than pure theoretical academia. In the words of the authors, “Our emphasis is not on mathematics, but on providing intuitive explanations of key ideas and procedures. We believe that illustrating our technical explanations with data from exemplary research studies makes this book widely accessible.”

In recent decades, developments in research methodology, administrative record keeping, and statistical software have significantly enhanced the capabilities of researchers to make well-informed evaluations of the causal impacts of educational interventions. Methods Matter offers professional wisdom that will continue to increase the number of well-designed impact studies and educated policymaking.