HGSE News: In Evidence Lies Change (Professor Carol Weiss)

"Governments don't often use research directly, but research helps people reconsider issues, it helps them think differently, it helps them reconceptualize what the problem is and how prevalent it is, it helps them discard some old assumptions, it punctures old myths."

—Whiting Professor Carol Weiss

After a dozen years of ignoring evaluation data, the Drug Abuse Resistance Education program (D.A.R.E.) finally began to consider studies that shed doubt on the long-term efficacy of their programs. The New York Times recently reported on controversial research that claims no decrease in and perhaps occasionally increased drug use among students who participated in popular drug prevention programs such as D.A.R.E.

With school drug abuse prevention programs generally showing little long-term effect on drug use, the Department of Education now requires evidence of effectiveness before it provides drug prevention funds to districts. Such research-based reform would seem a triumph for Whiting Professor Carol Weiss, who is a strong advocate of "evidence-based policy."

Weiss, however, is hesitant to reject D.A.R.E. and instead underscores the benefits provided to kids in the program. "The more I look at this case," says Weiss, "the more I think that knee-jerk reactions [to research findings] are not good. An important thing about D.A.R.E. is that it has an excellent delivery system. It uses police—who have received rigorous training—to take the content to the schools; the curriculum materials have been somewhat standardized. It would be too bad to scrap D.A.R.E., because it has good infrastructure. It makes more sense to help it improve."

Beginning a Tradition of Evidence-Based Policy
Weiss's support of D.A.R.E. in light of discouraging research might seem uncharacteristic, considering her career as a researcher and developer of evaluation methods, and her tradition of advocating what is sometimes known as "knowledge utilization." In a recent interview, Weiss recalls her frustration when her very first research project was shelved despite its potential applicability to real-world problems:

"I was asked to evaluate a program of the "War on Poverty" in the 1960s. Lyndon Johnson's policy to 'eradicate poverty' generated a whole range of new programs: education, health, mental health, job training, programs for the elderly and so on. It was the start of major government support for programming and evaluation and a social revolution in services. I was doing research for social agencies and I was asked to evaluate a program in central Harlem. One of the program's goals was to bring black college students from universities in the south to work in central Harlem, to work in the schools, the hospitals and social agencies. They were trained and then they spent the year working in the community."

"When I finished my evaluation of the Harlem program, the report came out in 3 volumes. We sent copies of the report to Washington: I never heard a word from them! I had the feeling I could have just dumped it into the ocean and it would have made no difference. So, I asked myself: "Why did they support and fund this evaluation if they were not going to pay any attention to it?" That's how I got interested in the uses of research: What was going on? What could researchers—or anyone else—do to encourage people to pay more attention to research?"

Although a disappointing reality-check about the tribulations of being a researcher, Weiss also remembers this very first project as the basis of what continue to be her two strongest academic interests today: evaluation, and the influence of research on policy.

Indeed, time—and experience—have softened the blow for Weiss when groundbreaking research does not seem to affect policy. She emphasizes that research can sometimes be the silent but indispensable partner motivating social change. In her opinion, D.A.R.E. research was certainly not lying fallow for the past twelve years, but instead was a necessary stepping stone for bringing about program re-evaluation. Says Weiss, "[Research-based policy] takes time."

Lessons from the U.K.
This summer, Weiss was keynote speaker at an Evidence-Based Policy conference in the United Kingdom. At the conference, she told the history of evidence-based policy, shared some of her D.A.R.E. study findings, and expressed optimism for the future of research utilization in policy. She returned to the states impressed with the strong weight given to research findings by leading British policy-makers—first and foremost Prime Minister Tony Blair. Weiss claims that the U.S. is not too far behind, with "bits and pieces" of governmental research offices dedicated to increasing the use of research for policy change.

With valuable educational research at hand supported by the U.S. Department of Education's Office of Educational Research and Improvement and other agencies and foundations, why wouldn't politicians take full advantage of academic findings in determining what's best for the country? Weiss points to a combination of competing objectives, and, in some cases, self-interest.

"Decision-makers have ideological commitments. You are not going to change their opinion about, say, the value of educating girls in Pakistan, just by giving them information. They have deep-rooted beliefs and values. Decision-makers also have interests. They must satisfy certain constituencies, gain support from other agencies, abide by the rules and routines of their institution, or even please some powerful political figure."

"A Question of Values"
A lot has changed since President Johnson's "War on Poverty." Weiss traveled to Pakistan to do research on management systems, wrote numerous textbooks on evaluative methods, and quite recently expanded her interests to an issue that hits home in Massachusetts: determining how the state made the decision of where to set the failing score of the MCAS tests. She is excited to see the evaluation of the revamped D.A.R.E. program. The new D.A.R.E. works with older kids and changes the role of police in the program. But if the new program is successful, shouldn't these reforms have happened a decade ago, when initial research doubting existing programs was first available?

One of research's strongest advocates speaks from vast experience when she explains, "Research can be used in policy-making. Whether it should be is a question of values." Ultimately, Weiss advocates patience and confidence that policy networks today that are informed with more than just empirical research are the richest, and most successful agents of social change.

Weiss notes that the crucial function of research in democratic policy-making is being accessible, and contributing to a decision, but not being the sole motivating factor behind a policy. Ultimately, evidence-based policy is part of complex, rich web: "It takes time and reconceptualization before research actually leads to a change in policy. In the meantime, lots of other things have happened. So it's hard to say that social science triggered a change. There had to be a lot of supporting and reinforcing conditions in place."

For More Information
More information about HGSE faculty members and their research is available in the Faculty Profiles.

P7 MenuMagic dwt Control Layer -Do Not Alter