Evaluations to Watch
Evaluation Based on Theories of the Policy Process
Harvard Family Research Project explains how it helps to ground evaluation
in theories of the policy process.
As the old saying goes, “There are two things you never want to see being
made—sausage and legislation.” Indeed, the policy process is often
not pretty, and it can be messy. Yet evaluators of advocacy efforts need to
understand how the policy process works. To do so, evaluators must find ways
of simplifying this typically complex process in order to evaluate the actors
and their actions within it.
There are a variety of different theories that can form the conceptual underpinning
of an evaluation involving the policy process. Some draw on intriguing-sounding
ideas like diffusion of innovation,1
while others, like punctuated-equilibrium theory, are more technical.2
One particularly well-known theory comes from political scientist John Kingdon.3
Kingdon's Agenda-Setting Theory
According to Kingdon, agenda setting is the first stage in the policy process.
The policy agenda is the list of issues or problems to which government officials,
or those who make policy decisions (including the voting public), pay serious
attention. Moving an idea onto or higher up on that agenda involves three processes:
problems, proposals, and politics.
-
Problems refer to the process of persuading policy decision makers
to pay attention to one problem over others. Because a policy proposal's
chances of rising on the agenda are better if the associated problem is
perceived as serious, problem recognition is critical. It can be influenced
by how problems are learned about (e.g., through data or indicators, focusing
events like a disaster or crisis, constituent feedback) or defined (e.g.,
framed or labeled). Budget crises are a special consideration in problem
recognition, as they often trump other problems.
-
Proposals represent the process by which policy proposals are generated,
debated, revised, and adopted for serious consideration. Because competing
proposals can be attached to the same problem, getting a proposal on the
“short list” typically takes time and the willingness to pursue
it by using many tactics. Proposals are likely to be more successful if
they are seen as technically feasible, compatible with decision maker values,
reasonable in cost, and appealing to the public.
-
Politics are political factors that influence agendas, such as changes
in elected officials, political climate or mood (e.g., conservative, tax
averse), and the voices of advocacy or opposition groups.
 |
These three elements operate largely independently, although the actors in
each can overlap. Successful agenda setting requires that at least two elements
come together at a critical time—that is, when a “policy window”
opens. For example, advocates may develop a policy proposal, wait for the right
problem to come along, and then attach their proposal to it. Or researchers
may identify a problem, but it will not get on the agenda until politics shift.
Policy windows are not just chance opportunities, however; they also can be
created.
Elevating an idea on the policy agenda requires investments in more than one
element and in the ways that the elements can complement one another. Investing
in research alone to define a problem, for example, has less chance of success
than investing in problem definition and advocacy for proposals that
get attached to that problem. The likelihood of successful agenda setting substantially
increases if all three elements—problem, proposal, and politics—are
linked in a single package.
Applying Kingdon's Theory to Strategy
The Children, Families, and Communities program area of the David and Lucile
Packard Foundation is funding a grantmaking program called Preschool for California's
Children (referred to here as the Preschool Program). Its goal is ensuring that
California makes quality preschool avail-able for all 3- and 4-year-olds in
the state.
Fundamentally, the Packard Foundation's Preschool Program strategy is based
on the notion that getting a policy idea—quality preschool for all of California's
children—recognized as an idea “whose time has come” requires
that it appear prominently on the policy agenda. Therefore, the strategy (see
figure below) invests in all three elements of Kingdon's theory. It attempts
to inform thinking about the problems that quality preschool can address by
investing in research and communications to frame them. It invests in proposals
by supporting the development of policy solutions that fit the problem, along
with leadership and engagement efforts to build support for those solutions.
It invests in politics by engaging influential constituencies to bring preschool
problems and solutions to the fore. Preschool Program grantees and their partners
are the actors in the policy process helping to drive and shape these elements.
Click here for a larger version
of the figure.
Related Resources
Birkland, T. A. (2001). An introduction to the policy process:
Theories, concepts, and models of public policy making. Armonk, NY:
M.E. Sharpe. Provides an overview of current thinking on the policy
process. Also offers a glossary of terms, annotated bibliography, and
guide to significant public policy research websites.
McCool, D. C. (1995). Public policy theories, models, and concepts:
An anthology. Englewood Cliffs, NJ: Prentice Hall. This edited
volume provides an overview and critique of public policy theories, models,
and concepts. Sections are on the theoretical foundation of policy studies,
participation in policymaking, the policymaking process, policy typologies,
policy subsystems, and conflict and choice in policy theory.
Sabatier, P. A. (2007). Theories of the policy process (2nd
edition). Boulder, CO: Westview Press. Offers overviews of widely
used theories of the policy process written by distinguished scholars.
Theories include institutional rational choice, the multiple streams framework,
the garbage-can model, the advocacy coalition framework, and punctuated-equilibrium
theory. The edited volume ends with chapters that compare policy process
frameworks, theories, and models, and offer thoughts on future directions
in theory development.
|
Applying Kingdon's Theory to Evaluation
Harvard Family Research Project (HFRP) is evaluating the Preschool for
California's Children grantmaking program. Because the Preschool Program's strategy
is based on Kingdon's theory, the evaluation also uses the elements of this
theory as touchstones against which to examine the strategy's progress. We use
a variety of methods to explore the following overarching questions about the
grantmaking program's progress, momentum, and likelihood of success:
-
Are the problems that quality preschool can address recognized and
perceived as pressing? What messages are audiences using to talk about the
issue?
-
How are proposals promoting quality preschool perceived? Are they
seen as technically feasible, fiscally viable, and in line with public and
policymaker values?
-
How are politics factoring in? Is support for quality preschool
perceived as being statewide and with broad constituency support? Who are
recognized supporters and opponents of preschool policies?
-
What is the likelihood that streams will converge to open a policy window?
Where is quality preschool on the general policy agenda? Where is it on
the children's policy agenda? What is the likelihood of success and what
forces are affecting that likelihood?
While there is no neat way to package the policy process to explain all of
its complexity and nonlinearity, evaluations of advocacy and other promotional
efforts that are based on theories of the policy process can help simplify the
process to help evaluators intelligibly assess advocates' actions and their
outcomes within it. While Kingdon's theory provides one way to do this, it is
just one of many theories on the policy process. The Related Resources box on
this page offers sources for other such theories.
1 Berry,
F. S., & Berry, W. D. (1999). Innovation and diffusion models in policy
research. In P. Sabatier (Ed.), Theories of the policy process (pp. 169–200).
Boulder, CO: Westview Press.
2 True, J.
L., Jones, B. D., & Baumgartner, F. R. (1999). Punctuated-equilibrium theory:
Explaining stability and change in American policymaker. In P. Sabatier (Ed.),
Theories of the policy process (pp. 97–115). Boulder, CO: Westview
Press.
3 Kingdon,
J. W. (1995). Agendas, alternatives, and public policies (2nd ed.). New
York: Longman.
Julia Coffman, Senior Consultant,
HFRP.
> Go to next article
> Go back to Table of Contents
|
