Beyond Basic Training
A Guide to Measuring Advocacy and Policy
Organizational Research Services identifies outcomes associated with advocacy
and policy work based on its new resource, A Guide to Measuring Advocacy
and Policy.
While many foundations and nonprofits are interested in measuring their advocacy
and policy work, currently no commonly accepted evaluation approach or practice
exists. To help remedy this, the Annie E. Casey Foundation commissioned Organizational
Research Services (ORS), a Seattle-based evaluation consulting firm, to create
a guide that would help both the Casey Foundation and other organizations better
define and document the effectiveness of their advocacy and policy strategies.
A Guide to Measuring Advocacy and Policy gives foundations and nonprofits
alike a way to identify and talk about the outcomes associated with advocacy
and policy work. In addition, it offers evaluation design suggestions that include
a broad range of methodologies, intensities, audiences, timeframes, and purposes.
The guide aims for wider acknowledgement about how evaluation fits into the
world of advocacy and policy, greater acceptance of evaluation's role in demonstrating
success and learning about progress, and increased confidence among those undertaking
evaluation in this area.
While outcome categories are fairly standardized and widely accepted in the
service delivery arena, such standardization does not yet exist for advocacy
and policy work. The guide highlights a core set of outcome categories and provides
concrete direction for those searching for what to measure about their
advocacy and policy strategies.
In developing the guide, ORS reviewed a broad range of advocacy and policy
outcome categories and indicators of progress. Repeatedly, the same categories
of outcomes emerged. Some represent the interim steps and infrastructure that
create the conditions for social change; others reflect the end goal-policy
adoption, funding, or implementation and enforcement.
ORS distilled these outcomes into six distinct categories, described below,
that represent the essential changes in lives, community conditions, institutions,
and systems that result from advocacy and policy work. The table at right also
presents these categories and the specific outcomes, strategies, and units of
analysis that relate to them. The order in which outcome categories appear does
not represent their relative importance or sequence.
1. Shifts in social norms. Social norms are the knowledge, attitudes,
values, and behaviors that comprise the normative structure of culture and society.
Advocacy and policy work increasingly has focused on this area because of the
importance of aligning advocacy and policy goals with core and enduring social
values and behaviors.
2. Strengthened organizational capacity. Organizational capacity
is another name for the skill set, staffing and leadership, organizational structure
and systems, finances, and strategic planning of nonprofits and formal coalitions
that do advocacy and policy work. Development of these core capacities is critical
to advocacy and policy change efforts.
3. Strengthened alliances. Alliances among advocacy partners
vary in levels of coordination, collaboration, and mission alignment and can
include nontraditional alliances such as bipartisan alliances or relationships
between unlikely allies. Alliances bring about structural changes in community
and institutional relationships and are essential to presenting common messages,
pursuing common goals, enforcing policy changes, and protecting policy “wins.”
4. Strengthened base of support. Nonprofits draw on grassroots,
leadership, and institutional support in working for policy changes. The breadth,
depth, and influence of support among the general public, interest groups, and
opinion leaders for particular issues are a major structural condition for supporting
policy changes. This outcome category spans many layers of culture and societal
engagement including increases in civic participation and activism, “allied
voices” among informal and formal groups, the coalescence of dissimilar
interest groups, actions of opinion leader champions, and positive media attention.
5. Improved policies. Change in the public policy arena occurs
in stages—including policy development, policy proposals, demonstration
of support (e.g., cosponsorship), adoption, funding, and implementation. Advocacy
and policy evaluation frequently focuses on this area as a measure of success.
While and important focus, improved policies are rarely achieved without changes
in the preconditions to policy change identified in other outcome categories.
6. Changes in impact. Changes in impact are the ultimate and
long-term changes in social and physical lives and conditions (i.e., individuals,
populations, and physical environments) that motivate policy change efforts.
These changes are important to monitor and evaluate when grantmakers and advocacy
organizations are partners in social change. Changes in impact are influenced
by policy change but typically involve far more strategies, including direct
interventions, community support, and personal and family behaviors.
A Guide to Measuring Advocacy and Policy is available on the Annie E.
Casey (www.aecf.org) and ORS websites (www.organizationalresearch.com).
In addition, The Innovation Network's website (www.innonet.org)
offers an online supplement to the guide with sample measurement tools directly
applicable to advocacy and policy work. We hope that the outcome categories
described here and in the guide begin to provide foundations and nonprofits
with a common approach to policy and advocacy evaluation.
Menu of Outcomes for Advocacy and Policy Work
|
| 1. Shift in Social Norms |
| Examples of outcomes |
• Changes in awareness
• Increased agreement about the definition of a problem (e.g., common
language)
• Changes in beliefs
• Changes in attitudes
• Changes in values
• Changes in the salience of an issue
• Increased alignment of campaign goal with core societal values
• Changes in public behavior |
| Examples of strategies |
• Framing issues
• Media campaign
• Message development (e.g., defining the problem, framing, naming)
• Development of trusted messengers and champions |
| 2. Strengthened Organizational Capacity |
| Examples of outcomes |
• Improved management of organizational capacity of organizations
involved with advocacy and policy work
• Improved strategic abilities of organizations involved with advocacy
and policy work
• Improved capacity to communicate and promote advocacy messages of
organizations involved with advocacy and policy work
• Improved stability of organizations involved with advocacy and policy
work |
| Examples of strategies |
• Leadership development
• Organizational capacity building
• Communication skill building
• Strategic planning |
| 3. Strengthened Alliances |
| Examples of outcomes |
• Increased number of partners supporting an issue
• Increased level of collaboration (e.g., coordination)
• Improved alignment of partnership efforts (e.g., shared priorities,
shared goals, common accountability system)
• Strategic alliances with important partners (e.g., stronger or more
powerful relationships and alliances)
• Increased ability of coalitions working toward policy change to identify
policy change process (e.g., venue of policy change, steps of policy change
based on strong understanding of the issue and barriers, jurisdiction of
policy change) |
| Examples of strategies |
• Partnership development
• Coalition development
• Cross-sector campaigns
• Joint campaigns
• Building alliances among unlikely allies |
| 4. Strengthened Base of Support |
| Examples of outcomes |
• Increased public involvement in an issue
• Increased level of actions taken by champions of an issue
• Increased voter registration
• Changes in voting behavior
• Increased breadth of partners supporting an issue (e.g., number of
“unlikely allies” supporting an issue)
• Increased media coverage (e.g., quantity, prioritization, extent
of coverage, variety of media “beats,” message echoing)
• Increased awareness of campaign principles and messages among selected
groups (e.g., policymakers, general public, opinion leaders)
• Increased visibility of the campaign message (e.g., engagement in
debate, presence of campaign message in the media)
• Changes in public will |
| Examples of strategies |
• Community organizing
• Media campaigns
• Outreach
• Public/grassroots engagement campaign
• Voter registration campaign
• Coalition development
• Development of trusted messengers and champions
• Policy analysis and debate
• Policy impact statements |
| 5. Improved Policies |
| Examples of outcomes |
• Policy development
• Policy adoption (e.g., ordinance, ballot measure, legislation, legally
binding agreements)
• Policy implementation (e.g., equity, adequate funding, other resources
for implementing policy)
• Policy enforcement (e.g., holding the line on bedrock legislation) |
| Examples of strategies |
• Scientific research
• Development of “white papers”
• Development of policy proposals
• Pilots/demonstration programs
• Educational briefings of legislators
• Watchdog function |
| 6. Changes in Impact |
| Examples of outcomes |
• Improved social and physical conditions (e.g., poverty, habitat
diversity, health, equality, democracy) |
| Examples of strategies |
• Combination of direct service and systems-changing strategies |
Jane Reisman, Ph.D.
President
Email: jreisman@organizationalresearch.com
Anne Gienapp
Associate
Email: agienapp@organizationalresearch.com
Sarah Stachowiak
Associate
Email: sstachowiak@organizationalresearch.com
Organizational Research Services
1932 First Avenue, Suite 400
Seattle, WA 98101
> Go to next article
> Go back to Table of Contents
|
