Out-of-School Time Evaluation Snapshot
Measurement Tools for Evaluating Out-of-School Time Programs: An Evaluation
Resource
Number 6, November 2005
Christopher Wimer, Suzanne
M. Bouffard, and Priscilla
M. D. Little
Download this brief:
Harvard Family Research Project's (HFRP) series of Out-of-School Time
Evaluation Snapshots distills the wealth of information compiled in our Out-of-School
Time (OST) Program Evaluation Database1
into a single report. Each Snapshot examines a specific aspect of out-of-school
time (OST) evaluation. This Snapshot describes instruments used by current
OST programs to evaluate their implementation and outcomes.
A growing investment in evaluation, for purposes ranging from continuous improvement
to accountability, has led to increased requests from the out-of-school time
(OST) community for practical evaluation tools. As part of Harvard Family Research
Project's continuing effort to help practitioners and evaluators choose appropriate
evaluation methods, this Snapshot describes instruments and tools that
can be obtained and used for on-the-ground program evaluation. From first-time
internal evaluations to large-scale national studies, these evaluation instruments
provide valuable ways to assess the characteristics of programs, their staff,
their participants, and other key information.
An evaluation instrument is “a means used to measure or study a person,
event, or other object of interest.”2
The instruments used by OST programs take a variety of forms, ranging from checklists
of program components, to survey questions measuring self-esteem, to assessments
of academic skills, and beyond. The instruments presented here are compiled
from evaluations of OST programs in the Harvard Family Research Project OST
Program Evaluation Database (see footnote 1).
Organization of Information in This Snapshot
The evaluation instruments are presented in tables organized by content area.
The instruments fall into five categories, some of which include several subcategories,
in order to make the instruments easier to find and use. The categories are
listed below. These categories are not necessarily mutually exclusive; because
some instruments measure more than one area, they appear in more than one category.
In addition, the multicomponent scales may contain subscales relating to other
areas. Some instruments were excluded from the tables due to the expense of
administering them or difficulty obtaining them (e.g., Stanford Achievement
Test, 9th edition [SAT-9]).
Whenever possible, bibliographic citations and Internet links are provided
to facilitate access to the instruments. In addition, the names of programs
whose evaluations used each instrument are also provided, so that readers can
see examples of how and where the instruments were used. Where feasible, a sample
item from each instrument is included.
Categories of Measurement
The entire set of tables use the following categories and subcategories:
Academics
Academic Achievement
Academic/Educational Attitudes and Value
Psychological/Social Development
Future Orientation/Future Plans
Life Events and Experiences
Mental Health and Behavior
Relationships
Self-Perception/Self-Esteem
Alcohol, Tobacco, and Other Drug Use Prevention
Program Quality/Program Environment
Multicomponent Scales/Comprehensive Surveys
How to Use This Resource
The information in these tables can help practitioners and evaluators find evaluation
instruments that match their program and evaluation goals and characteristics.
They can be used either alone or in conjunction with previous Snapshots
focused on performance measures3
and data collection methods4
to provide an overview of previous evaluations and to design future evaluations
for a range of purposes and stakeholders. It should be noted that this Snapshot
is intended solely as a resource, and is not intended as an endorsement of any
of the instruments listed. Technical assistance or consultation with professional
evaluators may be necessary before selecting or utilizing the instruments.
Table 1. Academic Achievement
Download this table:
Table 2. Academic/Educational Attitudes and Values
Download this table:
Table 3. Future Orientation/Future Plans
Download this table:
Table 4. Life Events and Experiences
Download this table:
Table 5. Mental Health and Behavior
Download this table:
Table 6. Relationships
Download this table:
Table 7. Self-Perception/Self-Esteem
Download this table:
Table 8. Alcohol, Tobacco, and Other Drug Use (ATOD) Prevention
Download this table:
Table 9. Program Quality/Program Environment
Download this table:
Table 10. Multicomponent Scales/Comprehensive Surveys
Download this table:
1 Our database
contains profiles of out-of-school time (OST) program evaluations, which are
searchable on a wide range of criteria. It is available in the OST section of
the HFRP website at www.gse.harvard.edu/hfrp/projects/afterschool/evaldatabase.html.
2 Weiss, C.
H. (1998). Evaluation (2nd ed.). Upper Saddle River, NJ: Prentice Hall.
3 Little,
P. M. D., Harris, E., & Bouffard, S. (2004). Performance measures in
out-of-school time evaluation. Cambridge, MA: Harvard Family Research Project.
Available at www.gse.harvard.edu/hfrp/projects/afterschool/resources/snapshot3.html.
4 Bouffard,
S., & Little, P. M. D. (2004). Detangling data collection: Methods for
gathering data. Cambridge, MA: Harvard Family Research Project. Available
at www.gse.harvard.edu/hfrp/ projects/afterschool/resources/snapshot5.html.
|
