Beyond Basic Training
Doing What Works: Scientifically Based Research in Education
In the midst of recent education reform policies, a new phrase has taken
center stage. Since its numerous appearances in the No Child Left Behind Act
of 2001, the phrase “scientifically based research” has sparked fervent
questions among researchers, educators, and policymakers. What exactly is scientifically
based research? What are the implementation challenges? How can research and
evaluation respond to these new demands? Suzanne Bouffard, consultant for Harvard
Family Research Project, examines the new science-based research standards.
According to the No Child Left Behind Act (NCLB), all federally funded education
programs, including out-of-school time programs that are funded under Title
I,¹ must be based on research studies
that meet scientific standards. Government publicity materials (see www.nochildleftbehind.gov)
refer to this as “doing what works.” These user-friendly versions
of the law cite the metaphor of medical research to explain how practice should
be informed by experimental studies (i.e., studies in which participants are
randomly assigned to treatment and control groups).
The primary goal of scientifically based research (SBR) is to ensure that programs
for children are based on methods that have been proven effective and are therefore
more likely to benefit other children, with a corollary goal of increasing the
overall quality of education research. According to NCLB legislation, the following
principles define scientific quality:
- Use of the scientific method with an emphasis on experimental control
(or comparison) groups
- Replication of results, using multiple studies by different investigators
- Ability to generalize results from one sample to other children in
the general population
- Fulfillment of rigorous standards with an emphasis on peer review
- Convergence (or consistency) of results between studies
Challenges to SBR Implementation
There is consensus that education research should follow the same general principles
as other sciences, yet experts note some unique challenges to implementing SBR
in educational contexts, including the realm of out-of-school time. For example,
diverse values and goals for education provoke debate, and randomizing students
to treatment and control groups poses ethical problems (Shavelson & Towne,
2002). Experts also question whether scientifically based research necessitates
a specific research methodology, with many believing that scientific quality
is determined not by a particular methodology, but by the appropriateness
of the methodology for the research question. In fact, a recent report on
SBR in education by the National Research Council (NRC) explicitly discourages
the exclusion of non-experimental studies, with the explanation that descriptive
studies may first be necessary in order to design effective experimental interventions
(Shavelson & Towne); furthermore, random assignment to experimental groups
is necessary only to establish causality. For some questions, for example
how family relationships affect school achievement, random assignment may be
neither desirable nor ethical (Raudenbush, 2002).
What Can Researchers and Evaluators Do?
The call for scientifically based research has been met with a number of responses.
The NRC report recommends choosing a research method based on the kind of question
being asked (Shavelson & Towne, 2002). Appropriate methods might include
surveys, observations, or experiments. Other experts have pointed out that improving
education research requires more than a strong methodology, and underscore a
need for the collaboration of professionals from various jobs and disciplines.
Some state that the best research is conducted by people with direct experience
in the educational system (Gardner, 2002), with teachers providing valuable
insight into questions concerned with activity in the classroom (Olson &
Viadero, 2002). Others highlight how research from cognitive and developmental
psychology can inform classroom instruction (Hirsch, 2002).
Experts also emphasize the role of a strong research community in increasing
the quality of education research. Debate, discussion, and the peer review process
are important, but researchers additionally stress the need for the accumulation
of studies in order to establish scientific certainty. They caution that this
process will require public patience and support, and should neither provoke
discouragement nor overshadow existing research (Raudenbush, 2002). The challenge
is to increase the quality of work while using the research that is currently
available.
Toward this goal, the U.S. Department of Education’s Institute of Education
Sciences has created a new “What Works Clearinghouse” of high-quality
research studies (see www.w-w-c.org) and
the independent, nonprofit Education Quality Institute will publish a collection
of policy briefs on education research (see www.eqireports.org).
These initiatives have resulted from an increased dialogue between researchers,
practitioners, and policymakers. Some feel that this dialogue, along with the
chance to improve the quality of education research and practice, may prove
to be one of the greatest opportunities resulting from the new focus on scientifically
based research.
Suzanne Bouffard, Consultant, HFRP
Email: suzanne.bouffard@duke.edu
For more information on scientifically based research in an out-of-school
time context see Evaluating the 21st Century Community
Learning Centers Program—A View From the States.
References
Gardner, H. (2002, September 4). The quality and qualities of educational research.
Education Week, 22(1), 49, 72. [Available at: www.edweek.com/ew/ewstory.cfm?slug=01gardner.h22.]
Hirsch, E. D., Jr. (2002, October/November). Classroom research and cargo cults
[Electronic version]. Policy Review, 115. [Available at: www.policyreview.org/OCT02/hirsch.html.]
Olson, L., & Viadero, D. (2002, January 30). Law mandates scientific base
for research. Education Week, 21(20), 1, 14–15. [Available at: www.edweek.com/ew/ewstory.cfm?slug=20whatworks.h21.]
Raudenbush, S. (2002, February). Scientifically based research. Paper
presented at the Scientifically Based Research Seminar, Washington, DC.
Shavelson, R. J., Towne, L., & the Committee on Scientific Principles for
Education Research. (Eds.). (2002). Scientific research in education.
Washington, DC: National Academy Press.
¹ Title I refers to the set of programs
in NCLB that relate to improving the academic achievement of the disadvantaged.
As the largest federal program supporting elementary and secondary education
(funded at $10.4 billion in FY 2002), Title I targets these resources to the
districts and schools where the needs are greatest. Title I provides flexible
funding that may be used to provide additional instructional staff, professional
development, extended-time programs, and other strategies for raising student
achievement in high-poverty schools.
> Go to next article
> Go back to Table of Contents
|
