Issues and Opportunities in Out-of-School Time Evaluation Briefs
Summer Success: Challenges and Strategies in Creating Quality Academically
Focused Summer Programs
Number 9, October 2006
Christopher Wimer and Rachel Gunther, Harvard Family Research Project
Download this brief:
Harvard Family Research Projects (HFRP) Issues and Opportunities
in Out-of-School Time Evaluation briefs highlight current research and evaluation
work in the out-of-school time field. These documents draw on HFRPs research
work in out-of-school time to provide practitioners, funders, evaluators, and
policymakers with information to help them in their work. This brief looks at
evaluations of 34 academically focused summer programs in order to distill challenges
and compile promising strategies for creating quality summer programs.
Introduction
Schools play an obvious and critical role in promoting learning, but we know
that children and youth spend the majority of their time outside of school (Larson
& Verma, 1999). A large portion of this nonschool time occurs in the summer,
which for many youth constitutes approximately a quarter of the year. Summer
represents an opportunity for experiences that enrich and complement the school
year and promote learning and development. It is well documented, however, that
during the summer, low-income and other disadvantaged youth fall further behind
academically than their more advantaged peersin part, due to a lack of
enriching opportunities (Heyns, 1978; Alexander, Entwisle, & Olson, 2001;
Burkam, Ready, Lee, & LoGerfo, 2004; Downey, von Hippel, & Broh, 2004).
Academically focused summer programs can meet parents' needs and desires to
support youth with fun and enriching opportunities (LeMenestral, 2003) and promote
learning when school is not in session (Fairchild, 2006). Given the potential
benefits of summer programs, practitioners and policymakers are looking for
ways to provide quality academically focused summer programs, particularly for
disadvantaged and academically at-risk youth. In December of 2005, Senator Barack
Obama of Illinois introduced the STEP UP Act, which would provide children with
the opportunity to participate in summer programs specifically designed to increase
the academic performance of poor and minority youth. Because of the opportunities
inherent in summer programs, it is important to understand how providers can
implement the highest quality programming possible.
Research Spotlight: The Benefits of Summer School Programs
In a meta-analysis of 93 studies of summer school programs,
Harris Cooper and his colleagues (2000) found that they led to increases
in participants' knowledge and skills. In particular, Cooper and his colleagues
concluded that:
- Programs aimed at remediation of learning deficiencies and programs
focused on learning acceleration both produced positive impacts on youth's
knowledge and skills.
- Programs had more positive benefits for middle class youth than they
did for more disadvantaged youth, perhaps because more resources are
devoted to supplementing programs in middle class families and communities.
- Impacts were greater when programs were run for a small number of
schools or in a small community.
- Impacts were greater when programs provided small-group or individual
instruction.
|
To date, little research has synthesized the large amount of implementation
data from existing evaluations of academically focused summer programs. This
issue brief fills that gap by assessing the formative, or implementation, data
from evaluations of 34 academically focused summer programs listed in HFRP's
Out-of-School Time Program Evaluation Bibliography. From these evaluations,
the brief identifies seven common challenges to implementing high quality summer
programs:
- Developing programming with intentionality
- Building positive and individualized connections with youth
- Recruiting and developing highly skilled staff
- Developing ongoing, mutually supportive relationships with schools
- Building strong, positive connections with participants' families
- Engaging community members, groups, and institutions in programming
- Incorporating a variety of fun and engaging program activities
While none of these challenges are unique to summer programming, the rich implementation
evaluations of summer programs offer a number of promising strategies to meet
them. This brief articulates successful strategies for overcoming common programming
challenges, with a particular focus on strategies implemented by academically
focused summer programs.
The 34 evaluations1
that were analyzed for this review met three criteria: The evaluations a) evaluated
a summer program, or the summer component of a year-round program; b) reported
formative/implementation evaluation findings; and c) evaluated a program with
a focus on improving learning and school performance. This last criterion was
defined broadly to account for the tremendous diversity of summer programs and
to capture information from many different program models.
As a result, some of the programs reviewed were traditional academic summer
programs that provided instruction in subjects like English/language arts or
mathematics, while others targeted specific academic areas, such as science,
oceanography, or book making. Other programs offered academic enrichment activities,
such as arts, science, literacy, and field trips. These different types were
not mutually exclusive; some programs combined remediation with enrichment or
targeted instruction in specific academic areas. By examining a range of academic
summer programs, this issue brief aims to document common challenges, recommend
promising strategies, and illustrate how specific programs implemented successful
practices to promote learning and development in the summer months.
Learning What Works: The Importance of Formative Evaluation
Formative, or process, evaluations are conducted during
program implementation in order to provide information that will strengthen
or improve the program being studied. Formative evaluation findings typically
point to aspects of program implementation that can be improved for better
results, such as how services are provided, how staff are trained, or
how leadership and staff decisions are made.
Funders and policymakers typically use outcome evaluations
to understand whether a given program is achieving its intended outcomes.
While this information is useful and critical, it is also important for
program leaders and staff to understand how their program is working,
what works well, and what needs improvement. Only with this knowledge
can programs make the necessary improvements in program delivery to achieve
their intended out comes. This ongoing process of integrating lessons
learned from evaluation into programming is commonly referred to as continuous
improvement.
Formative evaluation is also useful in helping programs
identify why they have achieved certain patterns of outcomes. For example,
while an outcome study might help funders understand that a science program
improved grades but not attitudes toward science, this type of study would
not help program personnel understand why that result might have occurred.
Collecting formative data could help the program find that youth who participated
in the program were successfully exposed to a wide variety of science
concepts (thus improving their science grades) but that this exposure
occurred in a relatively dry and unengaging way (thus failing to improve
youth's positive attitudes toward science).
|
Challenges to & Strategies for Successful Academic Summer Programming
A review of the 34 program evaluations from our Out-of-School Time Program
Evaluation Bibliography identified seven common challenges to creating and supporting
a successful summer academic programchallenges, which, if addressed, can
strengthen summer programming. This review also uncovered strategies that programs
have used to successfully address these challenges. This brief presents both
the challenges and strategies as examples of best practices for implementing
summer programs. The categories are not mutually exclusive, and the list does
not attempt to address every challenge a program might face during implementation.
Challenge 1: Developing programming with intentionality
Many summer programs aim to realize a variety of benefits for youth and their
families, but programs are likely to be more successful if they are developed
and implemented with intentionality from the outset. Improving the effectiveness
of out-of-school time (OST) programs requires the intentional linking
of program goals, program elements, participant outcomes, and evaluation (C.
S. Mott Foundation Committee on After-School Research and Practice, 2005). Specifically,
Halpern (2003) examined which after school literacy programs were most effective
and found the only difference between effective and ineffective programs was
intentionality in planning and program design.
For example, if a program's goal is to improve literacy outcomes for participating
youth, the program director and staff should continuously plan and design programming
with this goal in mind. Across many of the evaluations in this review, programs
were less successful when they were not developed or implemented according to
expressed goals and mission with intentional planning. Though maintaining intentionality
may be difficult due to tight implementation schedules and unexpected problems,
the evaluations revealed a number of promising strategies for ensuring that
intentionality is achieved.
Build in program planning and development time before the start of the
program.
Many program evaluations reported that challenges accrued because of a lack
of initial planning. For instance, the Summer of Challenge program in the Emery
Unified School District in California found that a lack of lead time for program
planning was a major implementation barrier. This need was even more pronounced
in programs with many stakeholders. In particular, programs that incorporated
the goal of involving parents and youth in the program planning process required
additional planning time.
To ensure sufficient planning time to achieve intentionality, program leaders
at the District of Columbia 21st Century Community Learning Centers summer program
recommended that program planning should (a) include multiple planning meetings,
(b) include time to train staff, and (c) begin at least 3 to 4 weeks before
the program's launch date. Other summer program experts recommend a longer planning
period, of approximately 6 months. While the exact length and design of planning
time will vary according to program needs and activities, all programs should
take care to build in planning time adequate to their program prior to the launch
date.
Build program planning and development time into daily program operations.
Planning time is critical not just at the beginning of a program but throughout
implementation. Ongoing planning allows programs to adjust to unforeseen events,
assess progress in meeting youth's needs, and design activities based on what
has worked and not worked so far. One of the strategies employed by the BELL
Accelerated Learning Program in Boston, New York City, and Washington, DC, was
to build in weekly assessments of participating youth. Staff used the results
of these assessments during ongoing planning time to tailor instruction and
activities for upcoming sessions in order to maximize each youth's learning.
Similarly, the Summer of Challenge evaluation noted that planning time built
into the fabric of day-to-day program operations allowed for collaboration between
staff to improve future programming.
Design activities and instruction purposively to achieve program goals.
Designing activities with program goals in mind is a critical part of building
a more intentional program. For example, if the program's mission is to keep
youth from failing in school or to ensure their proficiency in certain subjects,
activities should be purposively designed to achieve these goals. This does
not need to mean teaching to the test or dry instruction in core
academic areas. It could, instead, mean aligning activities and curricula to
school-district, state, and/or national standards.
The Verilette Parker Science Intervention Program in rural Georgia employed
an inquiry-based, active-learning instructional methodology to promote critical
thinking and cooperative learning, while intentionally aligning the curriculum
to national goals for science education. Similarly, the evaluation of an Oceanography
Camp in St. Petersburg, Floridadesigned to increase girls' interest in
science and science-related careersfound that activities worked best when
they were as active as possible. A common theme in the evaluations was the importance
of using methods like hands-on learning to spark interest and excitement.
Promoting Learning: Diverse Approaches to Common Concerns
When many people think of academics, they conjure
a set of standard images: teacher at the front of the room, students at
their desks, and teacher-directed instruction in key areas like math or
English/language arts. But with the exception of some strictly remedial
summer programs, very few programs reviewed in this brief embodied this
model of academic instruction. Even when programs incorporated teacher-led
instruction, effort was often made to select curricula that would be fun,
interactive, and engaging for youth. Below are some of the diverse ways
that, alone and in combination, programs in this review promoted youth's
learning and school performance.
Academic Remediation: Some programs promoted learning
through academic remediation. For instance, programs serving youth who
did not meet certain minimum levels on standardized tests and remedial
academic summer pro-grams often offered instruction in core academic areas
like math and English/language arts.
Specialized Curricula: Some programs used curricula
especially designed for after school and summer programming, such as KidzLit,
which targets literacy by exposing youth to engaging books and by encouraging
them to express their feelings and grapple with big ideas through discussion,
drama, art, movement, and writing.
Educational Field Trips: Many programs offered educational
field trips to places like museums or aquariums to complement the learning
taking place in the program.
Hands-On, Project-Based, and Thematic Learning Projects:
Many programs sought to enhance learning by offering hands-on, project-based,
or thematic learning projects. For instance, one program used book making
to build participants' literacy skills. Others used hands-on science projects
to spark interest in science and learning more generally.
|
Challenge 2: Building positive and individualized connections with youth
The majority of programs reviewed serve a diverse population of youth, even
within the particular groups some programs target for recruitment. For example,
a small summer program targeting African American students who are all behind
in math in a specific underperforming elementary school could serve participants
likely to be highly diverse in terms of a host of factors, including how far
behind they are academically, their learning styles, and their family situations.
To be successful in working with wide-ranging groups of youth, staff must learn
about participants' situations and needs and use this understanding to build
positive connections with them.
Many program evaluations found building positive, individualized connections
with all youth to be a major challenge. However, when programs were able to
build these connections, the benefits were manifold. Positive and individualized
connections can facilitate trust between staff and youth, make youth more excited
about and engaged in the program, and allow staff to tailor programming to youth's
interests and needs. Below are a few strategies that the evaluations found to
be promising ways to make connection-building easier.
Utilize preexisting connections with youth.
Some programs found it helpful to hire some teachers and staff who had preexisting
positive connections with youth. In Chicago's Summer Bridge program, teaching
staff were more likely to adapt the curriculum to meet students' needs and to
work closely with students outside of class when they knew a larger proportion
of their students before the start of the program. Across a number of the evaluations,
staff reported not having enough information on participating students' backgrounds
and needs. Hiring at least some staff with preexisting relationships with youth
could help deal with that challenge.
Create opportunities to ensure individualized connections with youth.
Many evaluations reported that daily operations ran more smoothly when programs
maintained smaller staff-to-youth ratios or provided opportunities for small-group
or individualized interactions between staff and youth. For example, in the
GEAR UP program in Austin, Texas, college students worked in small groups or
one-on-one with youth, which allowed for the identification of learning needs
and academic strengths of individual youth. Similarly, the evaluation of the
Camp Invention program of the National Inventors Hall of Fame found that small
groups allowed staff to spend more individualized time with each participant.
However, when determining appropriate staffyouth ratios, it is important
to consider the nature of the activity as well as the age of the participants.
For some activities, such as sports programs, a high staffyouth ratio
may be acceptable (e.g., one coach for a team of 10); similarly, some programs
for older youth may want to consider having slightly higher staffyouth
ratios that promote youth-to-youth bonding and interaction, rather than lower
ratios that could promote more intensive staffyouth relationships.
Develop mechanisms to provide staff with accurate information on students'
needs and backgrounds.
Programs found it easier to design and adapt programming if they had sufficient
information about students' needs and backgrounds. As noted above, the BELL
program conducted weekly assessments of youth in order to provide staff with
data on youth's status and needs and to determine areas in which they should
concentrate through the summer. In contrast, staff at the Summer Opportunity
to Accelerate Reading program designed their application forms to collect information
on the special needs of participating youth.
Harvard Family Research Project Out-of-School Time Program
Evaluation Bibliography
The Harvard Family Research Project (HFRP) Out-of-School
Time Program Evaluation Bibliography contains citations for all the out-of-school
time (OST) program evaluations that HFRP has identified to date. HFRP
provides basic program information as well as links to relevant evaluation
reports.
Types of Programs Included in the Bibliography
- Evaluations in the bibliography meet the following criteria:
- The evaluated program or initiative operates during out-of-school
time.
- The evaluation aims to answer a specific evaluation question or set
of questions about a specific program or initiative.
- The evaluated program or initiative serves children between the ages
of 5 and 19.
How Programs Are Categorized in the Bibliography
Programs are categorized by program type. Program type
can refer to a method of service delivery or a primary program goal. For
example, a program promoting health (a program goal) might use recreational
activities to achieve this goal (service delivery). Since programs may
fall into more than one category, the same programs may appear in different
lists; the program types that are most applicable to a particular program
(with a maximum of three, although more may apply) are listed in parentheses
following the program description.
How to Use the Bibliography
The bibliography is located in the OST section of the HFRP
website at www.gse.harvard.edu/hfrp/projects/afterschool/bibliography/index.html.
Users may click on any of the 19 program types or see a list of the most
recent entries and updates.
|
Challenge 3: Recruiting and developing highly skilled staff
Developing positive and individualized connections with youth is likely to
be much easier when program staff are highly skilled at working with youth.
Indeed, most strategies for implementing high-quality summer programming identified
in this issue brief would be facilitated by identifying, recruiting, and developing
a highly skilled workforce. However, many programs found recruiting and developing
highly skilled staff to be a major implementation challenge. Programs in this
review employed the following strategies for overcoming this challenge.
Recruit qualified and experienced school-year teachers for academic components
of the summer program.
Most teachers have the summers off and therefore present a large pool
of talented potential summer staff. This strategy is not without its challenges,
however. The Wake County Summer Academy's evaluation found that more skilled
and experienced teachers often prized their summers as a time for vacation and
relaxation, leaving an available pool of teachers younger and less experienced.
To address this challenge, programs may target more experienced teachers through
special incentive packages, such as extra compensation for experience, letters
of acknowledgement for teachers' personnel files, and professional development
opportunities. As noted above, in the example of Chicago's Summer Bridge program,
programs can also target potential staff who have preexisting relationships
with youth.
Tap into the talents of parents, community members, high school students,
and college students.
Each of these types of staff brings different strengths and assets to summer
programs. Parents possess knowledge of youth's strengths and needs and often
enjoy working with youth. Community members bring topical expertise in a variety
of areas, such as science, dance, or art, which can be harnessed to create exciting
programming. College students are a particularly promising source of quality
staff, as they often relate well to youth and are sufficiently easy to recruit
during their summer vacations. The Summer Career Exploration Program in New
Jersey and Pennsylvania found that most site coordinators had little trouble
recruiting college students, and often had many more applicants than slots to
fill. They were able to successfully recruit college students through local
newspapers, college placement offices, work-study programs, newsletters, and
word of mouth.
Program evaluations reveal that, when utilizing these diverse types of staff,
it is important to pay attention to issues of roles and responsibilities, lines
of communication, and proper staff training. The evaluation of Kids on Campus
in Ohio, for example, revealed that although parents and high school students
brought strengths to the program's workforce, they were often less capable than
other staff in effectively managing youth discipline and behavior. More training
was necessary to most effectively utilize these less-experienced staff members.
The same evaluation found that high school and college students sometimes clashed
with each other over issues of respect and power relationships, as the roles
and responsibilities of each were not always clearly drawn.
Provide adequate and timely training to staff members.
Though a majority of program evaluations found the lack of adequate training
time for staff members to be a program barrier, some identified training and
professional development opportunities as a program strength. One such program
was the Extended Learning Opportunities Program in Montgomery County, Maryland,
which provides staff training before the start of the program, focusing on understanding
the content and structure of the summer curricula, instructional planning and
strategies, and monitoring what students know and/or their progress. Staff also
receive reading and math guides to help with these types of activities over
the course of the program. Meanwhile, the evaluation of the Summer Opportunities
to Accelerate Reading program in Austin, Texas, found that training worked best
when it was responsive to the preexisting level of knowledge among the program's
staff. Many program evaluations found a need for increased or improved training
for staff in working with youth with behavioral challenges and special needs.
Innovations in Staffing: The Ascend Summer Youth Program
Limited resources make recruiting and retaining talented
staff an ongoing challenge for many youth programs. The Ascend Summer
Youth Program in Washington, DC, has tackled this problem head-on through
an innovative approach to staffing that both complements and improves
the program: employing former graduates of the Ascend program.
The Ascend Summer Youth Program was initiated in 2001 to
develop youth leaders in the field of information technology. Program
components include mentoring, workforce readiness awareness, and project-based
learning experiences. Staff use information technology to address a variety
of social, affective, cognitive, and academic outcomes necessary for postsecondary
success.
According to Joseph Davis, president of Ascend, the program
has found that recruiting former program graduatesnow usually college
studentsas staff helps the program accomplish its goals. Former
graduates know the program well and are more immediately ready to guide
activities adeptly because of their familiarity with the technology and
the program structure. In addition, the presence of former graduates shows
current youth participants what they can achieve. This in turn helps participants
develop self-efficacy and succeed within and after the program.
Former graduates demonstrate a commitment to the program's
goals and philosophy and often share and recognize the attitudes and cultures
of current participants. While not a magic bullet to the problem of staffing,
employing former program graduates can provide a meaningful boost to program
quality.
|
Challenge 4: Developing ongoing, mutually supportive relationships with schools
Many evaluations found that one challenge to developing quality academically
focused summer programming was building solid relationships with the schools
that educate their participants during the school year. Developing these relationships
is important for many reasons. First, programs need schools' help in order to
identify and recruit potential youth and staff for their programming. Second,
schools can provide summer programs with critical information about youth's
academic and social situations. This information is integral to creating summer
programming that addresses participants' unique needs. Third, many academically
focused summer programs are based in schools and need to establish strong lines
of communication with schools to operate effectively. School-based summer programs
need to communicate with schools to ensure adequate facilities and supplies
and to clarify roles and responsibilities.
Lastly, and perhaps most importantly, schools and academically focused summer
programs share the same missionpromoting the education and development
of program participants. Developing ongoing, mutually supportive relationships
is crucial to ensuring that schools and summer programs work collaboratively
to best accomplish this mission. Schools' missions are hampered when youth fall
behind academically during the summer months, just as summer programs' missions
are hampered when schools do not build on what youth have learned in summer
programs.
Relationships with schools were more often described as a program weakness
than a program strength. However, one specific OSTschool linkage strategy
emerged from the review:
Create formal partnerships with schools through early planning and transfer
of information.
The Break-Aways Camps in New York City created formal partnerships between summer
camps and schools. Together, the partners collaborated to plan literacy activities
in the context of camp experiences. These partnerships were able to enhance
the quality of the summer program because working with the school's teachers
allowed the program to identify potential participants, plan the curriculum
in a way that would best serve the needs of participating youth, and better
align activities with state standards.
Evaluators also recommended a number of potential strategies to make partnerships
with schools stronger. These recommendations generally fell under two overarching
strategies. First, programs should work with schools early, so as to ensure
that students and potential staff are recruited more easily and to allow enough
time to build relationships and engage in program planning. Second, programs
should develop formal mechanisms that will allow schools to transfer academic
and social information regarding students' academic and personal needs. Though
it is not always easy, through planning and partnerships, summer programs can
build strong relationships with schools to enhance the quality of youth's experiences.
Complementary Learning: Linking Multiple Supports for
Youth
Summer programs can be a critical component of what Harvard
Family Research Project calls a complementary learning strategy.
Complementary learning occurs when two or more institutions intentionally
link with each other to improve learning and developmental outcomes for
children and youth. These institutions include families, early childhood
programs, schools, out-of-school time programs and activities, higher
education, health and social service agencies, businesses, libraries,
museums, and other community-based institutions. HFRP calls this network
of supports complementary learning. Complementary learning is characterized
by discrete linkages that work together to encourage consistent learning
and developmental outcomes for children.
Many evaluations reported that forging such linkages was
both a priority and a challenge for summer programs. Academically focused
summer programs strove to link with schools, community members and organizations,
and families in order to strengthen their programming. The strategies
discussed in this brief under Challenges 4 through 6 can be viewed through
a complementary learning lens.
More information and resources on complementary learning,
including a guide on engaging families in after school programs, are available
at www.gse.harvard.edu/hfrp/projects/complementary-learning.html.
|
Challenge 5: Building strong, positive connections with participants' families
Establishing and developing strong, positive connections with participants'
families can provide academically focused summer programs with numerous advantages.
Getting to know youth's families can help programs understand the needs of the
youth they are serving. Programs can also encourage and facilitate families'
support of youth's learning and development. Because families are critical to
ensuring youth sign up for and attend programs, relationships with families
can help programs to recruit and retain youth (Lauver, Little, & Weiss,
2004).
Linkages between families and OST programs may also benefit parents' involvement
in children's education and schools and parents' relationships with their children
and may also improve implementation outcomes (Kakli, Kreider, & Little,
2006). Just as importantly, these linkages can help parents better support their
children's learning and development at home.
Many OST program evaluations have noted that they have found it challenging
to engage participants' families, and the academically focused summer programs
in this review were no exception. Programs, however, can adopt a number of strategies
to facilitate the development of connections with participants' families
Programs can create opportunities for parents to get involved.
Inviting parents and families to participate in program events and opportunities
can make parents more likely to be engaged and involved. For example, the Emery
Summer of Challenge, the evaluation of which found strong parent engagement,
provided an open invitation to parents to visit the program at any time. The
program also held several eventsincluding a 2-hour parent orientation
session at the beginning of the program, an open house toward the end of the
program, a Parent Night featuring exhibitions of youth's work, and an end-of-program
barbecue celebrationto encourage parent involvement. Parent attendance
at such events can facilitate communication about children's learning and development
to families and create engagement of parents in the program, which in turn can
support youth's active participation and engagement in the program.
Create strategies for coping with parents' schedules and child care situations.
Regular attendance by participants is critical to the success of academically
focused summer programs. However, attendance often suffers when families have
scheduling conflicts and child care challenges involving participants' siblings.
Many programs devised strategies to cope with these problems. Some found it
helpful to provide program slots for participants' siblings. The All-Around-the-Neighborhood
program in St. Paul, Minnesota, discovered that by incorporating siblings into
the program, it removed the barrier of some youth needing to care for their
younger siblings. As a result, the program was better able serve the needs of
youth and their families.
The District of Columbia 21st Century Learning Center similarly found that
parents appreciated the opportunity to have multiple children attend the same
program. Such strategies can improve not just youth attendance, but also parent
involvement. Some evaluations also suggested providing on-site babysitting as
a strategy for enabling parents to attend parent events.
Develop communication channels with parents to support their children's
learning.
Programs can be more successful if they work directly with parents to support
their children's learning. For example, the GEAR UP program in Austin, Texas,
makes developing connections with families a top priority. In this program,
each site employs a parent support specialist who is responsible for maintaining
consistent and frequent communication with parents. Parents are formally asked
to be partners in supporting their children's learning and development and sign
contracts with the program pledging their participation. These communication
channels have allowed the program to better support youth and have led to increases
in meetings with parents, home visits, personal phone calls, and referrals of
families to a variety of support services. Developing effective communication
channels between programs and families can mutually benefit both families' and
programs' abilities to serve the needs of youth.
Making It Fun and Educational: National Inventors Hall
of Fame Camp Invention
Camp Invention provides a series of programs that reflect
a mission of inspiration, teaching, and outreach through innovative hands-on
enrichment programs for students rising to grades 2 through 6. A summer
day-camp program offered in 5-day sessions, Camp Invention combines learning
and fun in modules that integrate science, math, history, and the arts.
According to Brenda Wojnowski of the National Institute Hall of Fame,
making the program fun and engaging is a big priority for the Camp Invention
program staff.
The program development team of Inventive Education, a subsidiary
of the National Inventors Hall of Fame, keeps the curriculum strong, fun,
engaging, and fresh. The team works to develop a large number of varied
modules, so that youth can participate in the program for up to 6 years
without ever having to repeat a module. While developing these modules,
the team brainstorms about each basic concept in order to build a unique
and fun curriculum.
After developing activities for the curriculum, the program
staff engage in the activities themselves and test the pro-gram in local
schools near the National Inventors Hall of Fame. If they do not find
the activities both fun and educational for students, the staff go back
to the drawing board. All modules are piloted and observed and only used
when the education staff feel confident that the program is fun and educational
and can be easily implemented by local teachers at the camps. Furthermore,
modules are continuously reviewed to see if they have become outdated
and are pulled if they are no longer current and engaging.
The formula seems to be working; this summer, there were
900 camps in 47 states. While the modules are designed to be fun, they
are also aligned with national, state, and district education standards.
External consultants and an advisory board of nationally recognized educators
and scientists review and revise all curriculum modules to ensure alignment
with these standards. The end result of all this effort is fun, engaging,
and educational programming that keeps youth excited about attending Camp
Invention, while at the same time learning valuable educational skills.
|
Challenge 6: Engaging community members, groups, and institutions in programming
Since summer programs often have limited resources and capabilities, linking
with other community entities can help them maximize the beneficial experiences
they offer to youth. By engaging actors in the community, programs can leverage
resources not usually at their disposal and thereby facilitate enhanced quality
programming. Businesses, colleges and universities, museums, and other community
institutions all possess valuable resources that can add to the format, content,
and location of programming. Coordinating with other community entities can
also help ensure a more seamless web of support for youth in the summer months,
as well as minimize conflicts between competing summer opportunities in the
community.
However, engaging and coordinating with other community settings requires commitment
and work and thus emerged as a sixth implementation challenge. Program evaluations
pointed toward some promising strategies for meeting this challenge.
Create linkages to colleges and universities.
Colleges and universities generally have more resources available for use in
the summer months than during the academic year, and many youth programs take
advantage of this opportunity to form partnerships with universities in the
summer. As noted above, college students, most of whom are not in class during
the summer, can provide an excellent source of program staff. College and university
resources also include classroom space and laboratory facilities.
Linking with the local university allowed the Summer Science Academy in Rochester,
New York, to gain access to modern lab equipment and technology, which in turn
enabled the program to provide exposure to advanced science topics and lab techniques
that youth could not get in school. One evaluation noted that the university
setting helped youth and their families to experience the college environment,
thereby contributing to their increased comfort with and excitement about the
idea of attending college.
Link and align with other complementary summer programs.
There are a variety of summer programs and activities available to youth. Such
overlap in programming can lead to competition for resources, staff, and student
attendance. Linking and aligning with other summer programs by forming community
partnerships can help reduce competition and maximize program potential. For
instance, the GEAR UP program in Austin collaborated with other academic summer
programs, referring youth between programs, sharing resources, and ensuring
that programs did not provide duplicate services.
How BELL Summer Program Strives to Meet the Seven Challenges
The BELL (Building Educated Leaders for Life) Summer Program
continuously strives to meet all the challenges articulated in this brief.
It offers full-day, 5-day-a-week summer programs in Baltimore, Boston,
and New York City. One hundred percent of the teachers conducting academic
instruction are state certified and receive an additional 32 hours of
training from the program. For outside enrichment, the program hires specialists
who bring knowledge, talent, and dedication to their areas of programming.
In the morning, youth learn core reading, writing, and
math skills from a staff of professional teachers and teacher's assistants.
In the afternoon, youth focus on strengthening social skills through daily
enrichment activities like art, music, drama, and dance. On Mentor
Fridays, youth learn from guest speakers and cultural presentations,
visit museums and parks, and engage with their communities in service
projects.
BELL operates with intentionality by conducting an evaluation
every year to assess program strengths and weaknesses and then taking
steps to address any areas in need of improvement. Recently, an internal
evaluation led the program to select a new curriculum that was as academically
intensive as the former curriculum but less labor intensive for staff.
At the same time, BELL develops individualized connections with youth
by conducting continuous assessments of youth's progress so that staff
can see where youth are excelling and where they need help. BELL follows
up on these assessments with thoughtful planning about how to make upcoming
activities responsive to the results of the assessments.
One of the biggest challenges BELL has faced is in working
with school systems to identify and secure facilities for program use.
Partnering with charter schools, private schools, and local colleges,
however, has reduced the need for reliance on traditional public schools.
BELL also develops linkages to families and community entities. The program
requires parents to sign reading logs to ensure that families
regularly support and engage in their children's learning and development
at home. In addition, the program provides an orientation that explains
program elements and requirements to families.
Throughout the summer, staff make calls to students' homes,
issue progress reports, and invite families to attend conferences. At
the end of the program, BELL holds a closing ceremony with food, student
performances, a choral presentation, and the culmination of a summer-long
jingle contest between different BELL campuses. The program
works with community-based organizations to connect alumni with high-quality
programs after they graduate from BELL and with other community agencies
to connect families with special services like health. Youth also complete
community service projects, which the program finds to be a critical component
in enhancing the program.
BELL makes sure to provide consistent fun enrichment activities
in the afternoons, as well as regular field trips on Fridays. These activities
are educational, but they also generate excitement from participants and
keep them engaged in the program, all of which makes accomplishing the
program's goals possible.
A recent random assignment evaluation of BELL found that,
as compared to students who did not participate, students in the treatment
group who received the 6-week program improved their reading skills by
about a month. The evaluation also showed positive effects on hours of
academic activities, books read, and the degree to which parents encouraged
children to read (Chaplin & Capizzano, 2006).
|
Challenge 7: Incorporating a variety of fun and engaging program activities
The summer has historically been a time of fun and relaxation for many youth,
and programs in this review reported greater success when they incorporated
a summer vacation spirit into their programming and attempted to
avoid remedial instruction-based activities. Indeed, some program evaluations
found that when programs emphasized remediation and conducted it in a dry, rote
manner, youth tended to become disengaged and attendance flagged. Since youth
engagement and attendance are critical to program success, finding strategies
to ensure a vacation spirit is of the utmost importance for summer
programs. The program evaluations revealed the following strategies.
Offer a variety of engaging and educational field trips.
Many evaluations indicated that field trips provided fun, engaging educational
experiences that generated enthusiasm among participants. Field trips are a
major component of the BELL program, which finds that trips to places such as
children's museums, zoos, and planetariums provide enriching experiences and
keep youth excited about attending the program week after week. Such field trips
can be an opportunity for summer programs to provide experiences that school-year
classes cannot or do not offer.
Design activities that are hands-on and focused on active learning.
Hands-on activities can also help keep youth excited, engaged, and learning.
The University of Virginia Summer Enrichment Program in Charlottesville, Virginia,
found that the active learning component of their program was critical, as youth
stressed the importance of actually doing something. The program
employed active learning approaches such as laboratory work, visits with experts
in science, or use of computer technology. An Oceanography Camp for Girls in
St. Petersburg, Florida, also found that its hands-on experiential learning
activities led participants to become excited about science. In some evaluations,
participants reported that they learned more about science in summer programs
than during the entire school year. Taking a thematic approach to hands-on learning,
such as focusing on oceanography, can also boost engagement in summer programs.
Summing Up for Summer: The Challenges and Strategies
|
|
Challenge 1:
Developing programming with intentionality
|
Strategies:
Build in program planning and development time before the start
of the program.
Build program planning and development time into daily program
operations.
Design activities and instruction purposively to achieve program
goals.
|
|
Challenge 2:
Building positive and individualized connections with youth
|
Strategies:
Utilize preexisting connections with youth.
Create opportunities to ensure individualized connections with
youth.
Develop mechanisms to provide staff with accurate information about
students' needs and backgrounds.
|
|
Challenge 3:
Recruiting and developing highly skilled staff
|
Strategies:
Recruit qualified and experienced school-year teachers for academic
components of the summer program.
Tap into the pool of talent available from parents, community members,
high school students, and college students.
Provide adequate and timely training to staff members.
|
|
Challenge 4:
Developing ongoing, mutually supportive relationships with schools
|
Strategies:
Create formal partnerships between summer programs and schools.
Work with schools early.
Develop formal mechanisms that will allow schools to transfer information
regarding students' academic and personal needs.
|
|
Challenge 5:
Building strong, positive connections with participants' families
|
Strategies:
Create opportunities for parents to get involved.
Create strategies for coping with parents' schedules and child
care situations.
Develop communication channels to work with parents to support
their children's learning.
|
|
Challenge 6:
Engaging community members, groups, and institutions in programming
|
Strategies:
Create linkages to colleges and universities.
Link and align with other complementary summer programs.
|
|
Challenge 7:
Incorporating a variety of fun and engaging program activities
|
Strategies:
Offer a variety of engaging and educational field trips.
Design activities that are hands-on and focused on active learning.
|
Putting It All Together to Implement High-Quality Summer Programming
While many summer programs face similar implementation challenges, each program
has different components and needs, and not every strategy is feasible or even
desirable for every program. Rather, the strategies presented in this brief
are intended to offer food for thought about how best to develop
and refine quality programming. Three major themes cut across the seven challenges
and accompanying strategies identified in this brief.
First, many evaluations identified the important role of linkages,
relationships, and partnerships. Most program goals can best be
achieved through coordination, leveraging resources, and building relationships
and partnerships. These connections can take many forms, from linking programs
to families, to building and leveraging relationships with community-based entities,
to developing partnerships with schools. Establishing such linkages can help
ensure that youth experience a more seamless network of supports that works
toward consistent learning and development.
A second theme that cuts across many strategies is youth engagement.
For academically focused summer programs to achieve their goals, youth need
to attend regularly and stay engaged in the program's instruction and activities.
Engagement can help youth become more invested in the learning process and therefore
achieve more positive outcomes. The goal of youth engagement drives many of
the strategies identified here, including intentional and ongoing program planning,
selecting and training staff members to work effectively with youth, and linking
with community members and institutions to provide active learning experiences.
A final theme common to most of these strategies is the importance of operating
with intentionality. Though this brief identified intentionality as one
of seven unique challenges, all six of the other challenges in fact depend on
a high level of intentionality and planning. This is consistent with current
thinking in the after school arena, which promotes a theory of change approach
to program design and implementationin particular, an approach that intentionally
links program goals, program elements, participant outcomes, and evaluations
in order to best develop effective OST programs (C. S. Mott Foundation Committee
on After School Research and Practice, 2005). Practitioners can use the challenges
and strategies identified here to think about similar strategies that they can
intentionally adopt in their own programs to maximize the quality and potential
of their summer programming.
Out of the scope of this brief, but nonetheless key challenges facing the
after school arena today, are sustainability of funding and responding to the
increasing demands for accountability with more and better ways to demonstrate
program results. However, the seven research-derived themes presented in this
brief are consistent with an emerging consensus about what constitutes quality
in after school and other out-of-school time programs (Eccles & Gootman,
2002; Rosenthal & Vandell, 1996; Miller, 2003). The specific strategies
outlined here are tailored to the needs and experiences of summer pro-grams,
but this set of research-based best practices adds to the mounting evidence
base that all quality OST programming should include opportunities for youth
engagement, skill-building experiences, practices that support positive relationships
with peers and adults, and connections with other settings such as schools and
families. Taken together, the quality indicators and the strategies identified
in this brief provide a framework for programs to continue moving toward providing
the highest quality services possible.
Widening achievement gaps between advantaged and disadvantaged youth over
the summer months are a major impetus for efforts to improve the accessibility
and quality of academically focused summer programming. Our review of program
evaluations builds on the findings about the academic benefits of such programs
to illuminate how programs can produce these meaningful outcomes. By assessing
the common challenges identified in our review and applying the appropriate
research-based strategies, programs can make the summer months an opportunity
for growth, rather than a time when at-risk youth fall further behind.
Acknowledgements
Preparation of this brief was made possible through the support of the Charles
Stewart Mott Foundation with additional support from the W. K. Kellogg Foundation.
The principal investigator for this study is Dr. Heather B. Weiss. We wish to
acknowledge Suzanne Bouffard and Priscilla Little from HFRP, and Brenda McLaughlin,
Deputy Director of the Center for Summer Learning, who offered a number of insightful
comments to improve the brief. We also wish to thank Joseph Davis of Ascend,
Brenda Wojnowski of the National Inventors Hall of Fame, and Tiffany Cooper
of BELL, for speaking with us about how they strive to create successful summer
programming.
Notes
1 The Appendix
contains a complete list of evaluations that were included in the review.
References
Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2001). Schools, achievement,
and inequality: A seasonal perspective. Educational evaluation and Policy
Analysis, 23, 171191.
Burkam, D. T., Ready, D. D., Lee, V. E., & Logerfo, L. F. (2004). Social-class
differences in summer learning between kindergarten and first grade: Model specification
and estimation. Sociology of Education, 77, 131.
C. S. Mott Foundation Committee on After-School Research and Practice. (2005).
Moving towards success: Framework for after-school programs. Washington,
DC: Collaborative Communications Group.
Chaplin, D., & Capizzano, J. (2006). Impacts of a summer learning program:
A random assignment study of building educated leaders for life (BELL).
Washington, DC: Urban Institute. http://www.urban.org/url.cfm?ID=411350
Cooper, H., Charlton, K., Valentine, J. C., & Muhlenbruck, L. (2000).
Making the most of summer school: A meta-analytic and narrative review. Monographs
of the Society for Research in Child Development, 65(1), 1118.
Downey, D. B, Broh, B. A., & Von Hippel, P. T. (2004). Are schools the
great equalizer? Cognitive inequality during the summer months and the school
year. American Sociological Review, 69, 613635.
Eccles, J. S., & Gootman, J. A. (Eds.) (2002). Community programs to
promote youth development. Washington: National Academy Press.
Fairchild, R. (2006). Making summer reading a priority. Presentation
at the Virginia State Reading Association 39th Annual Conference, Virginia Beach,
VA, March 911, 2006.
Halpern, R. (2003). Supporting the literacy development of low-income children
in afterschool programs. New York: The Robert Bowne Foundation.
Heyns, B. (1978). Summer learning and the effects of schooling. New
York: Academic Press.
Larson, R. W., & Verma, S. (1999). How children and adolescents spend
time across the world: Work, play, and developmental opportunities. Psychological
Bulletin, 125, 701736.
Lauver, S., Little, P. M. D., & Weiss, H. B. (2004). Moving beyond
the barriers: Attracting and sustaining youth participation in out-of-school
time programs. Cambridge, MA: Harvard Family Research Project.
Miller, B. M. (2003). Critical hours: Afterschool programs and educational
success. Quincy, MA: Nellie Mae Education Foundation.
Rosenthal, R., & Vandell, D. L. (1996). Quality of care at school-aged
child-care programs: Regulatable features, observed experiences, child perspectives,
and parent perspectives. Child Development, 67, 24342445.
Appendix: Program Evaluations
Included in This Review
References available at:
www.gse.harvard.edu/hfrp/projects/afterschool/bibliography/index.html
21st Century Community Learning CentersDistrict of Columbia
21st Century Community Learning CentersOrleans Southwest Supervisory
Union, Vermont
All-Around-the-Neighborhood
Arlington Public Schools Summer School
Ascend Summer Youth Program
Baltimore City Public School System Summer School
BELL Accelerated Learning Summer Program
Boys & Girls Clubs of Metropolitan PhoenixArtWeb
Break-Aways Partnerships for Year-Round Learning
Chicago Summer Bridge
Extended Learning Opportunities Summer Program
Gain the Edge
GEAR UPAustin, Texas
Gevirtz Summer Academy
Girls Math and Technology Program
InfoLink
Jobs for YouthBoston PLATO Summer Transition Program
Louisiana State Youth Opportunities Unlimited
National Society of Hispanic Masters of Business Administration's Summer
Enrichment Program
National Inventors Hall of Fame Camp Invention Program
New York City Summer School
Oceanography Camp for Girls
San Juan Unified School District Summer Intervention Programs
Summer Career Exploration Program
Summer of Challenge
Summer Opportunity to Accelerate Reading
Summer Science Academy
Summer Training and Education Program
University of Arizona SEEK Book Making and Writing Course
University of Virginia's Summer Enrichment Program Invention and Design
Verilette Parker Science Intervention Program
Voyager Summer Program
Wake County Summer Academy
Washington Reading Corps
|
Related Resources
Making the Most of Summer School: A Meta-Analytic and Narrative Review
This meta-analytic review of 93 studies of summer school programs examines
the effectiveness of summer school programs, as well as features of programs
associated with larger estimated program effects. Cooper, H., Charlton,
K., Valentine, J. C., & Muhlenbruck, L. (2000). Making the most of
summer school: A meta-analytic and narrative review. Monographs of
the Society for Research in Child Development, 65(1), 1118.
Summary available at: www.summerlearning.org/resourcesresearch/
sumschool.html
Making the Most of Summer: A Handbook of Effective Summer Programming
and Thematic Learning
Making the Most of Summer is a resource for providers who want
to improve the quality of their summer programs by meeting the academic
and youth development needs of their participants. The handbook contains
a variety of easy-to-use plan-ning tools designed to help summer programs
a) incorporate the characteristics of effective summer learning programs;
b) implement engaging thematic units that meet challenging academic and
youth development standards; c) improve the quality of summer staff development
opportunities; d) evaluate the success of their programs and services;
and e) develop a long-term strategy for sustaining their work. Fairchild,
R., McLaughlin, B. & Brady, J. (2006). Making the most of summer:
A handbook on effective summer programming and thematic learning.
Baltimore, MD: Center for Summer Learning. Available at: www.summerlearning.org/productsservices/index.html
Summer Learning: Research, Policies, and Programs
This book brings together up-to-date, research-based evidence concerning
summer learning and provides descriptions and analyses of a range of summer
school programs. The chapters present theory and data that explain both
the phenomenon of summer learning loss and the potential for effective
summer programs to mitigate loss and increase student achievement. Borman,
G. D., & Boulay, M. (Eds.) (2004). Summer learning: Research, policies,
and programs. Mahwah, NJ: Erlbaum. Available at: www.erlbaum.com
|
|