New Study of Boston Charter and Pilot Schools Finds Charter Schools Have Positive Effects on Student Achievement

01/06/2009 8:53 AM


Researchers from the Harvard Graduate School of Education and MIT have released the results of a groundbreaking study that suggests charter school students in outperform their peers at other public schools in . Results for were less clear; some analyses showed positive results at the elementary and high school level, while results for middle school students were less encouraging. The study uses an innovative design based on school lotteries that allowed for a direct comparison of charter and pilot school students with their peers.

The research team, led by Thomas Kane, director of the Project for Policy Innovation in Education and Professor of Education and Economics at the Harvard Graduate School of Education, and Joshua Angrist, Ford Professor of Economics at MIT, used data provided by the Massachusetts Department of Elementary and Secondary Education. The study is published in a report by The titled, Informing the Debate: Comparing Boston’s Charter, Pilot and Traditional Schools. The Rappaport Institute provided additional support.

“Fifteen years ago, the charter school movement in Massachusetts was launched to see if new models could lead to gains in student achievement,” said Kane. “The results of this study suggest that in Boston are making a significant difference.”

Positive effects of charter schools on student achievement were found at both the middle and high school levels and across subjects. The impact on middle school math was particularly dramatic; the effect here amounts to half of a standard deviation, an effect large enough to move a student from the 50th to the 69th percentile in student performance in one year. In fact, the effect of a single year spent in a charter school was equivalent to half of the black-white achievement gap. At the high school level, charter students showed stronger performance scores in English Language Arts, math, writing topic development, and writing competition.

“This report speaks to the promise of education reform – and to its potential impact,” said , president and CEO of the Boston Foundation. “Innovation in schools can help us to close the achievement gap that has clouded a stellar record for success among Boston’s schools, which have earned a reputation as among the best large urban systems in the country. The charge forward is to apply the ‘best practices’ of our charter and pilot schools to serve every family that wants to be a part of this culture of success.”

The team tracked the outcomes of students who had participated in the school admission lotteries, comparing those who had been offered a slot to those who had been denied. The school lotteries, which are required under the state’s charter law when a school is over capacity, provide a way to answer the common complaint that the charter school applicants are ‘different’ from their peers in the traditional public schools. “This study is unique because it makes use of unusually rich data on all schools in Massachusetts,” said Angrist. “The opportunity to use random assignment – the lotteries – to study and compare two important models for school decentralization means that the students we compared were very similar in terms of family background, motivation, and anything else you might think of except for the likelihood of attending a charter or pilot school.”

“At the time of admission, the only difference between applicants who were offered admission and those who were not was a coin flip,” said Kane. “The fact that there are large differences in subsequent performance suggests that the charter schools were indeed having an impact. The next step is to identify what’s working in charter schools that can be transferred back into the traditional public schools to improve student achievement.”

Read the technical report at http://www.gse.harvard.edu/~pfpie/pdf/Technical_Appendix_Final.pdf.

, , , , , , , , ,

  • Annie Brown

    This article makes me wonder how “student achievement” is being measured. Higher test scores are great, but at what cost? From my own experience, I know that charter schools, in order to retain their charters must “perform” which leads to intensive test prep. I have taught in Cambridge and Boston public schools and currently teach at a Los Angeles charter school (open to all and serving a low SES/ELL population). Factors like our small size and our project based curriculum help students achieve–as students, humans, and as test-takers. We outperform the district on testing, but to do that, there is a significant emphasis on standards and testing that are in direct contradiction to my firmly held beliefs. It was at the Ed School that my educational philosophy was molded by wonderful professors (Eleanor Duckworth and Lisa Schneier among others) advocating the Teaching for Understanding framework and emphasizing a respect for Multiple Intelligences and differentiated instruction. I just caution falling to heavily into the trap of measuring success by a test score. That diminishes our work as educators and student achievement that doesn’t fit into a bubble answer sheet

  • Anonymous

    So far so good, but could it be explained why “results for middle school students were less encouraging”? Is it because of the adolescent years?

  • Nini

    I agree with Annie Brown. Even though I don’t quite understand how a charter school runs, I think a single test is far from sufficient to reflect the multiple intelligence of a student.
    I am a student from China. I guess China can boast the largest population of the best test takers. In order to do that, when we were at school, we were forced to study at school from dawn till midnight and sacrificed the time playing with friends. Students who got the top scores are considered clever even though they have bad manners or don’t talk to people very often.
    I really hope that one day, there will be no tests, but a stage on which all of the students’ talents can be taken into full play.

  • Jon Dreyer

    The study correctly notes that charter students and traditional public school students are very different populations because charter students are only those who, with their families, took the initiative to apply. The authors should be commended for their lottery study in which they only considered students from that population. However, the study fails to consider a few extremely important points, which limits the applicability of its results.
    First, though the students in the lottery study may have been comparable, their cohorts were not. Lottery winners go to school only with students who applied. Losers go to school with a population that overwhelmingly did not apply anywhere. The student population of a school is one of the best predictors of student success. Thus, despite the care taken in the lottery study, we still do not know whether charters outperform traditional public schools because they do something better or because they cull the best and the brightest from the traditional public schools.
    Second, when evaluating the benefits of charter schools, it is important to consider not only the effects they have on their own students, but also the effects the culling has on the schools and students left behind in the traditional public schools. If charters really do cull the best and the brightest, the effect on traditional public schools may be devastating.

  • Molley Kaiyoorawong

    I agree with Annie Brown and Jon Dreyer brings up an excellent point. I attended a “public” magnet school which boasts a 100% college matriculation rate and a high position in national rankings but, in order to get in, students must pass IQ and pyschological tests. The fact that we did so well wasn’t because our administration and teachers were the cream of the crop; it was because our classmates were all reading well above grade level, were highly motivated and had well-educated, involved parents.
    I wonder which peer-reviewed publication this study will be allowed to be published in.

  • Cheryl Parchia

    After attending HGSE, I founded a charter school in Ohio which I currently serve as Superintendent. Upon admission the children who attend our school and other charters in the state most frequently perform academically 2-3 years behind their chronological age and grade level. In other words, the majority of these children have been unsuccessful in traditional public schools and their parents or caregivers are generally seeking a plausible remedy for their educational losses.
    It takes time and a dedicated staff to see the children make progress. If the student population in Boston charter schools is similar to ours, these strides have been made tediously and not because their parents “looked” for a school or because the children were carefully screened and selected for their talents.

  • Nell

    I would just like to point out that public schools have to accept all children regardless of their skills and desire to learn. The public school system is required to supply all students with the same educational opportunities. This is not a fair comparison as long as charter schools are allowed to select the type of student and can design their curriculum without outside interference.

  • Leslie Wade

    While it is true that some charter schools do (a) exclude students with special needs, and (b) fill the school day with test prep for the sake of stats, many charters are producing overwhelming results WITHOUT excluding students, while emphasizing the importance of character, citizenship and moral development. It is unfair to generalize about charter schools, as it would be generalize about the environment and issues in the public school classroom.
    The education of a child is a complex mix of teaching quality; parent involvement; an environment that supports learning; hunger, health and housing; and dozens of other components. To discount the success of charters based on the ideas that “it’s all test prep” or “their parents are motivated” discounts the fact that thousands of students around the country are being given a chance that they did not have in the traditional public school classroom. To marginalize charters creates a short sighted outlook that results in traditional school systems–many of which are using antiquated models–refusing to learn from the good work that is being done all across the country.

  • Joseph Ricca ’03

    As a school administrator myself, it is always wonderful to hear of student successes, no matter what the form of school they attend. However, as some previous posts point out, Charter Schools are comprised of students that are often selected through some form of screening, be it lottery or application. This fact makes Charter Schools inherently different from Public Schools – where all children are educated.
    While we should continue to focus our research and academic efforts on what seems to work in schools, we must also remember that it is our duty to provide the best possible education for all of our students (in all schools).
    To be sure, the vast majority of our children attend public schools – if best practices identified within the Charter School arenas translate effectively and practically in the public schools, then this is a victory for our children, if not – we will continue to experience a further schism between the these types of schools. I fear that this divide will, in the end, continue to stratify the “types” of students that are positively impacted by our research.

  • L.T. Marsh

    I agree with Joe Ricca, progress in education in any urban arena is a wonderful thing, but we certainly have to be cautious when we say “best practices.” As a Senior Business Consultant, having worked with public and commercial clients, I use this term often when there is an opportunity to leverage experiences/results (that are proven over time). However if the Charter Schools strategies of learning, reading, motivation, testing, etc. are truly best practices we should be able to apply it in the public urban school setting, even beyond Boston and achieve the same results. I look forward to reading how the Boston public schools are achieving with the implementation of the Charter strategies.

  • JC

    I agree with Jon Dreyer. I think the fact that you need to admit someone means that person and his/her family is already interested and engaged in the school. The population is biased.
    Jon’s comparison is much better. Another is to take over a school “as is” and call/run it as a “charter school”. Then, compare results.

  • GET

    Great comments regarding this information. I think however, there is such a reaction to this information that there is a sense of competition and doom for public education that is really amazing. Life is already stratified. If you don’t believe that I suggest you really don’t have a good grip on those skill sets of successful people. In institutional settings what is reflected back to society is often the result of what has been put into the setting. So one should not be surprised at this information gleaned from Charter Schools, given the demographics of the student population and the underlying parental background. The real issue at hand is how do we extend the profiles of charter school children and families to those in public schools? If one comes to a conclusion that it cannot be done, or cannot be done for a reasonable social and economic cost, then we have a more honest understanding of what is before us…not a comparison of Charter versus Public school education, but a question of whether a free capital based economy and society can ever really provide equal opportunities for education to all children. I believe it is the 800 pound guerrilla in the room…does anyone want to escort it out?

  • Libby Craig ’04

    I would love to know if the “age” of the charter schools studied was taken into account. I currently teach at a public elementary magnet school in its ninth year that is just transitioning out of what I would consider its first phase (i.e. most original families’ children have now moved on to middle school, members of the founding faculty and administrators have begun to move on to other jobs, retire, etc.). While the school continues to be largely successful, it is becoming clearer that the energy of a start-up school is fading to a certain extent, which inevitably presents challenges that may or may not influence students’ achievement. Given the number of charter schools that are founded and opened by predominantly younger educators and/or rely intensely on the high-energy commitment of their start-up teams, I would be interested to know if the results of the study are consistent once that “first phase” has passed.

  • Richard Fontaine

    The real question here is freedom. The Charter School movement is part of the freedom to chose that all Americans should have. As a parent of four and a tax payer I have a lot of strong opinions about where the public money for education should be spent. Too often well meaning politicans forget that it is the people’s money that is being spent. Parents should have a right to chose how their money is spent for their children’s education. It should not be held captive to a monopolistic and entrenched public school administration or a teachers union that wants to maintain the status quo. Any movement toward allowing the consumer, i.e. the parents to have a choice about the public education of their children is a huge step forward not only for those parents and their children but also for taxpayers.

  • Margaret P.

    When I read the article and comments, it strikes me that the big effect might come just from the relative size of the schools.
    Here in Rochester, MN, there are three big (about 1100 student) middle schools which serve geographic areas and one “choice” middle school with about 350 students. My children attend the choice school and I have noted that this school has a higher percentage of low-income, special education, non-English proficient, black, and Hispanic students than any of the other middle schools. That means a higher percentage of ALL the groups of students who are typically harder to educate, yet the choice school has better test scores and greater progress from year to year for ALL of these groups.
    The degree of choice here is that parents must apply for their children. The school site is near neighborhoods with large concentrations of poor, black, and Hispanic families and they apply because it is the closest school. It is possible that these parents are the most motivated, but in my volunteer work there I have worked with many struggling students who were getting no help or encouragement with their schoolwork at home. That is the concentration of “hard to educate” students.
    There is also a concentration of high-achieving students there because the former principal grouped the highest achievers in one section per grade and the other middle schools did no grouping at all. The current principal has been there for five years and she grouped the lowest achievers in another section and provided extra teacher and para support to that class and the teacher taught at least the fundamentals of the curriculum.
    I led an effort starting seven years ago to start a program for highly gifted middle school students and four years ago this principal asked for the program to start at her school. When the “pull” from her was added to the “push” from my group, the school board approved the program. The principal arranged for professors to come in and teach her and her entire staff about the needs of gifted students and how to educate them.
    She asked her teachers to teach “extended” versions of the normal classes and put the highest achieving students in them.
    Almost every teacher there now spent 18 months of Friday night and all day Saturday classes every other week and completed a certificate in gifted education.
    The highly gifted program started in September 2007 with one section of highly gifted 6th graders. They are now 7th graders and my daughter is in the second group of HG 6th graders. Next fall we may have two sections of HG 6th graders.
    With the high achievers and highly gifted students come involved parents. I am one of about a dozen parent volunteers and one of three who works directly with students. The others support the staff in a variety of other ways. The extra help in and outside the classroom means that the teachers can work more effectively. Sometimes I work with a handful of students during their “target” time when they can read or do homework and when announcements can be made. Usually these are students who are struggling with a few specific things and I can often help them get over their mental blocks and understand what they need to do because I can work with them in a small group. Most of the time, I help a couple of students first, who can then do their work and I am there to help if needed. Then I can work with the 1-3 others who need more help to understand the material and give lots of diagrams, examples, and other forms of help. Almost always I can get these student to understand what they need to be able to progress in the subject. That’s what individual or small group help can do for students.
    Several of the teachers make themselves available after school on designated days to provide extra help to students. The most struggling students stay after school M-Th for Afterschool Academy which combines healthy snacks and physical activity with lots of extra help.
    Thus, my recommendations for a school which works well and raises student achievement is addressing the needs of both the highest and lowest achievers in a small setting where more or less all of the teachers can know and keep track of all of the students. Then recruit enough volunteers so that the struggling students can get lots of extra help and attention. If charter school do this, they will succeed. If public schools do this, they will also succeed.

  • Harold Potler

    Isn’t it the “Chapman Report” of the ’60s still valid – that the socio-economic status of the parent/parents is the most important factor in the child’s academic success? I think that a lot of thought should be going into how our schools can produce students who can compete with those in Asia and parts of Europe.

  • Robert J. Byron

    Most charter school teachers commit to teaching in a charter school environment because of the choice presented. The range of options in learning approaches is so varied that teachers can usually find a school whose philosophy and methods are consistent with their own. This nurtures the sense of identity with a school’s mission and creates a sense of professional academic freedom. It comes close to the vocational calling of Nuns, Brothers, and Priests who used to dominate the teaching ranks of Catholic schools. Even today, most parochial schools achieve academic results superior to public school counterparts – even though their teachers are paid less. The motivational driver for most charter teachers is a zeal for professisonal excellence and student success – hard to measure except by results.

  • Erica C.

    I’ve taught in both traditional and charter urban schools. There is, indeed, an element of triage involved in the success of charter school students. Charter populations are diverse, but there is some self-selection on behalf students and there families.
    Still, my city’s traditional public schools are largely overwhelmed by the social problems they are expected to mitigate before they can even begin teaching well. At least charters offer a haven for some of those students. As we strive for social equity on a broader scale – the undertaking of centuries – effective charters might be a step in the right direction. They give urban kids from poor and dangerous neighborhoods a model of what a healthy school can look like. They plant a seed of hope and excellence in struggling communities. They keep teachers teaching who might otherwise leave the field.
    Of course, thorough studies must back up these personal observations. I’m too busy teaching to do it, yet I hope these lines of research are pursued and updated as urban charter schools mature and expand.

  • Laurel O’Donnell

    Hi, Jon! I remember discussing these issues about charter schools a year ago last fall in Kay Merseth’s school reform class. Last summer, I interviewed at a charter school in the city where I now live, and this gave me better insight into the arguments for and against charter schools. I agree with what you have to say about the student population differences in some school systems. The school where I interviewed, however, struggled at first to meet its enrollment requirement before the school year started. Although the school had great ideas for instruction, there was a feeling of animosity between the charter school teachers and the traditional public school teachers. This may have affected the slow-growing enrollment. Also, this school specialized in reading, which attracted ELL students–many from a Burmese population. I doubt that this charter school would have shown the population selection bias that the above study did.

  • Eileenmc

    My daughter, a 2nd grader, has attended an urban charter school in Boston since K2. Although she spent only a couple of weeks in the local public school before she got a seat in the charter, I was able to compare the two organizations. Many of the differences have already been noted in previous posts, but what stood out for me that my daughter’s charter school is a vibrant learning community while the public school was a bureaucracy.

  • Ashley

    I couldn’t agree more with Nini. I am a university student in Shanghai, China. Heavy school bags and thick glasses are the typical equipments of a Chinese student. When I was in senior high school, we had to stay up late ,to 1 or 2 am, and get up at 5 am ,just for studying.
    The exam-oriented education system and the National University Entrance Examination(same thing as SAT) are the reasons for our exhausting work.And it was every student’s wish to access the free and innovative atmasphere of universities in their imagination. However, when I got into the university, I found nothing had changed, we still have to face the endless homework. We are supposed to perform the calculations in mathematics but never to ask and understand why. We just follow what the teacher asked us to do! That’s horrible! We can see the wornout patterns and the immovable barriers of the enhancement of the education, but we can do nothing to change it ,it is sad.In China’s schools, the most valued is not how far your eyereach is or how capable you are, but how high your score is, score is everything. Our creativity was, is being and will still have to be killed by the educational system.

  • Andy M

    One thing that is still largely misunderstood about charter schools is that they “select” their students. The whole point of the lottery is that it is required by law if there are more applicants than there are spots. Some people, for their own reasons, spread a deceitful image of charter schools as being elitist institutions that only select the best and brightest. They do this to explain away the fact that some charter schools are doing extremely well. So please don’t listen to people who are either misinformed or have their own agenda.
    Let’s also remember that we have gotten to the point now where you cannot talk about charter schools as being all one type of school. There is large variety among charter schools now. Some charter schools actively advertise their school by going door to door to communities with the lowest achieving children,and I’m sure there are others who do the opposite. So of course there are going to be experimental biases depending on populations that are being drawn, but this study is controlling at least for the population as it exists at the time of the lottery. However, John Dreyer makes a good point above, that you cannot control for the effects of the cohorts between students selected by the lottery and those not.
    In the end this study is on the right track, and we need many more like it and on much larger scales. The charter school movement is providing us with a way to test new theories and practices, and we need good data to determine which of those practices are being most successful so that we can implement them across the board. The lottery is an ideal research tool, because rarely do researches get such a controlled random assignment without even having to do one themselves.

Latest Activity

Upcoming Events View All >


Jill Anderson

News Officer