Do Teacher Evaluation Systems Improve Teacher Effectiveness?OVERVIEW:
Recent research confirming the important role teachers play in promoting the academic achievement of their students has prompted an increased interest in and emphasis on teacher evaluation. Research suggests there is a wide variation in the ability of teachers to promote achievement and it can be difficult to identify the most and least effective teachers. Federal and state governmental agencies are encouraging school districts to develop systems for evaluating teachers. The research to date on evaluation systems has largely examined the extent to which current systems are effective at identifying teachers who are good and less good at promoting student test score growth. A second, less studied, question is whether the process of participating in evaluation helps teachers improve. Given that a number of mature systems provide direct feedback to teachers at numerous points in the evaluation cycle with the intent of improving performance, there is reason to believe that going through the evaluation cycle itself might have an impact on teacher quality. We examine the empirical evidence on this question using data from the Cincinnati Public Schools for 1999-2000 through 2008-2009. These data contain administrative record data on all teachers, evaluation information for teachers who underwent evaluation at any point during this time, and demographic and test score information on students taught during this time period. Given that the components of Cincinnati’s teacher evaluation system are characteristic of early proposals for the kinds of systems districts and states are being encouraged to adopt, information from this study is both timely and relevant.
RESEARCH METHODS:
Cincinnati’s Teacher Evaluation System (TES) program grew out of a 1997 collective bargaining agreement between the Cincinnati Federation of Teachers and the Cincinnati Public Schools. During the 1999-2000 school year Cincinnati Public Schools field tested the TES system that utilizes trained evaluators, a specified and research-based evaluation rubric, and includes multiple classroom observations of teachers during a year. The program was implemented in full the next year, academic year 2000-2001.
Cincinnati Public Schools maintains detailed records for each TES evaluation, including scores from each classroom observation and each portfolio review that contribute to the final score. Our data currently contain records on 2,071 teacher TES evaluations covering 2000-01 through 2008-09 with a high of 292 in 2006-07 and a low of 112 in 2000-01. We anticipate that TES records from academic year 2009-2010 will be made available to us for this proposed study.
Paralleling the TES program years, we have panel data on Cincinnati students for the 2000-01 through 2008-09 school years (and anticipate getting data for 2009-2010). When our data begin in 2000-01 Cincinnati enrolled approximately 21,000 students in grades 3-8, but enrollment had fallen over 30 percent to approximately 14,500 by 2008-09 (Ohio Department of Education, 2009). The student--by-year observations include information on the student’s gender, race or ethnicity, English proficiency status, participation in special education or gifted and talented programs, class and teacher assignments by subject, and, when applicable, standardized test scores. In these data we are able to match students to teachers enabling us to construct measures of a teacher’s ability promote student test score growth.
To answer our research question our plan is to combine these two sources of data so that (1) we can identify teachers who did and did not go through TES evaluation, along with their evaluation year, and (2) examine the extent to which teacher effectiveness, as measured by student test growth, changes as a result of going through TES evaluation. In examining this primary question we will be able to take advantage of a particular feature of the institutional history of the TES program in Cincinnati. Beginning in the 2005-2006 school year experienced teachers in Cincinnati (those who had been employed for more than three years) were phased into the TES program one or two cohorts at a time. The school year when an experienced teacher was first required to participate in TES was determined by the year she was hired into the district. The phase in was not sequential; for example, teachers hired in 1997 participated in TES in 2005-06 but teachers hired in 1995 or 1998 participated in 2007-08. Under the assumption that contract year is not directly related to a teacher’s effectiveness after cohorts have passed the early, steep part of the teacher experience curve, this process of assigning teachers to TES evaluation based on contract year essentially means that we have exogenous variation (approaching random assignment) in our question variable of interest, whether or not a teacher underwent TES evaluation in a given year. Thus, to continue the earlier example, we can compare 2005-06 and 2006-07 student achievement for teachers hired in 1997 (treatment group) to teachers hired in 1995 and 1998 (comparison group).
