Heather C. Hill is a professor in education at the Harvard Graduate School of Education (HGSE). Her primary work focuses on teacher and teaching quality, and the effects of policies aimed at improving both. She is also known for developing instruments for measuring teachers' mathematical knowledge for teaching (MKT) and the mathematical quality of instruction (MQI) within classrooms. Hill is co-director of the National Center for Teacher Effectiveness and also principal investigator of a five-year study examining the effects of Marilyn Burns Math Solutions professional development on teaching and learning. Her other interests include knowledge use within the public sector and the role that language plays in the implementation of public policy. She has served as section chairs for the American Educational Research Association and Society for Research on Educational Effectiveness conferences, and on the editorial boards of Journal of Research in Mathematics Education and the American Educational Research Journal. She is the co-author, with David K. Cohen, of Learning policy: When state education reform works (Yale Press, 2001).
Click here to see a full list of Heather Hill's courses.
This study describes an assessment strand late-stage design project that focuses on improving existing measures of teachers mathematical knowledge for teaching. Original measure development, which occurred at the University of Michigan during the period 2002-2010, had several goals: to identify the knowledge useful to teachers work with students and to explore the possibility that this knowledge is unique to teaching; to provide a set of measurement instruments that could be used in research on teachers knowledge; and to provide evaluators with an easy-to-use online administration and scoring system. These efforts resulted in widely disseminated instruments, numerous academic papers, and theoretical progress regarding the mathematical knowledge teachers use in their work.
We now seek to update these measures. One reason is their wide use: anecdotal evidence suggests that up to 25% of our target teacher population may have taken a version of these instruments in pre-service training or in-service professional development. To respond to this issue, we will create over 300 items and 10 new sets of parallel forms in the most frequently tested grades and topics. Another reason to update the measures relates to new mathematics content and practice standards; a review of existing forms suggests we could better align our item pools to this key instructional guidance. We also seek to respond to a variety of user requests, and to also make our online delivery system, the Teacher Knowledge Assessment System (TKAS), more flexible in both the item formats it can accommodate and in responding to additional form updates.
This study will produce a comprehensive synthesis of the research base that informs science, technology, engineering, and mathematics (STEM) teacher workforce and curriculum improvement efforts. The project team will utilize meta-analytic techniques to estimate the impact of STEM teacher professional development and novel curriculum materials on student outcomes, and analyze the relationships between program effectiveness and key moderators identified in the literature, such as duration, intensity, format, grade and disciplinary topic, and alignment with NCTM/NSTA standards. The completed synthesis will provide researchers, practitioners, and funders with evidence to inform future STEM workforce and curricular investments.
Identifying the characteristics of effective teacher professional development and curricular improvement efforts has been identified as a grand challenge facing contemporary U.S. STEM education (e.g. Wilson, 2013). A core intellectual contribution of the current study will be to produce the most comprehensive and rigorous synthesis to date of the causal research evidence on this topic. Prior reviews in this domain have suffered from a paucity of studies, which in some cases necessitated the inclusion of studies whose weak designs failed to rule out threats to validity. This project will improve upon prior reviews by synthesizing the substantial newly-available evidence from rigorous impact evaluations funded by the Institute for Education Sciences (IES) and the National Science Foundation (NSF), combined with additional causal studies identified via a comprehensive literature search. Researchers will utilize meta-analytic models and conduct theory-driven moderator analyses to identify characteristics that are associated with interventions effectiveness.
This project will advance the goal of improving K-12 teacher professional development and curricular interventions for STEM disciplines in the United States. The comprehensive research synthesis report will provide information to the National Science Foundation on characteristics of effective and ineffective professional development and curricular investments, which could inform funding priorities. In addition, the project team will make results available to researchers and practitioners. Researchers will be able to utilize the studys results to inform both theory-building and future study designs in these domains. To reach practitioners, the study team will produce short briefs and disseminate them via research-practice networks as well as professional organizations. Ideally, practitioners will utilize the studys results to guide their own school- and district-level resource allocation decisions regarding professional development and curriculum investments. By strengthening the evidence base that informs investments in STEM workforce development and curriculum improvement efforts, this research advances the ultimate goal of bolstering instructional effectiveness and US students achievement and outcomes in mathematics and science.
For the last 25 years, three major goals have animated the U.S. mathematics education community: more knowledgeable teachers, more challenging curricula for students, and more ambitious instruction in classrooms. Yet despite volumes of policy guidance, on-the-ground effort and research over the past decades, few comprehensive and representative portraits of teacher and teaching quality in U.S. mathematics classrooms exist. Instead, most research into these topics has been conducted with small samples, non-representative samples (e.g., Kane & Staiger, 2012) or by research projects that use diverse instruments, meaning that it is difficult to ascertain what, if any, progress has been made toward the three goals.
To provide information on such progress, we will collect data on teacher content knowledge, curriculum use, and instruction from a nationally representative sample of U.S. middle school mathematics teachers. A written survey component of this study will build on a similar study conducted in 2005 06 (Hill, 2007), allowing for the comparison of teachers curriculum use and content knowledge and more specifically, their mathematical knowledge for teaching (MKT) across time periods. An observational component will record and score videotapes of instruction, allowing for a description of current instruction as well as a comparison of current instruction with that observed during the TIMSS video study (Heibert et al., 2005). The new video dataset will also serve as a baseline for future studies of instruction, for instance ones comparing current instruction to that in 2025, to assess whether Common Core State Standards have been met.
This study is largely descriptive, as are many other studies of its kind (Hiebert et al., 2005). However, as in our past research (Hill, 2007; Hill et al., 2008), we can inquire into relationships between key variables (e.g., curriculum use, MKT, and instructional quality). Such inquiry can form the basis for future research. We can also describe instruction as it exists across a wide variety of U.S. classrooms, for example, asking whether as is often assumed instruction in urban districts is inferior to those in other areas and whether differences in instructional or teacher quality by academic track (honors, general, or remedial) exist. Finally, we believe that describing the mean and range of U.S. instruction can have a profound effect on the field, much as the TIMSS video studies (Stigler & Hiebert, 1999; Hiebert et al., 2005) did over a decade ago.
We argue that timely information on these topics would be useful for three reasons. First, mathematics educators have both researched and attempted to improve teacher knowledge, curriculum, and instruction for over two decades; examining current levels of each would provide evidence about the extent to which these goals have been met. Second, the Common Core has set new goals for teachers and teaching; it is important to establish a baseline for future research by describing the status quo as new curriculum materials and assessments reach classrooms. Finally, we believe that the work described in this proposal will demonstrate that teacher and teaching quality in the U.S. can be effectively tracked, similar to other national indicators such as NAEP.
CEPR will lead instrument development and survey administration work for the Knowledge Utilization Center. We will engage in an iterative cycle of instrument development, piloting and revision during year 1. This will include literature review and construct development, item development, cognitive interviews and analysis, and rostering 15-20 positions for a sample of 1000 districts. We will also plan a deep dive survey of 30 large districts where all positions within our sampling categories (positions likely to involve instructional decisions at the k-8 level) will be targeted, and conduct pilot testing with 200 respondents. In year 2, we will sample and administer surveys to one respondent each in 1000 mid-sized and large districts, as well as all positions rostered in the 30 deep dive districts. The surveys will be administered via mail, including a personally signed pre-letter, survey package, incentives and several follow-up contacts. We will supplement standard survey techniques with email contacts and the option for individuals to complete the survey online. We will participate in analytic work and contribute to paper
writing in year 3. We will also do a small survey of schools, targeting a few individuals in each of 120 schools, with the same effort to roster, mail and follow-up as noted above. We will contribute to technical reports and begin publishing measures on the project website. In years 4 and 5, a small team will work to add instruments and tools to the website, along with other dissemination activities.
This project will develop and field-test a mathematics-specific coaching protocol centered on the Mathematical Quality of Instruction (MQI) instrument, a Common Core-aligned observational tool. The coaching model will consist of three elements: a summer training workshop, personalized web-based coaching, and access to a library of resources to support coaching and instructional change. During the summer workshop, teachers will be introduced to the instrument, score high- and low-exemplar videos in small groups, and reflect on their own practice. Each teacher will then participate in 20 video feedback cycles with a personal MQI-trained coach. Teachers will record and upload a lesson which coaches will review, providing written feedback and choosing three short clips for discussion. The coach and teacher will then review the clips together, discuss feedback, set targeted goals for the next coaching cycle, and plan further review of exemplar clips from our video library of Common Core-aligned practices. We will work with 3rd through 8th grade mathematics teachers in the Appleton Area School District to implement and evaluate the coaching model. Because we believe the online format is limited in its capacity to address teacher content knowledge, we will exclude teachers with very low
mathematical knowledge for teaching, then randomly assign teachers to treatment and control conditions, stratifying on grade level. We will evaluate the effect of the coaching model in both the concurrent and post-intervention year, as past studies have found effects only after teachers have completed the full cycle of coaching (Allen et al. 2011; Campbell & Malkus, 2011). Our primary outcome measures will include teachers classroom performance as captured by the MQI, students perceptions of their teachers effectiveness, and student test scores.
In this exploratory study, we will develop and test professional development aligned with the Mathematical Quality of Instruction observational instrument. In the program, teachers will learn to score videotapes of instruction using the MQI framework first by completing online training and then through participating in a series of groups meetings with others at their school. These meetings will combine aspects of video clubs (Borko, Jacobs, Eiteljorg & Pittman, 2008; van Es and Sherin, 2006) and lesson analysis (Santagata & Angelici,
2010), using the MQI framework to view and score portions of lessons. A comparison group will enable us to understand the effects of the MQI on several measures, including teachers talk,
their analysis of videotapes, and reflections on their own lessons. Owing to cost constraints, we will leave the collection of data on instructional or student outcomes to a future study. However, we believe that the outcomes of interest noted above will provide adequate proxies to practice, in that they have been linked to practice or are precursors to the improvement of practice.
In addition, we will systematically vary the conditions under which the MQI-based professional development is delivered in order to gain knowledge about the best design for a scaled-up implementation. To do so, we will expose fifteen groups of fourth and fifth grade teachers to MQI-based lesson analysis. These fifteen groups will be randomly assigned into five conditions (three groups per condition), with each group engaging in weekly or bi-weekly lesson analysis using the MQI. However, the amount of facilitator intervention (less vs. more), the origin of the videotape (teachers own video vs. video from a library), and the delivery method (face-to-face vs. internet) will vary by condition.
AIR, in partnership with Harvard University, Clowder Associates and Measured Decisions Inc., is proposing to develop a rigorous impact study of a specialized mathematics PD intervention for grade 4 teachers designed to strengthen content knowledge and encourage the enactment of content knowledge in classroom instruction. The 42-month study, which is being designed in response to a request from the National Center for Education Evaluation (NCEE) at the Institute of Education Sciences (IES), is designed to estimate unbiased impacts of the PD intervention on teacher knowledge, pedagogy, and student achievement. The PD intervention includes the Intel Math program (80 hours), augmented by the Mathematics Learning Community (MLC) program (30 hours), and a half-time facilitator-coach per district. Key outcomes measures include an adapted form of the Mathematical Quality of Instruction (MQI), developed by Harvard University as well as teacher knowledge assessment items from the Mathematics Knowledge for Teaching (MKT) instrument. The experimental design uses teacher-level random assignment and will involve approximately 200 teachers in 6 districts. Primary data collection activities will occur during the 2013-14 school year.
In this research, the research team sheds light on twin problems facing efforts to improve teacher quality in mathematics. The first problem is theoretical: What mathematical knowledge do teachers need to effectively instruct children? Does teacher basic, advanced, or profession-specific knowledge (e.g, mathematical knowledge for teaching, or MKT) matter most to student outcomes? The second problem is more practical: Can a particular professional development program, Math Solutions, improve teachers mathematical knowledge for teaching, their instruction, and student outcomes? To address these twin problems, the researchers conduct a cluster randomized trial to examine the efficacy of Math Solutions on elementary teachers MKT, their instruction, and their students learning. We chose to embed the study of teacher MKT in an evaluation of professional development because it is not possible to randomly assign teachers to different levels of MKT. Instead, the tearchers are randomly assigned to Math Solutions with the goal of examining the contribution of changes in teachers MKT, as it develops in and through the professional development, to instruction and student achievement. Data collection takes place in the Albuquerque Public Schools over three years and involves 80 fourth and fifth grade teachers in approximately 12 schools. Half the eligible teachers are randomly assigned to the mathematics professional development and half receive coaching in another subject. Longitudinal data is collected on teachers mathematical knowledge, instruction, and their student gains, allowing us to examine the effect of Math Solutions on these outcomes.
Topic: Education Research and Development Centers: Teacher EffectivenessFocus: Mathematics Instruction, Grades 4 through 6Overview: The Harvard Graduate School of Education (HGSE) is well-placed to combine the disparate strands of research and accellerate the search for a valid, scalable measure of teacher effectiveness. The proposed center will have four primary goals: 1. Unify three disparate strands of research: value-added modelling, the direct study of math instruction, and non-instructional predictors of teacher effectiveness (e.g. assessments of teacher knowledge, student evaluations). The team of investigators includes leaders from all three areas.2. Develop an empirically-derived composite measure of teacher effectiveness, which efficiently combines estimated impacts on student achievement, ratings of pedagogical practice on both general and content-specific observation protocols, tests of teachers mathematical knowledge and student evaluations. 3. Operationalize the measures to ensure that they are usable in the field, with training videos for principals, test items for teacher assessments, student evaluation forms, etc. 4. Externally validate the estimated differences in teacher effectiveness by randomly assigning classroom rosters within teams of teachers working in the same grades and schools, comparing their students achievement gains. Partner Districts: District of Columbia and Charlotte-Mecklenburg Public SchoolsIdentification Study: Over two years, we will administer a student assessment in the fall and spring; collect 4 video observations per teacher in the study each year; rate those videos using both general and content-specific rubrics; assess teachers with the test of Mathematical Knowledge for Teaching; administer student evaluations. Using a novel empirical approach, we will form an Empirical bayes composite measure, efficiently combining all of the above in a manner which best predicts student achievement gains in a non-experimental setting.Validation Study: In a third year of data collection, we will randomly assign classrooms of students to teachers working in the same grades and schools. We will repeat the video collection and ask principals, using the training materials provided by the research team, to rate their teachers instruction. Using those videos, we will confirm that the principals ratings of instruction were consistent with the researcher ratings. We will also test whether the predicted differences in student achievement using the composite measures were borne out after random assignment. If not, we will use the student outcomes following random assignment to recalibrate the composite measure.
Hill, H.C. & Grossman, P. (2013). Learning from Teacher Evaluations: Challenges & Opportunities. Harvard Education Review 83, 371-384.,(2013)
Hill, H. C., Beisiegel, M., & Jacob, R. (2013). Professional Development Research Consensus, Crossroads, and Challenges. Educational Researcher, 42(9), 476-487.,(2013)
Hill, H.C. & Grossman, P. (2013). Learning from Teacher Evaluations: Challenges & Opportunities. Harvard Education Review 83, 371-384.,(2013)
Hill, H.C. (in press). The nature and effects of middle school mathematics teacher learning experiences. Teachers College Record.,(2012)
Hill, H.C., Umland, K. L. & Kapitula, L.R. (2011). A validity argument approach to evaluating value-added scores. American Educational Research Journal.,(2011)
Hill. H.C. (2010). The nature and predictors of elementary teachers Mathematical Knowledge for Teaching. Journal for Research in Mathematics Education, 41 (5), 513-545.,(2010)
Hill, H.C. & Shih, J. (2009) Research commentary: Examining the quality of statistical mathematics education research. Journal for Research in Mathematics Education.,(2009)
Hill, H.C. & Ball, D.L. (2009) The curious and crucial case of Mathematical Knowledge for Teaching. Phi Delta Kappan , 91, 68-71.,(2009)
Hill, H.C. (2009). Evaluating value-added models: A measurement perspective. Journal of Policy and Management, 28, 702-209.,(2009)
Delaney. S. F., Ball, D. L., Hill, H. C., Schilling, S.G., & Zopf, D. A. (2008). Adapting U.S. measures of Mathematical Knowledge for Teaching for use in Ireland. Under review at Journal of Mathematics Teacher Education,11, 171-197..,(2008)
Hill, H.C., Blunk, M. Charalambous, C., Lewis, J., Phelps, G. C. Sleep, L. & Ball, D.L. (2008). Mathematical Knowledge for Teaching and the Mathematical Quality of Instruction: An Exploratory Study. Cognition and Instruction, 26, 430-511.,(2008)
Hill, H.C., Ball, D.L. & Schilling, S.G. (2008) Unpacking Pedagogical Content Knowledge: Journal for Research in Mathematics Education,39, 372-400.,(2008)
Schilling, S.G. & Hill, H.C. (2007). Assessing Measures of Mathematical Knowledge for Teaching: A Validity Argument Approach. Measurement: Interdisciplinary Research and Perspectives (5), 2-3, 70-80.,(2007)
Hill, H.C., Ball, D.L., Blunk, M. Goffney, I.M. & Rowan, B. (2007). Validating the ecological assumption: The relationship of measure scores to classroom teaching and student learning. Measurement: Interdisciplinary Research and Perspectives (5), 2-3, 107-117.,(2007)
Hill, H.C., Dean, C. & Goffney, I.M. (2007). Assessing Elemental and Structural Validity: Data from Teachers, Non-teachers, and Mathematicians. Measurement: Interdisciplinary Research and Perspectives (5), 2-3, 81-92.,(2007)
Schilling, S.G., Blunk, M. & Hill, H.C. (2007). Test Validation and the MKT Measures: Generalizations and Conclusions. Measurement: Interdisciplinary Research and Perspectives (5), 2-3, 118-127.,(2007)
Hill, H.C., Ball, D.L., Sleep, L. & Lewis, J.M. (2007) Assessing Teachers Mathematical Knowledge: What Knowledge Matters and What Evidence Counts? In F. Lester (Ed.), Handbook for Research on Mathematics Education (2nd ed), p. 111-155. Charlotte, NC: Information Age Publishing.,(2007)
Hill, H.C. (2007). Mathematical knowledge of middle school teachers: Implications for the No Child Left Behind Policy initiative. Educational Evaluation and Policy Analysis (29), 95-114.,(2007)
Hill, H.C. (2007). Teachers Ongoing Learning: Evidence from Research and Practice. The Future of Children, 17, 111-128.,(2007)
Hill, H.C. & Lubienski, S.T. (2007) Teachers mathematics knowledge for teaching and school context: A study of California teachers. Educational Policy 21(5), 747-768.,(2007)
Learning Mathematics for Teaching. (2006). A Coding rubric for measuring the Quality of Mathematics in Instruction. Ann Arbor, MI: Authors.,(2006)
Hill, H.C. (2006) Language matters: How characteristics of languagecomplicate policy implementation. In M.I. Honig (Ed.), New directions in education policy implementation: Confronting complexity. Albany, NY: SUNY Press.,(2006)
Hill, H.C. (2005). Content across communities: Validating measures of elementary mathematics instruction. Educational Policy 19, 447-475.,(2005)
Hill, H.C., Rowan, B., & Ball, D.L. (2005) Effects of teachers' mathematical knowledge for teaching on student achievement. American Educational Research Journal 42, 371-406.,(2005)
Ball, D.L., Hill, H.C. & Bass, H. (2005) Knowing mathematics for teaching: Who knows mathematics well enough to teach third grade, and how can we decide? American Educator, Fall 2005, 14-22.,(2005)
Hill, H.C., Schilling, S.G., & Ball, D.L. (2004) Developing measures of teachers mathematics knowledge for teaching. Elementary School Journal 105, 11-30.,(2004)
Hill, H. C. (2004) Professional development standards and practices in elementary school mathematics. Elementary School Journal 104, 215-31.,(2004)
Hill, H. C. & Ball, D. L. (2004) Learning mathematics for teaching: Results from Californias Mathematics Professional Development Institutes. Journal of Research in Mathematics Education 35, 330-351.,(2004)
Hill, H.C. (2003) Understanding implementation: Street-level bureaucrats resources for reform. Journal of Public Administration Research and Theory 13, 265-282.,(2003)
Hill, H .C. (2001) Policy is not enough: Language and the interpretation of state standards. American Educational Research Journal 38, 289-320.,(2001)
Cohen, D.K. & Hill, H.C. (2001) Learning policy: When state education reform works. New Haven, CT: Yale University Press.,(2001)
Cohen D. K. & Hill, H.C. (2000) Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record 102, 296-345.,(2000)