Modeling Change and Event Occurrence

An Interview with Conant Professor Judith Singer and Eliot Professor John Willett

Applied statisticians and long-time research partners Conant Professor Judith Singer and Eliot Professor John Willett recently published Applied Longitudinal Data Analysis, which offers an integrated treatment of individual growth modeling and survival analysis. (They are poised to begin writing a companion volume that discusses the principles of longitudinal research design.) In this interview, they discuss some of the powerful methods for monitoring, mapping, and assessing human progress offered by the analysis of longitudinal data.

Q: Your new book describes innovative statistical methods for analyzing longitudinal data in education and the social sciences. Why did you choose to write a book about this?

A: In life, everything that is truly important is longitudinal"”simply as a consequence of the human condition. Children develop and learn. Organizations change as they are restructured and new policies are implemented. Adults pass through important transitions in their careers, their relationships, and their lives. The list is endless. Without powerful methods for analyzing longitudinal data, for addressing questions about change and duration, we couldn't monitor, map, and assess human progress.

Q: What particular methods are described and developed in the book?

A: In the book, we focus on new methodological advances in two very important areas: the analysis of change over time and the analysis of duration. The two approaches that we describe"”individual growth modeling and hazard modeling"”are methodologically complementary and provide ways for the researcher to address critical questions about "how things are changing" and "how long it takes." While we are being a little glib here, in the way that we are framing them, these are truly the most central questions that anyone can ask of the human condition, and they reoccur time and time again across all the major areas of substantive scholarship from education, to economics, to psychology, to sociology, to psychiatry and medicine, and beyond.

"Without powerful methods for analyzing longitudinal data, for addressing questions about change and duration, we couldn't monitor, map, and assess human progress."

Q: Methodologists have been working on these methods for a long time. What ds your work contribute to the ongoing conversations?

A: The history of quantitative research in the social sciences is littered with flawed approaches to measuring change and duration. For most of this last century, prominent methodologists actually argued strongly that change over time could never be measured well, and so they advised researchers to "ask their questions in other ways." Here are these critical substantive questions about the most important social and human processes that exist anywhere, and leading methodologists are saying "Hey, guys, forget that. Ask your questions in other ways!"

And then there's the research on duration, which was traditionally hamstrung by the presence of a problem called censoring. When you are observing a bunch of folk live their lives over time, and you are anticipating some important event like the onset of a developmental stage or a career transition, you often miss seeing the event occur for some of the folk in your sample because your data collection ends before they actually experience the event. This dsn't mean that they won't experience the event eventually; it just means that you won't record it in your dataset. So, how can you estimate the true average time to an event, and believe what you find, if you know you've missed out on observing the complete experiences of the most "long-lived" folk in your sample? It's tough. For the last twenty years, along with colleagues all around the world, we have been working on methods for solving these problems and have contributed to creative advances in both arenas. So, our book draws these advances together in an attempt to help empirical researchers handle their longitudinal data sensibly and well, given a well-documented prior history of disaster.

Q: What kinds of nuances are revealed through the use of these new methodologies?

A: It's hard to count the ways, but we are constantly coming across ways in which these new methods have led to insight and understanding after decades of confusion and bias. Some of the early work on the measurement of change"”which only tested participants at the beginning and end of a period of time"”led methodologists to many incorrect assumptions, including that change measurement is always unreliable. Now that we have these new methods, we know that none of these claims are true. Application of the new methods has also lead to more reasonable estimates of all kinds of important quantities from children's rates of achievement, to the duration of the doctoral career at HGSE, to realistic assessments of the positive impact of the GED on entry into the workforce, the duration of the teaching career, and so on.

Q: How might these methods have an important impact on educational research"”for example, in looking at student performance"”in a way that's not being done now?

A: One big issue in education today is high-stakes testing. Children, teachers, and schools are being evaluated on the basis of one-shot assessments of student achievement on a single occasion, rather than on the trajectories of their learning and progress over time. We think that this represents gross neglect, and we compliment those states and policymakers who are trying to collect longitudinal data and base educational policy on a more careful assessment of the trajectories of children's intellectual progress over their school careers.

"We must examine educational and social processes with all the different lenses at our disposal, both quantitative and qualitative, both experimental and observational."

Clearly, if you want to make decisions about quality of teaching and the impact of school programs, you must examine not what children know on one occasion, like today, but how their knowledge and skills change over the course of their entire exposure to the teacher or the program. You cannot do this without individual growth modeling. If you want to understand critical educational and life events"”like school graduation and dropout, entry into college or postsecondary training, matriculation with a degree and entry into the workforce, and so on"”you must use hazard modeling. If you do not use the right methods to analyze your data, then your intuitions and decisions may very well be completely wrong.

Q: You drew from a number of different disciplines to write the book. In your opinion, what can education researchers learn about methodology from researchers in other disciplines?

A: Nothing in science occurs in isolation; that's the beauty of our open system of research and scholarly publication. Advances are constantly made in one arena only to see ready and effective application in another. Part of being successful as a scholar is in recognizing powerful strategies that can be modified and brought from one substantive area to another"”with appropriate credit being given, of course. For instance, a lot of the early work on hazard modeling was done in medicine, where it was designed as a method to model "time-to-death" for patients with terminal diseases, such as cancer.

That's why the method is often referred to as "survival analysis," and why it is darkened by terms like "hazard," "risk," and "median lifetime." One of our key insights, about 20 years ago, was in realizing that the method wasn't useful simply for analyzing the event of physical death, but that it could be applied effectively to all manner of important events such as transitions betweens psychological states, important career transitions, even events at the "micro" level, like conversational turn-taking. The methods of individual growth modeling had their genesis in the biological study of human and animal growth. One of the early innovative statistical papers on growth modeling was by Wishart and was called "On the Growth of the Bacon Pig." It's just a small step from modeling crop yields in agriculture to the modeling of other kinds of important growth, both intellectual as well as physical.

Q: Do you think that the new methods that you describe hold the key to making major advances in educational and social research?

A: The methods in our book are powerful, rich, and flexible, and they will certainly support many important applications in educational and social research over the coming decades. New policies will undoubtedly be implemented that could never have been devised if these methods were not available.

That being said, education is a very complex enterprise. No single tool can ever successfully address all the questions that must be answered. We believe that there is a place in educational and social research for a viable and active partnership of methods. If we are going to ensure our progress as a civil society and as a species, we must examine educational and social processes with all the different lenses at our disposal, both quantitative and qualitative, both experimental and observational. Otherwise, we will never be able to successfully negotiate these reefs. We comment on this kind of thing in our next book, which should be out in a couple of years and, in which, we describe innovations in the design of longitudinal research that arise as a consequence of the new statistical methods that we have described here.