Beyond the Literacy Debate

Last fall's release of the 2019 NAEP reading assessment — the so-called Nation's Report Card for literacy — kicked off a new national debate about the best way to teach children to read. With two out of three children struggling to learn to read, and a widening gap between the highest and lowest performing children, state and district leaders (along with the general public) are again questioning what actually works. In this episode, Professor James Kim discusses why learning to read is so challenging, and he describes results of a pilot study of his new curricular model, called MORE, which offers another way.

TRANSCRIPT

Jill Anderson: I'm Jill Anderson. This is the Harvard EdCast.

National reports this past fall showed the two out of every three children in America are struggling to learn to read. Harvard professor James Kim has studied in design literacy interventions for years, knowing that we can't keep teaching reading the same way. He spoke to me about what makes teaching children to read so challenging and shared some of how his newest model called More offers another way. First I asked Jimmy if he was surprised about the latest national assessment of educational progress results.

James Kim: I think in some ways I was not surprised and I'd say that the NAEP scores, you can take the glass is half full or the glass is half empty and I think if you take more of a, the glass is half full perspective, I would call it stagnation, but what was really troubling is that disadvantaged kids were declining a bit more over time. And if you think about a state like Connecticut, which has very strong teacher induction systems, it generally performs very well. On the name across subjects, over the last five or 10 years, you've seen an increase of disadvantaged kids, kids from language minority families, low income children, students of color, and it's doing about as well as it did in the past. So you could say that's good news for Connecticut and a lot of other States.

I think the glass half empty perspective is that if disadvantaged kids are the ones who are falling behind, maybe it's something about teaching and learning curriculum and instruction and we're not adhering to what works. And I think that's what was alarming about these results is that people started wondering if what's happening in the classroom day-to-day is broken and that's why these kids are falling behind. I think that it's important to remember that NAEP is a thermometer, it just tells us what the temperature in the room is. It tells us nothing about the why. And so we should have a bit of humility when it comes to unique results in what we can infer.

Jill Anderson: There's a lot of speculation about how we're teaching kids to read a lot of judgment being passed about it and this question, how can it be 2020 and we still are struggling to teach kids to read?

James Kim: The first important thing about NAEP is that it's given in fourth grade, so it doesn't tell us much about the first couple of years. The other thing about NAEP that's important is how it defines reading. So if you asked someone was reading, some people will say, well it's the ability to read the words and to understand what's explicitly stated in the text. And we might say that's a very kind of narrow, simple view of reading. But other people might say that reading is being able to do that, but also connecting it to things you already know to learn something new. So one thing that is really hard about reading is that there's a distinction between memory and learning.

We actually do a reasonably good job I would say, looking at national data, for example, from the early childhood longitudinal survey, that's also a nationally representative survey, but we have information from kindergarten to third grade on helping kids read the words, make sense of what's literally stated and remembering enough information to answer a question that's directly in the text. But when it comes to learning where kids have to connect what they read to background knowledge, we really struggle. And I'll give you an example from NAEP. Should I quiz you Jill? Here's a classic item from the fourth grade NAEP. The growth of the blue crab larva into a full grown blue crop is most like a human baby into a teenager, an egg into a chicken, or the development of a tadpole into a frog? Well, it turns out that the passage that the kids read about blue crabs does not tell you the answer to that question. You have to know something about the life cycle of different organisms. You have to know something about metamorphosis.

And so the challenge really becomes building kids domain knowledge in disciplines like science and history so that they can not only read what the text says and faithfully recall what is literally stated in the text, but they have to connect that to background knowledge and be able to learn from texts. And that's what's so hard about reading. One thing that the NAEP board does is it thinks about the kinds of texts that kids should read, so if kids read narrative, informational text, but it also has something called cognitive targets. So some of the questions, some of the items are classified into this bucket called locate and retell. A second bucket is called integrate and interpret. And a third bucket is called critique and evaluate. And where we really struggle as a nation is in the second bucket and the third bucket, particularly the critique and evaluate. And that's kind of like the blue crab larva question.

It's where we're asking kids to use some prior domain knowledge to evaluate the text, where the knowledge needed to answer the question is found nowhere in the text. And so I think that's one of the things that makes reading so hard.

Jill Anderson: So it sounds like a lot of reading models don't include that domain knowledge as part of learning how to read.

James Kim: So you asked me the question that all researchers love to ask and that of course Jill is a question about theory. And theories, it's just a big word for some idea that explains a lot of different specific instances. And one view of reading that is widely known in the research world and now it's becoming increasingly known in the policy world is just called the simple view of reading. And that's the idea that reading comprehension is a product of decoding and language, so once kids have read the word, they have to understand what the word is in order to comprehend text.

And the interesting thing about the simple view of reading is that when the NAEP scores came out, one of the headline findings is that Mississippi was doing really well and had experienced large gains over time in three or four subjects it saw these gains and most importantly for equity purposes, all subgroups of kids, particularly disadvantaged kids made significant gains. And people pointed out, well Mississippi follows this science of reading, they follow the simple view of reading. They focus on practices to teach the code and also to build language, but it doesn't, as I just explained, have a role for domain knowledge. And I would say the theory that really guides a lot of researchers thinking about reading comprehension is, it's got a long name, but I'm going to go ahead and talk about it because it's what guides my own way of thinking about reading comprehension.

It's called construction integration. It's by a very renowned psychologist, Walter Kinch. And the idea is really simple. I mean if you think about those two words, the idea is that there's a construction phase where kids have to literally understand what the words and propositions mean and then there's an integration phase where they have to integrate that with domain knowledge to understand what's in the text. That process of constructing and integrating prior knowledge with what's in the text becomes especially important when the text itself becomes more complex. If the text is just a simple narrative that's very familiar to kids, it's less important to know something about blue crabs and lots of other topics. But once kids start encountering disciplinary knowledge, which I would argue should happen as soon as possible all the way in pre-K, they need to know something about the domain. And the proxy for domain knowledge is the words. The words tell you something about whether kids know something about the domain that they're studying.

Jill Anderson: So I want to hear more about your model of reading engagement more, we'll call it that and how it differs from some of the more traditional models that exist.

James Kim: Yeah. Thanks for asking Jill, so MORE stands for model of reading engagement. I'll say three things to talk you through the chronology. The principle determinant of how well anyone is going to understand the text is how much they already know about the text. So if I'm going to give you a passage still about any given topic, what's really going to drive how well you understand that passage is how much you already know. And that idea comes from schema theory. This was a very important theory back in the 1980s that readers have to have a schema that enables them to understand any given topic. It's kind of like a network where ideas are related to each other. So when we develop more, we worked out of this idea that it was very important to build a schema so that kids could visualize that schema and leverage that schema to better understand complex text. And the schema that we happened to choose with Arctic ecosystems because it's a very important schema and concept in the next generation science standards and generally is engaging.

So I think the first thing we thought a lot about was how could we build schemas that kids could leverage to understand complex text. The second thing that affected our influence more was the common core state standards. Although many people say that the common core is dying or is dead, they're quite influential. And I often joke with my colleagues and we'll ask the question, what's the most important page on the common core state standards? And in my humble opinion, it's page 33. What page 33 shows, it's a visual, it's a figure, and it says in order to build domain knowledge that helps kids reading comprehension, schools have to stay on topic within and across grades. And it shows a little visual, a little map where if the topic is a human body, in kindergarten you would choose books about the five senses, you would spiral upward and by fifth grade you're talking about the endocrine system.

We don't do that in U.S. public schools, because it's very hard for systems to be that coherent and coordinated. So as I thought about the challenge of building domain knowledge, we wanted to build a model of reading engagements so that it would be a spiral curriculum. Where in first grade kids would learn more basic foundational concepts and then it would spiral upward to more complex concepts. That was the second reason we built it. And then the third thing that we were thinking about, is you've probably heard of balanced literacy, it's a term that's lightly used. It's not always clear what that means. I'd like to think of balanced literacy as a great attempt by many reformers to make sure that there was a balance of reading and writing, make sure there's a balance of focusing on words as well as connected text.

But one thing that's important about balanced reading is I don't think it systematically builds domain knowledge because it privileges texts that are at kids levels and a wide variety of texts, narrative and informational. So, that was typical practice in the district that we were working in. I would argue that that was, and it still is typical practice in many US school districts. And so we wanted to develop a curriculum that moved as far away as possible from typical practice and that's how we developed MORE. And put simply what we did with MORE is, we said, "Let's curate a set of text around the topic of ecosystems or whatever science or history concepts you're studying and let's make those conceptually coherent, engaging, and challenging. And then let's choose a set of words that are related so that when kids have multiple exposures to those words, they're actually learning about the bigger concept of ecosystems." And that's how we built it. There's a lot of other things we do. We integrate reading and writing. There's an emphasis on engaging in motivation. Kids participated in read alouds, they have peer led research groups. So I guess the best way I could explain it to Jill is it feels a lot like PhD programs. Right? And we did that very intentionally.

Jill Anderson: Kids don't notice any difference, obviously.

James Kim: No. And the thing that was interesting is we did a study with first graders and the texts in MORE were much more challenging, much more difficult on many dimensions, both quantitative and qualitative indicators of text difficulty, but the kids actually did better. They did better on reading and writing outcomes, vocabulary and they liked it a little bit more. And so we felt like this was good news for the initial test. And I would say the final thing that was very important is the program was delivered by regular classroom teachers and not researchers, so there's a lot of what we call face validity in the actual study. And we thought that was good news, but of course you always want to replicate innovative findings and so that's what we're doing now.

Jill Anderson: Right. Can you give me an example of what MORE would look like compared to a more traditional curriculum?

James Kim: Let me give you a couple of examples. If you walk into a MORE classroom, you'll see fewer texts, but they're all related to the same topic. So going back to the study we just completed in first grade, all the books cohere around Arctic ecosystems, so how animals survive in extreme habitats. We picked the Arctic because there are cute and cuddly polar bears and penguins and all kinds of other things. So the first thing that you'll see is when you walk into a MORE classroom, you don't actually see bins of books. You don't see a hundred books, level a to Z, you see five or six books. The other thing that you'll see in MORE is an emphasis on concept mapping. So we start with the premise that knowledge is networked and what do I mean by that? You know, Jill, if you and I know what the word sale means, mast, star board, it's quite likely that we know something about the architecture of sailboats.

So words are the tip of the conceptual knowledge iceberg. They are proxies for something deeper. So what we do is we organize these texts and then we find words that are conceptually related. So in the first grade study, those words were not surprisingly, ecosystem, habitat, prey, predator. There's a set of words that are all related and we focus on words in these meaningful texts through read alouds, then through kids picking research topics of animals in the Arctic ecosystem that they're interested in. And then kids learn how to do argumentative writing, which is a type of activity that again is rarely observed in first grade classrooms. When I said that more in first grade, it's like PhD programs. We focus on argumentative writing because in argumentative writing you need to have discourse and domain knowledge. Argumentative structures have a particular type of discourse structure. There is a claim, there is evidence, there's closing arguments, there's attention to counter arguments, but you also need to know something about the domain.

You need to know something about Arctic ecosystems to make an argument. So, that's also important. And the last thing that I'd say that's different about MORE is in MORE we really focus on the scientific question of transfer. And by transfer, I mean if kids are taught about the Arctic ecosystem, they learn the schema, they learn the words that are related to that schema, they were exposed to all these books, they heard read alouds, they did writing, argumentative writing and research groups. Can they build the kind of domain knowledge that helps them understand something about the rainforest ecosystem, which was not studied? And in fact, in our study we found that there was evidence of transfer because one of the tasks was about the reinforced ecosystem. Neither the treatment or control group was exposed to that topic, but the treatment group, the MORE kids did better. And I think going back to your question about NAEP, the fundamental scientific and practical challenge we face in American literacy and education is understanding how much instruction fosters transfer.

And I would make the argument that NAEP is the farthest kind of transfer. It's so removed from the day-to-day curriculum that's enacted in the 9,000 plus US school districts. So one thing that we're trying to figure out at MORE is, if we do the things that I just described, can it promote MORE transfer? Not just knowledge of the topics that kids study, but passages that require knowledge of that schema. How far will the transfer go? And that's a very important scientific aim that we have in this project. And that's very different I would argue, from typical instruction where you'll have end of unit assessments, but they're very tightly linked to what the kids were studied. So we don't know how much transfer occurred. Those are the things that are, I would say different, somewhat unique about the MORE curriculum.

Jill Anderson: So you just completed this study and it showed some positive effects on the children, but it sounds like it's still in terms of how it affected reading engagement is still, there wasn't a huge-

James Kim: That's exactly right. In this study we wanted to open up the block box and really figure out what are the leavers in the block box that are moving, that ultimately lead to improved reading comprehension. And so we had several measures of domain knowledge, how well the kids understood the words, whether they could listen to a very difficult passage with those words and understand what it meant and how well they could write when we gave them an argument of writing task. We saw strong improvements on all of those measures relative to kids in the typical instruction condition which use the balanced literacy materials and curriculum. When we looked at engagement, which we measured using what we call situational engagement, right after the activity, did the kids find the activity interesting, their motivations, the extent to which they felt good about themselves as readers, if they valued reading and how much they liked the read aloud. We actually found no differences.

Just like you said, the MORE kids do as well as the typical instruction kids. My hypothesis there is that most first graders are pretty engaged. I don't know if you've been around first graders but they're not like you know seventh graders. I taught seventh grade where there's much more variation in motivation. And some scholars have called first grade the golden era of motivation because most first graders are eager to please their teachers are eager to learn and they really get into anything that's kind of interesting. So we kind of saw the what we call ceiling effects, all the kids were pretty high and there wasn't a lot of movement. It'll be interesting to see though in our follow up large scale replications if we do move the needle on dimensions of motivation and engagement. It's an important part of MORE, we just need to wait to see if there's an impact with the older kids.

Jill Anderson: You had actually teachers implementing this in their classroom on researchers, which you know may be more typical for this type of study and I wonder what feedback you got from teachers on this.

James Kim: We always get systematic feedback from teachers. Teachers take a survey, we talk to teachers, we do debriefs and one thing that was very interesting about the first grade study is that the teacher's reaction to MORE was very mixed. I would say that some teachers really liked it, they thought the kids were quite engaged. Other teachers thought it was so different, so radically different from what they were used to that it wasn't part of their grammar of literacy. A lot of the things that I was sharing with you about schema theory, domain knowledge, construction integration models and the importance of leveraging your prior background knowledge of disciplinary topics to better understand complex texts. It was all very new. And I think the lesson there is whenever you try something new, you're not going to get buy-in from everyone at the beginning. I think what you want to make sure you do is that teachers feel that they are your colleagues and collaborators, rather than the subjects and participants in your research.

So it was more important to me that I communicated as clearly as possible to teachers that they're just co-laborers with us, but we need to do something different. If we look at NAEP fourth-grade scores, the large gaps in achievement in literacy between children who have historically been served well and children who are not served well, we can't do the same thing for the next 30 years and I think all the teachers are resonated with that message. I would say now that we're doing followup studies, the reaction to MORE is still somewhat variable, but when teachers see not the kids' test scores or the outcomes, but when they see what kids are like in the moment they're delivering the instruction and they see greater engagement, that's what really causes teachers to embrace the program. And we'll see over time what teacher perceptions are and what the results are, but I think teachers are an absolutely critical part of our research endeavor.

Jill Anderson: What does this mean for literacy instruction going forward? You just mentioned that what we're doing obviously isn't really working, especially for disadvantaged kids.

James Kim: I really liked the question you asked earlier, Jill, about 20 years after the national reading panel, 20 years after the national research council report preventing reading difficulties in young children, why are we still struggling so much as a nation? I gave you kind of a class half-full, half-empty perspective, but I'll just make two final points. I think going back to my theme of the importance of building domain knowledge, it's not just important in terms of theories of reading, it's not just important for doing better on NAEP or the third grade integrated test in Florida and California, it's an equity issue. What we know from many surveys is that children in high poverty schools, children in disadvantaged schools get a very restricted curriculum in America. They do not get opportunities to learn science and history content because there is such an overemphasis on basic literacy and math skills.

I'm not opposed to that, I'm just saying we need to give all children an opportunity to learn and acquire important domain knowledge that is fundamental to being a citizen of the United States and participating in our economy and having knowledge and skills that enable all children to be ready for college and a great career. So I think one challenge that we have is trying to get the field school districts states to ensure that all kids have more opportunities to acquire domain knowledge in science and history and that we not cut out those subjects. The second thing that I'll say is you've probably heard of the reading wars Jill. You probably are aware that in America today we're having a big debate about the science of reading. What is the science of reading? Why don't some people follow the science of reading? Why do other folks reject the science of reading?

One of the things that is important about the science of reading is understanding what's unique about the English writing system. The English writing system is not phonetic. You know that there are many exceptions to these rules. The English writing system as we literacy scholars like to say, it's more full phonemic and all we mean by that is that the thing that's so fascinating and amazing about English is that the smallest units of meaning communicate meaning. And what English prioritizes is the regular spelling of these small units of meaning, which we call morphemes over the phonemes, which are the smallest units of sound. So I think when we talk about the science of reading and the reading wars and phonics, or just whole language, that's kind of a distraction. And what we have to do is to help teachers, practitioners, parents understand that when you teach reading, and by that I mean focus much more on words, you have to lead with phonemes and morphine's.

Jill Anderson: Professor James Kim has an expert on literacy intervention and experimental design at the Harvard Graduate School of Education, where he leads the READS lab, a research based collaborative initiative to identify and scale adaptive solutions for improving children's literacy. I'm Jill Anderson. This is the Harvard EdCast produced by the Harvard Graduate School of Education. Thanks so much for listening and please subscribe wherever you like to get your podcasts.

About the Harvard EdCast

In the complex world of education, we keep the focus simple: what makes a difference for learners, educators, parents, and our communities.

The Harvard EdCast is a weekly podcast about the ideas that shape education, from early learning through college and career. We talk to teachers, researchers, policymakers, and leaders of schools and systems in the US and around the world — looking for positive approaches to the challenges and inequties in education. One of the driving questions we explore: How can the transformative power of education reach every learner? Through authentic conversation, we work to lower the barriers of education’s complexities so that everyone can understand.