The Question of College Completion Rates

For the past few decades, college completion rates have slowly been going up. Economist Jeff Denning wanted to know why. As part of recent research, Why are College Completion Rates Increasing, Denning and co-researchers dug deep to understand this phenomenon. Denning, a Brigham Young University associate professor, says the reason isn’t what you might suspect like an increase in college enrollments or that students are studying more. Instead, his research shows that grade inflation is driving the change.

"[Whether it is] good or bad to inflate grades, I think is an open question," says Denning. "We show one side of this, which is: If you had told me before I wrote this paper, 'We have this policy lever, it's going to increase graduation [rates] and it's not going to cost any money, basically,' I'd be pretty interested in that policy lever." 

In this episode, Denning provides insight into how he conducted this research and also what this might mean for higher education. 

TRANSCRIPT:

Jill Anderson: I'm Jill Anderson. This is the Harvard EdCast. Economist Jeff Denning set out to better understand why college completion rates are increasing. What he didn't expect to discover is the reason why. He says it's not an increase in enrollments or even student characteristics driving this increase, but actually great inflation.

As part of his research, he looked at many different factors to attribute this change. I spoke to Jeff and wanted to know what this might mean and why he started looking at college completion rates in the first place.

Jeff Denning: There was a very famous paper, My Circles by [Don Livine Hymantan 00:00:50] that he showed from the 1970s to the 1990s college completion rates declined. That is if you went to college, how often did you complete? And that actually was declining from the seventies to the nineties.

And so that was a fact that I knew and that other researchers knew. And then I saw in more recent data, I saw that college completion rates had been increasing. Starting in about the nineties, they started increasing and suddenly over that timeframe. And so we said, "Oh, that's interesting, that's a new fact. And let's see if we can explain it."

One thing I would say is college graduation rates are increasingly show, but they're still low. By the end of our period they're almost 60%. So it's not like everyone who goes to college completes, it's just slightly more now are completing than they used to.

Jill Anderson: It's something like about a 7% increases what you saw?

Jeff Denning: That's right. So seven percentage point increase relative to a baseline of 52 percentage points. So a reasonably sized increase in graduation rates.

Jill Anderson: When you really read this and you go through it, it's shocking how much data you looked at to come to your conclusion, which was that there's some grade inflation potentially happening.

So why don't you talk to me a little bit about how you did the research came to this realization that grade inflation is part of this?

Jeff Denning: Yeah, so we started with the fact completion rates are up. Why is that? Our starting place was to use the methodology of this earlier paper and consider factors that they considered. Could those explain it statistically?

So they come away saying one of the reasons that completion rates declined is that students were going to different colleges that had lower completion rates and that there was less support for higher education as measured by student faculty ratios.

So we looked at those trends and other trends and it turns out none of those trends would predict increasing graduation rates. So students are going to the same types of colleges as they were basically, they didn't change dramatically the distribution of colleges. Colleges Weren't getting more resources per student or anything like that. It had about the same amount of resources per student, maybe a little worse, but not markedly different.

We use evidence from other papers that show students are more likely to be employed for wages, not working at school, but working to make money. So that would make you think that that's not a good thing for graduation necessarily. Prices are up, tuition is up over this time. I think that might affect it. That's going kind of the wrong way as well. Students are studying less over time.

There's another paper that shows that. We started looking at all these different trends and we're like, "Oh, these are going the wrong way. They're not going to help us explain why college completion rates are increasing." But we noticed that grades were up. So first year grades were up by a fair amount. And so we're like, "Oh, well, grades are correlated with whether or not you finish college."

And so why are grades up? That might be our story. And so then we said, "Okay, why are grades up? Well, our students more prepared for college." And so we took data from some national surveys. We took data from several colleges and we saw after we used statistical methods to see comparing people who were at a similar level of preparation over time, that is that they have the same SAT score for instance, did they have higher grades in the later years? And they did. So we couldn't explain the increase in GPA by increases in preparation.

Like I said, students were studying less, so that's not explaining it. My favorite piece of evidence comes from a single university, which we are allowed to call Public Liberal Arts College. At this Public Liberal Arts College. We have data on a lot of things, but importantly, we have your grade in the class, as well as your final exam score.

And we showed that people's grades are going up over time, even after we compare people with the same final exam score. So two people in subsequent years, they have the same exact score on the final exam, but one just comes later. The person that comes later to that school, we would predict, have a higher GPA in that class.

And even further we found for some classes, they offered the exact same tests over time. And so we could compare people who got the exact same score on the exact same test and see, did they have higher grades if they were coming at the later part of the sample?

And the answer is, yes. So over time people were having higher grades, even though they presumably learned the same amount of stuff, they had the same score on the final exam. So I think that is our strongest evidence for great inflation at that particular college, but the trends in graduation and GPA at that college mirror what we see and more nationally representative data as well.

Jill Anderson: So people are going to hear this and they're going to say, "Does this mean college has gotten easier?"

Jeff Denning: Yeah. That's a great question. Our answer would be, it seems like it a little bit. Our next question is, is that a bad thing or a good thing? We set how hard college was. I don't know when in the seventies, fifties, whatever. And the question is, is that the right level of difficulty. Our paper is one that suggests like, well, there are benefits to making college a little bit easier. More people graduate, college graduation is a good thing.

It helps people in a lot of ways. It helps them earn higher wages, et cetera. And so this seems to be a benefit. There may be costs as well, but this seems to be a benefit of making it easier for students to graduate.

Jill Anderson: What do you think that this might say in the larger discussion about the value of college? Does it change it in any way?

Jeff Denning: People often ask, this is a great next question, right? Like, "Oh, so a college degree, somehow devalued." Not our paper, but other papers have shown that the college wage premium, that is how much more you make if you have a college degree compared to someone who doesn't, it's very high and has stayed pretty constant over the last 10 or 20 years, I don't remember exactly how long, but for the last little while it's very high and it's not shrinking, it's not getting worse.

So people who go to college have higher wages, and there's a lot of good evidence to suggest that college has a causal effect on wages. It increases wages. So I think the returns to college are very strong, certainly on average. You're right to wonder, you can imagine if college changed dramatically or got way, way easier, then you might be worried about it. That doesn't seem to have played out, at least in aggregate trends that we see in the data.

Jill Anderson: What about higher ed leaders and people working in higher education? How can they take this research and use it, or think about what their goals are for students?

Jeff Denning: I think the first thing that we would say is that grades are a policy tool. That this is something that they can use to affect whether or not students graduate. It can affect other things too, like the incentive to study and that sort of thing can be affected by grades. So you want to think about grades, not as just kind of arriving like the dew or something, but something that you can actually affect and would have consequences for your students. So that's the first thing.

Is it good or bad to inflate grades, I think is an open question. We show one side of this, which is: If you had told me before I wrote this paper, "We have this policy lever, it's going to increase graduation [rates] and it's not going to cost any money, basically," I'd be pretty interested in that policy lever.

And that's basically what we found. Now, you might think there are trade offs in terms of student effort or signals to employers or that sort of thing. Our paper, I hope is one in a series of papers, that maybe other people will write, or that we'll write that say, "What's the right level here. What's the right level of grades that we should be giving out."

Jill Anderson: I don't know if there's any explanation you can give in practice. Do faculty members even realize that they're inflating the grade or is it just something that happens?

Jeff Denning: People mean different things by great inflation, but what we mean by great inflation is: Are grades higher for the same level of understanding or something like that over time, right? Could people talk about inflation across disciplines or something? We're talking about over time. So that's how we define it. Are grades higher than they were before, conditional on the same amount of knowledge or competency?

Now, why is this happening? I think is a really interesting question. When people ask me this, I usually answer it with a question is, "Why didn't it happen earlier?" Who was the constituency for tougher grades? As a professor I can tell you, it's not my students. They're not asking me to grade harsher, college administrators often are, they're interested in increasing graduation rates. And so they're I think skeptical of giving out really low grades.

So they're not really rooting for low grades often. Teachers who give out lower grades are likely to get lower student evaluations, which are important for decisions about promotion and that sort of thing.

So, there's not really an obvious constituency for lower grades within the university. So there are also other speculative reasons. I think there's some evidence to suggest adjuncts tend to give out higher grades than tenure track faculty.

There's a lot more information about professors and classes online than there used to be so you can perhaps there's incentives to find the teacher who will give higher grades or that sort of thing, or pressures to compete that way. We don't really know, but there doesn't seem to be a lot of incentives for lower grades that I can see in higher education.

Jill Anderson: Well, thank you so much Jeff.

Jeff Denning: Thanks for having me.

Jill Anderson: Jeff Denning is an associate professor at Brigham Young University. He's the lead researcher of, why have college completion rates increased, to be published in the American Economic Journal. I'm Jill Anderson. This is the Harvard EdCast produced by the Harvard Graduate School of Education. Thanks for listening.