New Research Finds That Online Courses Significantly Reduce Student Achievement

Having already written a defense of the lecture this week, I’m going to take the risk of seeming like a complete curmudgeon and share these findings about online education, from a new study coming out of Stanford University’s Center for Education Policy Analysis (CEPA): (H/T L.D. Burnett)

CEPA Logo…online courses do less to promote student learning and progression than do in-person courses for students at the margin. Taking a course online reduces student achievement by about one-quarter to one-third of a standard deviation, as measured by course grades, and reduces the probability of remaining enrolled by three to ten percentage points (over a base of 68 percent). Taking a course online also reduces student grade point average in the next term by more than one-tenth of a standard deviation. Additionally, we find that student achievement outcomes are more variable in online classes. The variation of course grades increases by as much as one-fifth in online sections, and the variance in student grade point average in the next term increases by more than one-tenth. By contrast, we find that the variation in professors’ contributions to student achievement and persistence are smaller in online classes than in-person classes.

Authors Eric Bettinger, Lindsay Fox, and Susanna Loeb of Stanford and Eric Taylor of Harvard studied online and face-to-face (F2F) courses at DeVry University, where about one-third of courses are taught F2F and two-thirds are online. Why DeVry?

DeVry University logoDeVry University’s approach to online education creates an intuitive, clear counterfactual. Each DeVry course is offered both online and in-person, and each student enrolls in either an online section or an in-person section. Online and in-person sections are identical in most ways: both follow the same syllabus and use the same textbook; class sizes are the same; both use the same assignments, quizzes, tests, and grading rubrics. The contrast between online and in-person sections is primarily the mode of communication. In online sections all interaction—lecturing, class discussion, group projects—occurs in online discussion boards, and much of the professor’s “lecturing” role is replaced with standardized videos. In online sections participation is often asynchronous while in-person sections meet on campus at scheduled times.

I suppose that last factor might lead online advocates to push back, or at least ask the effect of synchronous discussion on online learning. The authors acknowledge that “online asynchronous interactions change the implicit constraints and expectations on academic conversations….students may feel less oversight from their professors and less concern about their responsibilities to their professors or classmates in the online setting.” (My own limited experience has made me quite dissatisfied with asynchronous discussion and curious to explore alternatives.)

Still, this research seems like a significant challenge to the “no significant difference” claims regularly offered by advocates of online education. And it only adds to the mounting concern about the quality and integrity of for-profit schools like DeVry.

You can read the full paper here. Or watch Bettinger’s presentation below:

8 thoughts on “New Research Finds That Online Courses Significantly Reduce Student Achievement

  1. These studies come up all the time and fall short. I am not surprised that DeVry was used. Its online instructors are usually inexperienced. What other studies have revealed concerning online teaching is that the teacher matters more than anything. One of the ideas behind online teaching was that it would remove the instructor from the content delivery and make them a non-factor so that education would be delivered on a more even keel. Instead, the quality of the instructor was even more important than ever.

    Also, DeVry doesn’t attract the top tier of students that go for online learning any more if it ever did. If anything its student body is in the lower tier. This study suggests a problem with DeVry, not online education at large. There are too many studies out there that show the no significant difference results are quite real. This study and the fight over lectures show there are different lines of thought regarding higher education in multiple facets. The sides are going to put themselves in the best possible light.

    1. Well, yes: I think this probably says more about for-profit than online. But two points. (1) They’re comparing apples to apples: the same instructors teach the same courses both F2F and online; even assuming you’re right that the instructors aren’t great, shouldn’t the results then be similar in both modes? (2) What’s most troubling is that the students most affected seem to be those at the margins, precisely the group that online (because of its flexibility and relatively low cost) supposedly helps.

      1. Online education is not for everyone. This is a concern that gets overlooked in the conversation. Many students at DeVry really need help with their English areas of the curriculum and they are in an online course without getting the developmental help. This is not unique to DeVry either. I’ve brought this up in my own school. There are too many dollars out there that are driving the push for online education to be used without considering the other needs involved.

  2. I teach most of my courses in multiple formats: traditional, online, and hybrid with intensives. I really like doing synchronous sessions with Adobe Connect. I present PowerPoint, interact via webcam, etc. I do one hour sessions. If students miss, they watch the recording and have to write 1/2 page (133 word) response. Recording and posting recording url is instantaneous, no uploading. I survey about time to meet with before semester. Here are two most recent ones:

    I also have papers be posts so students learn from each other. I require 2 replies (to other students’ posts) and include them in grade.

  3. Oooh, you’re hitting close to home for me! 🙂 – My concerns/questions with this study would be:

    1) “Identical” courses in Online & F2F – One of the dangers I’ve spotted in my line of work as an instructional designer for online & blended education is the common assumption that courses traditionally delivered face-to-face can simply be replicated in the online environment. (They can’t.)

    2) Quality of course design – This is related to the first concern, but their seeming reliance in the online realm of asynchronous discussion for “participation” is limited & limiting. There are multiple ways to engage online learners, both asynchronously and synchronously – and there are also better and worse ways to design and facilitate asynchronous discussion forums.

    3) Capabilities & experience of instructors – Someone already commented on this, but I’ll echo it. In my experience, our school keeps wanting to do more and more online & blended delivery, but they’re throwing inexperienced teachers at it (job security for me). An experienced online instructor can make a huge difference on the educational experience and learning outcomes for students – so that relates to presence/connection between the instructor and students. – And if an instructor only has novice level mastery of even the most basic learning tools available to them in the online realm, they’re not going to do a very good job of teaching with those, much less using some of the more innovative and engaging tools available in the online realm.

    4) Preparation/Assessment of students prior to enrollment – As another commenter said, “Online is not for everyone.” If programs don’t do a good job of evaluating (or helping prospectives self-evaluate) their suitability for the online program, you’re asking for trouble later in the process. And if you’re trying to intentionally target “students at the margins” (as the report summary said), you better have really good support systems in place for those students to help them succeed, and good instructors and good online courses to help facilitate quality learning.

    1. Thanks, Brian. All four points seem well taken. The 4th especially resonates with my experience: some students have really thrived in the online summer course I teach, but others flounder (only to do well once they retake it in the fall F2F). But much of the expansion of online education is driven by a desire to boost enrollment (and, for DeVry, profits), so much so that it seems like there’s a built-in disincentive for institutions to invest in helping students with that kind of discernment. (I suppose you could argue that it’s in their long-term interest, since it helps with retention, but most of these program don’t seem to care about attrition if the volume is high enough.)

      1. I’d like to think that my small, private Anabaptist-Mennonite institution has its heart in the right place when it comes to design & delivery of online & blended programs & courses, and I think we do. I’ve seen other Christian schools dive into the “quantity over quality” model of online, and that’s certainly not where me nor any of my colleagues want to go (and we have discussions about that).

        So it’s an interesting question to introduce into this discussion: for-profit vs. private Christian institutions and how that shapes the design & delivery of programs, including the support & care of students and instructors alike in those programs…

  4. As a colleague I echo Brian’s comments. Another factor to consider is that not all disciplines lend themselves well to distant learning nor do all students possess the digital literacy to succeed in an online environment. An online environment adds another aspect of interpretation, or maybe cognitive engagement, that must be negotiated. It might be equated to taking a course in your native language as opposed to taking one in a language in which you have no, or very little, familiarity. When you compound the learning process of an unknown subject, with various other factors, such as actual language, cultural differences, and technical knowledge, you have created quite the pedagogical obstacle course to navigate. A Face-to-face course is much more familiar and eliminates several barriers. So, the question is not if we should offer online courses, but how we are to design them to reduce these obstacles.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.