About two years ago I was invited to give a talk to our faculty in recognition of having been awarded Bethel’s 2009 Faculty Excellence Award for Teaching. I chose to reflect on my experiments with academic podcasting as elements of two courses I teach. Today I’ll post the first (slightly updated) section of my talk; the conclusion will follow on Friday.
So it turns out that if you win a Faculty Excellence Award at Bethel, you get/have to give a talk related to the field for which you’re being recognized…
I honestly can’t think of many things worse than talking about my supposed excellence in teaching to a roomful of excellent teachers. My first inclination was to see if it was possible to decline and return the award money, but given the impending geometric growth in my family, I can’t afford to be so noble. So then I thought I’d push back the talk as far into November as possible, hoping that my wife would simply go into labor a minute before I was scheduled to start. But here we are, 35 weeks pregnant with twins and counting, and my cell phone’s not going off…
Then I thought that perhaps I’d select a historiographical title so dry that no one would show up. But finally I decided that, since I was being recognized in part because of my experimentation with academic podcasting, I would let the punishment fit the crime and give a talk on the sometimes contentious issue of online vs. face-to-face (F2F) education.
A year ago this very topic caused bitter debate on our faculty listserv, and a recent survey found our faculty divided. But I’d like to chart a middle course here: I’m a sucker for new innovations, but I also greatly enjoy F2F instruction and have no vested interest in seeing more and more classes move online.
First, let me give you a brief introduction to my experience with online education. On the one hand, it’s limited: I’ve never taken or taught a fully online course. (Though I will be doing this starting in the summer of 2013.) On the other, I have had the chance to experiment with integrating multiple types of podcasts into two different courses.
This experimentation began in 2005-2006, when Sam Mulberry, Stacey Hunter Hecht, and I began to produce weekly podcasts (“CWC: The Radio Show”) for GES130 Christianity and Western Culture. These podcasts were optional supplements to a lecture-based multidisciplinary course taken by 70% or so of Bethel 1st year students, meant to humanize professors whom most students encounter with 130+ colleagues in a lecture hall. It’s been hard to gauge listenership, but our goal was to get at least 15-20 students listening — the equivalent of an entire CWC section committing to an additional 15 hours of CWC-related material for the semester — and we’ve more than reached that target. (We’re now in the eleventh “season” of this podcast series. Click here to get episodes from iTunes U.)
We’ve also used podcasts in CWC as required elements of the course: first, replacing a lecture on Islam with a podcast built around interviews with scholars of religious studies and medieval literature; and second, a podcast modeled on NPR’s This American Life (featuring interviews with four professors from different departments who teach courses in the Bethel general education category that builds on CWC) that students drew on in writing their final essays.
But the closest I’ve come to teaching online was in 2007-2008, when I decided to replace one-third of the lectures in my HIS354 Modern Europe course with hourlong podcasts (“Radio Modern Europe” – click here to listen to episodes from Fall 2008). There might be good economic reasons for our administrators to encourage departments to develop hybrid or online courses (Bethel now offers stipends for this purpose, with the goal of having at least one online course per department), but that wasn’t relevant to my decision. Instead, I had grown somewhat dissatisfied with the way I was delivering several lectures and decided to try a different format (and reorganize material so that I could create room for more discussion of readings and films) in which 3-4 brief lecture-like segments (accompanied by a PowerPoint) would be broken up by clips of period music, interviews with faculty or historical “guests” (I took strange pleasure in impersonating Karl Marx, V.I. Lenin, and Neville Chamberlain), film or museum reviews (ideally by students, though they didn’t show much interest in being recorded for podcasts), or other less serious bits (e.g., my friend Sam and I “broke down” World War I and the Cold War in the style of an NFL pregame show; I did a “cooking” segment using a Victorian guide for housewives — it would have been much better with a video podcast).
These podcasts were scheduled for Mondays and Fridays so as to give students the equivalent of a long weekend, and then followed the next class session by a quiz covering the podcast and associated textbook reading.
I’ve taught this course every year since coming to Bethel, and so had already grown accustomed to using it as a “pedagogical laboratory” — making semi-significant changes to readings and assignments every year within the confines of a standard narrative structure. For whatever reason, podcasting seemed even more conducive to this spirit. I felt somewhat liberated to try things that I might have been averse to attempting “live” in the classroom. In practice, the result was something akin to what I normally do in January term with my World War I class, where I meet daily with students from 8:00-10:45 and break up lecture with film clips, singalongs, and roleplaying simulations. (All easier to do when you’re focusing on one class for a term; harder to arrange in-semester when you have two other courses to attend to.)
Now, before I tempt any colleagues to try something similar, let me first say: It was a ton of work. Especially that first time out in Fall 2007, developing ten hours of podcasts was, to say the least, exhausting. After all, I tasked myself not only to rethink what material I wanted to cover and how, but to teach myself how to use Garage Band and iTunes U. (I suppose if you just wanted to record lectures, that would be one thing; but that approach seems like an utter waste of the potential of podcasting and a failure of the pedagogical imagination.) The Fall 2008 series was considerably easier, since I was more accustomed to the technology and software, and mostly able to revise instead of create.
That said, I’m both convinced that it worked pretty well and that I don’t want to do it again for a while. I’ll get to the latter later, but to the former point…
One question here is whether there was any “significant difference” in student learning or enthusiasm between the podcasts or the more traditional lectures they replaced. “No Significant Difference” (NSD) is a school of thought in education that posits that there is, well, no significant difference in student learning between online and face-to-face education. (The implication being, I think, that most educators would expect — or prefer to believe? — that F2F is vastly superior to online.) One of NSD’s pioneers is Thomas L. Russell, now retired from North Carolina State University, who administers a No Significant Difference website that collects studies whose findings agree and disagree with the NSD hypothesis. (At least it was collecting such studies when I first gave this talk; I checked back and it hasn’t been updated since 2009.) For 2006-2009, for example, it features eleven studies: two haven’t yet been sorted; of the remainder, five found no significant difference, three found that online (or “technology”) classes yield significantly better results, and one found F2F superior.
Based on the feedback I received from students (I included several questions about the podcasts on the standard course evaluations for 2007-2008) as well as their performance on quizzes and other podcast-related assignments, I’d offer the following NSD-related observations:
- Students were very happy to have the ten long weekends created by replacing lectures with podcasts. No surprise. More surprising, and contrary to what usually happens with online education, students seemed to think that the class was better for the online component. In most of the NSD studies I perused, the gap between what students actually learn and their satisfaction with the learning mode is significant.
- I didn’t have terrific data on this, but judged by performance on individual assignments and by student self-reporting of learning outcomes on our course evaluation instrument, it seems clear that they learned the basic outline of modern European history just as well with podcasts as they would have with lectures. But perhaps they could do even better…
- What struck me as most interesting was that my students used the podcasts in the same fashion they did lectures: that is, they sat down, listened to it in one playing, and took notes. (One student even reported that she went to our vacant classroom at the usual time and listened to the podcast as if I were at the head of the room giving a lecture!) Few reported any creative use of the technology (e.g., listening to the podcast in smaller chunks; replaying segments). I’d be curious to know what those who research in this field have found: How do students adapt to online courses? How can instructors help them to do so?
In the end, I’m willing to buy the NSD thesis to the extent that well-designed online and hybrid education is able to help students gain basic knowledge and perhaps even to develop some intellectual skills (were I to return to this model, I would try to build in more exercises based on reading and analysis of primary sources, something I tried last spring with some success in my Cold War history course), and our department has entertained the possibility of using a fully online course to introduce new history majors to the methods, sources, and philosophies of the discipline, to be taken in tandem with one of our sophomore-level survey courses (taught F2F).
But it’s much less clear both that online teaching is able to foster the more advanced cognitive learning that one would expect to see highlighted in an upper-division course like Modern Europe. Not that I was willing to test that: the post-podcast quizzes (with one or two exceptions) focused exclusively on relatively low levels of Bloom’s cognitive taxonomy: knowledge and comprehension. They did not attempt to measure analysis, evaluation, or synthesis. This seems to be a common limitation of the NSD literature, with most of those studies focused on introductory courses.
And this doesn’t even touch on the affective learning (e.g., internalizing values) that is central to the liberal arts model in general and the Christian liberal arts model in particular. I remain highly dubious that an online environment can cultivate this kind of learning (to say nothing of psychomotor) as effectively as happens when you’re face-to-face with a professor and fellow students.
All that needs further study (by me, at least; I’m sure experts in the field are hard at work already), but for now, I find myself much less interested in typical NSD discussions about student comprehension than in two different significances of online education for colleges like Bethel: one of which suggests an advantage of online teaching; and one of which clearly points in the opposite direction.
We’ll consider those different significances in part 2 of this talk, coming Friday…
The Pietist Schoolman 