On College Rankings

U.S. News Best CollegesThat collective exhaling you heard this morning came from the chancellors, presidents, provosts, and other leaders of the 1378 colleges and universities that participate in the U.S. News Best Colleges ranking process, results of which were released today. As someone who indirectly participates in this process (by teaching at one of those institutions and so possibly affecting factors like its reputation and student retention) and who is indirectly influenced by its results (because prospective students and their parents may be more or less likely to opt for my employer, Bethel University, because of its U.S. News ranking, and we’re very much a tuition-driven school), I was at least mildly interested to learn Bethel’s annual ranking: #19 among Regional Universities (Midwest).

We were at #17 last year, so my first inclination, of course, was to offer to resign for having taught so badly in 2010-11 that I sunk our reputation and/or caused dozens of students to run off to neighboring schools.

But that impulse faded quickly. Leaving me, as usual, equal parts bemused, upset, and past caring about such results.

I know my part of the institution well enough to know how well we do our jobs, and I talk to students and alumni enough to know that they recognize this and appreciate it. But it’s a complex enough organization that I have no idea of our quality beyond my little neck of the woods. And for the most part, the U.S. News methodology makes quite a bit of sense: schools that attract a high caliber of undergraduate students and hold onto them (Retention is worth 25% of our score), in part by holding onto gifted faculty through competitive salaries and small class sizes, probably are better than those that fail on those counts.

But I strongly suspect that Bethel ranks as low as it does (and being in the top 20 of any category is no small accomplishment) primarily because of the Reputation score: 25% of the total for schools in our category, and based solely on a survey of presidents, provosts, and admissions deans at peer institutions. Now this is meant to measure intangibles like teaching quality (which wouldn’t come out fully in quantitative measures like selectivity of admissions, ACT/SAT scores, or even student retention), but as anyone who participates in this survey would tell you, that’s silly. Such leaders are better positioned than, say, me to know something about schools of similar sizes and missions, but the 1-5 rankings they assign are little better than a crap shoot. (I’d love to know on what basis the president of, say, Creighton University — #1 in our category — judges our faculty’s “dedication to teaching,” or how our president does the same for professors at Creighton.)

For that matter, I would hope that U.S. News does not put so much faith in the essential goodness of humanity that they’d let leaders of institutions effectively competing in the same category rank each other.

To get a sense of the fundamental goofiness of the college rating game, start by looking at the ten Minnesota schools to rank highest in the Regional University (Midwest) category: (this leaves out schools like Carleton, Macalester, and St. Olaf, since they’re in the National Liberals Arts College category and so ranked separately; likewise, the University of Minnesota-Twin Cities and University of St. Thomas are in the National University category)

  1. Hamline University (#11)
  2. St. Catherine University (#14)
  3. Bethel University (#19)
  4. College of St. Scholastica (#26)
  5. Augsburg College (#28)
  6. University of Minnesota-Duluth (#36)
  7. Winona State University (#62)
  8. Minnesota State University-Mankato (#70)
  9. MSU-Moorhead and St. Cloud State University (t-#81)

Now look at how those same schools do in one of the chief competitors to the U.S. News ranking, that published by Forbes magazine last month: (Forbes lumps all schools together and lists the top 650)

  1. Bethel (#177)
  2. Hamline (#268)
  3. St. Catherine (#574)
  4. MSU-Mankato (#599)
  5. St. Cloud State (#619)

Half of them don’t even make the list (including two of the top five). (For the record, in this methodology Bethel ranks ahead of National Liberal Arts colleges in Minnesota like Gustavus Adolphus, St. Benedict’s, and Concordia-Moorhead and even National Universities like the U of M and St. Thomas.)

Forbes claims to be different than U.S. News primarily because it doesn’t take “reputation” into account at all. In place of that administrator peer score, it seeks to make tangible intangibles like quality of teaching and learning. How?

Prof. Robert Gehrz
My uncle Bob - University of Minnesota (click on my "About Me" link for a "hotness" comparison)

Well, that’s where Forbes could also be accused of silliness. It gives just over a quarter (27.5%) of the total points to Student Satisfaction (which seems right to me), but most of that (17.5%) comes from the institution’s faculty’s score on RateMyProfessors.com! (The other 10% for retention.) Now, for the record, I do decently well in that system; though I’m peeved that my uncle Bob the physicist has been rated as “Hot” and I haven’t. But I’m not sure that nine anonymous ratings (three of which are more than five years old) are all that meaningful.

Forbes also emphasizes, understandably, Post-Graduate Success, which accounts for more than any other factor in the system (30% — the rest is from student debt, four-year graduation rate, and student success in nationally competitive awards like the Rhodes and Fulbright). This is not measured at all in the U.S. News formula (which instead looks to the quality of students before they’ve ever taken a class at their college or university), so Forbes seems to be on to something. But the way it does this clearly privileges schools that churn out people who aspire to enter high-paying white-collar professions. 15% of the ranking is based on alumni salary, 5% on inclusion in the Forbes corporate officer list, and 10% comes from… wait for it… looking at listings in Who’s Who’s in America?

Which would be wonderful if this were 1959, when my grandpa the corporate lawyer would have been thrilled to see his name appear in Who’s Who? But speaking as someone who routinely recycles his invitation to be included in that book, I daresay that it’s not an especially relevant measure of student success.

And in general, attributing so much value to colleges and universities that prepare waves of CEOs and law firm partners seems arbitrary. Especially if you’re a school like Bethel (or many other private colleges, especially of the Christian liberal arts variety) that prides itself on preparing those in service professions like teaching, nursing, and social work.

All of which leads me to two basic conclusions:

1. Chill, y’all. None of these rankings is fool-proof, but none is entirely without merit. And in any case, only the most gullible high schooler and her parents would circle a number in a magazine and say, “That’s where I’m going.” If nothing else, this society prepares its adolescents to be consumers, and American consumers (while not exactly rational actors) take in more data than this. So none of these rankings will either make or break a college.

2. At the same time… I fear that these rankings are perceived to influence admissions enough that they tempt trustees, presidents, provosts, and other administrators to change their institutions so as to nudge up their scores.

And that could be devastating to a Christian college like Bethel that has such a distinctive mission and identity, since what essentially makes Bethel Bethel shows up nowhere in any of these calculations: forming whole and holy persons who are converted to Jesus Christ and seek God’s glory and neighbor’s good in all that they do. (that’s my version, not the official mission statement, of course)

Not that such a mission is necessarily at odds with retaining students or being perceived as offering excellent teaching, but to improve our reputation among predominantly secular peers or boost our contribution to the upper middle class workforce could lead us away from that mission, and make us virtually indistinguishable from other schools trying to do the same thing.

If you’re interested, here’s how Bethel and the other twelve institutions in the Christian College Consortium did in the U.S. News and Forbes rankings this year. (Listed alphabetically)

  • Asbury University (U.S. News: #5 – Regional College/South; Forbes: #164)
  • Bethel University (USN: #20 – Regional University/Midwest; F: #177)
  • George Fox University (USN: #25 – Regional University/West; F: #309)
  • Gordon College (USN: #144 – National Liberal Arts College; F: #173)
  • Greenville College (USN: #33 – Regional College/Midwest; F: unranked)
  • Houghton College (USN: #133 – National Liberal Arts College; F: #205)
  • Malone University (USN: #58 – Regional University/Midwest; F: unranked)
  • Messiah College (USN: #5 – Regional College/North; F: #415)
  • Seattle Pacific University (USN: #16 – Regional University/West; F: #181)
  • Taylor University (#1 – Regional College/Midwest; F: #213)
  • Trinity International University (2nd tier – National University; F: #472)
  • Westmont College (#90 – National Liberal Arts College / F: #76)
  • Wheaton College (#57 – National Liberal Arts College / F: #66)

Calvin College (not in the CCC, but widely regarded as an elite evangelical institution of higher learning) ranks #102 in the U.S. News National Liberal Arts College category and #218 on the Forbes list.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.