The Pietist Schoolman

Another year, another new college ranking system:

Using unique measures of educational quality, affordability, and career outcomes, MONEY’s new value rankings will help you and your child find the right school at the right price.

I guess whatever Forbes, Washington Monthly, and the Chronicle of Higher Education can do, Money can do better. (Certainly better than U.S. News.) And, for what it’s worth, there are strengths to its system: (explained in depth — and, to its credit, with acknowledgment of limitations — here)

• First, and most importantly, Money quickly dismisses the majority of the country’s 1500-some four-year public and private, non-profit colleges and universities. If a school’s graduation rate was below the median (or if it served more academically fragile populations but didn’t significantly outperform its expected rate), out it went. Ditto if its bond rating with Moody’s indicated significant financial difficulties. That — and a somewhat strange refusal to consider the military service academies — left only 665 contenders standing.
• Graduation rate shows up again in the first category within the calculation, dictating half of the points in “Quality of Education.” We can quibble with the weighting, but it’s hard to argue that six-year graduation rates are unimportant indicators, or that colleges that outperform expectations (given their student populations) don’t warrant special attention here. Most of all, I like that Money focuses on what colleges do with the students they attract, rather than the profile of those students before they’ve ever taken a college course. (Historically, that’s been one of the chief critiques of U.S. News. Money does, like its competitor, use standardized test scores — an admittedly “imperfect measure” — since students’ learning is affected by the abilities of their peers.)
• Not surprisingly, Money pays a lot of attention to money. “Affordability” accounts for one-third of the rating, with net price, debt, and loan default risk factoring in. Even for those of us who think that “return on investment” (when defined solely or primarily in monetary terms) is a problematic way of thinking about the college decision, it’s increasingly obvious that families need to consider the cost of college and how it will be paid for.

A bit trickier is the third of the rating calculation allotted to “Outcomes,” since 90% of those points derive from career earnings. (And the other 10% has to do with access to career services counseling.)

• First, this makes no allowance for outcomes like, say, intellectual transformation, character development, forming of relationships, preparation for graduate school and continuing education, etc.
• Second, the earnings data come from PayScale.com, though Money doesn’t use its “meaningful work” data. (Previously discussed here.) While PayScale‘s 1.4 million users may indeed make it “‘the only game in town’ for anyone who wants to compare colleges’ outcomes nationwide,” the sample sizes for schools vary widely. (And, as Money admits, PayScale doesn’t include unemployed alumni; as a corrective to the inflationary effects on salary date, it only looks to alumni with bachelor’s degrees, leaving out highly-paid doctors, lawyers, and other professionals with advanced degrees.)

Yadda yadda yadda… I think we all know the profound limitations of attempting an objective analysis of something this subjective. (I recently saw admissions data for our school, and — as always — the most important factor explaining why an admitted student matriculated at Bethel or went elsewhere was… “feel.”) And that’s all the more true for the subset of four-year colleges and universities I’m most interested in: simply put, no system that needs to rank public institutions and avowedly secular private ones is going to properly assess the value of Christian colleges and universities.

But, since I’ve done this for several other ranking systems, here’s how the 117 American members of the Council for Christian Colleges & Universities (CCCU) fared in the Money report…

Most importantly, only thirty-four CCCU member-institutions even made it through the preliminary graduation rate/bond rating filter. That’s less than 30% of the consortium, substantially lower than the 45% or so nationally.

Ten of the thirty-four scored at least 3.0 on Money’s four-point, GPA-like scale. (Again, for the record, my own employer came in just shy of that mark: at 2.95, Bethel was the 12th best-scoring CCCU school, and #339 overall.) Here are the ten, with a couple of the indicators made public by Money:

CCCU Member	Score	Net Price	Annual Salary of Grads within 5 Years
62. College of the Ozarks	3.50	$72,420	$31,100
101. Wheaton	3.35	$144,023	$42,400
147. Messiah	3.26	$129,012	$46,100
202. LeTourneau	3.13	$148,903	$51,600
208. Calvin	3.12	$134,876	$41,800
223. Indiana Wesleyan	3.10	$133,712	$43,900
235. Mississippi College	3.08	$91,974	$38,300
269. Cedarville	3.03	$118,221	$44,200
280. Malone	3.02	$120,556	$41,700
285. Westmont	3.01	$160,959	$41,200

Having repeatedly criticized U.S. News for paying so much attention to “reputation,” I probably shouldn’t allow myself to be too surprised to see College of the Ozarks, LeTourneau, and Mississippi College do so well while CCCU stalwarts like Taylor (#400), Seattle Pacific (#409), George Fox (#503), Gordon (#511), and Asbury (not even rated) lag behind. It seems clear that the economic factors made a huge difference here, with College of the Ozarks (with its trademark emphasis on students avoiding debt and working on campus) and Mississippi College (whose net price was among the lowest in the country according to the Department of Education’s College Scorecard) doing especially well on affordability, and LeTourneau (like the marginally surprising Indiana Wesleyan and Cedarville) scoring well on starting salary.

To the extent that these criteria are significant, I would encourage prospective students to dig deeper. Visit PayScale itself, for example, and you’ll find data on how meaningful alumni find their jobs, their mid-career (rather than starting) salaries, and the overall 20-year return on investment (i.e., “total income that a graduate will earn after graduation in 20 years of working, minus both what they would have earned as a high school graduate and the cost of college… minus the average financial aid amount awarded to students at that school if that filter has been selected”): (Westmont didn’t appear in the ROI list at PayScale, so I replaced it with Bethel and added a few more higher-reputation CCCU schools for comparison’s sake.)

CCCU Member	20-Year ROI	Average Mid-Career Salary	% with Highly Meaningful Jobs	Number of Alumni Reporting (8/1/14)
LeTourneau	$436,000	$77,500	60%	214
Bethel	$275,200	$64,400	68%	163
Seattle Pacific	$273,600	$78,800	52%	212
Calvin	$264,300	$73,900	58%	166
Cedarville	$261,000	$69,200	65%	123
Indiana Wesleyan	$258,300	$61,800	58%	714
Messiah	$233,000	$72,000	55%	123
Gordon	$174,800	$76,500	53%	46
Mississippi College	$147,600	$63,700	60%	117
Wheaton	$145,600	$84,400	62%	122
George Fox	$104,700	$62,900	58%	142
Malone	$77,090	$62,200	60%	83
College of the Ozarks	-$18,700	$51,400	insufficient data	29

The sample size alone (let alone a negative ROI) casts all sorts of doubt on College of the Ozarks’ remarkably high Money rank, and Mississippi College doesn’t do quite so well — though better than CCCU flagship Wheaton. But you can see why LeTourneau would be so intriguing to the “return on investment” crowd; I suspect that its signature programs in engineering and aviation  boost average salaries considerably.