Law Schools, Media and Journalism, Rankings, Ridiculousness

National Jurist Needs To Delete Their Law School Rankings And Apologize To Us All For Wasting Our Time

National Jurist rankings are so bad they should apologize.

By  
on

Say what you will about the Cooley Law School Rankings, but at least they are internally consistent and objectively applied.

We’ve written before about the ridiculous National Jurist Best Law School Rankings. Many law bloggers have written about this list that looks like it was put together by getting the Sorting Hat drunk on goblets of fire water and forcing it to name law schools until it passed out.

We’ve all tried to reason with the National Jurist, but it turns out that effort was not unlike trying to convince an infant not to poop while you’re eating. We’d have been better off just ignoring it and cleaning it up later.

The publication came out with an “edit” yesterday, and while its revisions did a good job of highlighting how stupid these rankings were in the first place, I’m compelled to write about them just so nobody is fooled into thinking their “updates” have actually fixed anything….

Most of the criticism of the original rankings involves NJ’s use of Rate My Professors as a measure of student satisfaction. Professor Brian Leiter, a law school rankings guru, puts it well:

[T]he coup de grace is that 20% of the overall score is based on Rate My Professors, the notorious on-line rating site used mainly by undergraduates, and hardly at all by law students.

What kind of sample was this? In the case of the University of Chicago Law listings on Rate My Professor, it consisted of 54 responses total for ten faculty over a period of six years — but only 23 responses for actual full-time law school faculty! Indeed, Rate My Professor lists one person, Smigelskis, who has never even taught in the Law School here (he accounts for almost 20% of the responses, and also had the lowest scores). (Actual law faculty, like me, didn’t even appear, because Rate My Professors had me listed in the wrong unit!) I’ve gotten e-mails from colleagues elsewhwere reporting similar anomalies.

In short, 20% of the overall score is fraudulent on its face. And it’s that 20% that explains all the variance. Stanford, Harvard, Virginia, Chicago, Michigan, Yale all get A and A+ scores in all the employment categories (NLJ 200 partners, Super Lawyers, etc.), what differentiates them is the fraudulent Rate My Professors data. This means the National Jurist has one advantage over the Cooley rankings: its absurdity isn’t quite as obvious.

Sponsored

Using Rate My Professors as a measure of student satisfaction is dumb, but apparently the people at National Jurist thought their problem was simply incorrectly applying the stupid statistic they were using. So yesterday, they revised the rankings so that the Rate My Professors data would be based only on current law faculty (as opposed to non-law faculty or faculty no longer at the school).

Here’s what changed, according to National Jurist:

After the update, 33 percent of RateMyProfessors grades did not change; 16 percent shifted to “no grade” on account that there were fewer than 40 evaluations completed for the school; 33 percent changed by less than a grade; 14 percent changed by more than a grade but less than two grades; and eight schools — or 4 percent — changed by more than two grades.

The changes are significant. University of Chicago Law School jumped from #56 to #5. Boston University College Law School jumped from #48 to #10 (as Dean Vincent Rougeau noted in an email that we’ve reprinted on the next page).

This, of course, completely proves Brian Leiter’s point… that Rate My Professors, an unverified site predominately used by non-lawyers, had a huge effect on the National Jurist rankings. What a joke. Fixing an issue with using ratings for non-law professors is only part of the problem.

Sponsored

Then again, there’s no way to save these rankings. Remember, we’re talking about rankings that try to convince us that “Super Lawyers” is a quasi-legitimate indication of post-law-school success.

I didn’t think it was possible for somebody to be worse than Cooley at ranking law schools, but at least Cooley uses objective factors. The square footage of your law school library is a stupid thing to look at when ranking law schools, but at least it can be applied fairly over many institutions. Super Lawyers cannot. I mean, say what you like about the tenets of National Socialism, but Dude, at least it’s an ethos.

The National Jurist needs to stop the charade. They don’t need another update; they need an apology. Here, I’ll write one for them:

Dear Internet,

I’m sorry to have wasted all of your time. We’ve been told that people love rankings, and so we decided to make some up so that people would notice that we have a publication called the National Jurist.

It turns out that our plan worked, and people have talked more about our lazy and poorly executed efforts than anything else we’ve written about.

But we’ve got to stop this now. While we’ve proven our point, the thought that even one law student would draw value from these rankings forced us to examine our motives. We’re sorry. It was cynical, and again, incredibly lazy for us to do this.

Can’t promise we’ll do better next time, but we are sorry for wasting your time like this. We didn’t mean to hurt anybody.

Sincerely,
National Jurist

If they would just post this on their front page for a week, I think we’d call it fair and move on.

Best Law Schools updated, corrected: UChicago jumps into top 5 [National Jurist]
National Jurist in Competition to Displace Thomas Cooley Rankings as Biggest Joke in Legal Academia [Brian Leiter’s Law School Reports]

Earlier: Pure Ridiculousness: National Jurist’s Law School Rankings
National Jurist EIC Defends His Indefensible Rankings
Latest Cooley Law School Rankings Achieve New Heights of Intellectual Dishonesty

1 2Next »

Topics

, , , , , , , , , , , ,