Ask the Experts, Biglaw, Football, Job Searches, Sports

From Across the Desk: Good Enough for the NFL

Bruce MacEwen asks: What is wrong with the current hiring process employed by large law firms? Oh, where to begin....

By  
on

Ed. note: This is the latest installment in a series from Bruce MacEwen and Janet Stanton of Adam Smith Esq. and JDMatch. “Across the Desk” takes a thoughtful look at recruiting, career paths, professional development, human capital, and related issues. Some of these pieces have previously appeared, in slightly different form, on AdamSmithEsq.com.

I don’t know about you, but I find talent markets fascinating. They have several characteristics that make them quite distinctive from regular old goods and services markets:

  • Talent is extremely heterogeneous; it’s not as if there’s another Honda Accord where that one came from.
  • Talent is what economists call both “excludable” and “rivalrous,” meaning that if I hire you Suzie can’t hire you at the same time. (Knowledge is the classic non-rivalrous and non-excludable good; everyone can know the same thing at the same time without its impairing anyone else’s knowledge of that same thing, and without shutting off anyone else’s access to it.)
  • Talent is notoriously difficult to judge in advance, without actually experiencing it, that is to say, without actually hiring the individual and putting them to work in your organization. Some other markets approach this condition of “ignorance until purchased,” such as attending performing arts events or taking a vacation to a previously unknown locale, but the stakes tend to be much higher for all parties concerned in talent markets.
  • Once talent is hired, it’s stickier than most other purchases. You can walk out of the movie theater or reconfigure your travel plans, but once you hire someone, short of felonious or otherwise appalling behavior, you’re stuck with them for a decent interval.

All this leads to a number of devices and stratagems that attempt to mitigate uncertainty and delay serious resource commitments until some firsthand evaluation can be performed.

Sponsored

For example:

  • Performance bonuses paid in arrears, that is, after the activity you wish to reward has (or hasn’t) happened;
  • Likewise, commission payment structures; and
  • Deferred compensation in general.

Sponsored

Given all that’s at stake in the talent market, firms find it worthwhile to invest considerable imagination and resources in trying to assess new hires before they’re taken on. In the world of professional services — not to mention industries where it’s even more widespread — tools such as aptitude tests, psychometric profiling, compatibility assessments, and more, are universal.

What types of tests? It depends on the organization, of course, and what they’re looking for, but firms such as D.E. Shaw (an exotic quant trading firm where Jeff Bezos got his start before upping and outing for Seattle), Google, McKinsey, and Sanford Bernstein (an elite investment research boutique) use these tests religiously. Candidates might be asked questions that appear oddball but are designed to expose how they think. “How many pubs in Great Britain?” “How many cans of Coke sold in the US annually?” (They’re not looking for the most accurate answer; they’re looking for the rigor and imagination of your analytic process.)

An hour or two, even a day, of objective testing seems a small gate to require one to go through for one of these intensely desirable jobs.

But before talking about Biglaw, let’s go to a talent market where even more money is at stake: The NFL.

The New York Times reported, in advance of the annual NFL scouting “combine” (the recruiting event of the year for the hundreds of college players aspiring to the pro leagues), that a new segment of their extensive job interviewing will be:

an hourlong psychological assessment designed to determine and quantify the nebulous qualities that coaches have long believed make the most successful players — motivation, competitiveness, passion and mental toughness — and to divine how each player learns best. The new test, like the [previous test], is mandatory for the more than 300 players who attend, and it will be given for the first time Friday.

Here’s the protocol:

To determine their personalities, the test will ask players a series of questions about their preferences and behavior. To evaluate their cognitive abilities, it might tell them to look at four diagrams and figure out how they relate. Then, to measure how quickly they can adjust their thinking, the items they are comparing might change, forcing the players to determine their relationships anew.

To see how they learn best, the test will present questions in verbal and graphic form. Players will have an hour to take the exam on a computer.

Imagine if a comparable tool were available to law firms.

The criteria we’ve been using to evaluate entry-level talent has been unidimensional for decades: The students with the highest GPAs at the best law schools. Great in theory, but it turns out to have terrible predictive value of success. In fact, the proportion of associates from Tier 1 law schools exceeds the proportion of partners from Tier 1 schools by a sizable and statistically significant margin. And law review is negatively correlated with making partner. But these are the criteria we continue to hire on. What on earth are we thinking?

Again, given all that’s at stake, why wouldn’t we want to look for different, or at least additional, tools? At least for those who care to look at the actual evidence, we’re clearly doing something wrong. I hate to break this to you, but any head of recruiting at a Fortune 500 whose criteria consistently predicted failure and not success would have a very short tenure indeed.

Virtually every law firm I’ve talked to about this admits their criteria aren’t filtering everything that matters, although a few stoutly maintain that “the best predictor of success in future is success in the past,” resolutely ignoring the reality that academic success and worldly success are poorly (or, a la law review, negatively) correlated. And the vast majority wish they had better tools. The problems are three-fold:

  • They don’t know what those tools might be and they’re not in the business of designing them;
  • They refuse to be the first to put their applicants “through an extra hoop—they’d just go somewhere else”; and
  • Even if the tools existed and were offered by some independent third party, so as not to be tarred by association with any specific firm, they don’t think they’d know what to do with the results.

What if all three of those reservations could be solved? What if you had available:

  • A proven capability assessment developed by a firm that does nothing else;
  • Offered by a disinterested third party;
  • Which you could validate by comparing the clusters of candidates it generates to the clusters of lawyers in your own firm? (After all, you know who the winners, losers, and can’t-yet-tell’s are in your own firm.)

It may be on the horizon.

But just because it’s good enough for the NFL…

Topics

, , , , , , , , , , , , , , ,