Biglaw, Document Review, In-House Counsel, Technology

Inside Straight: Why Vendors Win The Document Review Work

Document review, which was historically an important profit center for large law firms, has moved permanently into the hands of third-party vendors. That sea change was not prompted by the recession, and things are not going to return to the old "normal" after the economy recovers. But why?

By  
on

When you’re in private practice, people ask you tough questions. “During the proxy fight, a trial court held that our proxy statement was false and misleading. We settled that case, so that judgment is final. We’ve now been hit with a 10b-5 shareholder suit, and the plaintiffs have filed a motion saying that the earlier proxy decision is binding on the question whether our statements were false and misleading. How do we defeat that motion?”

Then you move in-house, and the question changes: “How can we reduce the cost of electronic discovery and document review?”

Oh, how the mighty have fallen.

But, when you’re forced to think for a minute about electronic discovery and document review, you realize that the battle previously waged between law firms and third-party vendors to capture this work is now largely over: Document review, which was historically an important profit center for large law firms, has moved permanently into the hands of third-party vendors. That sea change was not prompted by the recession, and things are not going to return to the old “normal” after the economy recovers. Companies that continue to rely on law firms, rather than third-party vendors, to do large document reviews are probably making a mistake, and law firms that are counting on document review projects to resuscitate their profitability are betting on the wrong horse.

Why?

There are two reasons, and they’re tough to argue with. First, review by third-party vendors is cheaper than review by law firms. Second, review by third-party vendors is better than review by law firms. That about covers the waterfront, doesn’t it?

Sponsored

Vendors are cheaper for many reasons. Vendors pay their lawyers less than law firms pay associates. Vendors do their work in less expensive space than where law firms conduct reviews. Some vendors give their employees “an engineered work environment that is optimized for comfort, efficiency and mental accuracy.” William D. Henderson, “Three Generations of U.S. Lawyers: Generalists, Specialists, Project Managers,” 70 Md. L. Rev. 70, 112 (2011). (When you’re not quite sure what certain words imply when you’re typing a sentence, you’re always better off quoting someone else and sticking in a cite.) Vendors are developing knowledge management and intelligence gathering tools to permit reviewers to identify relevant fact patterns more quickly. Id. Unless law firms basically establish separate businesses devoted to document review, law firms won’t be able to compete with the specialized vendors on price.

Nor can law firms compete on quality.

I’ve heard the pitch — heck, not that long ago, I made the pitch — that a law firm simply must have the firm’s own lawyers review documents, because otherwise the firm wouldn’t have permanent institutional knowledge of what the documents say. And that may be true in a case that involves a couple of boxes of documents, where one or two lawyers can sit down, read every relevant piece of paper, and retain in a single unifying mind all of the key information.

But that argument is a bunch of hooey when the document review involves sitting 50 lawyers at computer terminals for months on end and having them plow through materials until their minds are numb and their eyes bleary. It’s a safe bet that, in a large firm, the partner at the helm of a case involving one of these massive discovery efforts is not speaking regularly to the junior associates performing the review. Whatever “institutional knowledge” the firm is acquiring isn’t being passed up the ranks in any meaningful way — or, at least, in any way that a vendor couldn’t replicate. Moreover, the team of junior associates won’t remain on this big case, and won’t remember the details of the many documents they’ve reviewed, six months from now. It’s fine to pontificate about the need for “institutional knowledge,” but it’s much harder to explain precisely what that means.

(I guess I should add a caveat here: Big cases frequently involve two levels of document review — a first-tier review to identify a manageable group of arguably relevant documents, followed by a second-tier review that’s more focused. In this column, I’m thinking largely about first-tier reviews; the things that I’m saying may not apply equally to second-tier reviews.)

Sponsored

If the argument about institutional knowledge is basically false, then the real question is who does a better job identifying and culling relevant documents — law firms or third-party vendors. Both common sense, and some limited evidence, point in the direction of vendors. The common sense is that vendors specializing in document reviews are, well, specialized in document reviews. They focus on that subject. They work hard to enforce quality control. Some of the vendors employ techniques borrowed from other industries to provide continuous feedback loops and statistically-driven quality control (such as determining what percentage of reviewed documents must go through a second level of review to be sure that the primary reviewers have been properly trained and are sufficiently attentive).

That’s just some vendors, of course. Other vendors do things the old-fashioned way. And even vendors at the cutting edge of the document review world are nowhere close to perfection. The question is simply how the vendors compare to law firms.

What do law firms do?

They too vary widely, of course. The traditional document review involved having a partner or senior associate explain to a gang of young lawyers what the case is about, putting a second- or third-year associate at the helm (to have “ownership” of the document review) and letting people go to a conference room (or, more recently, sit at their desks) and start reviewing. The second-year associate knew little about quality control or keeping people motivated in this environment, and the associates burdened with the document review process felt, well, burdened by having to participate. Those lawyers would have preferred to be doing something “real” with their time, like legal research or writing, learning how to take depositions, or anything that would advance their careers. Instead, they sat at the terminals going through documents for hours on end, with no emotional connection to the client or the litigation and little concern for work quality.

The results are predictable. Although I haven’t seen the studies themselves (and I suspect there would be plenty of room to quibble over methodologies and the precise meaning of the results), the reports that I’ve heard suggest that the specialized vendors are winning the race to quality. One vendor sports a study showing that Am Law 100 firms achieve only 85 percent accuracy in their document reviews, while the vendor achieves 99 percent.

It’s possible, of course, for law firms to improve their processes, implementing the technological and quality-control processes being developed by vendors. But, unlike many of the vendors, law firms do not specialize in document review, so it’s unlikely that law firms will lead the vendors in this regard.

Generalities are dangerous things. Third-party vendors surely vary widely in quality, as do law firms. But corporations should always consider using third-party vendors to help with large document reviews, and corporations should ask hard questions of either law firms or vendors about how they conduct those reviews and what quality assurance procedures they have in place.

Or at least that will be the rule for the next few years. Within a decade, artificial intelligence may well have progressed to the point where the idea of human document review seems positively quaint — along the lines of sundials, iceboxes, and the manual to your Wang word processor.

Mark Herrmann is the Vice President and Chief Counsel – Litigation at Aon, the world’s leading provider of risk management services, insurance and reinsurance brokerage, and human capital and management consulting. He is the author of The Curmudgeon’s Guide to Practicing Law (affiliate link). You can reach him by email at inhouse@abovethelaw.com.

CRM Banner

Topics

, , , , , , , , ,