Technology

Study Confirms: AI Outperforms Experienced Attorneys At Reviewing Contracts

LawGeex unveils a new study and it's harder than ever to dismiss the advantages of artificial intelligence.

By  
on

After years of hype, artificial intelligence, as a legal services tool, has reached the “put up or shut up” stage of maturity. While AI has delivered solid results in a variety of capacities for years, the potential of something greater just around the corner kept the hype machine inflating. As we crest the hype cycle, and the “trough of disillusionment” looms, AI companies need to lay their cards on the table and demonstrate concrete advantages to potential customers.

Earlier today, contract review automation company LawGeex stepped up to that challenge and released a study, with the results analyzed by professors at Duke, USC, and Stanford, to finally resolve the question: can AI outperform a seasoned attorney in identifying risks in a contract?

The study pitted the LawGeex AI solution against 20 US-trained top corporate lawyers with decades of experience, specifically in reviewing NDAs. The participants’ legal and contract expertise spanned experience at companies including Goldman Sachs and Cisco, and global law firms including Alston & Bird and K&L Gates.

Both the lawyers and the LawGeex AI analyzed five previously unseen contracts, containing 153 paragraphs of technical legal language (“legalese”), under controlled conditions precisely modeled on the way lawyers review and approve daily contracts. This is the first time that an AI has been tested with a typical task undertaken by lawyers on a daily basis.

LawGeex registered a 94 percent accuracy rate. The humans averaged 85 percent.

And took a lot longer.

You can request the full study here, but to break it down a little bit more, the best performing lawyer matched the LawGeex algorithm’s 94 percent accuracy, while the lowest performing attorney only caught 67 percent of the issues. On the one hand, humans can take heart that at least one of our number managed to keep up with the steam-powered hammer. On the other hand, LawGeex performed its check in 26 seconds, while the humans took between 51 and 156 minutes. Whether you’re paying by the billable hour or not, that’s a significant investment of quality attorney talent squandered on a task where attorneys aren’t providing any value add.

That’s the thing with AI as a legal support tool. It’s not really about outperforming humans at straightforward identification tasks — though it can do that too — it’s about delivering that completed work product orders of magnitude faster than humans ever could.

Sponsored

So the choice for clients is pretty simple — the cost of an hour’s worth of attorney labor for, at best, the same thing you could get in less than 30 seconds. When you put it that way, the prospect of embracing AI isn’t really a hard choice at all.

(You can check out a cool infographic for this study on the next page.)

Earlier: The Artificial Narrative Of Artificial Intelligence
Can AI Avoid The ‘Trough Of Disillusionment’ In Legal Tech?

Sponsored

1 2Next »
CRM Banner

Topics

, ,