I used to vacation with my family and friends at a swanky resort in a foreign country. The resort had several grand pools and overlooked the Pacific. It also had a very strict towel policy. To get a towel, you had to sign in with an attendant who recorded your room number, name, and the time that you took the towel. At the end of the day, you had to return the towel to the attendant and have him write down the time it was brought in. If you didn’t turn it in at the end of the day, the attendant reported that, and you were charged a significant fee.
Compound that by several pools and God knows how many towels, and you need a lot of human labor just to take care of the towels. The cost to pay all those attendants at all those pools had to be a lot. I naively wondered at the time whether it would be cheaper for the resort to just absorb the cost of missing towels or somehow use technology to replace part of what all those attendants did.
Then it dawned on me. At the meager wage levels of workers in that country, it was much cheaper to pay humans than to eat the cost of lost towels or buy the technology to replace them.
But as labor costs went up, all that changed. Now you grab your towel from a bin, and if you remember to turn it in, you just drop it in another bin on the way out. If you forget, no big deal.
Today’s Cost of AI Use
It’s similar to where we are with AI today. Like the resort, today’s users of AI assume it’s cheaper to use AI without restraint than to have humans do what the AI can do. There’s a notion that AI is more efficient than humans for any and all things. That there is no need to worry about or even talk about usage cost. But that begs the question of whether that will always be the case.
Few vendors and users are even thinking about what will happen if, and when, the usage cost goes up and goes up significantly. Much less talking about it. Not at ILTACON. Not any place.
Except perhaps NetDocuments.
The Legal Context Engineering Benchmark Report
Back in May, NetDocuments released a Legal Context Graph and, more recently, a Legal Context Engineering Benchmark Report about what the Graph showed. Both the Graph and Report were designed to measure how better context and content can make AI use more efficient and results more accurate at less cost. The Graph and Report purported to show that the cost of obtaining a correct AI answer can be substantially reduced by adding content and context.
It’s perhaps easy to dismiss what NetDocuments is doing here since it is in the business of document management. And it’s in its own interest to tout being able to use increased context to increase accuracy at a lower cost.
But that would be a mistake. What NetDocuments has done is important because virtually no one has gone to the trouble of looking hard at the relationship between AI cost and accuracy. And being able to determine that ratio is about to become critical.
Today’s Usage Pattern
Dan Hauck, NetDocuments Chief Product Officer, describes the current prompting process this way: a user types in a prompt in an AI platform hoping to get the answer they are looking for. If they don’t, they simply continue to prompt the tool over and over with additional information until they get what they believe to be an accurate answer. They can do that because the cost of repeated prompting is typically not much of a factor, and there’s no way to undertake a cost-benefit analysis in any event.
But What About Tomorrow’s Usage, And Its Cost
But that’s about to change. Here’s why: AI vendors are investing millions, if not billions, of dollars in developing AI models and platforms. They generally charge for these platforms on a subscription basis while losing boatloads of money. But the investors in these vendors are not going to let them get away with that forever. At some point, they will demand that the vendors show a profit. A significant profit.
Indeed, one vendor, Legora, is already talking about going to a consumption-based model. The more tokens you use, the more the cost will be. Under that kind of model, the ability to evaluate the benefit of the AI usage for a task against the cost could become important.
And other vendors across the AI world will no doubt follow. Why? Because quite simply, they can. Customers are relying more and more on AI platforms. They are changing their workflows and processes. Some are even changing their business models.
All that makes it very hard for them to leave the AI Garden of Eden, even if the usage cost goes up dramatically. It is a perfect recipe for changing the pricing model, charging more for tokens, and raising the overall price.
Think Amazon. In the beginning, Amazon charged a remarkably low cost for its eBooks. In doing so, it cultivated a huge customer base over the years while also losing money. But once that base was all in with eBooks, once we all bought our Kindles, Amazon began raising the price of those eBooks significantly. In doing so, it became extremely profitable. AI vendors will soon adopt a similar strategy. As Josh Baxter, NetDocuments CEO, succinctly told me while we were at ILTACON: “The reality is it’s coming.”
It’s a reality that will require application of cost-benefit analysis.
The Future Importance of the Cost–Benefit Ratio
That’s why what NetDocuments is doing is important, even if perhaps it’s before its time. It’s providing a framework for injecting an accurate cost-benefit analysis into the AI use matrix. Here’s what that could mean: when increasing the quality and accuracy of an AI output by 1% costs 10-20% more, the user may decide it’s not worth it.
I’ve experienced this with the insurance industry. Over time, carriers figured out that they could pay notoriously low rates to insurance defense lawyers, knowing that incremental increases in quality from using higher-priced lawyers was simply not worth the increased cost. My resort figured that out as well.
Yes, sometimes the increased cost for increased accuracy may indeed be worth it. Sometimes it won’t. But without some way to determine that relationship, it would just be guesswork. And the temptation to be right at whatever cost will be the default.
Baxter puts it this way: “As AI shifts to consumption pricing, the industry is swinging between two extremes: spending without limits and cutting without strategy.” With no way to figure it out.
It’s Not Always Rock and Roll
All of which is to say that NetDocuments is taking a step in the right direction. It’s tossing a dry towel on all the drunken AI gushing that’s going on.
And I’m not surprised that it’s NetDocuments that’s doing this. I keep coming back to what Baxter told me one time:“We’re not a rock band.” That’s what distinguishes the company in a sea of AI over the top hype.
And sometimes it’s nice to listen to some quiet jazz instead of Yungblud. Whoever that is. It’s the name ChatGPT gave me when I asked who the most over-the-top rock star is today.
At least his name isn’t Harvey.
Stephen Embry is a lawyer, speaker, blogger, and writer. He publishes TechLaw Crossroads, a blog devoted to the examination of the tension between technology, the law, and the practice of law.