The jury is still out on how much and how soon GenAI will impact the legal profession, as I pointed out in a recent article. But one thing is certain: GenAI is affecting what people are revealing, the questions they’re asking, and what advice they’re receiving. The implications for lawyers, or perhaps more accurately, their clients, are downright scary. People are talking too much and getting wrong advice that’s memorialized for future use and discovery
I had sounded this alarm before. And now a recent Washington Post analysis of some 47,000 ChatGPT conversations validates many of these concerns in alarming ways.
The Post Analysis
Here’s what the Post found:
- While most people are using the tool to get specific information, more than 1 in 10 use it for more abstract discussions.
- Most people use the tool not for work but for very personal uses.
- Emotional conversations were common, and people are sharing personal information about their lives.
- The way ChatGPT is designed encourages intimacy, and the sharing of personal things. It has been found that techniques that make the tool seem more helpful and engaging also make the tool more likely to say what the user wants to hear.
- About 10% of the chats analyzed show people talking about emotions. OpenAI estimated that about 1 million people show signs of becoming emotionally reliant on it.
- People are sharing personally identifiable information, their mental issues, and medical information.
- People are asking the chat to prepare letters and drafts of all sorts of stuff.
- ChatGPT begins its responses with yes or correct more than 10 times as often as it starts with no.
And of course, it still hallucinates. While the analysis focused on ChatGPT conversations, there can be little doubt that other public and perhaps closed LLMs are being used in many of the same ways and doing the same things.
The Problem
That means there’s a lot of scary stuff out there that could of course be open to discovery in judicial and regulatory proceedings. Indeed, as previously written, OpenAI’s CEO Sam Altman has recognized that the company would have to comply with subpoenas. And government agencies like law enforcement can seek access to private conversations with an LLM as well.
What the Post analysis tells me though is that people aren’t recognizing this danger. They seem to think that they stuff they put in and get out is private. Indeed, the Post got the 47,000 conversations because people created sharable links to their chats that were then preserved in the Internet Archive. OpenAI has now removed the option to have shared conversations discoverable with a mere Google search since people had accidentally made some chats public. That’s troubling in and of itself.
Worse, the answers given by ChatGPT since they tell the user what they want to hear, are wrong. One thing I learned in my years practicing law, is that clients usually start out convinced they are right. (Most never really change their minds.) Their mindset when their lawyer tells them they are wrong is that they would have received the answer they wanted if only they had a better lawyer.
Now we have the problem on steroids. The client walks in convinced they are right and thinks that their position has been confirmed by ChatGPT.
Perhaps even worse, people may be acting on the advice they are getting from the LLMs, getting themselves in even more trouble. Clients often held back acting on something because they knew enough to know they should consult a lawyer. But since that was expensive, they just didn’t do it out of an exercise of caution. Now they have what they think is confirmation. A green light.
Here’s Where We Are
Putting these facts — people putting discoverable and potential damaging stuff in an LLM thinking it’s private (which LLMs encourage), LLMs telling the user what they want to hear or making up answer which the user believes and might even act upon –together with some common situations demonstrates why these factors should be concerning to lawyers.
It doesn’t take much to foresee a C-suite officer, for example, using ChatGPT to seek to solve a thorny personnel problem by brainstorming with an LLM and commenting on the responses in a back-and-forth manner that creates a paper trail for a future wrongful termination case.
Or a disgruntled spouse venting in a conversation that becomes public in a divorce or custody decision. Or people seeking advice on how to hide documents. Or how to avoid discovery. Or taking advice to avoid paying taxes.
Or someone in a fit of rage writing something threatening even though they were just venting. And then getting charged with terroristic threatening.
I could go on and on.
And don’t forget, the tools are going to get better.
An Added Issue
I am sure that the Post got access to the 47,000 conversations in a legitimate way. But it also seemed pretty easy and carried the risk that some of the participants didn’t realize their conversations were public.
And that makes me uneasy. As we have seen over and over in the digital world, what many think is private somehow became public. I worry that many of the millions of conversations with LLMs might end up being not private at all, either through legitimate or illegitimate ways.
What’s a Lawyer to Do
Back in the early days of eDiscovery, there was a push by many lawyers to try to educate their clients about the perils of not being careful with what they say in things like emails, texts, and other digital tools. Even with that, people still screw up and say things they shouldn’t, thinking or assuming that just because it’s digital it’s somehow private. Now we have a tool that in essence eggs you on to perhaps say or do something you shouldn’t and help you do it.
It’s incumbent on all of us — lawyers, legal professionals, vendors, and even LLM developers — to do all we can to make ordinary people aware of the dangers. There can be little doubt that savvy lawyers will use the proclivity of people to say too much to their favorite bot to their advantage in litigation and discovery, as will government investigative and regulatory entities.
Based on experience, I know many aren’t going to get the message. But that doesn’t mean we shouldn’t try. We need to lead the way in training our clients about the risks, not the other way around, when the damage is already done. We need to sound the alarm in ways they can understand.
The Post analysis is a start toward an educational process. We owe it to our clients to do more. And don’t forget we are ethically and practically bound to understand the risks and benefits of relevant technology. It’s hard to run and hide from the relevance of GenAI anymore.
Stephen Embry is a lawyer, speaker, blogger, and writer. He publishes TechLaw Crossroads, a blog devoted to the examination of the tension between technology, the law, and the practice of law.