We’ve officially moved beyond the “ChatGPT made up a few cases” stage of the artificial intelligence ethics crisis. Now, the robots are apparently inventing witnesses and police testimony in murder appeals. Progress!
New Mexico attorney Stephen Aarons was held in contempt and fined $5,000 after filing an appellate brief containing “false testimony from wholly fabricated witnesses,” according to the state Supreme Court. Aarons represented Oscar Renee Sandoval, who was convicted of murdering the mother of his children and sentenced to life in prison.
As reported by Reuters, Aarons used ChatGPT to summarize a computer-generated transcript and other materials from Sandoval’s trial. He told the court that he assumed the chatbot would produce “a bulletproof summary.”
Narrator: It did not.
Instead, the brief included imaginary testimony, imaginary witnesses, and “fictional statements that the shooter was wearing dark pants and a white shirt.” That’s quite a leap from hallucinating a citation. This AI-generated material purported to rewrite the factual record in an appeal involving a client serving a life sentence.
The justices were understandably stunned that Aarons apparently trusted ChatGPT without confirming that the people and testimony it described actually existed.
Justice C. Shannon Bacon put it rather bluntly during an August hearing:
“Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what’s going on in the world? Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day.”
Ouch. Accurate, but ouch.
It’s difficult to argue with the judge’s assessment. Lawyers have been sanctioned across the country for submitting AI-generated briefs filled with fake cases, bogus quotations, and nonexistent citations. At this point, “ChatGPT sometimes makes things up” should not come as shocking news to anyone practicing law — particularly someone handling a murder appeal.
Aarons told Reuters that he had not understood the extent to which AI could hallucinate facts, calling the incident an “honest mistake” and “a lesson learned for all professionals who rely upon this powerful but sometimes unstable technology.”
Unfortunately for his client, this was one hell of a lesson.
The Supreme Court said Aarons had demonstrated “a lack of remorse and a lack of concern for his client.” In addition to holding him in contempt and imposing the $5,000 fine, the court said it would refer him to the attorney disciplinary board for further investigation.
Sandoval’s appeal remains pending, but it has since been reassigned to New Mexico public defender Kim Chavez Cook.
AI can be an incredibly useful tool for lawyers, but it is not a substitute for reading the record. It is most definitely not a substitute for confirming that every witness quoted in a brief is an actual human being who actually testified. AI cannot relieve lawyers of their obligation to verify every word they submit to a court.
That should go without saying. Yet here we are, saying it again.
Lawyer cites fake witnesses in murder case and blames ChatGPT [Reuters]
Staci Zaretsky is the managing editor of Above the Law, where she’s worked since 2011. She’d love to hear from you, so please feel free to email her with any tips, questions, comments, or critiques. You can follow her on Bluesky, X/Twitter, and Threads, or connect with her on LinkedIn.