Dear Law Review editors:
Over the weekend — a professor (no, I’m not going to share their name because this isn’t about that) posted something that suggested someone at an R1, T75 law school had submitted something to his school’s law review that was 79% AI generated (according to Pangram, the latest and purportedly greatest AI detection tool).
I suggested perhaps we not go rush to judgment. I wrote on X: “A reminder that AI detection software triggers when the author is on the spectrum. We need to be careful with making accusations without facts. I thought a recent death would have highlighted that.” The death I was referring to was Jason Arday’s.
AI Built for Litigation. Verified by Design.
Grounded in authoritative content and verified at every step, Protégé is the only legal AI tool that delivers work you can trust—without exception.
People rushed to the defense of Pangram. This is not your father’s Oldsmobile. This is the latest, greatest, AI detection tool. No errors! And so on. Pangram: It’s foolproof!
No. It isn’t. And the fact that you think so tells me a lot about the problems the legal academy has with AI.
Bottom line: I have a problem with AI detection software being used if that policy a) is not disclosed, b) produces false positives for human writing, and the author has no ability to challenge the determination, or c) enables more sophisticated users to bypass such policies (false negatives).
The reason I have issues with this is because I have seen neurodivergent students being falsely accused of AI use. I’m not a fan of people claiming, “Oh, this tool is so much better than the previous tools we touted that killed the careers of innocent people.” Usually, that just leads to more killing of careers of innocent people.
Managing Partners are Navigating the AI Inflection Point.
New executive research from Ari Kaplan explores how law firm leaders are responding to changing client expectations, evolving economics, talent transformation, and AI governance.
The notion that law reviews might be screening law review submissions and potentially rejecting them because the journals detect AI is problematic for a variety of reasons. I have questions, as usual:
1. Did your law review disclose a no-AI policy? If so, were the authors of the AI work told that before they or their school paid Scholastica? If not, why not? (Scholastica is an accomplice here, but we’ll save the problems with Scholastica for another blog post.) Some journals have AI policies, and you have to dig to find them. Maybe that’s the point. But it would be helpful if there were in fact some Scholastica selection boxes related to it.
2. Does your law review know that AI detection tools, EVEN Pangram, are not foolproof? There is some literature on this. (Pangram stans: Yes, I know if you are a true Pangram devotee there will be no convincing you.) But, perhaps look at some of the literature. For example, this piece suggests that:
Commercial detectors on published abstracts punish acceptable AI-assisted editing, compound risk for English-language learners who use AI for translation, and fail to catch humanized synthetic text. Light edits are flagged at 64-80% by Pangram and 38-49% by GPTZero. After humanization, more than 96% of AI-labeled rewrites evade Pangram and GPTZero. Together, these results define an integrity catch-22: high detector flags on assisted writing alongside near-total misses after humanization. Integrity programs therefore need transparent AI-use norms and process evidence, not standalone detector scores.
3. Assuming you find AI in the piece (AI-assisted editing? AI summarizing? AI writing? AI what?) and assuming you have a policy against AI, what policies do you have in place for appeal of the finding? A quick rejection without more conveys no information. Was the author rejected for content, for AI screen, because your journal closed, or because the author is not at a T10 law school? If the answer is AI, what if you have falsely rejected an article for AI use when in fact none was used (false positive)?
4. Did you know one reason AI detection tools can never be 100% accurate is human languages are not regular languages? The world of human words is infinite, and AI detection will never fully capture it all. So, how many false positives (nay, how many careers) are you willing to risk?
5. Did you know that other AI detection tools, and perhaps all of them, may be biased against neurodivergent people? Anecdotal evidence also suggests authors who have written entirely their own works have been subject to AI detection, even with Pangram (see the second experiment in the link).
In short, there are two problems: If you are opposed to AI in academic writing, you are potentially enabling it due to the problem of false negatives and overreliance on the AI detection tool: Your AI detection does not necessarily catch AI writing that has been cleansed. Your process also suffers from the problem of false positives. Writing that is not AI could potentially be labeled as partially or wholly AI generated.
Law reviews — my dear friends — I’m fine whichever way you come out: AI supportive, AI hostile, or some as yet undiscovered happy medium. The key issue is whether you have disclosed your view of scholarship and whether your detection tools provide accurate information with a chance of rebuttal from the author. Or, whether you are enabling more sophisticated AI users to better screen their usage despite your implied policy against such usage.
Ask yourself: If your professor detected AI in your work, would you be happy with your professor giving you an “F” and not disclosing why?
Law professors reading this: Now think about your AI policies with respect to students and how much those policies match the level of fairness you would seek in the law review process.
Also, hey, can we finally have that discussion about what is scholarship and its purpose rather than the usual discussion of how to measure how cool we all are in terms of metrics?
Disclaimer: It wasn’t my article. I don’t use AI for law review writing (or any writing for peer-reviewed journals). Okay, look, sometimes I use it to see if my puns work. AI falsely tells me they aren’t. You’re not funny, AI.
LawProfBlawg is an anonymous law professor. Follow him on X/Twitter/whatever (@lawprofblawg). He’s also on BlueSky, Mastodon, and Threads depending on his mood. Email him at [email protected]. The views of this blog post do not represent the views of his employer, his employer’s government, his Dean, his colleagues, his family, his AI generated wife, or his AI imaginary friends.