{"id":30465,"date":"2024-05-10T11:46:02","date_gmt":"2024-05-10T15:46:02","guid":{"rendered":"https:\/\/abovethelaw.com\/?p=1058070"},"modified":"2024-05-10T11:46:02","modified_gmt":"2024-05-10T15:46:02","slug":"generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence","status":"publish","type":"post","link":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/","title":{"rendered":"Generating Predictions For <em>Grants Pass v. Johnson<\/em> Using Artificial Intelligence"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-650078\" src=\"https:\/\/abovethelaw.com\/wp-content\/uploads\/sites\/4\/2020\/04\/GettyImages-1158859085-300x229.jpg\" alt=\"cartoon The Supreme Court architecture\" width=\"300\" height=\"229\" \/><\/p>\n<p>The philosopher Archimedes once wrote \u201cGive me a lever that is long enough and a fulcrum to place it on and I will move the world.\u201d Is there a metaphor for the Court within this quote and if so do certain justices control a lever? In fact, we may find that the control or the center of the Court shifts this term from its point in the three previous terms since Justice Barrett joined the Court. While there is no way to be certain of such shifts in advance of opinions, oral arguments provide useful hints. Along with a collaborator, I developed a novel prediction engine, currently known as\u00a0<em>Optimized Legal Audio (OLA)<\/em>, based on synthesizing and then implementing specific improvements to a variety of existing text and audio-based features.<\/p>\n<p>OLA is an artificial intelligence engine in its infancy that tries to hear what the judges say, read the language they use, and through this to infer their relative preference for one attorney\u2019s argument over another\u2019s. It then generates a vote prediction for each justice (or judge, as it is not designed to solely be used to examine Supreme Court oral arguments).<\/p>\n<p>The idea that a computer can complete sentences and write poetry, take the bar exam, or create art was unimaginable until the advent of GPTs. The idea that a judge\u2019s words, the transcript, and the oral argument audio can give a reliable estimation of the outcomes now appears not only possible, but feasible with a high degree of accuracy.<\/p>\n<p><strong>Argument of Interest<\/strong><\/p>\n<p>While this Supreme Court term did not begin with the same high-profile myriad of cases like in the 2021 term with\u00a0<em>Dobbs<\/em>\u00a0and\u00a0<em>Bruen<\/em>, this term will in all likelihood be no less momentous.\u00a0 With cases ranging from defining speech rights, to gun rights, the future administrative deference, and executive immunity, several of these cases will more likely than not be taught in constitutional law courses for years to come. Just last week, the Court heard arguments in three cases with potentially immense repercussions.<\/p>\n<p>In the first,<em>\u00a0<a href=\"https:\/\/www.supremecourt.gov\/oral_arguments\/argument_transcripts\/2023\/23-175_dc8f.pdf\">City of Grants Pass, Oregon v. Johnson<\/a><\/em>, the Court looked at whether laws limiting camping on public property are a form of \u201ccruel and unusual punishment.\u201d In the second,<em>\u00a0<a href=\"https:\/\/www.supremecourt.gov\/oral_arguments\/argument_transcripts\/2023\/23-726_6jf7.pdf\">Moyle v. U.S<\/a>.<\/em>, the Court examined the potential for future enforcement of Idaho\u2019s Defense of Life Act, prohibiting abortions except in instances that would save the mother\u2019s life, in light of the Emergency Medical Treatment and Labor Act. Finally, the justices heard arguments in\u00a0<em><a href=\"https:\/\/www.supremecourt.gov\/oral_arguments\/argument_transcripts\/2023\/23-939_f2qg.pdf\">Trump v. U.S<\/a>.<\/em> where they examined the doctrine of presidential immunity relating to criminal prosecution for official acts while in office.<\/p>\n<p>These cases are not likely to be resolved much before the Court ends its term in the last days of June. Between now and then folks ranging from legal pundits, academics, and lawyers (among others) will speculate about the potential outcomes in these cases with far reaching ramifications.<\/p>\n<p>While there is no tried-and-true method to predict case outcomes from oral arguments, techniques are improving by the day. Not directly related to oral arguments, in 2004, several leading political scientists published a\u00a0<a href=\"https:\/\/www.law.berkeley.edu\/files\/columbia04.pdf\">paper<\/a>\u00a0comparing the predictions of legal experts with a statistical model finding that the model predicted 75% of outcomes correctly compared with 59.1% from experts. Since then, things have improved, although the marginal gains are slight.\u00a0 Additional insights have been derived from oral arguments ranging from looking at how\u00a0<a href=\"https:\/\/chicagounbound.uchicago.edu\/cgi\/viewcontent.cgi?article=1266&amp;context=law_and_economics\">frequently justices speak<\/a>\u00a0(Epstein, Landes and Posner (2009)), to the\u00a0<a href=\"https:\/\/scholar.harvard.edu\/sites\/scholar.harvard.edu\/files\/msen\/files\/scotus-audio.pdf\">pitches<\/a>\u00a0of their voices (Sen and Dietrich (2018)).<\/p>\n<p>In the words of baseball star Yogi Berra, \u201c[i]t\u2019s tough to make predictions, especially about the future.\u201d The best time to predict an outcome is when you have the maximum amount of information available upon which you can base the prediction. After the completion of oral arguments, all of the information at the public\u2019s disposal is at hand. Still the 70-75% threshold for predictive accuracy is a high bar to reach and exceed.\u00a0 This article applies some readily available methods to generate predictions for\u00a0<em>Grants Pass<\/em>.\u00a0 The article concludes by comparing the inferences from past methods to the novel OLA method mentioned above.<\/p>\n<p><strong>What do the arguments tell us?<\/strong><\/p>\n<p>Oral arguments occur at a specific point in each Supreme Court case. Since they are heard after the justices receive case briefs, the justices have time before oral arguments to get a strong sense of how they may vote in a case.\u00a0 There is no clear consensus about the extent with which oral arguments affect Supreme Court decision-making with\u00a0<a href=\"http:\/\/users.polisci.umn.edu\/~trj\/MyPapers\/Johnsonetal2006.pdf\">some papers<\/a>\u00a0presenting evidence that they may play a large role and others showing that the justices may have their minds made up about their decisions prior to oral arguments.<\/p>\n<p>The justices\u2019 votes are also often at least somewhat predictable notwithstanding oral arguments, especially with how they rule on recurrent issues over time.\u00a0 Still, some justices\u2019 measurable\u00a0<a href=\"https:\/\/bpb-us-w2.wpmucdn.com\/sites.wustl.edu\/dist\/4\/1744\/files\/2019\/01\/Ideological-Drift-Among-Supreme-Court-Justices-Who-When-and-How-Important-152qghh.pdf\">preferences shift<\/a>\u00a0longitudinally more than others.<\/p>\n<p>We are still left with the question of what oral arguments can tell us about how the justices may decide\u00a0<em>Grants Pass<\/em>. While there are limitless ways to measure the justices\u2019 oral argument behavior, this article incorporates four measurable dimensions: the quantity of speech, when the justices choose to speak, the sentiment of their speech, and the complexity of the language they use.\u00a0<em>OLA<\/em>, our novel method, is then introduced at the end.<\/p>\n<p><strong>QDAP Word Counts<\/strong><\/p>\n<p>The\u00a0<a href=\"https:\/\/cran.r-project.org\/web\/packages\/qdap\/index.html\">QDAP library in R<\/a>\u00a0is helpful in breaking transcripts down into these dimensions and others.\u00a0 A heat map of some of the word statistics from the\u00a0<em>Grants Pass<\/em>\u00a0arguments is below.<\/p>\n<p>Petitioner\u2019s Argument<\/p>\n<div class=\"wp-block-image is-style-rectangular\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_pet.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13137\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_pet.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13137\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/wc_grant_pet\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_pet.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"wc_grant_pet\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_pet.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_pet.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>Respondent\u2019s Argument<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_resp.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13138\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_resp.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13138\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/wc_grant_resp\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_resp.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"wc_grant_resp\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_resp.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/wc_grant_resp.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>The terms used in the heatmaps are <a href=\"https:\/\/trinker.github.io\/qdap\/word_stats.html\">defined here.<\/a><\/p>\n<p>Since the shading is relative speech, the absolute measures (number of words, sentences, etc.) show more speech for the attorneys: Theane Evangelis and Kelsi Corkran than for the justices.\u00a0 Focusing on the justices \u2013 Sotomayor, Kagan, Jackson, Kavanaugh, and Barrett all spoke more to the petitioner\u2019s attorney which leads to the inference that they will vote to affirm the 9<sup>th<\/sup>\u00a0Circuit\u2019s decision below (holding that removing such encampments equates to cruel and unusual punishment). The four potential votes in the opposite direction (dissent) based on this measure alone are from Roberts, Gorsuch, Alito, and Thomas.\u00a0 In a largescale predictive model, more than just word counts would be applied to generate an understanding of how the justices may vote.<\/p>\n<p><strong>QDAP Speech ordering<\/strong><\/p>\n<p>Along with speech counts, we can also look at order and extent of each justices\u2019 speech chronologically across the arguments. In\u00a0<em>Grants Pass,<\/em>\u00a0the chronological order of speech look like the following.<\/p>\n<p>Petitioner\u2019s Argument<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_pet.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13149\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_pet.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13149\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/dia_grant_pet\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_pet.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"dia_grant_pet\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_pet.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_pet.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>Respondent\u2019s Argument<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_resp.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13150\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_resp.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13150\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/dia_grant_resp\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_resp.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"dia_grant_resp\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_resp.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/dia_grant_resp.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>Here we can see that most justices controlled a single segment of the respondent\u2019s argument while the justices tended to speak several, intermittent times during the petitioner\u2019s turn. These graphs add nuance to the earlier graphs. They also show the ordering of speakers so we can visually see when certain justices potentially follow up on points from other justices.\u00a0 They show how Sotomayor and then Kagan followed by Jackson managed the first part of the petitioner\u2019s argument while Barrett, then Gorsuch, and last Alito were the main justice speakers during the middle of the respondent\u2019s argument.<\/p>\n<p><strong>QDAP Speech Polarity<\/strong><\/p>\n<div class=\"wp-block-group\">\n<div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<p>Another way to think about the justices\u2019 speech is through the sentiment or valence of their speech. In QDAP this is referred to as speech polarity. The polarity of each speaker\u2019s contribution during the argument is seen below.<\/p>\n<\/div>\n<\/div>\n<p>Petitioner\u2019s Argument<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_pet.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13165\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_pet.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13165\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/pol_grant_pet\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_pet.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"pol_grant_pet\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_pet.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_pet.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>Respondent\u2019s Argument<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_resp.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13166\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_resp.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13166\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/pol_grant_resp\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_resp.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"pol_grant_resp\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_resp.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/pol_grant_resp.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>These data give us several pieces of information which allow for two main comparisons. The first comparison is within each justice and between each argument so we can tell when a justice uses more positive language. The second is between justices and within each argument so we get a relative sense of the justices\u2019 positive and negative linguistic tone. The differences in polarity show that the liberal justices tended to be more positive during the respondent\u2019s argument in\u00a0<em>Grants Pass<\/em>\u00a0along with justices Barrett and Kavanaugh.\u00a0 Justices Alito, Thomas, and Gorsuch were on the negative end.\u00a0 Thomas (who spoke minimally) is on the low end for the petitioner\u2019s argument as well, but Alito and Gorsuch came across as more positive towards the petitioner. These general brushstrokes accord with the inferences from word counts.<\/p>\n<div class=\"wp-block-group\">\n<div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<p><strong>QDAP Automated Readability Index (ARI)<\/strong><\/p>\n<\/div>\n<\/div>\n<p>Another way to think about the justices\u2019 speech relates to the complexity of language they use. While there are multiple ways to measure the complexity of language, QDAP has a function for the\u00a0<a href=\"https:\/\/readable.com\/readability\/automated-readability-index\/\">Automated Readability Index<\/a>\u00a0(ARI) which provides an academic grade level associated with the difficulty of a text (the speaker\u2019s words here). The ARI algorithm is based on [characters \/ words], and [words \/ sentences] and is used in the graphs below.<\/p>\n<p>Petitioner\u2019s Argument<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_pet.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13174\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_pet.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13174\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/ar_grant_pet\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_pet.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"ar_grant_pet\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_pet.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_pet.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>Respondent\u2019s Argument<\/p>\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-medium\"><a href=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_resp.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13175\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_resp.png?w=281\" alt=\"\" width=\"281\" height=\"300\" data-attachment-id=\"13175\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/ar_grant_resp\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_resp.png\" data-orig-size=\"810,866\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"ar_grant_resp\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_resp.png?w=281\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ar_grant_resp.png?w=810\" \/><\/a><\/figure>\n<\/div>\n<p>While Chief Justice Roberts is on the high end for ARI scores for both sides\u2019 arguments, Justice Jackson is at the high end for the respondent\u2019s argument and the low end for the petitioner\u2019s. Kagan was at the low end for the respondent\u2019s argument and the high end for the petitioner\u2019s.\u00a0 We can see that there is less consistency across each speaker\u2019s ARI scores (either high for one and low for the other or both high or both low).\u00a0 Here, this may indicate other elements of the justices\u2019 approach to oral arguments \u2014 either strategic or subconscious \u2014 that are not directly correlated with their potential votes.<\/p>\n<p><strong>OLA: Where do we go from here?<\/strong><\/p>\n<p>Putting this all together, with existing methodologies we should have at best around a 70% chance of predicting the ultimate direction of the cases above. In other words, we should get approximately 2 of the 9 votes (22% error rate) incorrect. That could mean the difference between a 5-4 and a 4-5 decision and therefore, the outcome may completely surprise us. The obvious goal is to move beyond this hurdle.<\/p>\n<p>With OLA we sought to build on existing methods by combining several of the measures above (and others) and then testing them on past cases. We then took our model and applied it to the federal appeals courts with three judge panels. The new algorithm continued to outperform the 70% threshold. Next we tested it several trial court cases with a single judge and found that it continued to provide accurate predictions. We analyzed a variety of parameters and then brought them all together. This is still very much an evolving process.<\/p>\n<p><strong>OLA and Grants Pass<\/strong><\/p>\n<p>Based on this prediction engine,\u00a0<em>Grants Pass\u00a0<\/em>seems to fracture somewhat on ideological lines, but not in the typical fashion. We find that the likely majority for\u00a0<em>Grants Pass<\/em>\u00a0voting to affirm the 9<sup>th<\/sup>\u00a0Circuit\u2019s decision in the case is made up of Justices Sotomayor, Jackson, Kagan, Barrett, and Kavanaugh with the Chief Justice, and Justices Thomas, Alito, and Gorsuch in dissent.<\/p>\n<p>How do we generate these predictions? An example might help. Below is a visualization from OLA and it is highlighting a specific interaction between Justice Jackson and petitioner\u2019s attorney Theane Evangelis.<\/p>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-13199\" src=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ola1-2.png?w=1024\" alt=\"\" width=\"1024\" height=\"699\" data-attachment-id=\"13199\" data-permalink=\"https:\/\/empiricalscotus.com\/2024\/05\/03\/generating-predictions-for-grants-pass-v-johnson\/ola1-2-2\/\" data-orig-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ola1-2.png\" data-orig-size=\"1294,884\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"ola1-2\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ola1-2.png?w=300\" data-large-file=\"https:\/\/empiricalscotus.files.wordpress.com\/2024\/04\/ola1-2.png?w=1000\" \/><\/figure>\n<p>This point on the graph reflects an interaction where Justice Jackson says: \u201cBut punishment is happening. In my hypothetical, people are going to jail because they\u2019re eating in public\u2026Why is the Eighth Amendment not implicated?\u201d<\/p>\n<p>This is an instance where Jackson appears frustrated with Evangelis\u2019s response and the high point on the vertical axis corresponding to this point in the argument correlates with Jackson\u2019s intonation and language use. Based on the aggregation of this and other justice\/attorney interactions, using multiple methods, and adjusting for some of the shortcomings of the previous methods, OLA aims to improve the predictive capacity of trial courts, appellate courts, and Supreme Court outcomes.<\/p>\n<p><strong>Concluding Thoughts<\/strong><\/p>\n<p>Oral arguments do not mark the end of each case.\u00a0 The justices form initial coalitions after the arguments, but\u00a0<a href=\"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1065912912442111?casa_token=D8uIPzozcBEAAAAA:SpyokjjTTP68YobqxiFrGy1rwv72auyBMenR10kcuN-yuQDI9qqoYnQ9oLoEU1zh_sETr5L8Q7yPIo0\">historically<\/a>\u00a0justices still shift votes up to around 10% of the time from their initial vote after oral arguments to their final vote on the merits.\u00a0 This is due, at least in part, to the justices\u2019 agreement or disagreement with positions in the draft of the majority or dissenting opinions. With this knowledge in hand, no prediction engine is likely to get all votes right in each case.\u00a0 As we get closer to that point, marginal gains are harder and harder to come by.\u00a0 We\u2019ll see how these predictions hold up when\u00a0<em>Grants Pass<\/em>\u00a0and other decisions in cases argued this term are finally released.\u00a0 Stay tuned. More predictions are likely to follow.<\/p>\n<hr \/>\n<p><strong><em>Adam Feldman runs the litigation consulting company Optimized Legal Solutions LLC. For more information write Adam at\u00a0<a href=\"mailto:adam@feldmannet.com\" target=\"_blank\" rel=\"noopener\">adam@feldmannet.com<\/a>.<\/em><\/strong><em> <strong>Find him on <a href=\"https:\/\/twitter.com\/home\" target=\"_blank\" rel=\"noreferrer noopener\">X\/Twitter<\/a>\u00a0and\u00a0<a href=\"https:\/\/www.linkedin.com\/in\/adam-feldman-j-d-ph-d-48b91313\/\" target=\"_blank\" rel=\"noreferrer noopener\">LinkedIn<\/a>. He\u2019s also on Threads @dradamfeldman and on Bluesky Social @dradamfeldman.bksy.social.<\/strong><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p class=\"summary\">Optimized Legal Audio is an artificial intelligence engine in its infancy that tries to hear what judges say, read the language they use, and through this to infer their relative preference for one attorney\u2019s argument over another\u2019s.<\/p>\n","protected":false},"author":146860,"featured_media":650078,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3157,11],"tags":[11007,930,9509,2306,11505,11508,448,7],"class_list":["post-30465","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-courts","category-technology","tag-ai-legal-beat","tag-adam-feldman","tag-artificial-intelligence-ai","tag-courts","tag-empirical-scotus","tag-optimized-legal-audio-ola","tag-supreme-court","tag-technology"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence - Above The Law&#039;s Legal Tech Non-Event<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence - Above The Law&#039;s Legal Tech Non-Event\" \/>\n<meta property=\"og:description\" content=\"Optimized Legal Audio is an artificial intelligence engine in its infancy that tries to hear what judges say, read the language they use, and through this to infer their relative preference for one attorney\u2019s argument over another\u2019s.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/\" \/>\n<meta property=\"og:site_name\" content=\"Above The Law&#039;s Legal Tech Non-Event\" \/>\n<meta property=\"article:published_time\" content=\"2024-05-10T15:46:02+00:00\" \/>\n<meta name=\"author\" content=\"Dr. Adam Feldman\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Dr. Adam Feldman\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/\",\"url\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/\",\"name\":\"Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence - Above The Law&#039;s Legal Tech Non-Event\",\"isPartOf\":{\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#primaryimage\"},\"thumbnailUrl\":\"\",\"datePublished\":\"2024-05-10T15:46:02+00:00\",\"dateModified\":\"2024-05-10T15:46:02+00:00\",\"author\":{\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/#\/schema\/person\/9e57cd355074d78ab6040f4b7db46595\"},\"breadcrumb\":{\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#primaryimage\",\"url\":\"\",\"contentUrl\":\"\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/#website\",\"url\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/\",\"name\":\"Above The Law&#039;s Legal Tech Non-Event\",\"description\":\"A Legal Tech Adoption Guide For Perplexed Lawyers\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/#\/schema\/person\/9e57cd355074d78ab6040f4b7db46595\",\"name\":\"Dr. Adam Feldman\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/cde36c20235788cc90c9946b9dcd831233038179e8237c15028f98a6274fbcf8?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/cde36c20235788cc90c9946b9dcd831233038179e8237c15028f98a6274fbcf8?s=96&d=mm&r=g\",\"caption\":\"Dr. Adam Feldman\"},\"url\":\"https:\/\/abovethelaw.com\/legal-innovation-center\/author\/afeldman\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence - Above The Law&#039;s Legal Tech Non-Event","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/","og_locale":"en_US","og_type":"article","og_title":"Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence - Above The Law&#039;s Legal Tech Non-Event","og_description":"Optimized Legal Audio is an artificial intelligence engine in its infancy that tries to hear what judges say, read the language they use, and through this to infer their relative preference for one attorney\u2019s argument over another\u2019s.","og_url":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/","og_site_name":"Above The Law&#039;s Legal Tech Non-Event","article_published_time":"2024-05-10T15:46:02+00:00","author":"Dr. Adam Feldman","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Dr. Adam Feldman","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/","url":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/","name":"Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence - Above The Law&#039;s Legal Tech Non-Event","isPartOf":{"@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/#website"},"primaryImageOfPage":{"@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#primaryimage"},"image":{"@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#primaryimage"},"thumbnailUrl":"","datePublished":"2024-05-10T15:46:02+00:00","dateModified":"2024-05-10T15:46:02+00:00","author":{"@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/#\/schema\/person\/9e57cd355074d78ab6040f4b7db46595"},"breadcrumb":{"@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#primaryimage","url":"","contentUrl":""},{"@type":"BreadcrumbList","@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/2024\/05\/10\/generating-predictions-for-grants-pass-v-johnson-using-artificial-intelligence\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/abovethelaw.com\/legal-innovation-center\/"},{"@type":"ListItem","position":2,"name":"Generating Predictions For Grants Pass v. Johnson Using Artificial Intelligence"}]},{"@type":"WebSite","@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/#website","url":"https:\/\/abovethelaw.com\/legal-innovation-center\/","name":"Above The Law&#039;s Legal Tech Non-Event","description":"A Legal Tech Adoption Guide For Perplexed Lawyers","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/abovethelaw.com\/legal-innovation-center\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/#\/schema\/person\/9e57cd355074d78ab6040f4b7db46595","name":"Dr. Adam Feldman","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/abovethelaw.com\/legal-innovation-center\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/cde36c20235788cc90c9946b9dcd831233038179e8237c15028f98a6274fbcf8?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/cde36c20235788cc90c9946b9dcd831233038179e8237c15028f98a6274fbcf8?s=96&d=mm&r=g","caption":"Dr. Adam Feldman"},"url":"https:\/\/abovethelaw.com\/legal-innovation-center\/author\/afeldman\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/posts\/30465","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/users\/146860"}],"replies":[{"embeddable":true,"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/comments?post=30465"}],"version-history":[{"count":0,"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/posts\/30465\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/"}],"wp:attachment":[{"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/media?parent=30465"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/categories?post=30465"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/abovethelaw.com\/legal-innovation-center\/wp-json\/wp\/v2\/tags?post=30465"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}