Fantasy SCOTUS

FantasySCOTUS: Testing the Wisdom of the Crowds

By  
on

Ed. note: ATL has teamed up with the 10th Justice to predict how the Supreme Court may decide upcoming cases. CNN has called FantasySCOTUS the “hottest new fantasy-league game.”
How large of a crowd do we need before the crowd becomes “wise”? In this week’s 10th Justice, we will be check how accurate our members predicted five cases recently handed down: Smith v. Spisak, Power Marketing, LLC v. Maine Pub. Util. Comm’n, Kucana v. Holder, South Carolina v. North Carolina, and Wood v. Allen. While these cases are not among the most important cases of the term, they still contribute important data points to test the predictive capabilities of FantasySCOTUS.net, and enables us to further understand the wisdom (and limitations) of our crowd of users. For a refresher on confidence intervals and margins of error in FantasySCOTUS.net predictions, check out last week’s column.


The first case we consider is Smith v. Spisak, which asked whether or not the 6th Circuit overreached by vacating Spisak’s death penalty and went against the Antiterrorism and Effective Death Penalty Act in the process. The Supreme Court reversed the 6th Circuit ruling in a unanimous decision. Out of 114 members who predicted this case, 67 correctly predicted the Supreme Court would reverse the Sixth Circuit. (59%). Only 8 users predicted the unanimous decision, which constitutes approximately 12% of users who voted for reversal. Compared to the other voting splits, a 6-3 split (22 predictions) was the most predicted, followed by a 5-4 outcome (17 predictions). Given a 90% confidence level (+/- 7.58% margin of error) with a sufficiently large sample size, so this was a relatively close case, as the actual range was somewhere between 51.42% and 66.58%. Had the lower range dipped below 50%, we would have been unable to confidently predict the outcome. For this case, the FantasySCOTUS.net members generally correctly predicted the general disposition, but mostly failed to determine the correct split.
The second case we considered is NRG Power Marketing, LLC v. Maine Pub. Util. Comm’n, which asked whether the Mobile-Sierra doctrine applied to the Federal Energy Regulation Committee’s review of wholesale electricity rates. The Supreme Court reversed the D.C. Circuit’s decision that the doctrine only applied when a party to the contract attempts a unilateral rate change in an 8-1 decision, with Justice Stevens writing the lone dissent. As to the split, only one member predicted that only Stevens would dissent from the majority. This member was also the only one to predict an 8-1 split. The most popular split among reversal was a unanimous decision, with 13 predictions. Due to the close nature of the predictions, we could not predict with any level of confidence that this outcome would be accurate. Members generally predicted the correct overall outcome, but had difficulty with the split.
What do these cases teach us about the reliability of our predictions? That, and three more predictions, after the jump.

Sponsored

Topics