In my previous post, I argued that probabilistic analysis was superior to pass/fail analysis as an approach to social choice theory. As a quick recap, pass/fail analysis tries to identify desirable criteria, then figure out which methods pass them and which methods don’t. Probabilistic analysis instead tries to identify how often failures of these criteria occur and how severe those failures are.
One consequence of pass/fail analysis is that it’s tempting to adopt what I would call a “dealbreaker criterion”, a voting criterion which a voting method must pass for you to even consider recommending it. If a voting method fails that criterion, it doesn’t matter if it is otherwise great; that failure is a dealbreaker. I’m not sure how many people actually do this, but I do know that it’s common to pick a criterion to emphasize as being crucial, which at the very least comes close.
A good example of this would be the favorite betrayal criterion, which is passed if voters are never incentivized to give less than maximum support to their favorite candidate. One of the leading voting reform organizations, the Center for Election Science, places a lot of emphasis on this criterion. In their FAQ page, the first bullet point after the question “How is approval voting better than our current system?” is “Voters can always vote for their favorite candidate, whether they have a good or bad chance of winning”. The two single-winner methods they support, approval voting and score voting, both pass the favorite betrayal criterion. At the very minimum this is quite close to being a dealbreaker criterion for them, and it’s highly likely that some of their members do hold it as a dealbreaker criterion even if the organization doesn’t quite do so.
Having a dealbreaker criterion means that you care about pass/fail analysis for that criterion, since you’re not interested in supporting any method that fails it. It doesn’t matter if a voting method fails favorite betrayal only 1% of the time; you want to feel 100% safe about voting for your favorite, not 99%! While this seems like a reasonable position to take, it leads to some surprising conclusions.
For example, while single-winner approval voting passes the favorite betrayal criterion, the proportional version that the Center for Election Science supports does not. In fact, essentially no proportional method passes this criterion, including every method currently in use, so adopting it as a dealbreaker pretty much means opposing proportional representation. This is not at all obvious, and I expect that most proponents of the favorite betrayal criterion haven’t really wrangled with this implication, especially if they consider favorite betrayal to be a dealbreaker.
This sort of thing happens often enough that it’s turned me off of having any dealbreaker criteria. As a couple more examples, the Center for Range Voting strongly supports the participation criterion, but recommends reweighted range voting even though it fails participation. Likewise, the Equal Vote Coalition strongly supports the cancellation criterion, but recommends allocated score voting even though it fails this criterion. It’s debatable whether the level of support for these criteria is high enough for them to truly be dealbreakers, but even if we assume they aren’t, it’s clear that adopting them as dealbreakers would have surprising consequences for those organizations’ recommendations.
This is another reason I dislike pass/fail analysis. The way it encourages people to adopt dealbreaker criteria can lead them to quickly reject voting methods without thinking through whether the failure of that criterion is really bad enough to dismiss the method. Under probabilistic analysis, this approach is much less natural than one that weights failures of criteria based on how frequent and severe they are, which captures a lot more information than simply having a dealbreaker criterion.