| Hey readers, It's been a week since Election Day and not all of the votes have been counted, so it's premature to draw any firm conclusions about who had the best election predictions. But I want to dig into the conversation anyway because we do know enough to be able to draw some soft conclusions, which may have implications well beyond predicting the next election. Sites like FiveThirtyEight use polls and their own heuristics from past elections to predict the outcome. They take into account things like the economy, incumbency advantages, the demographic makeup of a region, and correlations between states. In analyzing the polls, they evaluate by sample size, pollster track record, pollster house bias, and the sample (registered voters, likely voters, etc.). On the eve of Election Day, FiveThirtyEight was projecting a 90 percent chance of a victory for former Vice President Joe Biden, and other modelers were even more confident: The Economist's forecast gave Biden a 97 percent chance of victory. Modelers also expected Democrats to take the Senate and expand their margins in the House. Biden ended up winning narrowly, with many state polls off by startling margins. The Senate looks likely to stay Republican, and while Democrats will hold onto the House by a narrow margin, they lost many seats. That's led to some second-guessing about polling and forecasts. "The forecasts were off. We were forecasting Biden to get 54.4% of the two-party vote and it seems that he only got 52% or so. We forecasted Biden at 356 electoral votes and it seems that he'll only end up with 280 or so. We had uncertainty intervals, and it looks like the outcome will fall within those intervals, but, still, we can't be so happy about having issued that 96% win probability. Our model messed up," Columbia statistician Andrew Gelman, who worked on the Economist model, said after the election. (Properly scoring each forecast will have to wait until we have full certified results.) But there was another approach to predicting the outcome of the election: betting on it. If there's a straightforwardly available, low-fee, low-friction market to place bets on elections, the market is effectively a form of paying people to think deeply about what will happen in the election and predict it. In many contexts, these "prediction markets" outperform explicit forecasts at predicting future events. The idea is that prediction markets can aggregate all the same information modelers use but can also incorporate information that the modelers don't have. There are a lot of reasons to think a good betting market would outperform predictors like FiveThirtyEight. The stock market is a kind of betting market on the future prices of stocks, and therefore indirectly on the purpose of companies, and it's astoundingly hard to reliably beat the market with an explicit model. But there's one problem with the prediction markets we have. As I wrote during the primaries, the existing election betting markets are not much like an ideal prediction market at all. The only legal one in the US is PredictIt, which has high fees, caps on how much money you can bet, and caps on the number of people who can participate in the market. All those things make it not worth most people's while to correct even obvious mistakes in the market. (Also, it crashed on election night and was hard to access while the results came in, during what would ideally have been a perfect time for bet-placing.) In other countries, people can place bets on Betfair or cryptocurrency exchange FTX, which don't have the same restrictions on volume but which are still much harder to trade on than the stock market. These barriers make betting markets significantly worse than they could be, if still compatible with a flawed alternative like polling. This past spring, I wrote about how Michael Bloomberg was leading the betting markets on the Democratic primary and shouldn't have been. The prediction markets got that call wrong. In the leadup to the general election, markets had Biden as a 65-35 favorite — much lower than the FiveThirtyEight and Economist forecasts — leading to widespread discussion of whether they knew something the polls didn't or were simply confused. On Election Day, the markets swung around wildly, briefly giving Trump an 80 percent chance of victory. Overall, though not all the votes are counted yet, it looks as though you could have made money on the betting markets by betting FiveThirtyEight was right about everything — which is to say that FiveThirtyEight comes out as better than the betting markets. Considering all the problems with polling, it seems frustrating that markets can't significantly improve on polls-based forecasts. Prediction markets have a lot of potential to solve the problems that dog election polling, which has missed pretty badly in the last two elections. But to get there, they need to be genuinely open for participation. Fees need to be low and transactions need to be straightforward. As we go into 2024, we'll be doing so with a lot of justified frustration with current polling practices, which were badly off in both 2016 and 2020. Prediction markets could be a helpful tool, but they're not reliably more than that right now — and they won't be until we change the way they're regulated and run. —Kelsey Piper, @kelseytuoc | 
No comments:
Post a Comment