The iconic pundit admits that his projections based on his poll sampling needs to be changed for the better…..
The linked piece is LONG and detailed….
At the end of each election cycle, I like to consider what changes we might make to our models in forthcoming cycles. I’ll be brief here, since I’ve already touched on most of these.
One set of questions is concerned with how reliable the polling will be going forward and what changes we might make in response to that. The changes we made to our pollster ratings earlier this year — namely, no longer privileging live-caller telephone polls — will have some knock-on effects on our models, since our models use our pollster ratings to determine how much weight to assign to polls from different firms.
Beyond that, though, I think questions about polling accuracy are probably more pertinent to 2024 than 2022. That’s partly because our congressional model already relies on a mix of polling and non-polling indicators and so is less sensitive to inaccurate polling than our presidential forecast might be. It’s also partly because the polls did quite well in the previous midterm election in 2018, so it seems prudent to see how they fare in 2022 before necessarily concluding that congressional polling is broken.
Also, I’d note that our models are already fairly conservative with respect to how accurate they expect the polling to be. That is, they assume that polling errors are common. That’s how our forecasts managed to be well-calibrated despite a fairly large polling error in 2020.8
What else to look at before 2022? Prior to last year, we’d already increased the weight given to partisanship in the Deluxe and Classic models and reduced the weight associated with variables such as candidate experience and fundraising. That looks to have been a good first step, but I suspect when we throw the 2020 results in and reanalyze the data, it will compel some further changes in that direction. And as I mentioned, while our models already assume that the errors between different races are correlated, those correlations may be even higher in an era of greater partisanship, so we’ll want to look at that, too.
Finally, there’s the question of what might happen to election analysis in an America where the Republicans might seek to nullify or invalidate election results. In some sense, these questions are external to our model, which is intended to forecast the results assuming all legal votes are counted and that electoral outcomes are respected. But we’ll keep our minds open as to whether there are ways to present our forecasts that make the scope and mission of our model clearer, or different types of model outputs that could shine light on these questions. In the meantime, we’ll continue to cover the threats to electoral democracy through our reporting….