Wednesday, November 7, 2012, 08:22 AM
Now that polls have closed we are able to compute the MAE (Mean Absolute Error) of a sensible (albeit naïve) baseline for predicting these elections: that past results just occur again.
In this table
you can find:
- Number of votes for Obama and McCain in 2008.
- Number of votes for Obama and Romney in 2012.
- The error made by predicting that Obama would obtain exactly the same percentage in 2012 that in 2008 in each state.
- The electoral votes obtained in 2008 and in 2012.
- The % of popular and electoral vote.
- The MAE (Mean Absolute Error)
Data for 2008 Elections was obtained from Wikipedia
. Data for 2012 Elections was obtained from politico.com
Please note that I do not fully understand most of the subtleties of the electoral college so the number of electoral votes may be not accurate.
Nevertheless, this "groundhog-day" baseline
is really good (for the US Presidential Elections): it only missed 2 out of 51 states with an impressive MAE of 2.75%
So, in short, what would be a reasonable MAE for an algorithm to be credible (to me)? A 5% improvement over the baseline, i.e. MAE = 2.61%.
Of course, if your algorithm is able to be below a MAE of 2.48% (a 10% improvement) I would be impressed.
If your MAE is greater than 2.61%, I'm really sorry but your algorithm is useless
However, I think that any sensible prediction should take into account data from past elections and, therefore, it would be really difficult to tell the difference between just using the baseline and "icing" that historical data with some extra information from social media.
As usual, contact me on Twitter if you please: @PFCdgayo