So then, that General Election, eh? For the benefit of those of my half-dozen readers who don't live in the UK, the gag here is that we get to vote our Labour government back in to power so that it can carry on invading things, abolishing our civil liberties, and whatnot. In principle we could also elect the Conservatives, a party which hasn't been in power for eight years and also has a policy of invading things, abolishing our civil liberties, and whatnot; but that's not very likely. Venturing into the really hypothetical, there are also the Liberal Democrats, of course; but when they were last in power they too had a policy of invading things, abolishing our civil liberties, and whatnot. (To be fair, that was almost ninety years ago.)
Anyway, you people mostly come here expecting a graph, and I wouldn't like to disappoint you. Here's my estimate of the probable outcome of the election, based on recent opinion polls recorded on Anthony Wells's site:
The axes of the figure show the estimated fraction of the population intending to vote for each of the major parties; the white circle shows the current estimate from opinion polls. The coloured areas show the regions of the plot in which -- under the assumption of uniform national swing -- each of the corresponding major parties would win a majority in Parliament.
For those of you who like this sort of thing, I will keep a copy of this and a more conventional plot updated at http://election.beasts.org/ as the campaign goes on. (Many thanks to Anthony Wells and Martin Baxter for collecting and publishing the data which goes into those plots.)
Of course, given the futility of the electoral process, you might decide that you aren't going to vote at all. If so, well, you might want to consider using Not Apathetic to tell the world why. The world might not be listening, but you never know. Thanks very much to Matthew and Sam, the unpaid mySociety volunteers who did most of the work on that site.
Equally, if you aren't actually entitled to vote but want to get involved in all the fun, well, there's still time! Under the new system brought in by our glorious leaders, you can vote in our election; all you need to do is fill out a form, post it to any local authority, and they'll send you a handy postal ballot form which you too can use to re-elect New Labour. And if that's too complicated for you, on Tuesday The Times published a handy guide on how to cheat in an election; in case that link doesn't work for you, here's an extract:
- Forge applications for postal votes, ask for them to be diverted to bogus addresses and then fill in the ballot papers.
- Offer to collect completed postal ballots from voters homes. Open them, scribble out crosses for rival candidates and insert your own in their place.
- Bribe or threaten postmen to hand over postal ballots.
and the marvellous, Dickensian,
Comments
Posted by Roy Badami, Thursday, 7 April 2005 21:22 (link):
You missed (at least) one alegedly real technique:
Set fire to post boxes in areas where you expect strong support for whichever party's chances you wish to harm.
Posted by Roy Badami, Friday, 8 April 2005 00:19 (link):
Actually, this was alegedged in the context of all-postal ballots, so perhaps not relevent here.
I presume and hope that they are not experimenting with our voting system during a general election....
Posted by Chris Lightfoot, Saturday, 9 April 2005 17:23 (link):
Well, they sort-of are. The two relevant changes are (a) rolling electoral registration, and (b) a relaxation of the conditions on which postal ballots are made available. On (b), previously, you had to have a reasonable excuse (e.g. being house-bound, or away on the day of the election, or whatever) before you could get a postal ballot. Now you don't need any excuse. And the new electoral registration forms typically have a tick-box you can use to get postal ballots for all future elections. There are no immediate proposals for all-postal General Elections so far as I know, but the number of voters requesting a postal ballot has been rising.
As I understand it, the justification for this is to increase turnout. As with electronic voting and any number of other such innovations, not enough thought has been given to the trade-off between trustworthiness and increased turnout. Hopefully, recent events in Birmingham -- I was going to say `the Birmingham fiasco', but that's obviously not an appropriate description of a well-organised, large and sophisticated fraud -- should sharpen our appreciation of this trade-off.
Posted by Roy Badami, Sunday, 10 April 2005 01:15 (link):
I was aware of both of those changes, and neither of those are that new. And not experiments in the sense I mean...
What I'm refering to is trials of non-standard voting systems, such as all-postal ballots, phone voting, electronic counting, electronic voting machines, etc, all of which have been authorised in recent local and (I think) European elections... Hopefully no such trials are happening in this election...?
Posted by Chris Lightfoot, Sunday, 10 April 2005 18:42 (link):
I've not heard of any, certainly.
Posted by Pete Stevens, Friday, 8 April 2005 10:55 (link):
You should have a fuzzy blob with errors based on the polling data error, and a textual description of the form
Current polls suggest Labour will have a majority of 66 seats. However, the errors suggest it might be as many as 350 or that the lib dems might win by 12.
Posted by Roy Badami, Saturday, 9 April 2005 16:22 (link):
That's a huge error bar, if Oggie's triangle is to be believed... ie the swing required for a LibDem win must surely be larger than the error bars on the polls?
-roy
Posted by Chris Lightfoot, Saturday, 9 April 2005 17:35 (link):
Yeah. The circle on the triangular plot shows the rough error bars (at 95%). That said, I've been lazy here and assumed that all polls have a 2-point error margin and that the kernel density estimate I use to smooth the data doesn't affect that. The errors which come in from the uniform national swing and various other assumptions would swamp that error if the election were too close to call.
Posted by Nick Vale, Friday, 8 April 2005 13:30 (link):
I want to vote for a hung parliament because it would be more interesting and hopefully less extreme than the last few. Any thoughts on how I/we could achieve that?
Posted by Simstim, Friday, 8 April 2005 14:49 (link):
Well, voting LibDem where they're a viable challenger will be a start as they're very unlikely to gain a majority and any seat they gain is one less for the Big Two. Similarly with the ScotNats and Plaid.
Posted by Chris Lightfoot, Saturday, 9 April 2005 17:33 (link):
This seems to be roughly what www.StrategicVoter.org.uk is aiming for, though the headline rhetoric is about the war against Iraq. They've had an opinion poll done -- see this analysis by (who else?) Anthony Wells -- which suggests that 18% of the population wants to see a hung Parliament:
Of the 18% wanting a hung Parliament, most would prefer Labour to be the larger party.
It seems that the poll didn't also ask about voting intention. It would have been interesting to see how big an overlap there is between the 18% wanting a hung Parliament and the c. 20% intending to vote Liberal Democrat.
Posted by Roy Badami, Sunday, 10 April 2005 16:26 (link):
It seems that the poll didn't also ask about voting intention. It would have been interesting to see how big an overlap there is between the 18% wanting a hung Parliament and the c. 20% intending to vote Liberal Democrat.
Though it would still be difficult to interpret that information. How many are voting for LibDem solely to maximise the chances of a hung parliament, and how many are genuine LibDem supporters who would prefer a LibDem govenment but see a hung parliament as a more realistic goal?
I guess I fall into the latter category, though I might not go as far as to descibe myself as a 'supporter'. I vote for them, though, at the moment (and have done since soon after New Labour surplanted plain old Labour)
-roy
Posted by Roy Badami, Sunday, 10 April 2005 17:49 (link):
Having asked you the other day whether you knew what the predicted Cambridge constituency outcome was, based on current national polls, I note that Strategic Voter gives this data as:
1.LibDem 32%
2.Labour 31.7%
3.Conservative 26.3%
That's about as marginal as you can get... I'm looking forward already to sitting up all night in front of the TV watching the results come in... :-)
Posted by Roy Badami, Sunday, 10 April 2005 18:12 (link):
On the other hand, the Electoral Calculus site predicts Cambridge to be a safe Labour seat:
1. Labour 40.68%
2. LibDem 26.07%
3. Conservative 24.67%
I'm immediately suspicious of anyone who believes they can meaningfully quote their predictions to a precision of 0.01%, but ignoring that it's entertaining just how different two predictions can be based on (presumably) largely the same poll data but a different set of assumptions about how to interpret it...
Posted by Chris Lightfoot, Sunday, 10 April 2005 18:40 (link):
OK, here's the simple uniform swing calculation:
... which differs slightly from Martin Baxter's calculation (his model is a bit more complicated than the one I use here -- I couldn't see any value in complicating matters any more), but gives the same basic result. Strategic Voter seems to use a variety of additional fudges in their model, which isn't really fully explained. In particular I'm very suspicious of this remark:
The Economist wrote something on this idea ten days ago, but to be honest it just looks like over-fitting to me. Perhaps Anthony will be along in a moment to offer an insight....
Posted by Roy Badami, Sunday, 10 April 2005 19:01 (link):
Thanks for the calculation.
One interesting aspect of the Strategic Voter approach is that they calculate swing by comparing current polls with polls taken the same number of weeks before the 2001 election, rather than in the more usual way of comparing current polls with the actual 2001 election results.
As for the fudge factor, it looks to be pretty small. They calculate the 2001 polling error as the difference between the later polls and the actual 2001 result, and then apply an ad hoc correction on the basis that they believe polls have got better. But their fudge factors are a couple of percent, so they don't explain the huge difference in prediction -- I guess it boils down to the difference in their definition of swing.
Which just goes to show how huge the potential error bars are when you're trying to use polls taken four weeks before the election to predict the actual election results...
Posted by Anthony, Sunday, 10 April 2005 22:38 (link):
The important piece of the Economist article is the last two paragraphs or so where the caveats are. If was John Curtice (since he wrote it), I would have spent rather more time exploring those caveats rather than just tagging them on the end, since they really do undermine the rest of the article.
Posted by Daniel Davies, Sunday, 10 April 2005 23:35 (link):
For what it's worth, I'm working on a new slightly oddball model, and I have Cambridge as a complete dead heat; Labour take it by 50 votes from LibDem (L/C/LD 36%/22%/36%). My model seems to (at present; it's rather half-baked at the moment and I have no very good idea where I might get the data from to bake it properly) materially understate LibDem chances, so it might even be an upset!
Looking at my backtesting, I overpredicted the Labour majority substantially in '01 (16k votes predicted vs 8.5k actual) ...
Posted by Chris Lightfoot, Monday, 11 April 2005 00:18 (link):
Interesting. Will we be hearing more details?
It occurs to me that the normal uniform swing model ignores the relative sizes of the electorate in different constituencies. That is probably worth investigating....
Posted by Roy Badami, Monday, 11 April 2005 20:00 (link):
I remain unconvinced that size of consituency matters. Surely you'd still expect a small constuency to have the same swing as a large constituency (in the absense of any other information)?
And even where constituencies change size (eg inner city constituencies getting larger) I don't see what you can usfully do to predict the voting behaviour of the people who have migrated into the constituency withoug a lot of ad hoc local knowledge
Posted by Chris Lightfoot, Monday, 11 April 2005 20:14 (link):
No, I don't think so. So far as I can tell, the reasoning behind Uniform National Swing goes something like this: the national swing is made up of a sum of the swings in each constituency, so therefore a fair estimate for the swing in a particular constituency should be the national swing. But that's not quite right, because (inter alia) Labour seats tend to have fewer electors (and fewer votes cast) than Conservative seats, and because the swings of Conservative seats and Labour seats turn out to be drawn from (slightly) different distributions.
That said, I think this is still dominated by local effects. I'll see if I can write something more useful about this later in the week.
Posted by Roy Badami, Monday, 11 April 2005 20:38 (link):
But that's not quite right, because (inter alia) Labour seats tend to have fewer electors (and fewer votes cast) than Conservative seats
But I'm not sure that kind of thing really helps you. If you knew how to weight the swings accordingly based on the type of constituency, wouldn't you just apply different swings to labour and tory constituencies, or perhaps based on their history over many elections?
If you want to automatically classify constituencies in some way, then looking at their political history is probably going to be more reliable than looking for small abnormalities in the size of their electorate.
Posted by Daniel Davies, Monday, 11 April 2005 21:10 (link):
Yeah. That's what I've been working on with the "Allocated Regional Swing Estimate" model (as in "here's a forecast I just pulled out of my ARSE"). It's not so much "not ready for prime time" as "not ready for a slot on BBC4 in the middle of the night", but the idea is thus:
1. Get the swing for each constituency from the '97 and '01 elections. 2. Divide it by the national swing to get a "local swing multiplier" for each election 3. Average the swing multipliers for each constituency over the two elections 4. Adjust the multipliers back toward 1 using what ponces call "Bayesian shrinkage" and practical chaps like me call "the one-third distance to the goal line method". 5. For each constituency, multiply the national swing by the local swing multiplier to get the local swing 6. Bob's your uncle; the swings still add up to the national swing but now you have something which is at least in principle capable of using the constituency-level information.
I actually carried it out by converting each constituency's percentages to a three-horse race, calculating the individual party shares and the change in them from election to election and thus calculating CON, LAB and LIBDEM swing multipliers. Obviously there is a big problem with my cavalier treatment of nationalist parties, and also with the fact that I'm only using two swings to calculate the multipliers (and two highly atypical swings at that). But I have always followed the principle that "if something's not worth doing, it's not worth doing properly".
I will be teaching a course on "Business Statistics and Forecasting For People Who Don't Care About Goodness Of Fit Measures Or Significance Tests" at North Bank Poly in the coming semester.
I also want to do something which uses the fact that the Scottish opinion polls probably contain information about some English constituencies, but realistically this isn't going to happen.
So anyway ... the new model with the shrink on the multipliers (which I have just added) has Cambridge as LAB over LIBDEM by 4000 votes on Baxter's predicted national vote shares and by 2000 votes using a moving average of the last week's polls listed on Anthony Wells' site. FWIW, the shrinkage seems to have mitigated the problem of massively underpredicting the LibDems, but I still have them losing seats: I have 211 CON, 378 LAB and 40 LD on my favourite way of estimating vote shares and 197 CON, 390 LAB and 42 LD on Baxter's numbers. Baxter has 190 CON, 377 LAB and 52 LD using his adjusted UNS model so I think we can say that at least one way of allowing for constituency-specific factors doesn't exactly make a qualitative difference.
Posted by dsquared, Friday, 8 April 2005 15:40 (link):
Are you going to update the triangle-chart (which is one good-lookin' graph, btw) in a way that allows us to see the movement from poll to poll? Maybe plot past polls as grey circles and connect them with a dotted line?
Posted by Chris Lightfoot, Saturday, 9 April 2005 17:16 (link):
I'll give it a go, but the last time I played with this (trying to show the effect of events upon public opinion) it was pretty hard to show the time evolution in a way that wasn't horribly confusing. Here's one example showing election results in Cambridge constituency (from before I'd written code to do the homogenous coordinates properly), and another, showing opinion poll data from the last months of 2003 (note that the second of those is zoomed in relative to the one in this post).
If you have any good suggestions for how to show the timeseries on the triangular plot, I'd like to hear them!
Posted by Alex Evans, Sunday, 10 April 2005 19:32 (link):
a suggestion you will hate... but here goes: get rid of the nasty grey arrows, at most, put them on the black curve itself as little black arrowheads. but most important of all, lose the text. for a print-only version I'd do some sort of numbering with key-at-the-side. for the web the answer-you'll-hate-but-which-is-the-way-forward-no-really-it-is, is to mark the events as small blobs on the curve and show the text/explanation for each one when the mouse rolls over it (or some such). my thinking here is that the unadorned black squiggly line isn't actually that hard to follow - and nasty as interactive graphs are, it might be the best way to keep it unencumbered by arbitrarily rotated chunks of text (yuch! what were you thinking). :)
Posted by Matthew, Tuesday, 12 April 2005 19:59 (link):
Chris,
Could you not show the triangle with only the likely vote shares showing, i.e. lib 15 to 25, Con 30-40, Lab 30-45 (or whatever), then it could be zoomed in a lot.
Matthew
Posted by Chris Lightfoot, Tuesday, 12 April 2005 20:17 (link):
Just for you: triangle-big.pdf. I'm not really sure this adds all that much, but it'd be reasonably easy to make the script generate it. (I wrote the code to do so but then commented it out -- I think it seemed a waste of space.)
Posted by Matthew, Tuesday, 12 April 2005 20:00 (link):
And why aren't they triangles in the chart?
Posted by Chris Lightfoot, Tuesday, 12 April 2005 20:12 (link):
The error margins? Errr... I haven't thought about it much, but naively, I wouldn't expect so, because it's an isotropic coordinate system and there are only two degrees of freedom[*]. The contours of likelihood will surely be circles around the point.
Or do you mean something else?
[*] this is, of course, a lie. There are as many degrees of freedom as parties standing, less one. However, it turns out that the fraction of the population expressing the intention to vote for `others' is roughly constant, so it can be factored out. That's also why the gridlines don't meet at points but at little triangles.
Posted by Matthew, Tuesday, 12 April 2005 20:47 (link):
I meant I'd forgotten the other parties' share. Thanks.
Posted by Matthew, Tuesday, 12 April 2005 20:53 (link):
I think it's much better in large scale actually (modest person I am), particularly if you plot movements in polls.
Now is it easy to shade bits of the grey where the Lib Dems would hold the balance of power (most of it I guess?)
Posted by Chris Lightfoot, Tuesday, 12 April 2005 21:00 (link):
Hmm. Plotting movement in the polls won't be very interesting, though, because so far all the swings have been small and Labour-Conservative. And one of the advantages of the large-scale plot is that it demonstrates how well our electoral system works (more on this later the week, if I have time).
In almost the whole of the grey area between the Labour and Conservative regions, the Lib Dems hold the balance of power. That said, UNS won't work as a model for large excursions from the results of the previous election, so anything in the lower-left should probably be taken with a pinch of salt.
Posted by The Last Toryboy, Friday, 22 April 2005 12:14 (link):
Interesting. I assumed Cambridge was a safe Labour seat what with its history of recruiting communist spies. :)
I might have to hold my nose and vote LibDem of all people.
Post a new comment.
Comments copyright (c) contributors and available under a Creative Commons License. See also the comments policy.