Friday, June 17, 2011

Frank Graves of Ekos analyses what happened

Frank Graves of Ekos close enough to the 2008 election to not have worried about the gap between the result of the Conservatives and where he had them.   In this election that error, and that of other parties, was large enough that he has taken a look at what is going on.

I think all the pollsters should take a hard look at what is going and try to understand why none of them came particularly close to the election result.

He has looked at what is going on and why his forecast of the election was wrong.   He highlights three reasons that may have caused the error

  • Sampling error on the part of Ekos
  • There was a lot of movement of voters on election day
  • The people that vote and do not vote are not statistically the same.
The last one is the one I have wondered about for some time and been placated by pollsters saying that the people that vote and do not vote are the same.   I have been convinced that there is a serious systemic error going for some time.   I raised this several times with Frank Graves.

Frank goes through and shows his sampling methodology is accurate.
He also shows that there is no evidence in the data before or after the election to indicate a large change in the voters on election day.

In the end what he finds is that the people that vote are not the same as the people that do not vote.   The support among the voters is higher for the Conservatives than the non-voters.    He manages to figure this out using the data that only Ekos was tracking - how likely people were to actually vote.   When he applies those numbers his results come close to the national result.

I used his numbers to make my final estimate of party support on May 1st and I was closer than any pollster was in the nation.   I still pulled back from going has high on the Conservatives that my data showed or as low for the Liberals, it simply was too far off from what everyone else was saying.

In the last ten days of the election Frank and I were messaging back and forth on Twitter about polling methodology issues, I raised issues about the small sample sizes in BC and the issue of who will actually be voting versus the polling numbers.  

In closing, I highly recommend all of you polling number crunchers read the article.
Post a Comment