Angus Reid has come out and said that one of the reasons they were so far off on the election results is because their sample was weighted with too many younger people than actually voted. I like the fact he is trying to figure out where they went wrong and why but even when adjusting for much lower youth voter turnout the Angus Reid's survey still had the NDP in the lead and would still have been significantly off of the results on election day but I have to wonder about how they managed to adjust their data to get their "new" results.
(One small issue, Angus Reid says the following: "Our final election poll was completed on May 8 and sent to our news client a day later." But Angus Reid polled on May 9-10 and May 12-13 and no poll corresponds with the May 8th date so I am assuming his May 8th date is a typo.)
Pollsters have normally adjusted their poll samples to roughly reflect the demographic reality of the public on the ground. Because there are so many factors you could weight for most pollsters only weight for a couple one them, the most common being age and gender. For a long time I have raised the issue that who votes is not the same as the census demographics of the country. For a long time I have noted there is a large gap between who answers a survey and who votes. Most polls or surveys have 80-90% of the people express an opinion on how they will vote but the reality is that only about 50-55% of the people vote. There is a clearly a systemic problem going on here in the way the surveying is done but when I have asked what is going on I the only answer I have received back from any pollsters on this is that they assume the non voters are acting in-sync with the voters. I do not think this is the case.
The final Angus Reid survey had the following numbers:
NDP 45%
Liberals 36%
Greens 9%
Conservatives 7%
Others 3%
Angus Reid has said that when they adjust the numbers they had the NDP lead over the BC Liberals cut to only three points. I can not figure out based on their own public numbers how they managed to get to that. They had 803 respondents and adjusted for age, education, gender and region. They do not give any details of how much they already had to weight their numbers. They also did not say how many people expressed an opinion. The lack of details on all these fronts makes it very hard to judge the quality of their work.
When I try to adjust based on 2009 BC voting patterns based on age I do not get a dramatic change in the final results Angus Reid says they got. Remember, I do not have the full Angus Reid data to work with. One of the problems is that they report everything the nearest whole number, doing this adds a +-0.5 percentage point rounding error to all the results before I do anything with the numbers. The adjusted results I get from the final survey are still far off of the final results. As an example, in no age category did Angus Reid find the NDP below 41% of the vote, 2 percentage points higher than the election day results.
I can only get to a three point gap in the numbers if I do not look at any under age 35 responses at all.
When I look further at their results, the data says over and over again that the NDP was going to win big. One question looked at people that voted NDP and Liberal in 2009 and their plans for 2013. These numbers should have had a very low number of youth voter respondents. It showed a large win for the NDP that would have captured within it the low youth voter turnout
I think there is something much more fundamentally wrong with political public opinion surveys. It could be that opt in internet surveys do not work as reflections of public opinion. It might be that more and more people refuse to take part in any surveys. It could be that people are lying. Whatever it is, someone needs to do a lot more academic work on the issue and try to figure out what is going on.
No comments:
Post a Comment