Because polling has done rather badly over the last few years, I thought I would look at how well Forum Research did with their polling of the by-elections.
Overall their results from their final polls look better than I had expected, but they are still over sampling people aged 55 and older. One result of this is that it is really questionable if one can call this a random sample of public opinion.
Results that are reasonably close to the election results are in blue, ones that are not are in red.
Etobicoke Lakeshore - a sample of 565 decided voters
Liberal PC NDP Green Others
By-elect result 42.0% 46.6% 7.8% 2.3% 1.3%
Forum July 30 43% 47% 7% 3% 1%
363 of the respondents were aged 55 or older - 59.6%. They are only 34.7% of the adult population.
In total 1,952 people gave responses in one of the four Forum polls of which 1214 were 55 or older. This means 3.4% of all the people over 55 and older answered a Forum poll during this by-election
That all noted, their results are very close here.
London West - 495 decided voter sample size
Liberal PC NDP Green Freedom
By-elect result 15.8% 32.7% 41.9% 4.3% 5.0%
Forum July 31 15% 38% 36% 6% 5%
The sample had 337 respondents aged 55 or older, 68% of total respondents. Only 35.7% of the adults are aged 55 or older.
Here they managed to correctly capture the Liberal support and the vote for Al Gretzky of the Freedom Party. The results for the PCs and the NDP were outside of the 95% confidence interval. They should only be this far off once in 50 to 70 polls
Ottawa South - 488 decided voter sample size
Liberal PC NDP Green
By-elect result 42.3% 38.7% 14.3% 3.1%
Forum July 31 36% 52% 9% 3%
The sample had 345 of 488 respondents aged 55 and older which is 70.7% but they are only 33.1% of the electorate.
Here they utterly missed not getting anywhere close to the results for any of the three major parties. The result is so far off for the PCs that this should not happen more often than once in 10,000 polls. The numbers are wildly off
Scarborough Guildwood - 359 decided voter sample
Liberal PC NDP Green
By-elect result 35.8% 30.8% 28.4% 2.2%
Forum July 30 38% 31% 27% 3%
250 of 359 respondents were 55 or older - 70.4%. They are actually 33.8% of the adult population
These results are within a reasonable range of the election results
Windsor Tecumseh - sample size of 292 decided voters
Liberal PC NDP Green Other
By-elect result 11.9% 20.1% 61.3% 3.6% 3.1%
Forum July 31 12% 28% 52% 6% 2%
195 respondents were 55 or older - 66.8% who are only 38.0% of of the adult population.
While they got the winner right, the margin they missed the NDP and PC support levels by are way outside of reasonable. They should only be this far off in less than one poll out of 100.
So two of the polls are within a reasonable range and the other three are no where close to a reasonable reflection of public opinion. In one case it is so far that as a result it should never occur in a proper random sample survey of the public.
Two out of five as reasonably accurate is not good enough and it means we can not realistically know which results are going to close and which ones are not. I do think there is a major methodological problem with Forum because of the very high response rates they get from people aged 55 and older. It shows us that their sampling system is not creating a true random sample of the public. Why are they missed younger people?