A Tale of Two Polls of the BC Political Scene

Yesterday two polls were released about BC politics, one by Ipsos and another by a new entrant into BC polling, Campaign Research.   The two results were dramatically different

I was called by Campaign Research as part of their poll.

Party   Ipsos  CR  Difference
NDP      51%   38%   -13
Liberals 32%   33%    +1
Greens    7%   12%    +5
Cons      9%   13%    +6 
Others    1%    4%    +3 

Either one or both of the companies made a huge error in their polling.   The only area the two surveys come close to each other is in BC Liberal support, their numbers for the other parties are so far apart that statistically it can not have happened.  The only way for these results to occur is for one or both of the two companies to have done something wrong in how they surveyed the public.  I am leaning towards both companies have something not right about how they survey.

The companies used different methods to survey the public:

• Ipsos sampled 1000 people from a pool of people from their online panels
• Campaign Research did an automated phone poll of 1,112 from 36,234 households dialed

Neither of them is a traditional in person telephone survey and could have issues of accuracy because of that.

First Ipsos
Over the last year the defining difference of Ipsos surveys have been the high total the NDP and Liberals combined get in their polls.  Ipsos has been 5 to 10 points higher than other companies for the combined NDP and Liberal support in all five of their last polls in BC.

Ipsos        Angus Reid    Other pollsters
Mar 2013 83  Feb 2013 78 Justasson Feb 2013 74
Nov 2012 83  Nov 2013 76 EKOS      Dec 2012 65.1
Sep 2012 81  Sep 2012 71 Justasson Oct 2012 76
Jun 2012 77  Jul 2013 68 Forum     Jun 2012 70 
Mar 2012 76  Mar 2012 66 Forum     Feb 2012 66

Ipsos averages 8.2 percentage points higher for the NDP and Liberals combined than Angus Reid does.

This bias towards the two parties currently in the Legislature means that Ipsos's result is the lowest one the Greens have seen since the early fall of 2012.   They are also oddly low for the "other" category.  With five or six serious independents running campaigns that alone would be worth about 40,000 votes on election or around 2.35% of the vote.   Eight people answered "other" in their poll which is only 1%.

In the 2009 election Ipsos's results were very similar to the other pollsters.  It is only after the fall of 2011 that Ipsos's numbers and those of other pollsters diverge.  This is also when Ipsos moved to an online panel for their BC polling.

So why is this and what does it mean?   I am not sure, but I think it likely indicates that Ipsos's online panel may be over represented by people with stronger connections to the two largest parties.   What it does clearly indicate is that the Ipsos methodology is producing different results than the other polling companies - either Ipsos is right and everyone else is wrong or vice versa.

What Ipsos can show us from their polling of the past is that the BC Liberals have lost some support and the NDP has gained some support.

Second Campaign Research
I do not have much of a track record to look at for this company.   It is really hard to judge how well they are reflecting public opinion but based on being called by them and a few other bits, I wonder about the quality of their process because their result for the NDP is so far off what anyone else has been getting.   This is the first single digit lead for the NDP by any pollster since March 2012.

Campaign Research said they called 36,234 households and got 1,118 respondents over 18 or older.  Only 3.1% of households phoned responded.   That sounds very low to me though it could be what it takes to get 1000+ responses.   I also have to say that as someone surveyed in this poll, something felt off about the poll. I can not put my finger on it but it seemed somehow less than the other polls I have received on the home phone.

Some specific questions I have:

• Did they only call landlines?
• Was everyone on the do not call list excluded?   
• How much did they weight their data?  Because they rounded to the nearest whole number it is hard to work backwards from their charts to know how they weighted various categories.
• Why did they not publish the regional numbers?

I can not reconcile the low NDP number in this poll to others taken in the last couple of months.   I need another poll from them soon to give me an idea if this was a badly done poll by them or a trend of their polling method.

My Analysis
I think that neither company has conducted a survey that is an accurate reflection of BC political opinion at this time.  

The support levels for the Liberals do not seem realistic as the problems of the last two weeks should have had an impact on Liberal support.   I think they are both high on the BC Liberals.

The support levels for the NDP seem too high from Ipsos but much too low from Campaign Research.  

The support levels for the Greens, Conservatives and "Other" seem to low with Ipsos and too high from Campaign Research.