Digging into the details of this week's poll

There is a lot more detail in the Innovative Research poll that came out yesterday than what is out in the media and I thought I would look at some of the more interesting things I can find

Issues

If we group issues into larger broad categories - take the numbers with a grain of salt because I am using the existing rounded numbers and not the raw results

Economic Issues 33% - more than half of this is affordable housing
Govt programs 19%
Environment Issues 13%
Social Issues 6%

They asked how likely people were to vote for a party which I find an interesting measure because it shows if the support is deep or not.

Here are the results for "Definitely" and "Very Likely" versus "Would Never Vote For" for each party

Party     +    -   net
Liberals 26%  22%  -4
NDP      19%  27%  -8
Greens   14%  28% -14
BCCP     11%  29% -18

As you can see the Liberals enjoy a significant advantage

Looking at the two major parties in a couple of sub groups
      under 35  35-54  over 54
Party     +       +       +
Liberals 25%     24%     28%
NDP      24%     17%     19%

Among younger voters the NDP is very close to the Liberals but still behind as you get to older groups the NDP falls behind

by region
       Van/LM North/Int Van Isle
Party     +       +        +
Liberals 29%     23%      21%
NDP      22%     15%      20%

They also have a table looking at how people said they would vote compared to how likely or they were to support the NDP and Liberals

What this table shows is there someone that is willing to seriously consider the NDP but does not vote for them is more likely to vote for the Liberals than the Greens.

Another thing that stands out is the high result for the BC Conservatives in the Not Very/Would Never category for both parties.   It says to me there is a group of conservatives in BC that hate both the NDP and the Liberals.   

The Liberals have a significantly better number for getting the vote or the people that would seriously consider their party than the NDP does.   That together with the size of the category means the BC Liberals have a much larger safe core pool of support than the NDP.

Now if we look at people that are open to voting for a party, Somewhat likely or better in the poll, here are he results.   This is a good measure of the theoretical ceiling for each party

Party     +   

Liberals 55%   
NDP      46% 
Greens   41%  
BCCP     32%  

This is very bad for the NDP.   For them to win they need to get almost everyone open to voting for them to vote for them.   At the same time the Liberals can achieve a win with about 80%

However you slice it, this is a very bad poll for the NDP.   If the data is a good reflection of reality the NDP has to do something drastic to change the landscape to be able to win.

Finally a fix for political junkies in BC - 2 new polls

Hi, my name is Bernard and I am a political junkie.   I get my fix from election days and polls.   As much as a I know these polls are not real proper data, it is all I have to go on.   It has been six months since I last had a BC poll hit.  

Here are the results from Ipsos and Insights West

Party   All of BC   Metro Van   Van Isle    Rest of BC
       Ipsos Insit Ipsos Insit Ipsos Insit Ipsos Insit
Liberal 42    34    32    37    32    21    44    35
NDP     36    40    39    39    39    52    28    35
Green   10    14     6    13    25    22     9    13
Cons    11    10    10     9     4     5    18    18
Other    1     2     1     2     0     0     2     1

So what can we make of these numbers?    First, there is no clear leader, there is a race.   Second, the Greens have a measurable strength on Vancouver Island, high enough that more seats are a realistic target.  Third, the moribund BC Conservatives have reasonable strength in the interior and north.

The 10% or so support for the BC Conservatives says to me there is a desire by a part of the BC population for an actual right wing party.   This number is in my opinion people parking their support.    The party has no leader, no media presence and no obvious grass roots organization but it is polling at 10%.  

The NDP has good enough numbers that it could win the next election but the polls are not showing the sort of monster lead like in the run up to the 2013 election.    I have to wonder if it is seen as a race if more people will come out and vote for the NDP?

Federal Political Opinion in BC created with a Google Survey

I have been testing out Google Surveys as a possible polling tool.   Here are the headline results

April 30-May 2
n=914 - decided voters only
Liberals - 28.5%
NDP - 27.5%
Conservatives - 23.5%
Greens - 15.7%
Other - 4.8%

I did a smaller one roughly at the same time - April 29 - May 1
n=545 - decided voters only
Liberals - 33.7%
Conservatives - 23.9%
NDP - 22.2%
Greens - 16.5%
Other 7.7%

I did a first one a week earlier - April 23 to 25
n=432 - decided voters only
Liberals - 31.3%
NDP - 26.2%
Conservatives - 25.5%
Greens - 11.8%
Other - 4.4%

For comparison, here are the EKOS results from April 22-28
n=400 - decided voters only
Liberals - 30.9%
Conservatives - 26.4%
NDP - 21.9%
Greens - 19.6%
Other - 1.1%

The numbers I am getting are in the same ballpark as the EKOS results but that does not mean my numbers are a useful representation of public opinion.   It may only be luck.  What I have been testing is how good the Google Surveys sampling method is, can it give me a demographically accurate representation of BC?

I think it may be a useful polling tool but in the testing I have made some observations that leave me feeling mixed about the quality of the results.  I will discussing the results in detail with Google on Monday,

Here are some quick observations:

• Too many of the responses come with no demographic data which means when you start a survey looking for 1,000 responses you are likely to only 700, I am finding 30% of the responses are without demographic data.   The results above are weighted for age and gender which means all the responses with no demographic data are not counted in the results above
• The geographic distribution of the results are not an accurate reflection of BC, it is far enough off that I am not happy with it.  I would like to calculate the weighting that would need to go into that to correct the number but I do need more information about how the locations are set.
• The responses consistently have had more men than women answer meaning the female responses are given a lot more weighting and this makes the results less reliable.

I have lot more detailed analysis to do over the next couple days.  I also want to run one more test, this time targeted specifically to Vancouver Island, I want to see if choosing a specific region would remove most of the responses with no demographic information.

If Google Surveys can be made function as a opinion poll, I would like to run a weekly public opinion poll of federal voting intentions in BC till the election.  If I go ahead with this I will be crowdfunding to cover the costs, but feel free to drop me a line now if you would like to support this idea.

If you want the raw data, I will happily share it with you, just drop me a line.

-----------------------------------------
Google Surveys can be very cost effective.  A single question survey within one province or the whole country costs only $0.10 per responses - a 2000 response survey only costs $200.

Google gets their respondents from people wanting to read content that costs but which by answering a survey you can read it for free.   

This is how a Google Survey looks:

Regional results from latest EKOS poll

I find this chart from the latest EKOS poll interesting

Overall the regional results have been reasonably consistent over the four polls EKOS has conducted since the new year.   

Clearly Alberta and Saskatchewan will remain strong Conservative and Atlantic Canada Liberal.   Ontario is a serious battle ground.   Though what I find interesting is the four way tie in Quebec and the Liberal lead in BC.

I had not expected to see the four parties tied in Quebec.   The Liberals and Conservatives are at roughly the same levels of support they saw in 2008, the Bloc is in the range of their 2011 results.  The NDP are down from 2011, so what will this mean in Quebec?

Here in BC EKOS keeps having the Liberals in the lead but organizationally on the ground the party is not nearly as evident as their polling numbers would indicate.   I wonder if the Liberal support in BC is weak or are people parking their vote?  One reason I wonder if parking the vote is the case is because 5% of people in BC answered some other party.  Another reason I wonder is because I am not hearing an enthusiastic endorsement of Trudeau in BC.

In BC the support for the Green has been very stable at between 15% and 17% over the four polls.   this is higher than the Liberals achieved in 2011 and only marginally behind their 2008 vote.

 The sample size is in BC is between 400 to 450 so it is large enough to tell us something, I just do not know what it is telling us.

An interesting trend in the EKOS polling numbers from July 23rd to February 3rd

I have my issues with polling in Canada these days, one company I have a lot of time for is Frank Graves' company EKOS.    I believe Frank' numbers do a good job of reflecting the mood of the public as a whole though not of the people who will vote.  

Here is what I find interesting about the trend over the last six months.

In the late summer of 2014 the Liberals had a large lead over both the NDP and Conservatives.  Since then there has been a clear upward trend for Conservatives since the end of October.  What is interesting is where the Conservative support has come from.   The Conservatives rose close to 10 percentage points in the last four months of which the support seems to have come equally from the Liberals and the NDP.  

What this highlights is that politics is not a simple left/right spectrum.   The public is just as willing to move from supporting Mulcair to Harper as from Trudeau to Harper.   The hardcore NDP support would never move to the Conservatives.   The people in question are the less politically active.   A lot more of them are open considering any of the three major political parties.

When we look at the low point of all three major parties in the last six months and this comes up to 74.6% of the vote.   Another 4% seem to fairly securely Bloc and 7% for the Greens.   This leaves about 14% that are very fluid.  

So where could this 14% go?   Any of the three major parties which means any one of them could win the next election.   One wildcard to this is if these people decide that Elizabeth May is a breath of fresh air.   There is a realistic chance the Greens could finish the election with as much as 20% of the vote.

I think the Conservatives have a upper limit of around 40%, the Liberals of around 38% and the NDP at only 32%.

I honestly do not know how the 2015 election will turn out.   I think anyone that projects what the result will be on October 19th is doing little more than make a wishful guess.   I think that close to half the seats in the country will be in play in this election.

One Year Since the 2013 Election and we have a new poll

Insights West has released the third poll of BC politics since the last provincial election and their second one.
           All of BC     Metro Van    Van Island           
Party     May 14 Dec 17 May 14 Dec 17 May 14 Dec 17 
Liberals  38%(-1) 40%   39%(-1) 40%   24%(-10)34%
NDP       39%(+3) 36%   38%(+1) 37%   43%(+4) 39%
Greens    14%     14%   11%(-2) 13%   27%(+4) 23%
BCCP       8%(+2)  6%   10%(+3)  7%    4%(+2)  2%

The results have not changed much since December other than one regional result.  On Vancouver Island the BC Liberals are now in third behind the Greens.

What can we take from this poll?  Not a lot at the moment other than the government and NDP are not far apart in support.  It also shows us that the Green Party does have a concentration of support on Vancouver Island and knowing where the strength of the Greens is on the island, I suspect the Greens and the NDP are not far apart in support in Victoria.

The poll is online panel with 824 respondents to this survey.

Problems with pollsters, again

I think at this point accurate polling in Canada that can predict elections on the large scale is no longer possible.

Forum Research polled people in the four recent by-elections four or five times during the by-elections and their results were not really very close at all to the final results.   There is a systemic problem with polling in Canada and I am not convinced pollsters have any real interest in addressing it other than Frank Graves at EKOS.

So people understand that I do have some idea about I am talking about, in the past I have run numerous telephone polls and came close to launching a polling company in the late 1990s.  My educational background used a lot of statistically methods to analyse topics from Canadian history.   I started university with the intention of a physics or math degree before choosing history.

The idea with a poll is that you are getting a random sample which will reflect the opinion of your whole population.  The two important factors at work are: knowing what population you want to sample and how to get a representative random sample.

Random Sample
Telephone polling worked reasonably well for close to 50 years when almost all homes had a phone and were in the phone book.   You could use the phone book as your randomizer and be fairly sure you would get a decent representative random sample.   Since most homes had more than one voter in them, you did have to ask for specific people when calling such as the youngest female voter or oldest male voter to ensure the random nature of the sample.

One reason in 1948 there was the famous Dewey Defeats Truman headline in the Chicago Tribune is because they relied on polling that indicated Dewey had a strong lead over Truman.  One of the problems with the polling at that time was that many homes, especially poorer ones, did not have phones and were not polled.  1948 was too early for telephone polling.

With the rise of mobile phones, the phone book and landlines are no longer a reasonable random sample of the public.   Without a real random sample you have no idea if you sample reflects the overall population.

To get a good sample you also need have most people answer the call.  Caller ID and voice messaging have reduced the number of people that will answer calls.  Interactive Voice Response polling (aka robocalls) has further reduced response rates to polls.   IVR polls can have as much as 95% refusal rates.   What this means is that the majority of random numbers you call do not respond making your sample not random any longer.

These two factors have made telephone polling not a way to get a random sample of the public and therefore not useful as a way to poll.

I will not go into the use of Bayesian credibility intervals as a substitute for a statistical margin of error other than to say the AAPOR is not too keen on it.

Population You Want to Sample
Elections are decided by the people that vote so those are the people you want to sample.   When voter turnout rates were close to 80%, a random sample of the general public was a reasonable substitute for the voting public.

These with close to half the people are not voting, and in by-elections 2/3s are not voting, the general public is no longer a reasonable representation of the voting public.   The problem is how do you know who will vote and who will not vote?  How do you demographically weight the sample?  Do you base it on the voter demographics of the previous election?

Most pollsters get 80-90% of their respondents expressing an opinion on how they will vote.  This is clearly not accurate when only 55% of the people vote.  Either the pollsters are only reaching the voting public, which I do not believe for a moment, or they are ignoring the fact that 1/3 or more of their respondents are not telling them the truth.  OK, that is not entirely true, many people will answer with an opinion and are intending on voting but then do not vote.

Ideally you want to sample the people that will vote in this election, but as work by Elections BC shows, a significant portion of the public that intends to vote will change their mind on election day.

What all this means is that we have no functional way to figure out who should be in the population we want to sample.

Online Opt-in Panels are not a Solution
The problem with all the online panels is that they are a self selecting part of the public.   They are in no way a representative sample of the public overall or the voting public.  Online panels making surveying easy, but this does not mean the data is at all useful.

I have been trying to work out any functional online model for polling, but I can not find a functional way to get a random sample.

Frank Graves at EKOS has a probability based panel - this means they choose you to in the panel pool and not the other way around.   I do not have data to know if his model works well or not.

Hybrid Approaches
This is using a number of different sources to get respondents.   It is one way to get around some of the issues but the added complexity of the hybrid models will make them much more prone to possible problems in the sampling process and how the data is weighted from the different parts of the survey.  

The hybrid model should make the survey more closely reflect the population you are sampling but the problem is how do get anything approaching a random sample?   I do not know if the second issue can be overcome.

That said, I suspect any functional model moving forward will likely be hybrid in nature.  Not only that, the methods I am trying out are hybrids.

My Current Best Possible Solution
What I am working on testing is a model that combines home visits and street interviews.   The idea is that in neighbourhoods with single family homes you have people canvas a specific number of the houses randomly selected.   You then supplement this by street level interviews of random people at  random times in random locations where you find apartments and condos.

This model seems to be able to produce a reasonable random sample of the general public, but it is unclear if there is a way to only capture the voting public.   What I am trying is to sample according to past election turn out rates by age.    I am not certain this will capture the actual voting public or not.

This model should work for polling within a single city but is not easy to scale up to the provincial or nationwide level.

Two Nanos Federal Polls in August

Nanos released a poll on August 21st in relation to the telecoms industry but had within it federal horse race numbers and were in the field from August 12th to August 19th.   They then released a poll today that was completed on August 22nd about the federal scene.    I have no idea why they waited so long to release the data from the latest tracking poll.

I am assuming the two polls were done complete separately which means we can see if there is consistency in the results for Nanos over a short period.   Whatever methodology a pollster uses should produce consistent results but it is rare to have one pollster in the field again so quickly when there is no election on.

Poll              CPC   Libs  NDP  Greens Bloc Sample Size 
Aug 20th Telcom   31.9  35.3  22.8   5.9   2.5     1553
Aug 22nd Tracking 29.7  35.7  24.8   5.5   3.9      897

I do see a reasonable consistency in the numbers which gives me a higher degree of confidence in their numbers.   What I do not know is if these numbers reflect what the voting population would be at the ballot box.

The Nanos results are broadly consistent with the Forum results on August 23rd and July 23rd, but we have had a very thin data set this summer with most companies not being in the field with only EKOS releasing ann other data this summer.  I would like to have seen at least on poll from Leger, Abacus, Ipsos, Angus Reid and Environics.

What I take from the Nanos polls is that the federal Liberal party has not continued declining over the summer but is the most popular party in the country.   It is still two years till the next federal election, can the Trudeau Liberals maintain this lead?   Will the return of parliament impact the polls?  How will the four federal by-elections play out?   Will the Senate scandal have an impact?

Will we see fewer political polls in Canada?

Given the recent track record of polling firms in Canada to utterly miss the actual results of a number elections, how will the media and public react to polls between elections?

Here is how many federal polls there were in each given month over a number of recent non-election years:
     2013 2012 2010 2009 2007 2006 2005 
July   2    3    8    6    3    7    4
June   6    8    7   10    3    3    7
May    4    7   11    5   10    4   15
Total 12   18   26   21   16   14   26

For a non election year, 2013 has seen the fewest national public opinion surveys in a long time.   During the minority years you may have expected more polls but this does not explain why in 2012 we had 18 over the the May to July timeframe and only 12 this year.  Two polls in July 2013 is the fewest in any month going back a long time.

There are more polling companies active in Canada now than in the past, and the newer phone and internet technologies make it easier to get started in polling than ever before so should be seeing a consistent rise in the number of polls publicly released but we are not.

Is this drop a reaction to the wrong results pollsters got in the BC and Alberta elections?

Also, how does the media make a story out of numbers released in a poll if they have no way of knowing if these numbers are any reflection of reality?

Ontario by-elections and polling from Forum Research

Because polling has done rather badly over the last few years, I thought I would look at how well Forum Research did with their polling of the by-elections.

Overall their results from their final polls look better than I had expected, but they are still over sampling people aged 55 and older.   One result of this is that it is really questionable if one can call this a random sample of public opinion.   

Results that are reasonably close to the election results are in blue, ones that are not are in red.

Etobicoke Lakeshore - a sample of 565 decided voters
              Liberal   PC    NDP  Green  Others
By-elect result 42.0%  46.6%  7.8%  2.3%   1.3%
Forum July 30   43%    47%    7%    3%     1%
363 of the respondents were aged 55 or older - 59.6%.  They are only 34.7% of the adult population.
In total 1,952 people gave responses in one of the four Forum polls of which 1214 were 55 or older.  This means 3.4% of all the people over 55 and older answered a Forum poll during this by-election

That all noted, their results are very close here.

London West - 495 decided voter sample size
                 Liberal  PC    NDP  Green Freedom
By-elect result  15.8%  32.7% 41.9%  4.3%  5.0%
Forum July 31    15%    38%   36%    6%    5%
The sample had 337 respondents aged 55 or older, 68% of total respondents.  Only 35.7% of the adults are aged 55 or older.

Here they managed to correctly capture the Liberal support and the vote for Al Gretzky of the Freedom Party.   The results for the PCs and the NDP were outside of the 95% confidence interval.  They should only be this far off once in 50 to 70 polls
   
Ottawa South - 488 decided voter sample size
                Liberal  PC   NDP   Green
By-elect result 42.3%  38.7% 14.3%  3.1%
Forum July 31   36%    52%    9%    3% 
The sample had 345 of 488 respondents aged 55 and older which is 70.7% but they are only 33.1% of the electorate.

Here they utterly missed not getting anywhere close to the results for any of the three major parties.   The result is so far off for the PCs that this should not happen more often than once in 10,000 polls.   The numbers are wildly off

Scarborough Guildwood - 359 decided voter sample
               Liberal  PC    NDP   Green
By-elect result 35.8%  30.8% 28.4%  2.2%
Forum July 30   38%    31%   27%    3%
250 of 359 respondents were 55 or older - 70.4%.  They are actually 33.8% of the adult population

These results are within a reasonable range of the election results

Windsor Tecumseh - sample size of 292 decided voters
               Liberal PC    NDP   Green  Other 
By-elect result 11.9%  20.1% 61.3%  3.6%  3.1%
Forum July 31   12%    28%   52%    6%    2%
195 respondents were 55 or older - 66.8% who are only 38.0% of of the adult population.
While they got the winner right, the margin they missed the NDP and PC support levels by are way outside of reasonable.  They should only be this far off in less than one poll out of 100.

So two of the polls are within a reasonable range and the other three are no where close to a reasonable reflection of public opinion.  In one case it is so far that as a result it should never occur in a proper random sample survey of the public.

Two out of five as reasonably accurate is not good enough and it means we can not realistically know which results are going to close and which ones are not.   I do think there is a major methodological problem with Forum because of the very high response rates they get from people aged 55 and older.   It shows us that their sampling system is not creating a true random sample of the public.   Why are they missed younger people?   

Federal Polling Lately - the trends show the Trudeau honeymoon is running out

After the selection of Justin Trudeau on April 14th as their leader the federal Liberal party saw a significant increase

Three companies polled shortly before Trudeau was leader and at least twice more since then.   When I look at the results there seems to be a clear trend

Forum
        CPC  NDP  Libs Bloc Green Lib lead
Jun 18  30   20   38    6    4      8
May 22  27   20   44    4    3     17 
Apr 15  30   19   43    5    2     13
Apr  2  29   25   33    6    4      4
Mar  7  31   27   30    6    5     -1

EKOS
        CPC  NDP  Libs Bloc Green Lib Lead
July 9  28.4 23.4 30.4  6.3  8.5    2.0
May 26  26.3 20.9 34.7  5.4  9.2    8.4
May  2  26.2 23.9 38.6  4.2  6.3   12.4
Apr 10  28.8 23.3 29.1  6.4  9.0    0.3
Feb 10  29.3 26.3 24.6  7.2  9.5   -4.7

Ipsos 
        CPC  NDP  Libs Bloc Green Lib Lead
June 25 30   28   33    6    3      3
May  28 30   27   36    4    4      6
Apr  30 32   25   35    5    3      3
Apr   3 31   27   32    6    4      1

Each pollster had their highest Liberal results during May.  All them show a significant increase after the selection of Trudeau which continued upwards till sometime mid to late May and then a steady drop since then.

The polling shows a clear honeymoon pattern to the support for Trudeau. 

What is interesting is that the support levels for the  Conservatives are actually quite steady with all three companies.

The question then becomes, where are the Liberal numbers headed and who is benefiting from the end of the honeymoon?  All of them still have the Liberals in the lead but the margin is narrow enough in the case of Ipsos that third place is possible for them.

If the three major parties remain close through to the election, what is going to matter more than anything during the 2015 election is how well the parties are organized on the ground.  Here in BC there is no indication of any life from the federal Liberals after the leadership race, I have not heard of any serious work being done to build the party on the ground. 

In western Canada there will be 104 seats in the next election but the Liberals only have any organizational strength in a handful at most.  Moderately better polling than the 2011 election results is not going win any more seats for the Liberals in the west.   Without taking more than four or five western seats, the Liberals can only become the official opposition if they win a lot of seats in Ontario and Quebec - they need to directly take 30-40 seats from the NDP.  

Federal and Provincial Party support analysis

Ekos has recently done a poll of the political landscape in Ontario.   One of the questions they asked is federal voting intention and the results are interesting

Fed      Fed % Prov Lib  NDP   PC   Green  
Liberal  35.1%    68.7% 16.1% 10.9%  3.7%  
CPC      31.6%    11.5%  9.2% 75.8%  0.7%  
NDP      22.9%    14.1% 69.7%  5.8%  7.8%  
Green     7.9%    17.4% 14.1%  6.3% 62.2%  
T
The rows are federal voting intention, the columns after the first one are how that translates into provincial voting intention in Ontario.

The first figure that stands out to me is that 75.8% of people saying they would vote for the Harper government are planning on voting for the Ontario PCs, this is the highest part loyalty between levels of government.   

Next I find it interesting that 16.1% of federal Liberal voters intend to vote for the Ontario NDP - is this a Trudeau effect?   These federal Liberals/provincial New Democrats are the largest single vote shifting block on the whole table.  

I also find it astonishing that 9.2% of people intending to vote for Harper federally intend to vote for the Ontario NDP.   This is not dramatically lower than CPC supporters willing to support the Ontario Liberals.

I am not convinced by Angus Reid's statement of why he thinks he got it wrong in BC

Angus Reid has come out and said that one of the reasons they were so far off on the election results is because their sample was weighted with too many younger people than actually voted.   I like the fact he is trying to figure out where they went wrong and why but even when adjusting for much lower youth voter turnout the Angus Reid's survey still had the NDP in the lead and would still have been significantly off of the results on election day but I have to wonder about how they managed to adjust their data to get their "new" results.

(One small issue, Angus Reid says the following: "Our final election poll was completed on May 8 and sent to our news client a day later."   But Angus Reid polled on May 9-10 and May 12-13 and no poll corresponds with the May 8th date so I am assuming his May 8th date is a typo.)

Pollsters have normally adjusted their poll samples to roughly reflect the demographic reality of the public on the ground.   Because there are so many factors you could weight for most pollsters only weight for a couple one them, the most common being age and gender.   For a long time I have raised the issue that who votes is not the same as the census demographics of the country.   For a long time I have noted there is a large gap between who answers a survey and who votes.  Most polls or surveys have 80-90% of the people express an opinion on how they will vote but the reality is that only about 50-55% of the people vote.  There is a clearly a systemic problem going on here in the way the surveying is done but when I have asked what is going on I the only answer I have received back from any pollsters on this is that they assume the non voters are acting in-sync with the voters.   I do not think this is the case.

The final Angus Reid survey had the following numbers:

NDP  45%
Liberals 36%
Greens 9%
Conservatives 7%
Others 3%

Angus Reid has said that when they adjust the numbers they had the NDP lead over the BC Liberals cut to only three points.   I can not figure out based on their own public numbers how they managed to get to that.   They had 803 respondents and adjusted for age, education, gender and region.   They do not give any details of how much they already had to weight their numbers.   They also did not say how many people expressed an opinion.   The lack of details on all these fronts makes it very hard to judge the quality of their work.

When I try to adjust based on 2009 BC voting patterns based on age I do not get a dramatic change in the final results Angus Reid says they got.    Remember, I do not have the full Angus Reid data to work with.  One of the problems is that they report everything the nearest whole number, doing this adds a +-0.5 percentage point rounding error to all the results before I do anything with the numbers.     The adjusted results I get from the final survey are still far off of the final results.   As an example, in no age category did Angus Reid find the NDP below 41% of the vote, 2 percentage points higher than the election day results.

I can only get to a three point gap in the numbers if I do not look at any under age 35 responses at all.

When I look further at their results, the data says over and over again that the NDP was going to win big.  One question looked at people that voted NDP and Liberal in 2009 and their plans for 2013.   These numbers should have had a very low number of youth voter respondents.   It showed a large win for the NDP that would have captured within it the low youth voter turnout

I think there is something much more fundamentally wrong with political public opinion surveys.   It could be that opt in internet surveys do not work as reflections of public opinion.   It might be that more and more people refuse to take part in any surveys.   It could be that people are lying.  Whatever it is, someone needs to do a lot more academic work on the issue and try to figure out what is going on.

A difference to be aware of in polling during an election

When companies poll between elections it is reasonable to ask about support for all the serious political parties, but when it comes to during election pollsters have to ask more detailed questions, spefifically they have to ask about the actual candidates on the ballot.

One nice thing about using online panels is that as a surveying method it is very, very easy to make sure that people get questions that have the actual candidates they can vote for.   The company has the address of all their panelists so knows which riding they are in.  They also know who all the candidates are and can very easily customize the survey to each riding.    I can not imagine any serious pollster with any understanding of statistics not doing this especially if they are using online panels.  

Can you imagine a company being to lazy to ask the right questions and then still publishing their results?   With the ability to so easily ask the right names for each riding no one could possibly be stupid enough to ask people about parties they can not vote for.  

So why is this so relevant in this election?  Because the Greens and Conservatives are not running full slates.   The Greens have 61 candidates and the BC Conservatives have 56 or 60 - 4 BC Conservative candidates are on the ballot without the party name.    The province wide numbers for the Greens and the BC Conservatives will look lower because they are not running full slates.   I means their support levels are higher in the ridings where they are running candidates than the average.

In the case of the Greens on Vancouver Island, the party is running 11 candidates in the 14 ridings and getting results in the 20% range for the whole island.   But since in three ridings there are no Greens, the real level of support on the rest of the island is closer to 25%.

So to read the polling numbers of the Greens and Conservatives more closely to what they would be if they ran full slates, you need to account for the ridings they are not running in.  

Jane Sterk - most trusted leader in BC, but that is not the point of this posting

OK, I am involved with the Green campaign so let me delight that in yesterday's Ipsos Daily Poll Jane Sterk was the most trusted leader in BC.

Here are the numbers:
Leader       Trust Distrust Net
Jane Sterk    46    31      +15
Adrian Dix    43    54      -11
Christy Clark 31    65      -34
John Cummins  24    58      -34

I do find it interesting that Jane Sterk is doing so well when it comes to approval of her as leader or the trust in her.   On a personal level I have been impressed with Jane since I first met her nine years ago and I am glad to see other people in BC are begining to see this

Now here comes my issue of concern with this survey from Ipsos.   They had 500 people from their poll of online panelists respond and they still had to weight their responses because they got too many people aged 25-54 answering and not enough aged 18-34.   This seems methodologically wrong to me because of the reason for moving to online opt in panels in the first place.   I am already concerned that public opinion polling in the run up to this election is not being done very well.

One of the the reasons you have an online panel is that you can find the demographically correct people for the survey, you should not have to weight your results but Ipsos did and most surprisingly Ipsos could not get younger people to respond.    Only 93 people under the age of 35 answered when it should have been 142. They only achieved 2/3s of the numbers they needed for the survey.

I wish I knew how big the Ipsos pool of panelists is and how it breaks down by age and location.   I am also interested in knowing how many people were asked to respond, finally, have any of these people answered a survey for them in this election?

As much as I like being able to say "Jane Sterk, most trusted leader in BC" this survey leaves me further concerned about the quality of public opinion polling in BC

On all the polls lately

I know I have not said much about the polls and surveys that have come out lately.  The main reason for this is that I am significantly concerned about the quality of the polling going on.  It is also one reason I am not updating my forecast of the election because I do not have the confidence I would like when it comes to the polls of what they are saying.

Just so that all of you know, I have run polls in the past and came close to starting a polling company at one point.  I have a good background in mathematical and statistical aspects of polling though I am not a Nate Silver, this is mainly because my life is too short to analyse the data in the degree of detail that he does.  Eric Grenier tries but I think he takes the results of the polling companies at face value.

More and more more pollsters are no longer conducting random samples of public opinion but are using online panels.    Online panels might be a good measure of public opinion but there is still very little data to show what it takes to have a good online panel and how to understand what the results mean.

The problem with all the online panels is that people have to opt in to be part of the panel and then they have to have to choose to take part in the survey.  The questions to ask about the panels are the following:

• How many panel members does the company have?   Abacus stated in their latest survey that they have 500,000 people in Canada in their online pool - this means I suspect they have between 30,000 and 40,000 in BC since they are not all that active here.  This is not nearly a large enough pool to gauge public opinion.   
• Are the panels accurate reflections of the province?  It is important to remember that the work by Angus Reid, Ipsos, Abacus and Insight West are all samples of their panels and not random samples of the public such as telephone polling has managed to achieve in the past.  It also means that is wrong to ever assign a statistical margin of error to an online survey which is something Angus Reid does all the time.   
• How do they make sure the panels are not stacked by the politically active?  I am a member of all the online panels and I have been part of more than a few surveys.   I am not an average random member of the public.
• How do they know their panel members represent either the public that votes or the census demographics of BC?   Since none of the companies give us any details on their panel membership we just have to trust that it is somehow a reasonable reflection of reality    I suspect all the online pollsters have very low numbers for people aged 65+ and that their small pool of people drive opinion results.
• How do they ensure the survey is answered by people other than those that are online 24/7?   The people that are online for too many hours of the day are not the ones that should form the core of the respondents to the online panels.

Overall I am remain skeptical of the online panels as an accurate tool of measuring public opinion.   I have been experimenting with models that would use social media to give us accurate results but so far I have not been able to develop one that I have statistical confidence in.  I have been reading all the academic literature on public opinion sampling methods and there is a major issue of how to poll without the traditional land line that every household had and everyone answered.

If not online panels, what about telephones?   Telephone polling is no longer a very good tool to get a random sample of the public because of the very high refusal rate and the ever increasing number of people with no land lines.

Companies like Ekos and Forum use IVR (Interactive Voice Response) to call people and get results for their polls.   I have used IVR polling in the past and used it in this election to get regional results for the Victoria area.   IVR has some major flaws.

The recent Forum IVR poll had the NDP and the Liberals only 4 points apart.  In one respect I am happy about Forum because they have a lot more detail in their report than other companies have but this means I can also see the flaws.   The poll reached a lot more old people than young people.  Forum weighted the results based on census demographics, but there are many ways they could weight the data and each way has a significant impact on the results.  

They asked how people voted in 2009 and what their voting intention is in 2013.  The results to this question on page 4 of the report showed only 28.7% of their poll respondents voted for the NDP in 2013 when in fact 42.2% actually did so.  It means their IVR sample was significantly light in NDP voters which means the overall result showed a close race.   If you weight for how people voted in 2009, the results would have been more like NDP 45% and Liberals 32%.

Ekos also used IVR and had some results that were not really in sync with the other companies.   Does this mean they are more accurate?   I do not know.

The problem with IVR polling is that you miss everyone with a mobile phone and you get a huge percentage of people refusing to answer.  When I have done IVR polling I have had between 85% and 90% refusal rates, others have told me they have had as high as 95% refusal rates.   These high rates mean that the IVR polls are not random samples of the public.

So in the end I am not convinced that any of the results to date are true representations of public opinion in this election.   That said, I do think I can glean some trends from the results:

• The NDP is marginally less popular than it was a few months ago, 44% instead of 46%
• The Liberals are marginally more popular  - 33% instead of 31%
• The Greens have a strong concentration of support on Vancouver Island which if it is concentrated on the South Island means the Greens are at about 30% in the Victoria area.
• The BC Conservatives are unchanged over the last few months with no data indicating any region with a consistent stronger result
• The results for "other" is high enough and concentrated enough that four independents could win their seats.

My one final thought on the polling, take them all with a grain of salt.   That means take my gleaned observations with that grain of salt as well.

New Angus Reid Survey for BC

Angus Reid was in the field again this week and released the results of their survey of their online panel members.

• NDP           48% (+1)
• Liberals      28% (-3)
• Greens        11% (+1)
• Conservatives 11% (+1)
• Others         2% (-1)

This not a dramatic change from last month.   Though I do like the fact we have one company in the field on a consistent basis.  It means I can see some possible trends.  

I thought I would average the last three surveys they have done to see what we get as a result because generally the numbers have looked stable since January.    This would be a sample of 2414 people over 60 days.   I know that since I do not have the survey results down to the tenths that I am compounding the +- 0.5 rounding error from each of the numbers and means even though it is a larger sample, there is an added +- 1.5 percentage point margin of error in each of the results.

Party      Avg result 2009  change
NDP           47.0%   42.2%  (+4.8)
Liberals      30.0%   45.8% (-15.8)
Greens        10.3%    8.1%  (+2.2)
Conservatives 10.0%    2.1%  (+7.9)
Others         2.7%    1.7%  (+1.0)

Interesting though is the regional breakout from Vancouver Island averaged over the last three surveys - that should have included close to 400 people over those three samples.

Party     Avg result 2009  change
NDP          52.3%   50.5%  (+1.8)
Liberals     23.0%   38.4% (-15.4)
Greens       18.0%   10.1%  (+7.9)
Conservatives 6.0%    0.3%  (+5.7)
Others        1.0%    0.8%  (+0.2)

I find the Vancouver results interesting because there is a rise in the Greens into the sort of popularity level where could expect a party to have a reasonable chance of winning a seat.   Is the Green support concentrated enough in some locations to bring about a win?

I also find it interesting that the NDP gain on the island has been quick small.

Leader Approvals
Among leader approval ratings, Clark and Dix have about as many people approving of them as there is support for their party.  This has been roughly true for the two of them since September of last year.  Dix has a net positive approval rating of 8 points at the moment which is in the ballpark of where he has been for most of the last nine months.   Christy Clark has a net approval/disapproval of -38 points, close to the worst she has seen in the last nine months and 13 points worse than her numbers in January.

 John Cummins is rising in approval numbers so that he is now 7 points ahead of his party, his highest lead in the last nine months.    What he suffers from is a large portion of the population that disapproves of him.   He has had a net negative approval/disapproval on par with Christy Clark's.   This does not bode well for the Conservatives to grow their support.

The one that really stands out is Jane Sterk, she is consistently scoring 13-16 points ahead of the Green Party.   She has a net of just into the negative but has by far the largest portion of the public that have no opinion of her though this has dropped to its lowest level in the last nine months.   The much higher approval numbers for Jane Sterk does say to me that there is space for the Greens to grow their support.

A Tale of Two Polls of the BC Political Scene

Yesterday two polls were released about BC politics, one by Ipsos and another by a new entrant into BC polling, Campaign Research.   The two results were dramatically different

I was called by Campaign Research as part of their poll.

Party   Ipsos  CR  Difference
NDP      51%   38%   -13
Liberals 32%   33%    +1
Greens    7%   12%    +5
Cons      9%   13%    +6 
Others    1%    4%    +3 

Either one or both of the companies made a huge error in their polling.   The only area the two surveys come close to each other is in BC Liberal support, their numbers for the other parties are so far apart that statistically it can not have happened.  The only way for these results to occur is for one or both of the two companies to have done something wrong in how they surveyed the public.  I am leaning towards both companies have something not right about how they survey.

The companies used different methods to survey the public:

• Ipsos sampled 1000 people from a pool of people from their online panels
• Campaign Research did an automated phone poll of 1,112 from 36,234 households dialed

Neither of them is a traditional in person telephone survey and could have issues of accuracy because of that.

First Ipsos
Over the last year the defining difference of Ipsos surveys have been the high total the NDP and Liberals combined get in their polls.  Ipsos has been 5 to 10 points higher than other companies for the combined NDP and Liberal support in all five of their last polls in BC.

Ipsos        Angus Reid    Other pollsters
Mar 2013 83  Feb 2013 78 Justasson Feb 2013 74
Nov 2012 83  Nov 2013 76 EKOS      Dec 2012 65.1
Sep 2012 81  Sep 2012 71 Justasson Oct 2012 76
Jun 2012 77  Jul 2013 68 Forum     Jun 2012 70 
Mar 2012 76  Mar 2012 66 Forum     Feb 2012 66

Ipsos averages 8.2 percentage points higher for the NDP and Liberals combined than Angus Reid does.

This bias towards the two parties currently in the Legislature means that Ipsos's result is the lowest one the Greens have seen since the early fall of 2012.   They are also oddly low for the "other" category.  With five or six serious independents running campaigns that alone would be worth about 40,000 votes on election or around 2.35% of the vote.   Eight people answered "other" in their poll which is only 1%.

In the 2009 election Ipsos's results were very similar to the other pollsters.  It is only after the fall of 2011 that Ipsos's numbers and those of other pollsters diverge.  This is also when Ipsos moved to an online panel for their BC polling.

So why is this and what does it mean?   I am not sure, but I think it likely indicates that Ipsos's online panel may be over represented by people with stronger connections to the two largest parties.   What it does clearly indicate is that the Ipsos methodology is producing different results than the other polling companies - either Ipsos is right and everyone else is wrong or vice versa.

What Ipsos can show us from their polling of the past is that the BC Liberals have lost some support and the NDP has gained some support.

Second Campaign Research
I do not have much of a track record to look at for this company.   It is really hard to judge how well they are reflecting public opinion but based on being called by them and a few other bits, I wonder about the quality of their process because their result for the NDP is so far off what anyone else has been getting.   This is the first single digit lead for the NDP by any pollster since March 2012.

Campaign Research said they called 36,234 households and got 1,118 respondents over 18 or older.  Only 3.1% of households phoned responded.   That sounds very low to me though it could be what it takes to get 1000+ responses.   I also have to say that as someone surveyed in this poll, something felt off about the poll. I can not put my finger on it but it seemed somehow less than the other polls I have received on the home phone.

Some specific questions I have:

• Did they only call landlines?
• Was everyone on the do not call list excluded?   
• How much did they weight their data?  Because they rounded to the nearest whole number it is hard to work backwards from their charts to know how they weighted various categories.
• Why did they not publish the regional numbers?

I can not reconcile the low NDP number in this poll to others taken in the last couple of months.   I need another poll from them soon to give me an idea if this was a badly done poll by them or a trend of their polling method.

My Analysis
I think that neither company has conducted a survey that is an accurate reflection of BC political opinion at this time.  

The support levels for the Liberals do not seem realistic as the problems of the last two weeks should have had an impact on Liberal support.   I think they are both high on the BC Liberals.

The support levels for the NDP seem too high from Ipsos but much too low from Campaign Research.  

The support levels for the Greens, Conservatives and "Other" seem to low with Ipsos and too high from Campaign Research.



   

Latest Angus Reid survey of public opinion on politics in BC

Angus Reid released their latest survey of the BC political landscape and little has changed since their previous survey in mid January.   Here are the headline results:

• BC NDP      47%(+1)
• BC Liberals 31%(+-0)
• Greens      10%(+-0)
• BC Cons      9%(-1)
• Other        3%(+-0)

In fact there has been little movement from their November survey.

The survey was done after the throne speech and after the budget.   It seems that they have had little impact on public opinion.

I normally do not pay much attention to what the opinions are about the leaders but in trying to get some more idea of what might be happening I thought I would look more closely at them than in the past.

What is interesting is that when you look at if public opinion improved, stayed the same or worsened of the parties and leaders over the last three months, it was worse for all the parties.   Christy Clark and John Cummins both have had dramatic negative shifts in public opinion, "momentum scores" of -36 and -24 respectivly.  Adrian Dix had 21% say their opinion improved and 28% worsened for an overall negative momentum for -7, which is much better than the other two but still going the wrong way.

No one seems enthused about any of the parties and there is no indication of any change that would indicate the Liberals have any hope of doing well in the election.  In fact the data says to me that the Liberals may not have hit rock bottom.

When asking about approving or disapproving of the leaders, here are the results:

Leader        Approve  Disapprove Not sure
Christy Clark 31%(+-0)   58%(+2)   11%(-2)
Adrian Dix    43%(-3)    41%(+7)   16%(-4)
John Cummins  15%(+2)    49%(-2)   36%(+-0)
Jane Sterk    24%(+1)    26%(-3)   49%(+1)

What is interesting in this table is that the approval numbers are not far from party support levels for three of parties, yes the NDP is 4 points higher than Adrian Dix and John Cummins is 6 points higher than his party.  It is the Green Party results that look out of sync with the others.   Jane Sterk is 14 points ahead of her party.

The numbers are not directly comparable because the total of all the approval numbers do not add up to 100 but saying you have a positive opinion of a leader should indicate you are willing to consider voting for the party.   It says to me that the party that has space to grow significantly are the Greens and this is not only because the leader is more positively received by the public than their party but because the party has the largest number of people out there without an opinion yet.

Ipsos survey on the BC Budget - It should scare the shit out of the Liberals

Ipsos released their survey on public opinion about the BC Budget today.   I tend to not like polls on complex subjects like budgets because so few people pay enough attention to have an informed opinion.  It means that in these sort of polls the uninformed overwhelm the informed in the poll and the results are not a good reflection of what the public is thinking.

Someone that is well informed will tend to have a lot more interest in the issue and it will matter to them a lot more.  Someone that is uniformed may not really care about it one way or the other.  Polling really works best for limited discrete choices such as which party would you vote for.

Ipsos used an online panel or 1200 people that said they heard something about the budget.  There are a few questions that do interest me.

Do people approve or disapprove of the budget is a good measure of if people are possibly supportive of the government or not.   Only 5% strongly approve though 36% somewhat approve for a total of 41% approving.   This to me is the absolute upper limit the government could draw from for support.   The 5% strongly approve is worrisome for the Liberals because all the online panels have a lot more active politicos answering the survey than any normal public opinion poll and the Liberals did miserably low.

The next question that is interesting is if people believe the budget is balanced or not.   Supporters of the "Free Enterprise Coalition" should have answered Yes to this but only 12% said they thought it was balanced and 72% did not.   This means most current BC Liberal supporters do not think the budget is balanced.   This result means a lot of people considering the Liberals are really holding their nose, that is not good.  You need a strong base and then pull in a few more points with the hold your nose crowd.   The BC Liberals could very likely see their vote fall in coming polls.

The next question is one of the dumbest ones in surveying  "Overall, what impact does this provincial budget have on your likelihood of voting for Christy Clark and the BC Liberals in the provincial election to be held in May this year?"  

It is a dumb question because someone who hates the Liberals is likely to say they are less likely to vote for the BC Liberals now even though there was zero chance of voting for the BC Liberals before the budget.   The only number that might tell us something is the More Likely to Vote for the BC Liberals one and that was 13%.   Once again, this is a pathetically low number for the government and should scare them.

The final question was if people thought Adrian Dix would have been better with the finances.   I think the results give us a bit of insight into how people align themselves politically   40% thought the finances would have been worse with Dix and only 24% thought they would be better.  This is a partisan split that is not really beneficial for him.

What this survey says to me is that the 30% that I thought was the probable floor of Liberal support is clearly no longer the floor.   I think the BC Liberals could potentially fall to 20%.   I also think that the NDP is not the place a lot of people want to go, I think a lot of people out there want another choice.