Dodgy Poll

Written by Ben Ross - Talking Auckland

Not all what to seems

 

[Please note I am not taking aim at Digi-Poll in any shape or form and I do respect their credibility as a polling company highly. I do however, take aim at the NZ Herald for their “reporting” and not laying out the fall statistical facts about the poll clearly in their article]

 

I did notice the Herald Digi-Poll out this morning citing a poll about Mayor Len Brown. You can read the article here: Thumbs down for Len Brown – poll

What the online piece did not show was the actual graphs which can be seen below from Twitter earlier today:

The Herald Digi-poll subsequently brought a stinging rebuke from Stats Chat which said:

Beyond the margin of error

Now, the Herald-Digipoll is supposed to be a real survey, with samples that are more or less representative after weighting. There isn’t a margin of error reported, but the standard maximum margin of error would be  a little over 6%.

There are two aspects of the data that make it not look representative. Thr first is that only 31.3%, or 37% of those claiming to have voted, said they voted for Len Brown last time. He got 47.8% of the vote. That discrepancy is a bit larger than you’d expect just from bad luck; it’s the sort of thing you’d expect to see about 1 or 2 times in 1000 by chance.

More impressively, 85% of respondents claimed to have voted. Only 36% of those eligible in Auckland actually voted.

,.,,,

So, how could the poll be so badly wrong? It’s unlikely to just be due to bad sampling — you could do better with a random poll of half a dozen people. There’s got to be a fairly significant contribution from people whose recall of the 2013 election is not entirely accurate, or to put it more bluntly, some of the respondents were telling porkies.  Unfortunately, that makes it hard to tell if results for any of the other questions bear even the slightest relationship to the truth.

——–

You can read the full Stats Chat piece here: http://www.statschat.org.nz/2014/03/20/beyond-the-margin-of-error/

 

The bits in bold is what gives suspicion the poll was slanted to produce a “story” that was not truly there. Then again check this:

Thumbs down for Len Brown – poll

By Bernard Orsman 5:30 AM Thursday Mar 20, 2014

The journalist running the story is not known to be “objective” from time to time and has been pulled up before for either slanting or giving a misrepresentation (that is his opinion rather than a claimed fact).

So I would be very careful in trotting out this poll which has misrepresentations and a bad case of slanting as proof of fact against the Mayor. Using such a poll in that method will not do your credibility any good – although two of the oppositional five Councillors had (although expected).

 

Back to City Building we go as there is nothing to see from that Orsman piece.

 

One thought on “Dodgy Poll”

  1. DigiPoll are a good polling firm and I would expect they have sampled respondents pretty carefully by age and gender. Possibly their numbers have been affected by two factors.

    1) When you survey Auckland you tend always to get an over-representation of the North Shore and the Easter Suburbs. I’ve done lots of surveys and this always happens unless you set quota samples by WARD or similar. (This involves a lot more field work.) I’m expecting that given the Herald’s budget we may have this effect on the results.
    2) Asking how people voted last time is somewhat problematic. There is a lot of literature covering the tendency for voters to “mis-remember” when asked later on. Some may genuinely forget or may offer socially acceptable answers. This is a common problem and may – I can’t say with certainty – affect the results here.

    I have respect for DigiPoll and the other major polling firms such as Ipsos (for whom I once worked) Colmar Brunton, Reid Research and UMR. They work hard to deliver trustworthy results. However on small regional surveys (248 Aucklanders) there is only so much one can do, given a budget. I do think the NZH should adhere to standards (as do USA Today and many US papers) to report more than simply the sample size.

    Duncan Stuart
    Life Member and Fellow of the Research Association
    duncan@kudos-dynamics.com

Comments are closed.