March TNS-BMRB poll: an analysis


By Dave Taylor

The latest poll suggests that the independence debate still hasn’t fired the population. Other polling has suggested that most Scots don’t feel they have enough information to make up their minds yet, and the status quo is the normal default position for most people.

The three TNS polls in 2013 have Yes at 28%, 33% and now 30% : Noes at 48%, 52%, 51% : Don’t Knows at 24%, 15%, 19%. With a 3% margin of error in polls of this size, that implies little change in the underlying opinion that polls try to measure – no matter how much the rival campaigns try to spin any result that seems to be in their favour.

As Chris Eynon, for the pollsters points out, “Any reading of this as the settled will of the Scottish people would be misplaced. The contest for hearts and minds on Scottish independence has not even begun.”

And now for the polling geeks – others can look away now!

The Herald hasn’t paid TNS-BMRB for their polls since 2003, so any spin in their poll reporting comes directly from them. Sensibly, the pollster report concentrates simply on the the overall result, and not on the tiny sub groups.

However, let’s play Magnus Gardham’s game and look at the sub groups.

In his Herald article he noted that “Those aged 45 to 54 are most likely to support independence, with 41% in favour compared with 42% against. The youngest and oldest voters are among the strongest supporters of staying in the UK. Among 16-to 24-year-olds, 29% said they would vote Yes, compared with 52% voting No. Among the over-65s, 22% said they would vote Yes compared with 60% voting No.”

Since 16-24 year olds are more difficult for pollsters to contact than their elders, it’s quite normal for them to weight up the responses they do receive from those young folk that they do manage to contact very significantly, while they simultaneously weight down the responses of older folk. It’s an acceptable statistical technique – but inevitably reduces the reliability of the figures for under-represented groups. Comparing them over time is even less valuable.

Just for clarity, TNS-BMRB was only able to contact 100 16-24 year olds, instead of the 144 that were statistically required. Consequently, each of those actual responses was counted as 1.44 opinions. Nothing wrong with that as a statistical process, but only an innumerate would think that number meaningful in any way.

I accept that I may be being unjust to Gardham. He may have passed the equivalent of ‘O’ Grade Arithmetic in his schooldays.