By a Newsnet reporter
A survey on Scottish independence which appeared to show support for Yes falling to an eight month low has been called into question amid claims of flaws in its methodology.
The ICM poll carried out on behalf of the Scotland on Sunday newspaper showed support for Yes falling by five per cent in one month, wiping out all gains made since last September. The same survey found support for No had increased by four points.
The survey, which was headlined as the worst result for Yes for eight months by the newspaper, was hailed by the anti-independence Better Together camp as evidence that momentum had changed in the referendum campaign. The dramatic swing in favour of the No campaign came despite widely reported problems within the campaign, with reports that its leader Alistair Darling had been sidelined.
However it has now emerged that the pollster ICM, which described its latest findings as “something of a reversal in the advance of Yes,” had altered its methodology from that used on previous surveys.
The voting expert Professor John Curtice commented on the change in methodology by ICM, saying the pollster had “refined their methodology”.
However online polling analyst James Kelly of Scot goes Pop, challenged the academic’s claim that the change was a ‘refinement’. Writing on his blog, Mr Kelly said: “the methodological changes have gone way beyond the one flagged up by John Curtice.”
The blogger revealed that the independence question, usually asked by pollsters first in order to minimise any influential effect from other questions, had in fact been asked third.
Mr Kelly added: “Indeed, John Curtice has spent a fair bit of the last few months rubbishing a poll conducted last summer by the BPC-affiliated firm Panelbase, simply on the basis that it asked the referendum question third.
“A few cynics might wonder why he pounced on methodological bad practice in a poll that showed movement towards Yes (and in retrospect he was probably right to do so, because it was completely out of line with other polls), but hasn’t said a word about the same problem in a poll that appears to show a favourable trend for No.”
Professor Curtice has previously criticised a Panelbase poll carried out on behalf of Newsnet Scotland, describing questions on devolution as “less than satisfactory”.
Data tables released by ICM show that those taking part in the survey were first asked if they felt uncomfortable being asked about independence.
Before being quizzed on independence, respondents were asked: “Thinking about the referendum on independence for Scotland, do you feel comfortable or uncomfortable being asked which way you might vote?”
According to Mr Kelly, the introduction of a question asking respondents if they felt uncomfortable may have influenced how they then answered the independence question.
He wrote: “Frankly it’s hard to think of a dodgier question to ask right at the start, because when respondents come to the referendum question a few seconds later they’ll be thinking to themselves, ‘ah, this is what some people are uncomfortable about being asked’, which is bound to influence their reaction in at least some cases.”
The poll featured widely across the Scottish media this weekend with Better Together claiming it represented a shift in momentum. However the anti-independence campaign has been taken to task for making exaggerated claims which include comparing polls from different pollsters and using different methodologies in order to suggest an increase in No support.
ICM was recently at the centre of a row after it emerged the pollster had carried out a secret independence survey on behalf of the Westminster Government. The coalition government refused to publish the survey findings amid rumours that it had found a rise in support for Yes.